Hello Everyone,
Hope someone will be able to help me out here.
I have a text file exported from my DTS package and it requires an '!' as a custom column delimiter.
Does anyone have any idea how I can use the '!' mark instead of the Tab or Vertical Bar as my Column Delimiter?
Would appreciate any suggestions.
Thanks,
Kay
hi guys,
i am working on a project witch involves creating packages on-the-fly to import data from txt/csv/xls files according to some definitions on the database.
so far, i have been doing fine.
now we are planning the ASP.net page that enables the customer to define the input file format that will be imported to the system. we want it to have the same listBox as the FlatFileConnectionManager Editor has to define some properties, such as - column delimeiter.
the code to set the column delimiter looks like this:
SSISRunTime.IDTSConnectionManagerFlatFile90 myFilecn = null;
myFilecn = (SSISRunTime.IDTSConnectionManagerFlatFile90)package.Connections["InputFileConnection"].InnerObject;
DtsConvert.ToConnectionManager90(package.Connections["InputFileConnection"]);
SSISRunTime.IDTSConnectionManagerFlatFileColumn90 col
col = myFilecn.Columns.Add(); //.....
string colDelimiter ="|" ; // it actually gets the data from the database... but it is the same thing
col.ColumnDelimiter = colDelimiter;
when we deal with the simple characters- "," , ";" , "|" ... we have no problems with setting the delimiter. but how can i set the delimiter to Tab? or {CR} ? {LF}?
i tried to look at the dtsx- XML , and i see that the column delimiter that is defined when i choose Tab is: _x007C_, but when i try to do something like this:
col.ColumnDelimiter = "_x007C_" ;
it doesn't work. the same happens when I try "{t}" or "Tab".
how do i solve this problem, and enable the user to select Tab as a column delimiter?
Thanks!
Good day everyone,
I have a package that reads data from a CSV file, transforms it and finally loads it in a destination DB table.
My current problem lies in the parsing of the input flat file. I shall illustrate it using a small example.
Source File:
P;Product-1;Short Description for product 1
P;Product-2;Short Description for product 2
Problem:
I configured the flat file connection manager to use semicolon as the column separator. But then I have received some sample flat files where I found that the semicolon might be sometimes used as content of a column data.
Possible Solutions:
I have thought about 3 different solution and I would like to get your feedback and recommendations about them.
Alternative 1:
Use a complex column delimiter, which wouldn't be used in the data.
Example:
P#~#Product-1#~#Short Description for product 1
P#~#Product-2#~#Short Description for product 2
Question 1:
- Is it possible to define such a customized column delimiter for the Flat File Connection Manager?
- If yes, how can I do this?
Alternative 2:
Use double quotes around the data, which the Flat File Source Adapter must somehow recognize and trim before pushing the data down the Data Flow.
Example:
"P";"Product-1";"Short Description for product 1"
"P";"Product-2";"Short Description for product 2"
Question 2:
- Is it possible to configure the Flat File Source Adapter to work as described?
- If yes, how can I do this?
Alternative 3:
Use a Script Component and write the needed code for parsing the Flat File.
Question 3:
- Do you have further suggestions/ideas for solving this parsing problem?
Thanks in advance and my regards,
Samar
hi, i had been removed the row counts and the column spaces... but ifailed to remove the extra tabs between rows of data return from theosql output.how to detect the row delimiter?i noticed i can use bcp to have a more decent output file, but i don'twant to do too much of file read-write. the programming enviroment isquite easy to "capture" the output from the comand prompt.secondary, char(252), 253, 254, 255 is not being read by the commandprompt, for example, 253 turns into 132 when i decode the input.thankx.from alan.
View 1 Replies View RelatedI am getting an error importing a csv file both using SSIS and SSMS. The csv is comma delimited with quotes for text qualifiers. The file gets partially loaded and then gives me an error stating The column delimiter for column "MyColumn" was not found. In SSIS it gives me the data row which is apparently causing the problem but when I look at the file in a text editor at the specific row identified the file has the comma delimiter and it looks fine. I am using SQL Server 2008.
View 9 Replies View RelatedI put this together to export the user name /password to a csv file to test my SP to output the user name/password.
DECLARE @user_name varchar(50)
DECLARE @psswrd varchar(10)
SELECT @user_name ,@psswrd
FROM ngweb_bulk_enrollments
EXEC master.dbo.xp_cmdshell 'bcp NGDevl.dbo.ngweb_bulk_enrollments out C: est.csv -Sserver1 -T -t, -r
-c'
This works but I don't get the headers in the file. How can I include the headers?
I am transferring data from an OLEDB source to a Flat File Destination and I want the column width for all of the output columns to 30 (max width amongst the columns selected), but that is not refected in the Fixed Width Flat File that got created. The outputcolumnwidth seems to be the same as the inputcolumnwidth. Is there any other setting that I am possibly missing or is this a possible defect?
Any inputs will be appreciated.
M.Shah
I'm in a flat file connection manager editor and I have a flat file where the columns are separated by a space. Does anyone know how to specify a space in the column delimiter option? I've tried {' '} and {s} but these don't work. Not sure what the syntax is for indicating a space. Thanks in advance
View 1 Replies View RelatedHi,
I have a problematic file. It's vertical-bar delimited, but the file conn mgr can't find ANY of the pre-defined row delimiter types.
The data looks like this:
col1|col2|col3|||||||||
With a bunch of vertical bars on the end of each line.
How should I handle this file?
Thanks!
Hi Friend,
I am stuck with a problem and need your help. As we know, all columns that go to error flow of flat file source connection are displayed as a single column e.g. FlatFileSourceErrorOutputColumn, but my requirement is to extract the first column value from this FlatFileSourceErrorOutputColumn, my data is dilimeted by "|" pipe operator. I have created a script component to deal with this. However if we take FlatFileSourceErrorOutputColumn column as input column in script component, it comes as BLOB data. I wrote below code in transformation script component to extract BLOB data from column in string form and then do a Left function search to take first column out.
When I am running this script component I am getting '??????????' question marks as a result in Row.Pname.
Can anyone please help me understand if I am doing anything wrong in this script or suggest a better way to take the data out?
I appreciate your help.
Public Overrides Sub Input0_ProcessInputRow(ByVal Row As Input0Buffer)
'
'
Dim Error_Data As String
Dim Column_1 As String
Dim Len As Integer
Dim Buffer As Byte()
Buffer = Row.FlatFileSourceErrorOutputColumn.GetBlobData(0, CInt(Row.FlatFileSourceErrorOutputColumn.Length))
Error_Data = System.Text.Encoding.Unicode.GetString(Buffer)
Len = Error_Data.IndexOf("|")
Column_1 = Left(Error_Data, Len - 1)
Row.Pname = Column_1
End Sub
Thanks,
Kul
how do I make the output columns padded with extra space ? I intentionally set my output width larger than the input width, but the generated file is still jamming all the columns next to each other
View 2 Replies View RelatedI have a table which has few columns as Numeric value. I need to export the output into a csv file as a report with column headers. I have used bcp command. Here column name and Column Header name is the same. The BCP query which i have used is Below
bcp "SelectCompanyCode,MonthId,ActualityCode,CompanyCounterpartCode,LocalProductCode,LocalCustomerBillTo,
DestinationCountryCode,LocalCostCenterCode,LocalGLAccountCode,LocalProjectCode,TXCurrencyCode,
TXAmountYTD,LocalCurrencyCode,LocalAmountYTD,SourceSystemFromTable(nolock)" queryout D: est.csv -c -t ";" -r -S -T
Is there any way to export bcp with header or any other command to generate the csv file.
Hi,
is there any way to change the delimiter in a FlatFile Source Adapter dynamically? You can do that with the row delimiter but there is no expression for the column delimiter... Is there any other way besides changing the XML code?
Thanks,
Hello,
Does any one know a simple way of changing the delimiter value for a CSV report delivered via email from SQL Server 2000 Reporting Services? The default is comma.
From the research I've done it seems that it can be done by writing a VB script that calls the Render method, but I'm not a developer and it seems crazy to have to go to those lengths just to change one attribute of a rendering extension that is available out-of-the-box.
I hoped that I would be able to change a value in a config file (e,g, RSReportServer.config), but this seems not to be the case.
Thanks
Lempster :S
Hi,
I'm trying to design this package where i take data from a source and need to transform it into a flatfile with some extra static information.
I use a SQL script like this (ex.):
SELECT '
BS0220131264202400000130001'+cast(wa.perf_applicant_number as nvarchar)+'000000000' + wa.perf_firstname + ' ' + wa.perf_lastname + CHAR(13)+
'BS0220131264202400000330001'+REPLICATE('0',(15-LEN(wa.perf_applicant_number)))+cast(wa.perf_applicant_number as nvarchar)+'000000000' + WAPD2.strvalue+ '
BS0520131264202410001130001'+REPLICATE('0',(15-LEN(wa.perf_applicant_number)))+cast(wa.perf_applicant_number as nvarchar)+'000000000 tekst der skal stå på kortet' as nvarchar
FROM dbo.WAIT_Applicant WA (nolock)
This makes the text (from one record) split up over several lines in the output.
I succeded with this in a SQL2000 DTS package and the flat txt-file looked liked I wan't it to. But now i tried doing it in 2005. And now it is not workin' anymore
In my Flat File Connection Manager Editor i chose {LF} as the row delimiter and the preview looks really nice. Like this:
BS0220131264202400000130001000000015826727000000000Søren Hesth
BS0220131264202400000330001000000015826727000000000adfasdf
BS0520131264202410001130001000000015827207000000000 tekst der skal stå på kortet
But in the file that is created it doesn't split up over several lines. Instead of a carriage return it puts a [black box] - a sign which counts as the carriage return.
I don't know if I have explained this well enough, but I hope that someone can help me. I've been trying for 3 days now.
I have a date file with no delimiter like bellow
0080970393102312072981103378000004329392643958
0080970393102312072981103378000004329392643958
I just know 5 first number in a line is for example "ID of bank"
or 6th and 7th number in a line is for example "ID of employee"
How can I create a XML format file?
I have a text file that loads just fine with DTS 2000, but in SSIS it does not seems to recognize the row delimiter. For example, most rows in the text file have 10 columns, but some have 8 columns. For those with 8 columns, the SSIS is adding the data from next row, not padding the columns with nulls. Please help...
Thanks in advance.
I am building a generic SSIS where it takes a text source file and converts to a destination source file . However in the process I want to set the row delimiter and the column delimiter dynamically through the package variable. So that I can use it for any file transfer.
I saw the option for row delimiter in the file connection string property but did not see any column delimiter option.
I am trying to import data from a text file to SQL database through a DTS package. i am always getting "Column Delmimeter not found" error at record number 8000. i copied the 8000th record into the beginning of the file to test if there is any problem with data but still i got the error at 8000 th record. i know it was a problem before and was fixed with sp1. i installed SP4 and still gettting the same error.
View 4 Replies View RelatedHi,
I have a dts package that imports a number of text files into a SQL Server 2000 database table. The package has been set up to accept a text file with a row delimiter of carriage return and line feed ({CR}{LF}). Some of the text files I receive only have a line feed ({LF}) as the row delimiter and the dts package fails the file. Is there an activex script i can use that will scan the file and change the row delimiter as required?
i was going to use the filesystemobject which allows me to read a line at a time, however the Readline method doesn't read the new line character. The text files are too big to read into one variable and then do a replace.
Any help would be appreciated
I am using SQL Server 2000 SP3, Windows Server 2000 and Windows XP Professional. All systems are fully patched
Regards Justin
I have a flat file that is row delimited by x00 x0D x0A. Any ideas on how to specify the row delimiter in the Columns section of the Flat File Connection Manager?
View 2 Replies View Related When I use SQL 2000 DTS Export to create a fixed length flat file, the data rows are delimited by carriage return-line. Which means that when I open the flat file in a text editor like UltraEdit or WordPad, the data rows are broken out nicely (row ends at the max row length position and new row starts at position 0).
But when I use SSIS to create the file, the whole file is displayed as one line in WordPad. The data rows don't end at the max row lenght position in ultraEdit neither. From Flat File Connection Manager's Preview page, I can see the data rows are displayed properly.
Now I wonder if the flat file destination is a true fixed length file.
Hi Folks,
I would like to write my table to a delimited file but I seem to have no choice but to use comma as the delimiter. Is there any way I can choose the delimiter ?
Thanks.
Sid
I have this SP:
set ANSI_NULLS ON
set QUOTED_IDENTIFIER ON
GO
ALTER PROCEDURE [dbo].[GetSessionInformation]
@CustomerID int,
@Success bit OUTPUT,
@Email VarChar(55) OUTPUT,
@FirstName VarChar(55) OUTPUT,
@LastName VarChar(50) OUTPUT,
@PhoneNumber VarChar(50) OUTPUT,
@CompanyName VarChar(50) OUTPUT
AS
SET NOCOUNT ON
DECLARE @UserKey AS int
SELECT @CustomerID = CustomerID
FROM Customers
WHERE CustomerID = @CustomerID
IF @CustomerID IS NULL
BEGIN
SET @Success = 0
END
ELSE
BEGIN
SET @Success = 1
END
BEGIN
SELECT customerID, Email, FirstName, LastName, PhoneNumber, CompanyName
FROM Customers
WHERE CustomerID = @UserKey
How do I test it in management studio?
When I run a EXECUTE GetSessionInformation 56
I get this error:
Procedure 'GetSessionInformation' expects parameter '@Success', which was not supplied.
Thanks for any help!
I have a gazillion text files, each with dozens of records. The fields are pipe-delimited, the end of each record is marked by ASCII char code 28.
example:
05|11900307|1|CO|Gervais|Neil|NG8880|F|540|0|0|0|T|0|||F|||F 05|11900307|2|AO|MARSHALL|BRAD|BM7843|F|510|60|0|0|T|0|||F|||F
The " " is the Chr(28). The line above should parse into 2 rows...
**** time passes ****
An interesting discovery - if I paste that " " directly into the row delimiter box, without quotes or brackets, I get two records!
So I found at least one answer.
Is there another?
TIA
Hi,
I am trying to figure out how to override the column delimiter from a script task for a flat file connection.
Before outputing the data into a file, I have a variable which contains the delimiter to be used... but as it seems that this property cannot be changed through an expression, I assume it has to be done throught code...
However, the connectionmanager object doesnt seem to have the delimiter as an available property.
Any help would be appreciated
I am working on an OLAP modeled database.
I have a Lookup Transformation that matches the natural key of a dimension member and returns the dimension key for that member (surrogate key pipeline stuff).
I am using an OLE DB Command as the Error flow of the Lookup Transformation to insert an "Inferred Member" (new row) into a dimension table if the Lookup fails.
The OLE DB Command calls a stored procedure (dbo.InsertNewDimensionMember) that inserts the new member and returns the key of the new member (using scope_identity) as an output.
What is the syntax in the SQL Command line of the OLE DB Command Transformation to set the output of the stored procedure as an Output Column?
I know that I can 1) add a second Lookup with "Enable memory restriction" on (no caching) in the Success data flow after the OLE DB Command, 2) find the newly inserted member, and 3) Union both Lookup results together, but this is a large dimension table (several million rows) and searching for the newly inserted dimension member seems excessive, especially since I have the ID I want returned as output from the stored procedure that inserted it.
Thanks in advance for any assistance you can provide.
Do you know of some SQL that I can test a table to see if a column exists or not inside a stored procedure?
What I have is a table that contains data for a 10 year history. I asked at the time and was told at the beginning of every new year the table would create 2 new columns so I dynamically set up some of my stored procedure to provide for that….well as of Jan 22 they have not updated them so I have 2 options:
1) hard code the first year in there and change it when they tell us
2) test for the field if it is there start the countdown to grab the rest of the historic info….if not check for the next year until it finds the field in the DB.
I’d rather do 2.
Thanks for any help.
Hi,
I have a column in table which tells me whether Tax should be calculated on a price. This is stored in a column called taxIncluded. I have a 'price before tax' column e.g. grossPrice. I want to calculate the price after tax e.g. netPrice if the taxIncluded column is set to 1. How do I form my sql statement to test whether the taxIncluded is set to 1 & therefore add the tax at say 20%.
e.g.
id name grossPrice taxIncluded netPrice <--- calculated within SQL
-- ------------- ------------------- ---------------- -------------
1 Product1 10.00 1 12.00
2 Product2 20.00 0 20.00
A test database that we used in one of our implementation pilots wasabandoned around 4 months back. The database when abandoned had a logfile size of less than 500MB. The log file has been steadily bloating(just came to my attention) and has reached 8.5GB. The database hasnot been used since abandoned 4 months back at 500MB. Considering thefact that our live DB has a combined (data + log) file size of lessthan 2GB I do not want to delete the database before exploring thecause.Any ideas??? Thanks and appreciate your helpSQL Server 2000
View 1 Replies View RelatedIs there a way to check for System.BDNull when I use the column name instead of column index? The column Photographer (in the excample below) sometimes contains the value null, and then it throws an error. Or do I have to go back to counting columns (column index of the returned data) again? try { connection.Open(); using (SqlDataReader reader = command.ExecuteReader()) { if (reader.Read()) { albumItem = new Album((int)reader["Id"]); if (reader["Photographer"] != null) albumItem.Photographer = (string)reader["Photographer"]; albumItem.Public = (bool)reader["IsPublic"]; } } } Regards, Sigurd
View 2 Replies View RelatedWe are setting up a test lab environment with 100 machines. We want one master testing db that gets replicated to each to run scripted application tests nightly.
My goal is to minimize the amount of work to move this thing to each of the 100 test machines. I am wondering if we need to even have the sql local and invest in a monster db server with 100 copies of the db we restore and each test machine point to their own db on that server, or if I should use db mirroring or something to get the master test db to each of those machines instead.
I had an existing table with lots of indexes.
As a test (fro speed) - I added a non clustered column-store index.
When I run test queries it always ignores my new column-store index. Why?
Should I remove the old indexes, leaving just the column store?