Breaking Column Data Using File Layouts
Apr 8, 2004
Hi DBA's,
I need to break up a 34 byte column into several columns. Here is my problem.
Table A:
Col1..Col2..Col3..Col4..Col5..Col6..Col7..Col8..Co l9..Col10
Col 9 is char (2) and Col10 is char (34). It is this column that needs to be broken up into several columns depending on the value in Col 9.
Col1 to Col3 is the key to the record.
So say if record 1 has Col 9 value 'AA' then Col10 ( 34 bytes) is to be spilt into 10+10+10+4 (four columns).The value 'AA' can repeat for several records and the value in Col 10 can change for the same value 'AA'.
Now say record 27 has Col 9 value 'BB' then Col 10 is to be split as 5+25+4 (3 columns).
There are 15 such unique values of Col 9. I have the file layouts for Col 10 for each distinct value of Col 9. So using the file layouts and Table A which exists in my database how do I proceed.
Need I make 15 tables ( one each for the 15 unique Col 9 values). These structure
Col1..Col2..Col3...Col9 (the key fields and Col 9) will be common to every table. Plus the file layouts will serve as additional columns specific to each of these tables.
Kindly help.
Vivek
View 14 Replies
ADVERTISEMENT
Apr 4, 2007
I am trying to load a file using SSIS that contains records with two different layouts in one data file but in the flat file connection I can only specify one layout and this is causing the records with the second layout to be loaded incorrectly.
The different record layouts can be identified by the first character of the record. Example: If Field begins with "A" then assign one layout; "B" assign second layout.
Has anybody come accross this issue, if so some guidence would be appreciated.
Thanks,
ire_ssis
View 10 Replies
View Related
Sep 18, 2006
We have a flat file format generated from a vendor. It contains a "mainframe" view of the data with a header record, batch header record, detailed records, batch trailer record and trailer record. It arrives as a .dat file. What is the best approach to extract the necessary columns out of this file to populate the corresponding SQL server tables and rows?
View 3 Replies
View Related
Aug 9, 2000
I currently have a Name column in one of my tables that is in the following format:
Name
Doe, John P
and I need to break this down into three separate fields in a different table:
Last_Name First_Name Middle_Name
Doe John P
Could someone please help me with this issue?
Thanks,
Peter
View 1 Replies
View Related
Dec 21, 2005
I have a column that has text delimited by a percent sign that I wishto turn into rows.Example:A column contains ROBERT%CAMARDA, I want to turn that into two rows,one row with ROBERT and antoher row with CAMARDA.I will have source rows that have zero, one, or many percent signdelimiters that will correspond to that many rows (One percent signwill create 2 rows, 2 percent signs will create 3 rows and so forth).Any thoughts?TIARob
View 1 Replies
View Related
Sep 18, 2007
I’m okay with simple queries but as I’m no expert and have failed to find perhaps the correct wording to describe this method, if at all possible to do, so I have come to ask here.
What I would like to do is take a column from a query and then break down that column into separate results.
So the full query results:
36,18/09/2007 10:00:00,NULL,000102000304,NULL
The column I would like to brake down is (Unique Reference Number):
000102000304
And I would like to break it down to get the last 2 parts (0003 and 04):
0001 | 02 | 0003 | 04
Is this possible to do?
If so where should I be looking or what should I be looking at?
Many Thanks
View 3 Replies
View Related
Jun 19, 2007
Does anyone have a routine that takes a row of data from database, duplicates/triplicates it, appends some information to it and writes it out as 2/3 CSV rows.
Basically I need to do the following.
Selected from database:
Row1 Col1 Col2 Col3
Output:
Row1 Col1, Col2, Col3, abc
Row1 Col1, Col2, Col3, def
Row1 Col1, Col2, Col3, ghi
Thank you
View 5 Replies
View Related
Jan 21, 2004
Hi,
I have a text file (5 MB). It appears as a single line in a text editor. But actually it has records of 1500 byte length each.
I want to strip it down to 1500 byte records. So 1500*3500 = 5 MB (approx). The record size is always 1500 bytes.
Does anyone have a script that I can run on this file to achieve this break.
Thanks
View 3 Replies
View Related
Jul 20, 2005
In my database there is a text field type that is used to enter streetaddress. This address could be a few lines long, each line with acarriage return at the end.Is there a way to search for these carriage returns and break out whatis in each line seperately?Thanks.Mike
View 1 Replies
View Related
Jul 31, 2007
Hi everyone,
I am using SSIS, and I got the folowing error, I am loading several CSV files in a OLE DB, Becasuse the file is finishing and the tak dont realize of the anormal termination, making an overflow.
So basically what i want is to control the anormal ending of the csv file.
please can anyone help me ???
I am getting the following error after replacing the '""' with '|'.
The replacng is done becasue some text sting contains "" wherein the DFT was throwing an error as " The column delimiter could not foun".
[Flat File Source [8885]] Error: The column data for column "CountryId" overflowed the disk I/O buffer.
[Flat File Source [8885]] Error: An error occurred while skipping data rows.
[DTS.Pipeline] Error: The PrimeOutput method on component "Flat File Source" (8885) returned error code 0xC0202091. The component returned a failure code when the pipeline engine called PrimeOutput(). The meaning of the failure code is defined by the component, but the error is fatal and the pipeline stopped executing.
[DTS.Pipeline] Error: Thread "SourceThread0" has exited with error code 0xC0047038.
[DTS.Pipeline] Error: Thread "WorkThread0" received a shutdown signal and is terminating. The user requested a shutdown, or an error in another thread is causing the pipeline to shutdown.
[DTS.Pipeline] Error: Thread "WorkThread0" has exited with error code 0xC0047039.
[DTS.Pipeline] Information: Post Execute phase is beginning.
apprecite for immediate response.
Thanks in advance,
Anand
View 1 Replies
View Related
May 15, 2006
good morning,
I want to load data that i receive everydays from my customers in .xls file format (excel) or cvs file format, to the database that i have created on this purpose. but when trying to do that whith SSIS; i got an error message .... that i can't import redudant data in my database column.
Best regards.
View 1 Replies
View Related
Aug 28, 2015
I have to value [CreateDate] in the data pump of my Flat File Source into my OLE DB Destination SQL Server Table. With a Variable within the SSIS Package or with a Derived Column task within the Data Flow between the Flat File Source and OLE DB Destination?
View 2 Replies
View Related
Jul 9, 2007
Is it possible to say for example have the first page that contains a chart in portrait layout and the rest of the report that is a table in landscape layout?
View 3 Replies
View Related
Apr 30, 2007
Opening DimTime table of AdventureWorksDW sample database in MS SQL Server Management Studio shows me values in FullDateAlternatKey like
View 3 Replies
View Related
Apr 16, 2007
Hi, all experts here,
Thank you for your kind attention.
I am wondering if we can drill down reports in a different page with different layouts and columns? e.g. I have too many columns desired to see in a report, so I want to direct the users to another page with different layouts and columns, so the first view in the report wont display so many columns in a page. Is it possible to make it in SQL Server 2005 Reporting Services? And if it is possible then how? Hope my question is clear for your help.
Thanks a lot in advance and I am looking forward to hearing from you.
With best regards,
Yours sincerely,
View 5 Replies
View Related
Sep 18, 2007
I have been searching for the answer to something I thought would be easy to find. But, no luck.
I need to load a csv file into a SQL Server table. The rub is that I need to also parse the filename for a couple of pieces of data also.
Example filename: c:importCustomerX_LocationY_1234.csv
For each record in the csv file I will call a stored procedure the has a parameter for each column in the file plus a parameter for the customer name and a colummn for the location name.
I assume that I will need to use a Script Component to parse the filename fro the information. What is not clear is how I get the filename from the csv connection object and how to get to result of the script component to the inputs of the OLE DB command component that I am using to call the stored procedure.
I have succeeded in using the Derived Column component to put constant values in for the customer and location. But, I cannot figure out how to get to the next step of deriving those two values from the input filename.
Thank you n advance for any assistance,
Jim
View 4 Replies
View Related
Sep 29, 2007
Hi,
I have a data file in the folloing format
SubjectId1|class1
SubjectId2|class2
SubjectId3|class3
I just wanted to insert only SubjectIds into my table 'Subjects' which has the follwing schama ignoring the classes
The row delimeter is "
" and the column delimeter is ' | '
Table Subjects
{
ID (Autoincrement)
SubjectId varchar(20)
}
Can any one provide the format file for doing this or suggest anyway to do this?
Please do note that the file may contain millions of records
Thank u
~mohan
View 5 Replies
View Related
Jun 27, 2006
Hi,
for testing purposes I'm inserting a flat file into a sql-server table using BULK INSERT unsig the following code:
BULK INSERT rsk_staging
FROM 'c: empulk
sk.txt'
WITH (
FIELDTERMINATOR = '',
ROWTERMINATOR = '
',
CODEPAGE = 'RAW',
DATAFILETYPE = 'char',
BATCHSIZE = 100000,
ROWS_PER_BATCH = 1925604,
TABLOCK
)
I have two versions of "rsk.txt" one with 1.9mill rows and one with the first 2000 rows only. The files have one column only with 115 characters that I'll split in to several columns later using SUBSTRING. The one with 2000 rows fires in to the database with no problems whatsoever using this exact code, the other one throws the following error:
Server: Msg 4866, Level 17, State 66, Line 1
Bulk Insert fails. Column is too long in the data file for row 1, column 1. Make sure the field terminator and row terminator are specified correctly.
How can I resolve this problem?
EDIT: I tried several different row- and fieldterminators but this exact one works for the small data-file so I assume it should also work for the large one...the large one is however copyed directly using binary ftp from a unix-filesystem and the small one is manually copied into a new txt-file using UltraEdit.
View 1 Replies
View Related
Sep 4, 2007
Hey all-
I put togehter a package that opens a flat file, parses the data based on the semi-colon delimeter, and imports the rows into a database table. Thats the fun easy part.
What I cant figure out is how to add a variable that will hold a constant ID value that will be persisted with the same value to all rows inserted to the DB. Making the problem harder, I would like that this value be defined in a properties file or database table of some sort so that I can do a lookup based on the file name / location to find out what value should be used.
Any suggestions? I hope my explanation makes at least some sense - but basically I want to do a look up in a configuration of some sort, pull out a single value, and add it to a data import.
let the fun begin!!
View 4 Replies
View Related
Dec 10, 2007
Hi Friend,
I am stuck with a problem and need your help. As we know, all columns that go to error flow of flat file source connection are displayed as a single column e.g. FlatFileSourceErrorOutputColumn, but my requirement is to extract the first column value from this FlatFileSourceErrorOutputColumn, my data is dilimeted by "|" pipe operator. I have created a script component to deal with this. However if we take FlatFileSourceErrorOutputColumn column as input column in script component, it comes as BLOB data. I wrote below code in transformation script component to extract BLOB data from column in string form and then do a Left function search to take first column out.
When I am running this script component I am getting '??????????' question marks as a result in Row.Pname.
Can anyone please help me understand if I am doing anything wrong in this script or suggest a better way to take the data out?
I appreciate your help.
Public Overrides Sub Input0_ProcessInputRow(ByVal Row As Input0Buffer)
'
'
Dim Error_Data As String
Dim Column_1 As String
Dim Len As Integer
Dim Buffer As Byte()
Buffer = Row.FlatFileSourceErrorOutputColumn.GetBlobData(0, CInt(Row.FlatFileSourceErrorOutputColumn.Length))
Error_Data = System.Text.Encoding.Unicode.GetString(Buffer)
Len = Error_Data.IndexOf("|")
Column_1 = Left(Error_Data, Len - 1)
Row.Pname = Column_1
End Sub
Thanks,
Kul
View 5 Replies
View Related
Sep 10, 2007
Hi,
I have a excel file which i want to import the data to sql server... The sql server Data type for that particular column is
varchar and it has a contraint too like the data should be in this fashion 00000-0000 or 00000...
but when i try to import the data from the excel to sql server... 08545 just becomes 8545 (cause excel is treating it as a float) and so my insert fails...
what can i do to rectify the problem...
regards
Karen
View 8 Replies
View Related
Mar 13, 2008
Hello guys,
here is my issue.
I created a ssis package which exports the data from oledb source to flat file (csv format). For this i have OLEDB source and Flat File as destination. I generate the file and filename dynamically with the column names in the first row. So if the dynamically generated file name already exists , then i want to append the data in the same existing file. But I dont want to append the column names again. I just want to append the rows to the existing rows.
so lets say first time i generate a file called File1_3132008.csv.
Col1, Col2
1,2
3,4
After some days if my ssis package generates the same file name i.e. File1_3132008.csv, this time i just want to append the rows to the existing file. So the file should look like this-
Col1, Col21,23,45,67,8
But instead my file looks like this if i set Overwrite propery to false
Col1,Col2
1,2
3,4
Col1,Col2
5,6
7,8
Can anyone help me to get the file as shown in the highlighed
Any help would be appreciated .
Thanks
View 3 Replies
View Related
Jan 12, 2007
I'm having a problem using the Flat File Source while using the underlying .Net classes to execute SSIS Packages. The issue is that for some reason when I load a flat file it Empty's out columns randomly. Its happening in the Flat File Source Task. By random I mean that most of the times all the data gets loaded but sometimes it doesnt and it empty's out column data. Interestingly enough this is random and even the emptying out of columns isnt a complete empty, its more like a 90% emtpying. Now you'll ask that is the file different everytime and the answer is NO. Its the same file everytime. If I run the same file everytime for 10 times it would empty out various columns maybe 1 of those times. This doesnt seem to be a problem while working with dtexec or the Package Executor utility. Need Help!!
View 9 Replies
View Related
Apr 9, 2015
I am having one store procedure which use to load data from flat file to staging table dynamically.
Everything is working fine.staging_temp table have single column. All the data stored in that single column. below is the sample row.
AB¯ALBERTA ¯93¯AI
AI¯ALBERTA INDIRECT ¯94¯AI
AL¯ALABAMA ¯30¯
After the staging_temp data gets inserted into main table.my probelm is to handle such a file where number of columns are more than the actual table.
If you see the sample rows there are 4 column separated by "¯".but actual I am having only 3 columns in my main table.so how can I get only first 3 column from the satging_temp table.
Output should be like below.
AB¯ALBERTA ¯93
AI¯ALBERTA INDIRECT ¯94
AL¯ALABAMA ¯30
How to achieve above scenario...
View 1 Replies
View Related
Jul 17, 2007
I've got a Table that has over 500,000 row in it. Now I need to convert the whole thing into Excel to import into another application. So I need to break the table into 10 different tables. How can I do that?
Thanks
View 11 Replies
View Related
Jul 23, 2005
I hope I can get this across clearly.I have a table that needs to be broken into 3 tables.Col1 Col2 Col3 Col4 Col5 Col6 Col7Col1 and Col2 need to go into LookupTable1Col3 and Col4 into LookupTable2If Col5 is twice the width.... haha just kidding...so Col5 and Col6 go into LookupTable3There is a 4th table which is made up of foreign keys which are the PK ofLookupTable1,2,3My questions is, how to get the data from the columns of each row and add itto its respective lookuptableand sequentially step throw the table to repeat the above step until I'veprocessed each rowthanks folksT.B
View 3 Replies
View Related
Nov 2, 2007
What is the best way to page break and I am having trouble. I am building a form based report and need to always break after the last field.
View 3 Replies
View Related
Dec 6, 2007
right now we have 2 node cluster . so if i break a cluster it will hamper or not for SQL server. Can you please give me suggestion
View 2 Replies
View Related
Mar 15, 2006
Is it possible by any kind of workaround to break the 8kb limit on user-defines datatypes?
My datatype can contain an arbitrary number of double-precision points meaning that I in best case only can store 512 points (2 x 8 x 512). there's a few extra bytes used for something else, but this is roughly the maximum, which is far from what I in many cases need. I serialize the object myself to ensure that I only store what I really need.
View 1 Replies
View Related
Aug 6, 2004
I have been working on a pretty ugly stored procedure recently, while debugging I added a char(10) to the end of each line of the SQL query so I could copy it to query analyzer(QA) and debug the SQL syntax output from of the stored procedure.
It had no effect on the stored procedure working, but when I copied the query to QA it got the error below, so I removed them all and added them in one line at a time to find the problem.
--Server: Msg 170, Level 15, State 1, Line 3
--Line 3: Incorrect syntax near ','.
Below are the 2 querys, the only difference is the Char(10) between Amt6 and Amt7!
http://www.rakbiz.com/download/broken.txt
http://www.rakbiz.com/download/working.txt
I also tried char(13) with same results.
any ideas why this is happening or how to add line breaks for readability without this problem?
Thanks for your help!
View 3 Replies
View Related
Dec 20, 2007
Hi, I have a report that is frustrating me. I've built this report, and it is not yet used in production. What it does is page break in places that I don't understand.
The FIRST area it would break after a line that had wrapped text to the line below. Even though there was plenty of room to fit it, and the entire rest of the group on the page.
The second area I honestly have NO idea why it breaks.
View 3 Replies
View Related
Mar 20, 2008
On a webpage, there are filters to choose from. Like date, amount, SSN (multiple filters can be choosen).I have a single query so far. SqlCommand cmd = new SqlCommand("SELECT [column1], [column2], [column3], [column4], [column5] FROM [table] WHERE [column4] = 'condition4' AND [column5] = @total_bill AND [last_change] >= @txtStartDate AND [last_change] <= @txtEndDate ", Conn) ; cmd.Parameters.Add(new SqlParameter("@total_bill", total_bill1.Text)); cmd.Parameters.Add(new SqlParameter("@txtStartDate", txtStartDate.Text)); cmd.Parameters.Add(new SqlParameter("@txtEndDate", txtEndDate.Text)); I want to break the query so that it executes on the basis of different sets of conditions (filters). If I dont select date filter, then the above query will not execute properly.Please help.
View 2 Replies
View Related
Jan 23, 2012
I want to show a third-level overview of how many to-do items are in various statuses (pending declined and completed) for the third level individuals.
In this example there's 2 regional leaders (level3), each with local leaders (level2) and individual team members (level1)
Status "1" is a new or pending to-do item
Status "2" is a declined item
Status "3" is a completed item
Here's what I want the output to look like (I may need to add columns for the final report but it should look as follows):
LEVEL3 | Status 1 | Status 1 | Status 2 | Status 3 |
LEADER | TOTAL | 2012 | TOTAL | TOTAL |
-------|----------|----------|----------|----------|
paul | 3 | 1 | 1 | 0 |
roger | 1 | 1 | 1 | 0 |
Here's some sample data similar to my live DB:
CREATE TABLE #items (id int, datestamp datetime, userid int, status int)
INSERT INTO #items (id,datestamp,userid,status) VALUES (1,'12/26/2011',1,1)
INSERT INTO #items (id,datestamp,userid,status) VALUES (2,'12/26/2011',2,1)
INSERT INTO #items (id,datestamp,userid,status) VALUES (3,'1/2/2012',4,1)
INSERT INTO #items (id,datestamp,userid,status) VALUES (4,'1/3/2012',1,2)
[Code] ....
I have a view called hierarchy that shows me the hierarchy for each level 1 user. Here's the source for the view:
SELECT
a.id 'level1',
b.id 'level2',
c.id 'level3'
FROM #users a
INNER JOIN #users b ON a.reportsto = b.id
INNER JOIN #users c ON b.reportsto = c.id
I've run into table/view limits when using a straight query and timeouts when trying to create a cursor that looks through all level3 users and populates a temporary table with an INSERT and multiple UPDATE statements to get the individual counts for each column for each user.
(Just a note on the actual database in case it's relevant: It has about 1000 LEVEL1 users, 50 Level2, and 15 Level 3 users. It will generate about 1400 to-do items per week.)
View 4 Replies
View Related