Help Importing File With Varying Column Widths
Nov 7, 2007
True story. Thanks. Dont think that fixed or ragged is going to work for me though. File does not seem to be structured.
I have what appears to be a fixed length file that I would like to import using SSIS. However it does not look loike the spacing between fields is constant. Is there a way to say from character 1 to 10 is X, and 11 to 15 is Y, etc etc? Below is a couple of rows that all pertain to the same record.
01000000225672101242253 55232101242253 5TKTT / 0101 FFVV 5235MCYSNR/CE MUSSETT/BRIAN.A.MR 20071017U7700002277000022 U00000000ZAJNB JNB ZA 00000000 00000000 00000000 SITII Y06 405
02000000235672101242253 55232101242253 5ICE 20071017 1407000017OCT 17OCT
03000000245672101242253 55232101242253 5 PLZPLZ ZAR 1000 ZAR 72ZAZAR 140ZVZAR 562XTZAR 1774 ADT PENALTY APPLIES
04000000255672101242253 55232101242253 50000017740000000100000ZAR200000000000 0000000000000000 0000000000000000 00000000000000000000000000000000 000000000000000000000000000000000ZA 00000007200ZV 00000014000EV 00000001600YR 00000054600 00000000000 00000000000 00000000000 00000000000 00000000000
05000000265672101242253 55232101242253 51 OPLZ DUR CE CE 0633 0633 K K 18OCT18OCT18OCT1625 1740 OK KOW 2 DUR PLZ CE CE 0602 0602 B B 20OCT20OCT20OCT1000 1115 OK BOW
07000000275672101242253 55232101242253 5CCIK5221 000001774005221591015192982 0308M067781 00000000000 00000000000 00000000000
08000000285672101242253 55232101242253 509PLZ CE DUR550CE PLZ450ZAR1000 END CE XT16EV546YR 1 2 3 4
What do you reckon is the best option? SUBSTRING in a Dervied Column? Below is an example of the text.
View 5 Replies
ADVERTISEMENT
Apr 24, 2008
How do i import a Varying Column Width Flat file into a Table using SSIS?
I have a flat file that has 4 columns with varying width
Like I should read the file as
Col 1 - (1 to 10 Characters)
Col 2 - (12 to 21 Characters)
Col 3 - (22 to 35 Characters)
Col 4 - (36 to 38 Characters)
At the end of the record is a "LF"
I think "Fixed Width" Columns allow me to define a standard column length for all the columns.. Right?
Any thoughts on how to?
View 9 Replies
View Related
May 13, 2007
I am able to get reports going with tables sized properly. They look fine on the ReportServer website and I adjust the column widths so that the headings and data look nice. When I set up a subscription to be delivered by "Report Server E-Mail," though, the table formattings get completely distorted.
In particular, I have two tables, with some column headers being two short words (e.g. Max Height). When rendering on the site, I adjust the columns so the full column header is visible on one line. When I receive the email and read it in Outlook, the header row is now about twice as tall and everything is scrunched together. Both the headings and the data in the fields do not format the same as on the website.
The two tables tend to actually have the exact same width in the email version, although occasionally they are a little different (in the web version one is about half as wide as the other). I have tried just making the columns bigger and that has not worked. I've tried making the font sizes smaller, which didn't work. If I do that, leaving the columns the same width, the email version just gets scrunched into a smaller area with the same text-wrapping problems.
If I open the email in a browser (in a web mail interface) the report renders perfectly as on the site.
I have almost all the default settings, and haven't been messing around with page sizes and things like this (except after, to see if that would fix the problem).
Any ideas, similar experiences, or suggestions? If there is a book I should read or any reference you could point me to in order to figure this out would be helpful. I haven't been able to understand this either using web searches or the two SQL reporting services books I have.
View 4 Replies
View Related
Feb 12, 2007
The reason I say this is because a subtotal of a dollar amount will take up more space than other values. Right now, I'm forced to make all columns the same larger width because it appears to be all wrapped into 1 column width setting. I can try to change the value of the subtotal column, "matrixcolumn4", but it reverts to the other value after I press enter to apply the changes.
View 1 Replies
View Related
May 15, 2006
good morning,
I want to load data that i receive everydays from my customers in .xls file format (excel) or cvs file format, to the database that i have created on this purpose. but when trying to do that whith SSIS; i got an error message .... that i can't import redudant data in my database column.
Best regards.
View 1 Replies
View Related
Oct 5, 2000
I receive several TXT files daily that need to update information in SQL Server databases. The process requires that all TXT files be appended to a master file and also update individual files' information based on the TXT file name. For example:
File TABLE1_x_ddmmyy.TXT (ddmmyy = date, x = "O" or "B") is to be appended to the master file and also update SQL table "TABLE1" by setting a flag for those records in the table that match a unique key that is provided in the TABLE1.TXT file.
In VFP, I had the following process in place:
a) open the TXT file.
b) read its file name and open the corresponding VFP file
c) update the VFP file based on the key provided in TXT
d) append the key to the master file.
e) repeat c-d for next record in TXT
f) repeat c-e for next TXT file
Using the same process with ADO takes a considerable time since I am processing one line at a time.
Is there any way to do this using a DTS package of some sort? How can I read the TXT file names in SQL Server?
Thank you.
View 3 Replies
View Related
Apr 18, 2007
Hi
SSIS is brand new for me.. Playing with since a few hours..
Iam trying to import a Flat File into the SQLSERV DB using SSIS..
One of the column is in this format -- "YYYYMMDDHH24MISS"
How do i get around this to import the data in a readable fashion into the Destination?
Thanks!
MKR
View 6 Replies
View Related
Oct 23, 2014
I get error reports in simple text files like the one below in relatively the same format. The only thing that varies is the number error reasons as there can be any number of error reasons for a file. Usually there is only one but there can be a handful. What is the best way to capture the error description and count of errors no matter how many there are? I want to take these items and update a table I have in sql server 2008r.
Report Message example:
************************************************** *****
Original File Name: some.file.YYYYMMDD.d.incr.02of02.1.dat
Source File ID: file02YYYYMMDD
File Receipt Date: 10/17/2014
Total records received: 1331136
Total records loaded: 1329987
ERROR REASONS
Error code: EBBW002 Error desc: Duplicate Record Total records: 1146
Error code: EABC001 Error desc: Invalid Length Record Total records: 1
Error code: ERRCM10 Error desc: Missing First Name Total records: 2
Total number of Errors encountered during the ODS update processing: 1149
************************************************** *****
View 2 Replies
View Related
Dec 2, 2003
Hi all.
I'm trying to export some tables in fixed width format for a client that needs it this way. The first time I did this, using Import/Export in Enterprise Manager, integer fields were 12 chars long. However, when I saved this as a DTS and ran it again later, it made integers 4 chars in the text file (which matches the lengths of the datatypes you see in enterprise manager.)
Why is this difference there? What do I need to do differently? Also, is there a list of how wide each data type is when exported as fixed delimited text?
thanks,
Craig
View 2 Replies
View Related
Jul 20, 2005
Hi everyone,I encountered an error "Need to run the object to perform this operationCode execution exception: EXCEPTION_ACCESS_VIOLATION" When I try to import data from Oracle to MS SQL Server with EnterpriseManager (version 8.0) using DTS Import/Export Wizard. There are 508 rowsin Oracle table and I did get first 42 rows imported to SQL Server.Anyone knows what does the above error message mean and what causes therest of the row failed importing?Thanks very much in advance!Rene Z.--Posted via http://dbforums.com
View 1 Replies
View Related
Mar 7, 2008
hi when I try to export data to a fixed width txt file I get the following error:
Error 0xc02020a1: Data Flow Task: Data conversion failed. The data conversion for column "postcode" returned status value 4 and status text "Text was truncated or one or more characters had no match in the target code page.".
even though my sql for postcode is LEFT(postcode,8) and the field length is 8 !
do you now why ?
View 1 Replies
View Related
Jul 9, 2007
Hi,
Is it possible to fix the max width of the columns ?? Sometime the report only generates 1 or 2 columns and it looks a bit odd as they're stretched to the width of the page.
Also, when we have a large number of culumns, say 20 odd, the legend text get's real squashed up and un readable, are there any tips/tricks so i can have more flexibility over what's displayed ?
thanks
Steve
View 2 Replies
View Related
Jan 3, 2008
Hi
I have a column in a .txt file that I am trying to import.
The field shows the time of a transaction in the format of "12:20 AM"
Does anyone know a way of importing the field as a time field as opposed to a VARCHAR type? I have tried importing it as a datetime type, but this doesnt seem to work.
Thanks
View 3 Replies
View Related
Sep 19, 2006
I have a spreadsheet that has all of the data in the first column. The data is delimited by asterisks and there are no column headings. My goal is to import the data into a SQL table. I get the error message "External table is not in the expected format" when I try to select the name of the Excel sheet in the Excel Source Editor. I tried reading using a flat file connection instead of Excel connection, but that didn't work.
In Excel, I can use the Text to Columns feature to convert the data to columns in the spreadsheet. But I have to go into the file manually to do that. Is there a way in SSIS to do the same thing automatically? Or is there a way to save the XLS as a TXT file? Then, I could just use Flat File Connection to read the delimited data.
Thanks.
cpat
View 1 Replies
View Related
Jun 22, 2007
Hi i'v e installed the file: SQLServer2005_DTS.msiBecause i've heard that i need that to import a csv file into a ms sql database. now i have no idea how to work it, like to even make it open.Where would i open it from? I reckon i'v eread enough literature about using it, to be able to have a bash... thanks in advance...
View 3 Replies
View Related
Oct 5, 1999
Hi, this is my first time and was wondering what's the best way to do the following:
i've sql server 6.5. also have a sql database(.DATfile) in a folder.
i need to import this database to my server. should i just LOAD/import it? or do i need to first create a database device on my server(how much size should i allocate etc) and then load it??
thanks a lot
rohit
View 3 Replies
View Related
Jun 24, 2002
Hi guys,
I have to write a Store procedure which will pickup a txt file from a destination, read it and update some table.
How to pick and read a txt file in a Stored procedure.
Thanks in advance
Vineet
View 1 Replies
View Related
Jun 11, 2007
Hi,
i am having problems while importing a .csv file.
when i import a .csv file in sql server I get the folowing error message.
"Cannot create an OLE DB accessor. Verify that the column metadata is valid".
Any help would be appreciated
View 3 Replies
View Related
Jul 23, 2005
I have a .txt file that I need to add to an existing table, which ismade up of Varchar, Char, numeric and int fields.What is the best way to do it.The first thing I tried was importing the .txt file, and then goinginto the design and changing the field type, but I hit problems on theNumeric fields.Then I tried changing the field types from varchar to different typesat the transfom stage of importing, but that failed too.Regards,Ciarán
View 2 Replies
View Related
Nov 22, 2007
I have a CSV file that i need to import into a SQL table. The problemis the values in the first column are not brackited in "". There areover 700K rows. Is there an easy way to fix the data or have SQLcorrectly import the the data?The data looks like this1, "xxx","zzz"2, "aaa","bbb"an so on...Thanks
View 1 Replies
View Related
Sep 19, 2006
Hi all-
I am in need of some help importing a .CSV file into a SQL Server 2005 Express database. I can't use SSIS because it's SQL Server Express. The operation needs to run as a parameterized Stored Proc which I will call from ASP.NET.
The problem is that I need to get the only 2 columns from the text file, then "tack on" 3 more columns that will have data that changes, which will come in as parameters to the T-SQL stored proc. This prevents me from using a straight BULK INSERT.
The data in each of these columns will be the same for each record in that column - that is, every record in every field in column 3, for example, will be the same.
Some of the files will have 4 million records and up, so speed is of the essence here. I tried using BULK INSERT to dump the data into a #temporary table - which took 38 seconds and was acceptably fast - but then, my next step was to dump the additional data into the other columns using UPDATE... SET. I gave up on this after the query ran for THIRTY MINUTES! My next step was going to be to move the data from the temporary table into the target permanent table somehow, but I never got that far when I saw how long the previous step took...
It's a little odd, because I can do the same thing in MS Access in under 5 minutes by using a SQL statement like this in my ASP.NET code:
"INSERT INTO [Target_Table] (field1, field2, field3, field4)" & _
"SELECT F1, F2, '" & strSource & "', Now() AS DateTimeStamp " & _
"FROM [Text;HDR=NO;DATABASE=" & strPath & ";].[" & strFilename & "]"
strSource is a string that the User enters (properly vetted for security); strPath and strFilename are strings holding the path and filename to the .CSV file. I create a unique filename from the file the user uploads. This works in under 5 minutes for several millions of records in MS Access, as I said above. I've had no luck getting anything similar to work in SQL Server, though.
Anyway, does anyone have any ideas? This is somewhat urgent, as the project was about to go "live" when it was discovered that the actual, live data had grown to the point where Access couldn't hold it, and a move to SQL Server Express was necessary.
Thanks in advance,
-Andrew
View 9 Replies
View Related
Sep 19, 2005
I need to be able to create a DTS package that imports a CSV file which is loated at URL. I.E. HTTP://www.url.com/csv/thefile.xls I tried copying the URL an pasting it in the file location when in SQL wazird but I got an error message.
View 1 Replies
View Related
Jun 4, 2007
I'm a newbie to SSIS, so there is probably an easy solution:
I'm importing from a csv file into a db table, but would like to remove the quotation marks from around the data before inserting into the table.
Are there any suggestions of how to remove the "data" marks? Would a Foreach Loop container work for this?
View 1 Replies
View Related
Jun 16, 2003
When I try to import a Pervasive table into SQL Server 2K, I get the error msg of 'Invalid column number'.
Is that because SQL Server cant handle more than 1024 columns ?
I think my Pervasive table has more than 2000 columns
Thanks in advance
View 5 Replies
View Related
Nov 29, 2007
I am trying to import a table from a remote database which has a new column added, which I want along with the rest of the data. When I try doing the import via the wizard (Tasks -> Import Data...), the option to drop and re-create the destination table is disabled. Under "Mappings", the newly added column appears as the last row, but for the Destination column, it says "<ignore>" and presents no other option.
How can I bring this new column over? I've checked the table via the dependency viewer, there are no objects which depend on it and no objects on which it depends.
View 1 Replies
View Related
Jul 31, 2007
How do i import a Excel file into a table i have created in my database in SQL server 2005???
View 10 Replies
View Related
May 1, 2008
Happy Thursday all,
I am importing a text file to sql and most of my fields look like this:
"M","NEW ADDRESS",
and my other field looks like this:
"firstname Lastname"
but I need it like this:
"firstname", "Lastname"
Can anyone help me understand a better way of making this happen?
Thanks in advance
View 2 Replies
View Related
Jul 22, 2004
Can someone please help me.
I need to import a csv fiel to sql server and I know that the column delimiter is
and the newline delimiter is but I don't know what the rowterminator is or the fieldterminator. How can I import the file into an empty table in an existing database.
Any suggestions would be greatly appreciated.
View 1 Replies
View Related
Sep 15, 2004
HI Guys,
I am doing the following to read the data in a text file and inserting it into SQL.
1) Open db connection
2) Open Text File
3) loop through text file all along inserting each row into the db
4) close the text file
5) close the db connection
However, the text file has over 400 rows/lines of data that need to be inserted into the db. Each line in the text file is a row in the db. At anyrate, the above script times out. Is there a better, faster way to do this? I can't use Bulk Insert due to permissions previlages.
Thanks in Advance!
View 2 Replies
View Related
Sep 4, 2001
Hi all,
Please let me know if it is possible to import an Oracle .dmp binary file into SQL using DTS. DTS has ODBC and OLE DB drivers for Oracle but both require specifying the Oracle database name, user name and password. i.e., it looks like the only way to import the .dmp file into SQL would be to first load the .dmp file into Oracle and then tranfer it from Oracle to SQL using DTS. Is there any way to load the .dmp file directly into SQL?
The .dmp file is a binary file that has been generated from Oracle using the 'Export' utility.
Thanks in advance,
-Praveena
View 2 Replies
View Related
Dec 6, 2000
I have large flat files that I need to import using DTS. The record counts are between 500,000 and 3000,000. Or, 16 to 116 MB. When creating the DTS package I need to use to import, the wizard is haulting at the point where I should place the fixed points for the fields. I thinks it is because, the wizard will only recognize excel or text files. I cannot save these files as either txt or xls, because of their size. does anybody have any suggestions?
View 3 Replies
View Related
May 19, 2004
I am building a aspx/c# application with SQL Server 2000 backend. Now i want to have the option for "Importing" the data into one of the tables in my database.
The source file for the import is a text file , CSV format. I want the users to click on the "Import" button placed on my webform and supply the souce file and the data should get imported into the SQL Server 2000 database table.
I want to know the various ways to implement this. Is it possible to invoke the DTS and then DTS will itself guide the users do the import? or if i need to write a SQL query , what would that be like??
View 2 Replies
View Related
Feb 14, 2007
I have an app written in delphi v5 with a MS SQL Database and I'm struggling with an import feature i need
The user needs a function to import Contact data from a txt/csv file into the Contact Table
Details of feature:
the user enters the different parameters into the Delphi app e.g.
File Location
Files Includes Column Headings or not
Whether the file Comma or Tab Seperated
Mapping the fields
Then i need to use those parameters and run some sort of import routine putting the data into the specific table.
The tables consists of 3 fields - First Name, Surname, Mob Number - but these fields can be in any order in the file. for example Col 1 (in file) needs to go into Field 3 in DB. this is sorted in the Mapping Fields above
How do I do this using MS SQL??
I've been looking at using the BULK INSERT command e.g
BULK INSERT Contact
FROM 'C:Import_data.csv'
WITH
{
FIELDTERMINATOR = ',',
ROWTERMINATOR = ''
}
but at the minute i just get error -
[Microsoft][ODBC SQL Server Driver]Syntax error or access violation.
Is this the correct command to use??
Do you know any websites that can point me in the right direction??
I have an app written in delphi v5 with a MS SQL Database and I'm struggling with an import feature i need
The user needs a function to import Contact data from a txt/csv file into the Contact Table
Details of feature:
the user enters the different parameters into the Delphi app e.g.
File Location
Files Includes Column Headings or not
Whether the file Comma or Tab Seperated
Mapping the fields
Then i need to use those parameters and run some sort of import routine putting the data into the specific table.
The tables consists of 3 fields - First Name, Surname, Mob Number - but these fields can be in any order in the file. for example Col 1 (in file) needs to go into Field 3 in DB. this is sorted in the Mapping Fields above
How do I do this using MS SQL??
I've been looking at using the BULK INSERT command e.g
BULK INSERT Contact
FROM 'C:Import_data.csv'
WITH
{
FIELDTERMINATOR = ',',
ROWTERMINATOR = ''
}
but at the minute i just get error -
[Microsoft][ODBC SQL Server Driver]Syntax error or access violation.
Is this the correct command to use??
Do you know any websites that can point me in the right direction??
Hope all this makes sense
Thanks
View 1 Replies
View Related
