Best Practice For Ragged Delimited Text Import

Aug 9, 2007

I have tab delimited text files which may have optional fields (meaning they can be not present at all) to the right of the required fields that I care about. It would appear that using a Flat File Connection with Delimited Format (tab) set will choke if it is initially configured with a file that has something like:

data data data

and it then encounters

data data data optionaldata

It chokes. I know this could be parsed line by line, but that seems silly. It seems like there should be a way to ignore columns beyond a certain point (e.g. Format "Delimited Ragged Right").

Is there some way to do this with a directly with a flat file connection?

Thanks,

--Andrew

View 1 Replies

Import A CSV Delimited Text File Into A Table

Jan 18, 2007

Hi,

Could you help me to write a script to import a CSV delimited text file into a sql server table.?

Thanks,

carlos

View 2 Replies View Related

Import Into Comma Delimited Text File (very Urgent)

Aug 28, 2001

I want to join differnet tables and import the data into comma delimited text file. There will be lot of checks like if then else to manipulate data. I want to use stored procedure but don't know how to output to text file. Is there any utility which can be used in stored procedure. In future this will be run as an automated job.
Thanks in advance.

View 1 Replies View Related

T-SQL (SS2K8) :: Import To Table From Varying Tab Delimited Text Files

Feb 10, 2014

I need to import data to a MSSql table from massive (read: a million and a half rows, every single day) logs that come in .txt format separated in tabs with a ";" symbol and then have some stored procedures analyze that data to generate some reports in an excel file with that info. The text files include the column headers in the first row and the data starts on the second one.

The challenge is that the text files differ in column order and count every single day.

The analysis that I need to do only needs about 15 columns from the nearly 90-120 that those files include, and those columns sadly happen to be in a different order in those files.

View 8 Replies View Related

How To Import In Special Character Delimited Text File By Using SSIS ?

Sep 26, 2007

Hi,

I would like to know how to import in the custom delimited text file by using SSIS.
For example, instead by using tab or comma delimited, I use this character : '�'
The reason is the delimited format that SSIS provided is too common such as colon, semi colon, tab, comma and pipeline.
I have the data that the user also key in the pipeline there. So I am thinking to separate the field by using this special character, but cannot see if there is anyway to import in by using SSIS.

Please help to share the solution on this :

A�B�C
1�2�3

thanks
best regards,
Tanipar

View 8 Replies View Related

Reporting On Delimited Fields.... Best Practice? Suggestions?

Feb 3, 2004

Hi guys... I need to do some statistical reporting on the answers to registration questions. The answers are stored in pipe-delimited varchar fields to allow for one-or-many answers to a single question... here's an example:

Q: What are your fav colors?
A: [checkbox list of colors]

Now the data is stored like this:

Reg001
-----------
red|blue|green|yellow

Now I need to determine how many people selected each individual answer, so like, how many people picked red, how many people picked yellow.... to report like this:

Red - 12
Blue - 23
Green - 15
Yellow - 9

What is the best, or suggested way, to go about splitting this data up and reporting on it? Can it be done in T-SQL, or will it need to be a combination of T-SQL and .Net language?

Thanks!

-e

View 1 Replies View Related

Best Way To Import ASCII Tab Delimited File Every Week (5,000 Records)...

Jul 20, 2005

I need to import an ASCII tab delimited file that has roughly 5,000 recordsonce a week into a SQL Server table. I have researched BCP and it seemslike the way to go. Am I headed in the right direction?Thanks in advance,James

View 3 Replies View Related

Comma Delimited Text File

May 18, 2004

Hi I'm pretty new to using Microsoft Visual C# .NET and I want to upload a comma delimited text file from my local machine into a table in an sql server database through a web app. How would I go about programming this and what controls do I need? Any help would be much appreciated. Thanks in advance.

View 4 Replies View Related

Reading Tab Delimited Text File

Mar 28, 2008

Is there anyway Sql Server reads a "Tab Delimited Text File" and Compare each record with the Column in a table..

my question is..

I've a Country_Code table which has 3 letter Country Code and the Actual Country names are listed in a Tab Delimited Text File "Country Data" with Country Code and Country Name, how do i read each record and compare to get the Actual Country Name for Display.

any ideas/suggestions.

thanks

View 3 Replies View Related

COMMA Delimited TEXT FILE

Jul 20, 2005

Hi,On SQLServer 2000, I have a table with a following structure:MYTABLEcol1 char,col2 date,col3 numberMy Objective:------------Externally (from a command line), to select all columns and write theoutput into a file delimited by a comma.My method:---------1. Probably will use OSQL or BCP to do this.2. Use the following syntax:select RTRIM(col1) +','+ RTRIM(col2) +','+ RTRIM(col3)from MYTABLE;My 3 Problems:-------------1) If there is a NULL column, the result of concatenating any value withNULL, is NULL. How can I work around this? I still want to record thiscolumn as null. Something like say from the example above, if col2 isnull, would result to: APPLE,,52) The time format when querying the database is: 2003-06-24 15:10:20.However, on the file, the data becomes: 24 JUN 2003 3:10PM. How can Ipreserve the YYYY-MM-DD HH:MM:SS format? Notice that I also lost theSS.3) Which utility is better? BCP or OSQL?For OSQL, it has a "-s" flag which gives me the option of putting acolumn separator. But the result is:"APPLE ,14 JUN 2003 , 5"I don't need the extra space.While for BCP, there is no column separator flag.You will notice from my inquiry above that my background in SQLServer isnot very good.Thanks in Advance!!RegardsRicky*** Sent via Developersdex http://www.developersdex.com ***Don't just participate in USENET...get rewarded for it!

View 1 Replies View Related

How To Extract Data From Delimited Text!!!

Jan 8, 2008

First of all I am a novice here. I am working on a table with a column of URL. I want to seperate the data in the URL delimited by '/'.
Eg:
http://www.simpletech.com/upgrades/aopen/s661fxm/s661fxmintelp4/

Here I want aopen as manufacturer, s661fxm as model_number and intelp4 as submodel_number.
I solved this problem in Oracle using substring and instring. But I have no ides how to achieve this in SQL server.
Please..advice me.
Thanks in advance.

View 17 Replies View Related

Export As Tab Delimited Text File

Dec 5, 2007

Hi,

I am trying to export as a tab delimited text file. For that I have changed my config file as :

<Extension Name="TXT" Type="Microsoft.ReportingServices.Rendering.CsvRenderer.CsvReport,Microsoft.ReportingServices.CsvRendering">
<OverrideNames>
<Name Language="en-US">TXT (Tab Delimited Text File)</Name>
</OverrideNames>
<Configuration>
<DeviceInfo>
<FieldDelimiter>	</FieldDelimiter>
<Extension>TXT</Extension>
<Encoding>ASCII</Encoding>
<NoHeader>true</NoHeader>
</DeviceInfo>
</Configuration>
</Extension>

I got this code from another one of the MSDN forms. When I run the report and try to export using this format, it still gives me a csv file instead of tab delimited file.

Can someone please help me fix this code so I can get tab delimited text files.
Thanks a lot,
-Rohit

View 8 Replies View Related

Help In Using ADO.NET To Read From Text Files(CSV Delimited)

Mar 2, 2007

Hello there

I have written a small code that is used to read records from a txt or a csv file, but when i try to read from a txt or csv file which is in unicode, i get the following error

No value given for one or more required parameters........this error is recieved on the following line

dtadpt.Fill(DataSet.Tables("table2"))

The code for reading from a file is as follows

Dim dtrow As DataRow

Dim cmdtxt, file, path

Dim dtadpt As New System.Data.OleDb.OleDbDataAdapter

Dim Command As System.Data.OleDb.OleDbCommand

Dim Conn As System.Data.OleDb.OleDbConnection

file = 'name of the file'

path = 'path of file'

Conn = New System.Data.OleDb.OleDbConnection("Provider=Microsoft.Jet.OLEDB.4.0;" & _

"Data Source=" & path & ";" & "Extended Properties=""text;HDR=YES;FMT=Delimited""")

For Each dtrow In DataSet.Tables("table1").Rows

Dim id

id = dtrow.Item("id")

cmdtxt = "Select * from " & file & " where id = " & id

Command = New System.Data.OleDb.OleDbCommand(cmdtxt, Conn)

dtadpt.SelectCommand = Command

Next

DataSet.Tables.Add("table2")

dtadpt.Fill(DataSet.Tables("table2"))

this code works if i choose a csv or txtfile which is in ANSI format. But i want to read from a unicode file because i have some information in that unicode format that i want to store

any idea how this can be possible

thanks and best regards

Saad

View 1 Replies View Related

Best Practice For Storing Long Text Fields

Sep 12, 2006

As we all know, there is a 8060 bytes size limit on SQL Server rows. Ihave a table which requires a number of text fields (5 or 6). Each ofthese text fields should support a max of 4000 characters. We currentlystore the data in varchar columns, which worked fine untill ourappetite for text fields increased to the current requirement of 5, 6fields of 4000 characters size. I am given to review a design, whichesentially suggests moving the text columns to a separate TextFieldstable. The TextFields table will have two columns - a unique referenceand a VARCHAR (4000) column, thus allowing us to crossreference withthe original record. My first impresion is that I'd rather use the SQLServer 'text' DB type instead, which would allow me the samefunctionality with much less effort and possibly better performance.Can anyone advise on advantages and disadvantages of the two optionsand what the best practice in this case would be.Any advise will be well appreciated.Tzanko

View 7 Replies View Related

Export Data Into Text File Using � Delimited

Sep 8, 2005

Hi,
I need to export data from SQL server 2000 database into text file uisng � Delimited. Because my destination database will be teradata. Could you let me know if you have any method for this.
Thanks

View 1 Replies View Related

Can MSSQL Load A Tab Delimited Text File?

Jul 23, 2005

Hi all...I would like to know if SQL SERVER can load a tab delimited text file.If yes, how?A search on the web did not return me the "load data" command as mysqlor other.Thank you all.

View 5 Replies View Related

2 Sql Tables To A Text File With Comma Delimited

Aug 10, 2006

hi ,

I have 2 sql tables. 1 is the header table and another is the detail table. How can I have the header record being appended in the text file and then have the detail records being appended to a same text file again with comma delimited ?

View 3 Replies View Related

Issues Importing A Text File (tab Delimited) To A SQL Table

Dec 7, 2005

I have a text file I am trying to import to a table. This text file is in a tab delimited format. I am using DTS to import the data to a new table I made. The fields are varchar and are set to allow nulls & allow 8,000 characters per field.

The error I am getting is that the data exceeds the allowed amount (or something like that) in col4.

Now I have checked everything in column 4 and nothing exceeds 5,000 spaces/characters combined. I have checked the entire sheet (in excel) for that fact, and there is not one single column/row/cell that exceeds 5,000 spaces/characters combined.

What the heck could be causing SQL to tell me I am trying to import too much data in one column when there is nothing that even comes close to 8,000 characters & spaces combined?

View 3 Replies View Related

Load Data From Comma Delimited Text File

Feb 29, 2004

Hello, i need to load some data from a long comma delimited text file, How can a i do that, using t-sql?, thanks for your help!!!!!

View 5 Replies View Related

Bulk Load From Text Delimited File To SQL Table

Dec 13, 2007

Hi,

I am new to SSIS but i have avg working knowledge in sql.
My problem is as follows ,I have a text pipe dilimited file in some folder and the number of columns and the name of the column is not consistant. It can have N number of column and it can have any column names. I need to load this text file data into a sql table. All that i want is to load this file to SQL Database with some temp name. Once i get the table in SQL Database, i can match the column names of both taget table and this temp table and only push those column which matches with the target table. For this i can frame Dynamic SQL. This part is clear to me.

Now the problem is , I developed a SSIS pacakge to push the text file to SQL Table. I am able to do this. But if i change the column names or added new column SSIS is not able to push the new columns. Is this functionality available in SSIS, is it can be dynamic like this?

I hope i am clear with my prob... if need any clarification please let me know

thanks in advance

Mike

View 3 Replies View Related

Integration Services :: (OutputColumnWidth) Size In A Delimited Text File?

Sep 26, 2015

I received a pipe-delimited file that I need to import. (It has the equivalent of 650+ fields on a single row). While I had no issue importing it (SSIS 2008) I noticed that the input connector, Advanced option, shows an "OutputColumnWidth" of only 50 for all fields.

I say only 50 because some of the pipe-delimited fields can supposedly have a max of 250 characters so I'm concerned about potential data truncation. Unless someone has another thought I plan to manually set those OutputColumnWidth fields to 250.

View 5 Replies View Related

Exporting Data To A Comma Delimited Text File, FORMAT Function

Jan 15, 2001

Hi. Im new to SQL and I need to export a SQL table as a comma delimited text file which is straight forward. However two of the fields are integers and I need these to be right justified with zero's.
In Access I would use something like format(columnname, "00000000") to get it to work, but SQL Server doesn't like this.
How can I do this?

View 2 Replies View Related

T-SQL (SS2K8) :: Export Query Results To Pipe Delimited Text File

Sep 5, 2014

I've got a query that returns the data I need. I want to put the query in a stored procedure such that, when the SP runs I get a pipe delimited text file on disk. I don't really want to mess with SSIS, etc. Is there a Q&D way to do this?

View 1 Replies View Related

Problem Importing Csv Delimited Text File Into A Sql Server 2005 Table

Apr 25, 2006

I am using the Bulk Insert command and trying to import a CSV delimited text file into a table and I am having problems with the quote field delimiters ", " The command below works but it takes in all the "" quotes as well and the field delimiter comma , works only if the commas are the separators only. If I have a comma within a address field for example then the data gets imported into the wrong fields. What can I use to identify that the text qualifier is ". I don't see where I can use the bulk insert command to determine this. Is there another command that I can use or am I using this command incorrectly. I thank you in advance for any response or suggestion you may have.

BULK INSERT AdventureWorks.dbo.MbAddress

FROM 'a:mbAddress.txt'

WITH (

DATAFILETYPE = 'char',

FIELDTERMINATOR=',',

ROWTERMINATOR='',

CODEPAGE = '1252',

KEEPIDENTITY,

KEEPNULLS,

FIRSTROW=2)

Here is a sample ascii file I am importing as well you can see that 6330 has a extra comma in the address line.

"AddressAutoID","Memkey","Type","BadAddress","Address1","Address2","Address3","City","State","Zip","Foreign","CarrierRoute","Dpbc","County","CountyNo","ErrorCode","ChangeDate","UserID"
6317,26517,1,0,"1403 W. Kline Ave","","","MILWAUKEE","WI","53221","","",0.00,"MILWAUKEE",79,"",1/25/2006 0:00:00,"admin"
6318,26225,1,0,"501 Dunford Dr","","","BURLINGTON","WI","53105","","",0.00,"RACINE",101,"",1/25/2006 0:00:00,"admin"
6319,20101,1,0,"2115 Cappaert Rd #35","","","MANITOWOC","WI","54220","","",0.00,"MANITOWOC",71,"",1/25/2006 0:00:00,"admin"
6320,23597,1,0,"728 Woodland Park Dr","","","DELAFIELD","WI","53018","","",0.00,"WAUKESHA",133,"",1/25/2006 0:00:00,"admin"
6321,23392,1,0,"7700 S. 51st St","","","FRANKLIN","WI","53132","","",0.00,"MILWAUKEE",79,"",1/25/2006 0:00:00,"admin"
6322,26537,1,0,"W188 S6473 GOLD DRIVE","","","MUSKEGO","WI","53150","","",0.00,"WAUKESHA",133,"",1/26/2006 0:00:00,"admin"
6323,25953,1,0,"3509 N. Downer Ave","","","MILWAUKEE","WI","53211","","",0.00,"MILWAUKEE",79,"",1/26/2006 0:00:00,"admin"
6324,19866,1,0,"10080 E. Mountain View Lake Rd. #145","","","SCOTTSDALE","AZ","85258","","",0.00,"MARICOPA",13,"",1/27/2006 0:00:00,"admin"
6325,25893,1,0,"W129 N6889 Northfield Dr. Apt 114","","","MENOMONEE FALLS","WI","53051-0517","","",0.00,"WAUKESHA",133,"",1/27/2006 0:00:00,"admin"
6326,26569,1,0,"8402 64th Street","","","KENOSHA","WI","53142-7577","","",0.00,"KENOSHA",59,"",1/27/2006 0:00:00,"admin"
6327,24446,4,0,"83 Sweetbriar Br","","","LONGWOOD","FL","32750","","",0.00,"SEMINOLE",117,"",1/30/2006 0:00:00,"admin"
6328,19547,1,0,"4359 MERCHANT AVENUE","","","SPRING HILL","FL","34608","","",0.00,"HERNANDO",53,"",2/8/2006 0:00:00,"admin"
6329,26524,1,0,"264 Lakeridge Drive","","","OCONOMOWOC","WI","53066","","",0.00,"WAUKESHA",133,"",2/10/2006 0:00:00,"admin"
6330,23967,1,0,"3423 HICKORY ST","100 Tangerine Blvd., Brownsville, TX 78521-4368","Texas Phone Number: 956-546-4279","SHEBOYGAN","WI","53081","","",0.00,"SHEBOYGAN",117,"",2/15/2006 0:00:00,"admin"
6331,25318,1,0,"3960 S. Prairie Hill Lane Unit 107","","","Greenfield","WI","53228","","",0.00,"MILWAUKEE",79,"",2/20/2006 0:00:00,"admin"
6332,24446,1,0,"83 Sweetbriar BR","","","LONGWOOD","FL","32750","","",0.00,"SEMINOLE",117,"",2/21/2006 0:00:00,"admin"
6333,26135,1,0,"P.O. Box 8 127 Main Street","","","CASCO","WI","54205","","",0.00,"KEWAUNEE",61,"",2/21/2006 0:00:00,"admin"

View 7 Replies View Related

Ragged Right Not Working Properly

Dec 25, 2007

Hi,
I am exporting a table to a fixed width flat file using "Ragged Right" format. Though i specify carriage return for the last column, the text file is not taking it up.

I am doing the following for connection
cmFlatFile.properties("Format").SetValue(cmFlatFile,"RaggedRight")
cmFlatFile.properties("RowDelimiter").SetValue(cmFlatFile,vbCrLf)

and while adding the column i am setting up the width as follows

vColumns = flatFileDestination.InputCollection(0).GetVirutalInput().VirtualInputColumnCollection

For cols = 0 to vColumns.Count -1
Dim col As IDTSConnectionManagerFlatFileColumn90 = ff.Columns.Add()
if cols = vColumns.Count - 1 Then
col.ColumnDelimiter = vbCrLf
end if
cols.ColumnType = "FixedWidth"
cols.Columnwidth = 20
......
Next cols

Still the rows are exported next to each other just like when we use FixedWidth.

View 4 Replies View Related

Flat File Source With Ragged Right Problems

Aug 27, 2007

For some reason, when I try to use the Flat File Source and set the record type to Ragged Right it does not seem to recognize 'short' records. It seems to be confused by the CRLF set delimiters and not recognize these in 'some' records. The input does not seem consistent. What am I missing?

View 1 Replies View Related

Loading Fact Using Dimension W/ Ragged Hierarchy (geography)

Jul 27, 2007

I'm loading a fact table that has several geographic attributes - some are at the state level, some are at the county level, and then some are drilled farther in that that. I understand the basic concept of the dimension with the ragged hierarchy, but unsure of how to load to the fact table using lookups based on these geographic units. For example, if my geographic dimension contains 200 records for the state of Wyoming, basically a record for each fine-grain place (i.e. city/town), then how do I go about doing a county lookup. Wyoming only has 23 counties, but because of the repetitive nature of the dimension attributes that are not at the finest grain, I'll get more records in the lookup than I need. This activity repeats of course while I move up the geographic scale to state, then country. How do I configure/fill my dimension to handle these differing scales of data?

View 2 Replies View Related

Need Help On Ragged Right Flat File Data Loading Into SQL Server DB

Oct 30, 2007

Iam trying to load a RR Flat File(Ragged Right) into SQL Server DB Destination.

The file feeds the data into two tables and the logic looks like this..

If Position 30 = ''

Record Set goes into Table 1
Else

Record Set goes into Table 2

I have no clue where to start with my limited SSIS Exposure.. Any Ideas as to what i shud be starting with (Transformations)??

Thanks in Advance.

View 1 Replies View Related

Import Text

Nov 9, 2000

We are trying to import a fixed length text file. It has two fields. The first is ten characters, the second is the rest of the row, which may be several hundred characters. Each row is terminated by a {lf}{cr}
The problem is the DTS text import utility. It generates a red line (column seperator) at the 85 character mark. This causes the second field to wrap. We can not move this line, or delete it.
Any idea what is going on?

View 8 Replies View Related

Flat File Destination , Ragged Right Output Column Length

Sep 14, 2007

how do I make the output columns padded with extra space ? I intentionally set my output width larger than the input width, but the generated file is still jamming all the columns next to each other

View 2 Replies View Related

Analysis :: Ragged Hierarchy - Unary And Custom Rollup Calculations

Aug 13, 2015

Any working example of what a ragged hierarchy should look like for the unary and custom rollup calcs to work?

I have a parent-child hierarchy that works as expected but can't manage to get the same with a flattened ragged hierarchy. Parents disappear if any of the children aren't in the fact table (even though the children are set to '~' and the parent has its own custom rollup calc).

Do I need to replicate the unary and measure formula all the way to the end in the same way i do the member name/value (so i can set ignore if same as parent)? Setting unary to null seems to work when the key doesnt exist in the fact table but the default behaviour for null is '+' so i end up getting wrong results if the key does exist!

Repro:

In words

I have an EAV style fact table. The measure name is defined in a 'dim measure' table. Some measures exist in the fact, some are manufactured using Unary ops or custom rollup formulas.

Here's a diagram of the hierarchy

I want to recreate the functionality as a flattened hierarchy (the number of levels is fixed), but can't seem to do it... Here's what I've gotten for the same hierarchy created using the two different approaches

Below are queries to recreate the tables from adventureworksdw2014. The parent child hierarchy is set up the standard way... measureparentkey is parent. set name, unary and formula as expected. Set NonLeafDataHidden for the parent attribute. I've tried various combinations for the ragged hierarchy but none work 100% of the time. I want to have a user defined hierarchy for the performance benefits.

--Fact view
CREATE View dbo.FactSales
AS
select fis.ProductKey
, fis.OrderDateKey
, 1 AS MeasureKey -- salesAmount
, fis.SalesAmount AS MeasureValue

[Code] .....

View 3 Replies View Related

Text File Import

Apr 17, 2000

SQLServer 7.0

Hi,

I have a small project on that involves importing a series of csv files held within an ftp directory into our Datawarehouse. Every day a series of csv files will be added to the directory. These will be named something like:

Audit1.csv,Audit2.csv etc.

I would like to automate this process as this can involve up to 400 files at a time. The proecedure would need to identify a valid file, import it into the database, delete the file and then move onto the next one.

Does anyone know of a way to achieve this? I was thinking along the lines of using a cursor and bcp but I'm not sure how to identify these files to the database i.e. how do i make it step through the directory and process the files?

Any help would be greatly appreciated.

Thanks

Rob

View 1 Replies View Related

DTS Import Text File

Oct 20, 1999

In a DTS package I have a text file import object, a data pump, and a SQL object. The text file import object has been set up to splice a 500 character wide file into 20 columns. The data pump task does a copy column for all the columns into the appropriate table. What I need to do is have a way of changing the file name I specify in the text import object. I have 12 months worth of data in seperate files (DBF0199.TXT, DBF0299.TXT, DBF0399.TXT, etc..)
which all use the same format. Is there a way to change the text import objects file name inside the package using an active script task or something?

Any help is greatly appreciated.

Thanks,

Todd

View 4 Replies View Related