T-SQL (SS2K8) :: Importing CSVs With Unpredictable Format And Name

Dec 5, 2014

I'm having a bit of an issue with a project I'm working on presently. Originally, I was supposed to be importing Excel files placed into a folder with unpredictable names, but consistent format. That was easy enough, and that's handled.

However, the project has now been expanded; I need to also import CSV files, still with unpredictable names (easy), but both the Excel and CSV files can now have unpredictable columns.

For example, I was originally told I'd be working with columns A,B,C,D,E, but now I'm working with files that can have that format, or A,B,F,G,C,D,E, or any variation as such.

For the Excel files, I can easily just do a SELECT * INTO... with a staging table and OPENROWSET, so that's no problem. The CSVs, however, I'm not so lucky with; I can't seem to use OPENROWSET for it. I could use BULK INSERT, but I can't use it in the form of a SELECT * INTO...

Compounding this is that there can be typos in the column headers, which I also just ran into

Am I missing something here? Is there means of easily adapting to the varying column header names and positions?

View 1 Replies


ADVERTISEMENT

SSIS And CSVs With Quotes

Jun 11, 2009

I have a CSV with some columns containing quoted text with the text itself containing quotes.

I'm trying to use a Flat File Source in SSIS. I've set a double quote as the text qualifier (") and comma as the delimiter. The trouble I'm having is getting to text that contains a double quote like in the following example;

123,"xyz","Don't you hate when MS say's it's a "feature" and not a bug","more data"

SSIS is seeing the first quote as a text qualifier and the second quote as the end of the text. Because the next delimiter is not found (in this case a comma), it returns;

[staging_output [4195]] Error: The column delimiter for column "Column 3" was not found.

How I can get SSIS to treat the whole column as a single piece of text that just happens to contain quotes?

View 11 Replies View Related

DTS -- Unpredictable Order Of Records -- Why?

Oct 9, 2006

Using bcp to load data from a flat file into a database table will load the records in the exact same order as they are in the flat file; whereas, a DTS package may not load the records into the table in the exact same order as they are in the flat file. Why? ... the table has no keys or indexes ... the DTS package was created with MS SQL 2000 and is now being used in MS SQL 2005 ...

View 14 Replies View Related

Unpredictable Queue Order - PLEASE HELP !

Sep 8, 2006

Hi There

I am having alot of trouble with the order in which messages are being delivered, here is my scenario.

I have a transaction action table :

message 1 - xml schema A.

message 2 - xml schema A.

message 3 - xml schema B.

message 4 - xml schema D.

I have the following SP:

BEGIN TRAN

BEGIN DIALOG CONVERSATION

SEND ON CONVERSATION message 1



BEGIN DIALOG CONVERSATION

SEND ON CONVERSATION message 2

COMMIT

AT the target i get message 1 in the queue first them message 2. But then i try this.

BEGIN TRAN

BEGIN DIALOG CONVERSATION

SEND ON CONVERSATION message 3



BEGIN DIALOG CONVERSATION

SEND ON CONVERSATION message 4

COMMIT

At the target i get message 4 first it has the lower queuing_order on the queue, somehow it got ont he queue first.

I have tried turning validation off on the mssage types as i thought it may have something to do with the xml, but same problem.

I then tried to do a commit after each message like this:

BEGIN TRAN

BEGIN TRAN

BEGIN DIALOG CONVERSATION

SEND ON CONVERSATION message 1

COMMIT

BEGIN TRAN

BEGIN DIALOG CONVERSATION

SEND ON CONVERSATION message 2

COMMIT

COMMIT

But message 4 still gets on the queue first . WTF is going on ?

The only way i can get this to work is like this.

BEGIN TRAN

BEGIN DIALOG CONVERSATION

SEND ON CONVERSATION message 1

WAITFOR DELAY '00:00:01'

BEGIN DIALOG CONVERSATION

SEND ON CONVERSATION message 2

COMMIT

Why does this work ? and why do i not have this issue with message 1 and 2? If it has soemthing to do with the xml what is it?

Please help.

View 3 Replies View Related

SQL 2012 :: Importing Xer Format Files Through Wizard Takes Longer Time?

Aug 9, 2015

We are importing xer formats through the wizard to sqlserver database and It takes upto 35-45 mins for each import (single project), any option to reduce the time.Is they any other import options - which can give us faster results.

View 0 Replies View Related

Integration Services :: Importing Data From Word Document (Table Format)?

Aug 4, 2015

how to import data from word document into sql server table in SSIS.

View 18 Replies View Related

T-SQL (SS2K8) :: Export Column To TXT Format

May 21, 2014

I have table1 with col1 varchar,col2 int , col3 xml , col4 bit ...best way to fetch the col3 into file (.txt or .sql) into seprate file for each col3 record . I want to export this in different files for each record if possible with date and time stamp .

View 5 Replies View Related

T-SQL (SS2K8) :: Date Format Conversion

Jun 17, 2014

i am trying to convert a string like this 'le dd/mm/yyyy' into a datetime.I have removed the 'le ' part and used covert(datetime, 'dd/mm/yyyy',103) to convert into datetime. This works for example for 'le 22/11/1799' but for 'le 09/11/1716' it does not work.

select convert(datetime,RIGHT('le 22/11/1799', LEN('le 22/11/1799') - 3), 103) -> it works
select convert(datetime,RIGHT('le 09/11/1716', LEN('le 09/11/1716') - 3), 103) -> it does not work

View 3 Replies View Related

T-SQL (SS2K8) :: Date Format - Convert Function

Mar 12, 2014

How to get this out put.

Details:

declare @deadline Datetime = '2014-03-23 15:30:10.000'
SELECT CONVERT(VARCHAR(30),@deadline, 100) AS DateConvert
----With this I am able to produce that like
----o/p: Mar 23 2014 3:30PM

declare @deadline1 Datetime = '2014-03-03 15:30:10.000'
SELECT CONVERT(VARCHAR(24),@deadline1, 100) AS DateConvert

--o/p: Mar 3 2014 3:30PM
--expected O/p: Mar 03 2014 03:30PM

What is the correct date format to achieve this.

View 2 Replies View Related

T-SQL (SS2K8) :: Export Records Of A Table In XML Format

Jun 5, 2014

I need to export the records of a table in xml format.

create table ##prova
( Valuta varchar(2),
Misura float
)
insert into ##prova values ('EU',1000)
insert into ##prova values ('$',2000)

The final result must be something like this:

<root>
<obs id=”0”>
<dim name=”Valuta” value=”EU” />
<dim name=”Misura” value=”1000” />
</obs>
<obs id=”0”>
<dim name=”Valuta” value=”$” />
<dim name=”Misura” value=”2000” />
</obs>
</root>

View 2 Replies View Related

T-SQL (SS2K8) :: Create Format File Without Delimiter

Feb 16, 2015

I have a date file with no delimiter like bellow

0080970393102312072981103378000004329392643958
0080970393102312072981103378000004329392643958

I just know 5 first number in a line is for example "ID of bank"

or 6th and 7th number in a line is for example "ID of employee"

How can I create a XML format file?

View 2 Replies View Related

T-SQL (SS2K8) :: Get Date Format In DB2 As Expression In SSIS

Sep 15, 2015

I am working on a ssis package ,where I have date parameters as format below

"2015-09-01-00.00.00.000000"

How to get this date format in TSQl expression in SSIS package?

View 1 Replies View Related

T-SQL (SS2K8) :: How To Make A Column Comply To A Specific Format

Dec 4, 2014

I need to insert and update a table. One column of the table needs to conform to a format as: XXXXXXX.XXXXX

If any data is not complying to the format an error needs to be thrown.

How to implement this constraint!

View 3 Replies View Related

T-SQL (SS2K8) :: Convert Datetime To W3C Date And Time Format

Dec 21, 2014

I have a standard datetime and I need to convert it to the client specification of:

YYYY-MM-DDThh:mm:ssTZD
eg: 2009-04-16T19:20:30+08:00

I am not sure of the easiest way to do this.

The test code below gets me part of the way but I am unsure on how to get the offset on the end without hardcoding to much.

DECLARE @datetime DATETIME = '2014-12-20 12:30:00'
SELECT CONVERT(VARCHAR(30),@datetime,127)

View 2 Replies View Related

T-SQL (SS2K8) :: DB Mail Query From Mentioned Data Format Table

Apr 28, 2014

I'm sending report based on below DB Mail query from mentioned data format table. But i'm getting invalid format output in csv file.

Table Data:

CREATE TABLE Sales_Data
(
ITEM_CODE varchar(25),
Price float,
Purchased_Date Datetime

[code]...

View 5 Replies View Related

T-SQL (SS2K8) :: How To Display Stored Procedure Output In Html Table Format

Mar 16, 2014

i m creating one google map application using asp.net with c# i had done also now that marker ll be shown from database (lat,long)depends on the lat,long i wanna display customer,sales,total sales for each makers in html table format.

View 2 Replies View Related

Data Format Issue While Importing Data From Excel To SQL

Jul 17, 2007

hi



when i m importing data from excel to Sql using DTS the column which has text content was not imported as same in excel sheet. whereas a special character is appearing in between the lines. the text field contains multiple lines but the conetent is imported in single line .

ex:









ARIZONA
ALABAMA
STATE


but i m getting imported

as :
ARIZONA ALABAMA STATE

How to Format a single column while importing?



Regards

Raj

View 1 Replies View Related

Reporting Services :: Exporting SSRS Output To Word Format And PDF Format Differs

Aug 19, 2015

I have created SSRS report which has many overlapping objects, the output in PDF format seems to good but in word format it is not giving the required output.

View 5 Replies View Related

Conversion Of Date From Legacy Systems With 7 And 6 Digit Format To DD/MM/YYYY Format

Nov 19, 2014

We are migrating data from old DB2 systems to sql server 2012, the DATE FORMAT in those systems is in decimal format with 7 digits. CYYMMDD format.

I need to convert this into DD/MM/YYYY format.

View 9 Replies View Related

Date In String Format Has To Be Changed Datetime Format

Jun 15, 2005

I have date coming to one page as a string in the following format"May 4 2005 12:00AM"
I need to query one of my tables using this date in combination of other nondate values. How can I convert this date into valid sql server datetime format before I query a database tables
Please help
 

View 3 Replies View Related

Transact SQL :: How To Format A String In A Format Coming From A Table

Jun 4, 2015

I have a table which stores date-of-birth in varchar 19861231(yyyymmdd). A view takes this data. I want to store this date as mmddyyyy in the view. How can we achieve this?

View 18 Replies View Related

Converting Csv Files From One Format To Another Format With Differing Columns

Dec 19, 2007

Hi,


I have a set of csv files and a set of Format Specification files for each of the csv files. I need to convert the csv files into another format of csv files as specified in the Format Specification files. All the columns of the input csv files do not have a mapping with the columns of the output csv files. How can I achieve this using SSIS ? This is an urgent requirement. Please reply asap. Thanks.

View 1 Replies View Related

How To Format Leave Detail Into Tabular/pivot Format?

Jun 16, 2006

Hello Expert!

I need help to I translate this data...

Table "LeaveDetail"

StaffNo | StartDate | EndDate | LeaveType |
1 | 23/04/2006 | 26/04/2006 | AL |
2 | 24/04/2006 | 25/04/2006 | MC |
3 | 26/04/2006 | 27/04/2006 | EL |
1 | 30/04/2006 | 02/05/2006 | EL |

Into this format...

|Apr|Apr|Apr|Apr|Apr|Apr|Apr|Apr|May|May|May|May|
StaffNo |23 |24 |25 |26 |27 |28 |29 |30 |01 |02 |03 |04..
---------------------------------------------------------
1 |AL |AL |AL |AL | | ... |EL |EL |EL |
2 | |MC |MC | | |
3 | | | |EL |EL |

Parameter:
Date From e.g. 23/04/2006 to 23/05/2006

Using only query statement...

Is this possible??

TIA

Regards.

View 7 Replies View Related

Adapter To Convert CSV Format File To SAP IDOC Format

Nov 16, 2006

Hi All,

I am stuck at one place, where I have to convert CSV format file data into SAP IDOC format file. In SSIS we don't have any such SAP adapter (though we have .NET Data Provider for mySAP suite [SSIS SAP Adapter] but this is still not fully supported by Microsoft, plus it doesn't have feature to convert data into IDOC format) that can do this. Can someone here please provide me some pointers on any third party adapters available in market to do this job or if anyone has already developed some custom approach to achieve this task?


Your quick response on this is highly appreciated.

Regards,

Kuldeep Chauhan

View 2 Replies View Related

Date Format From Sql Without Millisecond ----12 Hr Format

May 4, 2008

dear all can anybody help me soon....
i am using visual studio 2005 webapplication based on  sql server 2005 database.
i can get one date from sql using one query.
I am selecting my field based on following code CONVERT(varchar, Oman.Positions.Datum, 9) AS LastUpdate
this case my output is May  4 2008  3:19:45:000AM.....
this output is correct but from this output i want to avoid millisecond part.
ie i want the output like May  4 2008  3:19:45 AM....
how i can do this
regards
 

View 4 Replies View Related

How To Convert Long Date Format To Short Date Format In Store Procedure.

Feb 1, 2008

E.g, i have a store procedure. The start date is long date (4/15/2007 3:00pm). i want to select the start date with a particular date (short date format 4/15/2006). Thanks in advance.

View 1 Replies View Related

Transact SQL :: Cast Or Convert Date In Format YYYY-MM-DD Into New Format Of MM/DD/YYYY?

Nov 27, 2015

I have a table that has a DATE field named. AccountingDate that is in the format YYYY-MM-DD. It's not a VARCHAR field. I simply want to convert this date field into the format MM/DD/YYYY and call it New_Accounting_Date.

I've played with various combinations of CAST & CONVERT but haven't been able to get it to work.

Below is my latest effort which returns the error:

Incorrect syntax near the keyword 'as'

What code would work to return a MM/DD/YYYY value for New_Accounting_Date?

Select GLBATCH.AccountingDate,
convert(GLBATCH.AccountingDate as date),101) AS New_Accounting_Date
from GLBATCH

View 11 Replies View Related

SQL's America Date Format Conflict With Australian Date Format

Nov 14, 2006



Hi

I am trying get my VB6 application to insert a record into a table (SQL Express) which has a datetime column but it would not process if the data format is differ to *American Date format*.

The date() function in VB returns 15/11/2006 which is in Australian Date format (DD/MM/YYYY) according to my setting in "Reginal and Lanuage Option-> Locale 0> English (Australia)" setting.

I get the following error:

Msg 242, Level 16, State 3, Server KITSQLEXPRESS, Line 1
The conversion of a char data type to a datetime data type resulted in an out-of-range datetime value.
The statement has been terminated.

My computer's locale is set to English (Australia) and I expect the datetime format would follow what is set in system locale


I've read an article somewhere on the net about how SQL 2005 eliminate the confusion of date conversion when read/write datetime records into a table...but it seems to me that it is still as in-flexible as MS Access


Is there a setting in the database that takes care of it?

Thanks

View 6 Replies View Related

Converting SQL Date Format To Oracle Date Format

May 28, 2008

Hi,

I have a column date in my database which I should send it to Oracle database. The Date format in Oracle is number. I don’t know how should I convert the date to that format?
Example :
SQL FormatOracle Format
02/16/05 105046

Thanks.

View 6 Replies View Related

I Want Ot Convert Xml Data Format Into The General Data Format

Jun 9, 2006

hi siri have table hh .it has two columnsone is hhno is integer datatype another hhdoc is xml data type likehh tablehhno hhdoc---------------------------------------------------------------------------------------------100<suresh>sfjfjfjfjf</suresh>....................................101<ramesh>hhfhfhf</ramesh>..................................how to convert the xml data format into the general data format plshelp me with examples

View 1 Replies View Related

Importing..

Sep 14, 1998

I am trying to import a database into Sql 6.5 from Access 97`.

How shall I go about this ?


Thanks.
mark.

View 3 Replies View Related

Importing From XML

Aug 26, 2004

Hello everyone and thanks in advance.

Ok, here's the deal. I am 1 of 2 developers at my organization, the other is a COBOL programmer. Long story short, every night he sends me a 22MB XML file containing our company's inventory. This contains about 40,000 items. In C#, I'm basically looping through all the XML nodes and on each node I load up an object with the particular properties (manufacturer, SKU, serial number, etc.) and call a Stored Procedure to to save the item.

In saving the item, I'm checking to see if it was already in my inventory and I either perform an update or an insert on the table as approriate. The query does a couple of other things like retreiving the Purchase Order Number, Cubic Feet, etc. from other tables. All in all, the query takes about 1-2 seconds to complete. I've optimized the query as much as I know how (including the addition of indexes), but this still takes at least 6 hours to complete the 40,000 items. Is this the best approach? Any other suggestions?

So far the only things I've thought about changing is:
1. Break it out into multiple queries to avoid poor compilation of execution plans

2. Multithread the C# app so that I'm running like 10 queries at once.

3. Get the COBOL programmer to give me a delimited file and do a BULK INSERT on it into an empty HEAP, then run a single INSERT and a single UPDATE (using CASE statements)

Again, Thanks in advance!

View 4 Replies View Related

Importing XML

Mar 17, 2004

Hi everybody:

I want to know how to import to SQL Server an XML File

Thanks for your help and quick answer.

Cristopher Serrato

View 1 Replies View Related







Copyrights 2005-15 www.BigResource.com, All rights reserved