There's Just No Escaping It... Data Containing Quotes Ruining CSV Files
Jul 11, 2006
Much to my surprise and dismay, after burning hours and hours developing various SSIS packages to extract data from my SQL 2005 DB to CSV flat files, I found out the hard way that SSIS doesn't properly escape quotes (") in the data (my output file is required to have quotes as text qualifiers).
It seems like I'm pretty much forced to create derived columns to test for the existence of quotes in my data, then either escape them or replace them with another character.
This is really NOT COOL. Not only will this significantly hinder performance, it makes my packages a heck of a lot more complicated (especially when I'm exporting lots of text columns).
Am I doing something wrong, or does SSIS really not escape the text qualifier in a delimited flat file? If so, someone PLEASE tell me this will be fixed in the very near future.
View 21 Replies
ADVERTISEMENT
Jun 16, 2004
Hi,
I need to have an varchar value with single quotes. For eg: the below code throws compilation error.
Declare @val VARCHAR(20)
SELECT @val = ''+name+''
print @val
Error: Invalid column 'name'
I want to print name enclosed with single quotes. Please guide me.
Regards,
Sam
View 1 Replies
View Related
Feb 7, 2008
Hi,
I have a parameterized query. The parameters contain data from my tables. Some of the parameters could include single quotes. The single quotes are wreaking havoc in my parameterized query. How can I replace single quotes with double quotes inside of my SQL stored
procedure?
I know that it's something similar to REPLACE(@variablename, '''''', ''''''''), but I can't get the number of quotes right.
All of the examples that I am seeing are converting the quotes inside of an application. This is not an option for me, as I am calling this stored procedure from a SQL job that will run daily.
Thx.
View 2 Replies
View Related
Jun 4, 2007
I have some code (C#) that runs an SQL update query that sets thevalue of a column to what the user passes. So, this causes an errorwhen anything the user passes in has a ' character in it. I'm surethere's other characters that'll break it too. So, I was wondering,how do I get around this? Is there some commonly accepted regexpattern that will make the value safe to run in an SQL query? How canI take care of any values that need to be escaped?I'm not using any fancy ado.net objects:string sql= [whatever the user passes in]SqlConnection connection = newSqlConnection(ConfigurationManager.ConnectionStrin gs[Utils.GetConnectionString].ToString());connection.Open();SqlCommand command = connection.CreateCommand();command.CommandType = CommandType.Text;command.CommandText = sql;try{int result = command.ExecuteNonQuery();if (result != 1){Response.StatusCode = 500;Response.Write("The file has been uploaded, but wecould not update the DB");Response.End();}}catch (InvalidOperationException){Response.Clear();Response.Write("error");Response.StatusCode = 500;Response.End();}connection.Close();
View 2 Replies
View Related
Apr 27, 2007
I am running into an issue with the SSIS when I try to load a CSV file that contains double quotes wrapped around a field (CSV files have double quotes when field contains a comma; example: "Streams, Inc")
Has anyone worked around this issue?
View 1 Replies
View Related
Aug 31, 2015
i have csv files, it contains 5 columns.
i need to add the double quotes in all the records from start and end.
source data
col1 col2 col3 col4
1 abdul this is email it was very good ,and very relative posts.
Target data
col1 col2 col3 col4
"1" "abdul" "this is email" "it was very good, and very relative posts"
View 2 Replies
View Related
Aug 8, 2006
I've got a flat file data source, that is to large to edit with most Windows apps on my server that contains both single and double quote characters that I need to load in a varchar column.
So I attempted to do it with a Replace in data transformation, but I can't get SSIS to allow me to use a variable or pair of single or double quotes within the replace.
If I don't replace the single quote characters with a pair then the records containing these characters all end up in my failed records output file.
Here are 5 example property legal descriptions from my FLAT FILE data source:
COM 441'6" N OF SW/C OF NW4 OF SEC 22-29-20 ELY1340' N200' CROSSING THE CNTR OF TR AT 100 WLY1240' S200' TO POB CONTAINING 6 3/10 ACRE MOL
N 50' OF S 330' OF W 122' OF E 735' OF SW4 OF NE4 OF SEC 28/28/18 A/K/A LOT
271 BLK "M" OF$PB 14/36-T
LOT 9 BLK "BA" OF$PB 39/1
OVERCODED POST LTS 17 21-42 47-55 & 69 PB 27/110 "ALL" SECS 16-21, 28 29/31/19 & "ALL"
N 100' OF S 815' OF TR "H" OF PB 28/58 LESS W 15' FOR ESMT ESMT DESC AS W 15' OF S 815' OF TR "H" OF PB 28/58
View 2 Replies
View Related
Feb 5, 2004
Hi!
Running a SQL2K-server on a Win2K(adv) I have sort of an enigma (to me atleast!):
One of my SPs sometimes seems to hang. Arrgh. This is of course not the total extent of my problem - naah, I've had/created my share of faulty SPs which hang on a regular basis - no, the enigma is that the code inside my SP *never* hangs when executed from the Analyser? The test is done using ctrl-a/copy&paste - remove 'Create...' and putting a 'Declare in front of the only parm, then inserting a select parm='value_of_parm'. As far as my knowledge goes this 'running code from SP in Query Analyser' should introduce no extra room for slack (which in turn could help me explain why the code works in Query Analyser)?
Btw, when I say 'hang' it's not the whole truth - using the debugger I can see the code (mostly a bunch of updates on one table) being executed - but extremely slow. The code takes approx. 1 min in Query Analyser but hasn't finished after 60 mins when executing the SP.
A little info on the SP:
The user runs another SP which gathers some data from an Oracle DB and inserts these data into a SQL2000 table. Then the user activates the above flamed SP in vain hope of marking the newly fetched data with a statusid.
The problem allways arises when the user tries to let the SP engage newly fetched data - and rarely when engaging old (already marked, hence 'remarking') data. I suspect this could be a clue for anyone with some insight?
I'm looking for 1) a solution, 2) a finger pointing out the bug(s) 3) any suggestions on how to do further research into this evil SP!
Thanx in advance!
Ult
View 6 Replies
View Related
Jul 14, 2006
I have text data files from a third party and they use comma as field delimiters and enclose the text for each column in double-quotes. Not a problem for most of the data files until they start sending files where there is " within the column values. SSIS package fails with the error:
The column delimiter for column "Column 1" was not found.
Any ideas on how to resolve this issue will be greatly appreciated.Thankspcp
View 15 Replies
View Related
Aug 11, 2015
Example of data in CSV are as follows:
"XXX","0001",-990039739 ,0 ,0 ,0 ,0 ,0 ,0
"ABC"," ",-3422054702 ,0 ,481385 ,0 ,0 ,0 ,0
"JJZ","0001",0 ,0 ,0 ,0 ,0 ,0 ,0Here's my format:
12.0
10
1 SQLCHAR 0 0 """ 0 "" ""
2 SQLCHAR 0 5 "","" 1 OKCCY SQL_Latin1_General_CP1_CI_AS
[Code] ....
View 5 Replies
View Related
Jan 3, 2002
I had a procdure in SQL 7.0 in which I am using both single quote and double quotes for string values. This proceudreused to work fine in SQL 7.0 but when I upgraded SQL 7.0 to SQL 2000, this proceudre stopped working. When I changed the double quotes to single quotes, it worked fine.
Any Idea why ??
Thanks
Manish
View 2 Replies
View Related
Apr 9, 2015
I have data in a trace file, and I need to extract some info such as phone number.The problem is the phone number could be varying lengths, and various positions in the row.
For example:
@City='New York', @Phone='2035551212' (10 characters, no dashes)
or
@City='San Francisco', @Phone='918-555-1212' (12 characters, with dashes)
or
@City+'Berlin', @Phone='55-123456-7890' (14 characters, with dashes)
I can use CHARINDEX to search & find @Phone=' so I know where the phone number starts, but stuck on a programatic way to find the data between the quotes since it can vary.
View 4 Replies
View Related
Jun 17, 2014
I imported data from flat file to SQL Server database table. After execution one of column got data with double quotes. It look like:
22222.....02/14/2014....."Smith, John"
333........02/14/2014....."Brownies, Alian"
How to remove quotes?
View 3 Replies
View Related
Jul 23, 2007
I have to import a flat file with commas and double quotes as the text qualifier in a SSIS package. However, when I try to import the data into a table, the data moves the information to right, therefore, the last field will capture mulitple fields worth of data. When I create the Connection Manager for the flat file, I have the format as [Delimited] and Text qualifier as ["]. I do not check the unicode button, but under Advanced, make each field a Unicode string [DT-WSTR]. I have included a sample of my data below.
"Internal Sales Document ",9/23/2005 0:00,0.58,"STORES ISSUES","TAPE, PACKING, 2" X 55 YD, CLE ","EP0079771","US363800","2065431980"
"Internal Sales Document ",10/7/2005 0:00,3.76,"STORES ISSUES","Post-It Note Pads, 3"x3", Cana ","EP0079770","US363799","2065431980"
As I highlighted above, I will have a description field that will have double quotes and commas within the text before the end of field.
Please let me know if this is user error. Thanks,
View 8 Replies
View Related
Aug 2, 2007
Hello!
I have a CSV file that encloses all the data fields with quotation marks. Here is a sample:
"08/01/2007","3","021200012","123","0.03"
Is there any way in SSIS that I can tell the Flat File wizard to ignore the quotation marks? I don't want to import the quotes in the database since that will really mess up other applications that need to use the data.
Thanks in advance,
Harry
View 11 Replies
View Related
Nov 10, 2006
Hi, i'm having problems executing the followingSET @SQLAH = 'SELECT sub_id WHERE 'SET @SQLAH = @SQLAH + 'VENUE_TYPE = Hotel'EXEC(@SQLAH)Its getting stuck at Hotel. I realise that it should include an apostrophe either side like so:..SET @SQLAH = @SQLAH + 'VENUE_TYPE = 'Hotel' '..But this escapes the string, how would i escape an apostrophe in a string?I thought maybe:SET @SQLAH = @SQLAH + 'VENUE_TYPE = 'Hotel' 'But no joy :confused: Thanks
View 4 Replies
View Related
Mar 29, 2007
Hi,
i have a field VPATH which holds virtual path of files from a .net project. i have created a search for a Library section.
i need to search only those pages which are inside "library" folder.
folder name: library
i tried to create a query using VPATH LIKE '%/library/%'. but it is showing result from all the folders. can anyone help me asap?
mail to :priya.interactive@yahoo.co.in.
View 6 Replies
View Related
May 21, 2008
myCommand2.CommandText = "INSERT INTO testimonials(name,email,testimonial,approved,time) VALUES ('" + exp.escString(txtName.Text) + "','" + exp.escString(txtEmail.Text) + "','" + txtTestimonial.Text + "','" + false + "','" + DateTime.Now + "')"; ok the form has a field txtTestimonial.Text these strings work when i submit them thoughthats all folksthat"s (double quotes) that\"sthat\"s these failthat's that's that\'s that\'s tried up to 7 just to make sure Exception Details: System.Data.SqlClient.SqlException: Incorrect syntax near 's'.Unclosed quotation mark after the character string ')'. I figure the " works because it ends up being ' " ' in the sql statement and it doesnt mind thatI tried researching this. I could not find any info stating that perhaps sql escapes things differently. I was on the assumption that in the sql itself the database would reconise the before the '.I read i can use html decode but it seems that then i will have to undecode everytime I read from the database and it could be a major pain if I am using a datagrid or something and a big mess. Any help would be greatly appreciated. Jim
View 5 Replies
View Related
May 29, 2007
Y'all:I am needing some way, in the SQL Server dialect of SQL, to escape unicodecode points that are embedded within an nvarchar string in a SQL script,e.g. in Java I can do:String str = "This is au1245 test.";in Oracle's SQL dialect, it appears that I can accomplish the same thing:INSERT INTO TEST_TABLE (TEST_COLUMN) VALUES ('This is a1245 test.");I've googled and researched through the MSDN, and haven't discovered asimilar construct in SQL Server. I am already aware of the UNISTR()function, and the NCHAR() function, but those aren't going to work well ifthere are more than a few international characters embedded within astring.Does anyone have a better suggestion?Thanks muchly!GRB-----------------------------------------------------------------------Greg R. Broderick Join Bytes!A. Top posters.Q. What is the most annoying thing on Usenet?---------------------------------------------------------------------
View 3 Replies
View Related
Jan 9, 2015
I proposed on a new server that we separate Data Files, Log Files, tempDB, Backups, etc. onto separate LUNS on a SAN with High Speed Solid State Drives.I was told that with the new technology with solid state SAN's that it would decrease performance and that it did not work the same way as it did when you had RAID 5's etc.I thought that if things were cared out correctly by a SAN Administrator they would know how to configure for optimal performance.
View 2 Replies
View Related
Jun 3, 2014
I'm working on a script to produce XML. There are two parts to the XML: an envelope (called "message") and a payload ("payload). Here's an example of what the result should look like:
<?xml version="1.0" encoding="utf-8" ?>
<message guid="cb9f7927-a4c7-44f8-9e55-bd5dd3e85894">
<client name="BNS CASL Client" guid="3dc500f3-dffb-455f-a071-12c4fa37a1eb" endPoint="http://localhost:51873/CASLWS.aspx" />
<payload>
<![CDATA[
[code]....
(Note that I had to add a space between the '&' and 'lt' and 'gt' to get it to post in the forum without the forum converting them back to < and >!)You can see that SQL escaped the < and >. Also the CDATA escape sequence is missing.I want to see < instead of & lt and > instead of & gt. Also, I need the CDATA escape sequence.
View 5 Replies
View Related
Feb 6, 2007
Hi,
We are facing a problem while passing a string containing the "&" character into Full Text search CONTAINS predicate. The records that do have this character are not being returned by the search.
I'd like to raise two questions:
1) Is there any special way to escape this character?
2) Does FTS index it at all?
We have tried all known (to us) ways of escaping like doubling the character, using char(38), using ESCAPE etc..Nothing seem to work. Any help would be appreciated.
Thanks,
Alex
View 2 Replies
View Related
Feb 26, 2008
We have a .NET drop down, which gets populated as the user types in letters(last name). If the user types in the single quote we get the error about not escaping the single quote. Question is, which way would it be easier to fix, in the .NET code or in the SQL procedure? I am not to sure if we have full access to the source code since that is a 3rd party control, so if that is not feasible how would I fix that in the stored procedure? This is the current proc that we are using:
Code Snippet
select @str = 'SELECT DISTINCT TOP ' + @Top + ' e.DisplayName
as DbComboText,
e.EmployeeID as DbComboValue
FROM DepartmentDirectory.dbo.Employees ee
INNER JOIN DataMart.dbo.Employees e ON ee.UIN = e.UIN
WHERE e.LastName like ''' + @LastName + ''' AND e.FirstName like ''' + @FirstName + '''
ORDER BY e.DisplayName'
Any help is greatly appreciated.
View 5 Replies
View Related
Sep 17, 2004
Hi!
I'm using replication with two database on SQL 2000,when begin, the log files size is 50mb and the data files size is 150mb. But now the log files size is 2Gb and the data files size is 4Gb. I would like to decrease the log files and the data files ??? How do i do this???
(I using Truncate and shrink doesn't change )
Thanks!!!
View 2 Replies
View Related
Jul 20, 2005
Hello all. Before my arrival at my current employer, our consultantsphysically set up our MSSQL 7 server as follows:drive c: contains the mssql enginedrive d: contains the transaction logdrive e: contains the data filesNo filegroups were set up and the data files consist of only 1 largephysical file. Currently, our data file is >10GB. When I was trained onthe physical aspects of sqlserver, I was told to never create physical files[color=blue]> 2048MB each. If I did, I could expect inefficient physical storage of[/color]data and slower performance (due to the OS).Our server has 2 RAID-5 arrays. Drive c: and e: are located on the firstarray and drive d: on the second. We're running Windows 4.0 NT Server SP6with NTFS.Can someone comment on the use of 1 single large data file vs. more smallerdata files?
View 2 Replies
View Related
Aug 7, 2015
I have an MVC asp.net application that stores many records in a table on sql server, in its own system. used the system for 2 months, worked fine accessing, changing data.
Now that other users are logging in? there is cross coupling going on. one user gets the data from another users sql search.
In the mvc app it had used the get async method to read the ID record from the db, i set that to synchronous. no effect; the user makes their own login id but that does nt matter either.
View 8 Replies
View Related
Mar 10, 2008
i am really in need of help. i have a text file consiting of some data.i want to update my database from that text file periodically say 12 hours.the text file is being updated by another server program in every 12 hours can any one help me in this case? i am lost for this scenario?? help me please.....
View 1 Replies
View Related
Dec 10, 2005
Hi,I have(had) an old Win2k Server server with about 30 web site databases(SQL 2000) that just went under due to hardware problems. Thankfully, Ihave backups of all the databases plus the MDF and LDF files from thehard drive.I want to move all of these sites and their data to a newer server(Win2003) running SQL2000.What's the best way to copy the database from the old server hard drive(now mounted as an extrnal drive to a local machine; I'm currentlyFTPing all of the web site directories from it to the new server)?Just upload the original data to the new server and then mount the MDFand LDF files within the new SQL server? Or do I restore the backupfiles in the new SQL2000?All of my previous data migrations have been DTS operations from onelive server to another, so no experience with either of the abovescenarios. I'll certainly have a lot more experience at one of them bythe time this weekend is through.Thanks for any help you can offer.
View 1 Replies
View Related
Apr 5, 2005
below is the code of my sp
it's giving my wrong result: 1 instead 0 and vice versa
could you please check what's wrong with it: i think there is something related to @var2 in the code colored in orange.
Thanks!
Code:
ALTER PROCEDURE sp_name
(
@param1 int = NULL,
@param2 int = NULL,
@Result int OUTPUT
)
AS
DECLARE @var1 AS varchar(255)
DECLARE @var2 AS varchar(255)
DECLARE @x AS varchar(1000)
IF @param1 IS NOT NULL
BEGIN
SET @var1 = (
SELECT t1col1
FROM tabl1
WHERE t1col2 = @param1)
SET @var2 = ''
DECLARE crsr CURSOR FOR
SELECT t2col1 FROM tabl2 WHERE t2col2 = @var1
OPEN crsr
FETCH NEXT FROM crsr INTO @x
WHILE @@FETCH_STATUS = 0
BEGIN
SET @var2 = @var2 + '''' + @x + '''' + ', '
FETCH NEXT FROM crsr INTO @x
END
CLOSE crsr
DEALLOCATE crsr
SET @var2 = SUBSTRING(@var2, 1, LEN(@var2) - 1)
END
ELSE
BEGIN
SET @var2 = (SELECT t2col1 FROM t2 WHERE tb2col3 = @param2)
SET @var2 = '''' + @var2 + ''''
END
PRINT @var2
IF EXISTS (SELECT * FROM tabl3 WHERE t3col1 IN (@var2))
OR
EXISTS (SELECT * FROM tabl4 WHERE t4col1 IN (@var2))
BEGIN
SET @Result = 1
END
ELSE
BEGIN
SET @Result = 0
END
View 1 Replies
View Related
Mar 19, 2008
Hi all,
How do you put quotes in quotes so it looks like this below?
exec('select IDENT_CURRENT('table')')
Not
exec('select IDENT_CURRENT('table')')
I just don't want the middle quotes to end the initial quote..
thanks in advance..
View 1 Replies
View Related
Aug 8, 2001
Hi,
Your help would be appreciated if you could give me some idea about why SQL 2000 takes single quote(') as a default for string searching instead of double quotes("). I have couple of stored procedures which were developed with string comparison with double quotes in SQL 6.5. If any one knows how I can change default as a double quotes(") instead of single quote(') in SQL 2000, would be great. Thanks
View 1 Replies
View Related
Jan 13, 2005
This is the sql statement I have:
Code:
select * from tblCalls where TNum = [NUMBER] and Street like '[TEXT]'
The problem is that the Street field contains street names with quotes (ex. DE L'ACADIE). When I run this query using this street name, it doesn't work because of the ' between the L and the A. How can I tell SQL to "ignore" the ' in the text?
View 9 Replies
View Related
Jul 25, 2007
Hi,
I am creating a flat file connection to a .csv file
In the columns section of the flatt file connection manager editor, I am not sure why the texts in the .csv file are shown with double quotes arouond them.
They do not have "" in the .csv file.
Thanks
View 1 Replies
View Related
