XML Source Adapter Inserts NULLS Into SQL Instead Of Blanks
Jul 3, 2007
I'm working on a new project using SSIS in SQL Server 2005 and have an issue that I need resolved.
I'm loading an XML file into SSIS using the XML Source Adapter. This maps to an OLE DB destination which reads the data into a SQL table (please see below for the table structure). In between this stage is a data conversion to convert the Unicode characters from XML into non-Unicode characters.
CREATE TABLE [dbo].[XMLTest](
[Forename] [varchar](50) COLLATE Latin1_General_CI_AS NULL,
[Surname] [varchar](50) COLLATE Latin1_General_CI_AS NULL,
[PostCode] [varchar](50) COLLATE Latin1_General_CI_AS NOT NULL
)
The problem I have is with blank strings in xml are being inserted into the SQL table as NULL values and not blanks (see the xml below).
<Root>
<People>
<Forename>Duncan3</Forename>
<Surname></Surname>
<PostCode>ME16 0WH</PostCode>
</People>
</Root>
Correct me if I'm wrong but a null value in xml means the node does not exist (See the xml below)?
<Root>
<People>
<Forename>Duncan3</Forename>
<PostCode>ME16 0WH</PostCode>
</People>
</Root>
Please also find the schema attached for reference.
<?xml version="1.0"?>
<xschema attributeFormDefault="unqualified" elementFormDefault="qualified" xmlns:xs="http://www.w3.org/2001/XMLSchema">
<xs:element name="Root">
<xs:complexType>
<xsequence>
<xs:element minOccurs="0" name="People">
<xs:complexType>
<xsequence>
<xs:element minOccurs="1" name="Forename" type="xstring" />
<xs:element minOccurs="0" name="Surname" type="xstring" />
<xs:element minOccurs="1" name="PostCode" type="xstring" />
</xsequence>
</xs:complexType>
</xs:element>
</xsequence>
</xs:complexType>
</xs:element>
</xschema>
View 5 Replies
ADVERTISEMENT
Jul 26, 2006
I wrote a simple data flow with an OLE DB source and destination. I do a direct mapping of the columns ( colA - > colA) with no transformations needed. I found that colA on the destinatin does not allow NULLS (required by the program that accesses that database) while colA on the source supports and has NULLs, Is there any accomodaiton for handling NULLS (like mapping them to blanks) in the direct copy approach or do I need to read each row and test colA_ISNull?
TIA,
barkingdog
View 3 Replies
View Related
Dec 6, 2004
Hi,
I'm trying to create a view from a table in which I want to concatenate to columns to a new column. The concatenation part works fine when data "supports" calculations, i.e. no nulls. The data however is "infected" in two ways I have Null-values and I have blanks alongside with some real information. I concatenate via a CASE statement but I can't figure out how to take the 3 situations into account (nulls, blanks and data) and create one meaningfull column in my view.
The result should be
column1 + "-" if column2 is null or blank
column1 + "-" + column2 if column2 has meaningfull data
Both the involved columns are text (varchar).
If anyone has been in this situation before could you please provide som code that handles the situation.
Regards
Kim Hansen
View 2 Replies
View Related
Feb 27, 2008
I have an Excel spreadsheet with a field that may contain blanks (empty). When I filter on that field to show only the blanks, I get 4000 records returned. But if I import the spreadsheet into SQL Server, the test in there for blanks ('') returns nothing. However, if I test for Nulls in that field, I get 5000 records returned. I don't know which one (Excel or SQL Server) is giving me the right answer. Can anyone help me understand what's going on?
Thanks!
View 1 Replies
View Related
Oct 19, 2007
Hi,
With DTS, the default behavior is to import blanks ("") in a data file as nulls in the table.
However, in SSIS, the default behavior is to import blanks as blanks.
I need to ensure that the packages I'm migrating import the data in the exact same way. I am wondering what is the easiest way to convert blanks to nulls?
Right now I'm using a script to convert the blanks to nulls before they are written to the table, but just wondering if there's an easier way.
Thanks
View 30 Replies
View Related
Apr 18, 2007
Hi,I've got a program that has a form. Ā On the form there are several fields which are optional that I would prefer to remain as NULL in the database (rather than a default value) if the user doesn't fill them out. Ā I'm using stored procedures for my inserts (sql 2000 + C#). Ā How do most people handle these situations? Ā Right now I have a seperate function which receives the parameter values as params and then handles creating the parameters and executing the stored procedure. Ā I'm not sure how to handle passing that I want one of those to be null, at least not without having multiple functions and possibly various stored procedures for different possibilities.Thanks.
View 1 Replies
View Related
Jun 22, 2006
I am currently writing a custom source adapter that extracts data from a JD Edwards OneWorld system. In the custom user interface of the source component I need to allow the user to set a query (a custom property), and then refresh a list of output columns that will be extracted based upon that query (similar to the list shown in the advanced editor).
My question is, can I apply the custom property change to the component and build my output column list without closing and restarting my custom UI form? I understand that the IDTSComponentUI interface being implemented allows for transactional editing of the component, in that the changes to the component are not applied until I have returned a result in the implemented Edit() method. However is there a way to apply changes without returning this result (and closing my UI)?
Essentially I am looking to have similar behaviour to that of the 'Refresh' button in the advanced component editor form.
Thanks
View 3 Replies
View Related
Jan 4, 2007
Hi All,
Using a data access mode of SQL Command, I have a stmt that returns one date ( e.g select max(LastChangedDate) from TblA ).
I want to store the result in a variable. How do I do this?
Cheers,
Tamim.
View 13 Replies
View Related
Oct 30, 2007
I am using an "XML Source Adapter" in an SSIS Package to try and load an XML file into a database table. But having trouble with a date field.
The XSD defines the field as:-
<xs:element minOccurs="1" name="DateTime" type="xs: string" />
The XML looks like this:-
<DateTime>12/31/2007 10:12:14.123</DateTime>
And the database table column is defined:-
[DateTime] [datetime] NOT NULL,
So in the Data Flow I have a "Data Conversion Transformation" converting [DT_WSTR] to [DT_DBTIMESTAMP].
However, when I run the package I get:-
Error: 0xC02020C5 at Load XXX File, Data Conversion 1 [13550]: Data conversion failed while converting column "DateTime" (15100) to column "DateTimeDT" (15422). The conversion returned status value 2 and status text "The value could not be converted because of a potential loss of data.".
So what how do I fix it?
I don't mind losing some miliseconds of precision, so I have tried configuring the error output of the Data Conversion for this field to "ignore failure" for "Truncation". But that seems to have no effect.
I'd be happy to change the XSD to:
<xs:element minOccurs="1" name="DateTime" type="xs:dateTime" />
But the "XML Data Flow Source" doesn't seem to support the dateTime data type.
Any suggestions appreciated. Thanks.
View 1 Replies
View Related
Apr 3, 2006
I'm having alot of trouble figuring out the proper way to log from inside my custom source adapter.
I couldn't find my useful information in MSDN, it mostly applies to logging from inside a script task.
I'd like to log my messages along with all the other SSIS package log entries, which I have going to the dts log table.
I'm assuming I should be using some functionality from Microsoft.SqlServer.Dts.RunTime, probably the LogProvider.
Can anyone advise?
View 3 Replies
View Related
Nov 21, 2007
I am using the "XML Source Adapter" in an SSIS package.
I notice that although you can specify the XML filename as an expression, the XSD appears to have to be a fixed file path.
This is a problem for me since the path for the XSD is different in my development than it will be in production (in production it's on drive E:, which I don't have).
I'd like to have the file location specified in the config file, but since I can't make it an expression how can I do that?
Also, since they don't have Connection Managers I can't switch DelayValidation on.
Thanks.
View 24 Replies
View Related
Apr 16, 2008
I am a business user trying to build an incremental ETL package with SSIS. I have a working prototype on SQL Server 2005 where I select the max(ID) from the last successful run and pass that value into a variable. Then, in my Data Flow step, I select an OLE DB source adapter and use this variable in a custom select statement.
Here's my challenge....the live data is actually in a Postgres DB so I have to use a Data Reader Source adapter. When I try to pass my variable to this adapter the job bombs out. Does anyone know how to dynamically update the query text inside a Data Reader source adapter using variables or otherwise?
View 3 Replies
View Related
Jun 1, 2006
Hi,
Quick question on how SSIS handles queries from Data Source in a Data Flow. I noticed that when I run a particular query from Query Analyzer it takes forever. But, when I run the same query in SSIS data source in a data flow. The query results are immediate.
The query plan is already cached in SQL.
Is this just something which I am seeing incorrect or is there some bit of optimization in there in SSIS. As per my understanding SSIS does not optimize the source query.
Thanks,
Gaurav
View 3 Replies
View Related
Feb 9, 2006
Hi
I am using Oledb Provider for Db2 from IBM. (There is another one from microsoft)
Through this provider I am constructing a oledb connection manager.
This connection manager I am Using in Oledb Source adapter.
Now when I set Data Access mode as Table or view I Am able to preview the data.
But when I use Data Access Mode as SQL Command try to preview I get this Error
TITLE: Microsoft Visual Studio
------------------------------
Error at s_NEWMAPPING10 [EMP [1]]: An OLE DB error has occurred. Error code: 0x80040E21.
------------------------------
ADDITIONAL INFORMATION:
Exception from HRESULT: 0xC0202009 (Microsoft.SqlServer.DTSPipelineWrap)
------------------------------
BUTTONS:
OK
------------------------------
I am not able to figure out how to resolve this.
View 3 Replies
View Related
Nov 16, 2015
I want to use the excel sheets located on a share point site as source, for this i downloaded the adapter from the following link URL....IĀ got the connection manager to work, but when coming to load the excel files to the destination location its not doing what is intended. For example i want the destination location my local drive for the excel files, how can i do that which destination should i use?
View 2 Replies
View Related
Jan 25, 2007
Hello SQL Server Community.
I'm looking for a way to keep my database in Subversion so I can maintain it in source control.
Seems like the best way to do this would be to dump the database to a text script file. This works fine for creating the database structure and managing those change, however, that database structure is useless without the actual data in the tables.
Is there a good way to generate the inserts for the actual data? Why isn't this an option in the script wizard.
Also, does anyone have a better idea of source control of SQL Server databases in Subversion?
Thanks,
Ryan.
http://www.dynamicajax.com
View 8 Replies
View Related
Nov 20, 2007
I have a very simple SSIS task whcih opens a flat file data source, checks the first two characters of each row and if it is equal to "06", inserts that row into an OLDB destination table. However it is inserting EVERY row, even the ones not equal to "06" in first two columns. So, how do I get it to NOT insert the rows I don't want? Any help is very much appreciated and the following is my code:
Public Overrides Sub Input0_ProcessInputRow(ByVal Row As Input0Buffer)
Dim InactiveDate As String
If Left(Row.Column0, 2) = "06" Then
Row.FilingNbr = Mid(Row.Column0, 3, 10)
Row.Address1 = Mid(Row.Column0, 13, 50)
Row.Address2 = Mid(Row.Column0, 63, 50)
Row.City = Mid(Row.Column0, 113, 64)
Row.State = Mid(Row.Column0, 177, 4)
Row.Zip = Mid(Row.Column0, 181, 9)
Row.ZipExt = Mid(Row.Column0, 190, 6)
Row.AgentCountry = Mid(Row.Column0, 196, 64)
InactiveDate = Mid(Row.Column0, 264, 2) + "/" + Mid(Row.Column0, 266, 2) + "/" + Mid(Row.Column0, 260, 4)
If IsDate(InactiveDate) And InactiveDate <> "00/00/0000" Then
Row.AgentInactiveDate = InactiveDate
Else
Row.AgentInactiveDate_IsNull = True
End If
Row.AgentLastName = Mid(Row.Column0, 268, 50)
Row.AgentFirstName = Mid(Row.Column0, 318, 50)
Row.AgentMiddleName = Mid(Row.Column0, 368, 50)
Row.AgentSuffix = Mid(Row.Column0, 418, 6)
End If
End Sub
Karen
View 5 Replies
View Related
Jun 8, 2007
Ok, I think this may have a simple answer. Basically I have no problems in setting up QueryString/Control/etc parameters when I use SELECT in the Configure Data Source Wizard as it prompts me for the necessary parameters. But when I try to use the Configure Data Source Wizard with an UPDATE, INSERT or DELETE it does NOT prompt me for the required parameters.Is this a bug or am I just missing something? Do I have to put them in manually or something?Thanks!
View 5 Replies
View Related
May 16, 2006
When i do a select on my emplee table for rows with null idCompany i dont get any records
I then try to modify the table to not allow a null idCompany and i get this error message:
'Employee (aMgmt)' table
- Unable to modify table.
Cannot insert the value NULL into column 'idCompany', table 'D2.aMgmt.Tmp_Employee'; column does not allow nulls. INSERT fails.
The statement has been terminated.
This sux
View 4 Replies
View Related
Feb 17, 2008
is there an elegant way to use one equals sign in a where clause that returns true when both arguments are null, and returns true when neither is null but both are equal and returns false when only one is null?
View 4 Replies
View Related
May 15, 2008
I have two SSIS packages that import from the same flat file into the same SQL 2005 table. I have one flat file connection (to a comma delimited file) and one OLE DB connection (to a SQL 2005 Database). Both packages use these same two Connection Managers. The SQL table allows NULL values for all fields. The flat file has "empty values" (i.e., ,"", ) for certain columns.
The first package uses the Data Flow Task with the "Keep nulls" property of the OLE DB Destination Editor unchecked. The columns in the source and destination are identically named thus the mapping is automatically assigned and is mapped based on ordinal position (which is equivalent to the mapping using Bulk Insert). When this task is executed no null values are inserted into the SQL table for the "empty values" from the flat file. Empty string values are inserted instead of NULL.
The second package uses the Bulk Insert Task with the "KeepNulls" property for the task (shown in the Properties pane when the task in selected in the Control Flow window) set to "False". When the task is executed NULL values are inserted into the SQL table for the "empty values" from the flat file.
So using the Data Flow Task " " (i.e., blank) is inserted. Using the Bulk Insert Task NULL is inserted (i.e., nothing is inserted, the field is skipped, the value for the record is omitted).
I want to have the exact same behavior on my data in the Bulk Insert Task as I do with the Data Flow Task.
Using the Bulk Insert Task, what must I do to have the Empty String values inserted into the SQL table where there is an "empty value" in the flat file? Why & how does this occur automatically in the Data Flow Task?
From a SQL Profile Trace comparison of the two methods I do not see where the syntax of the insert command nor the statements for the preceeding captured steps has dictated this change in the behavior of the inserted "" value for the recordset. Please help me understand what is going on here and how to accomplish this using the Bulk Insert Task.
View 2 Replies
View Related
Aug 31, 2005
Nevil Mascarenhas writes "Hi SQL team,
I am just a beginer in SQL
Here is sample output of a SQL query
Alarm No Site Name Startdate Starttime
7767 ABC 20-08-05 00:00
7765 XYZ 20-08-05 00:00
7762 ASD
5453 QWE 22-08-05 01:00
In this above example, I would like to replace blank fields in Startdate and Starttime coloumn with XXX
Output required is
Alarm No Site Name Startdate Starttime
7767 ABC 20-08-05 00:00
7765 XYZ 20-08-05 00:00
7762 ASD XXX XXX
5453 QWE 22-08-05 01:00
Can you please guide me."
View 1 Replies
View Related
Jan 18, 2006
Quick Question I hope.
Does having a lot of blanks in a column cause errors ?
In 2 or 3 packages where I get 95% of the data in the Error Colomn I can find nothing wrong with the data at all ? it all either seems perfect or there is no data in the column in question usually I would have maybe 800 rows where data would have been inserted but in the other 40, 000 rows the column is blank
Using the "Retain null values from the source as null values in the destination" doesnt seem to make a differance.
This is the error description
The data value cannot be converted for reasons other than sign mismatch or data overflow.
Anyone know of a solution / reason why this keeps happening
Thanks
View 12 Replies
View Related
Mar 5, 2004
I posted this scenario earlier and ndinakar said my code worked on (I guess) his own machine but has not worked on mine! Does that not sound funny? I need to ge through with this simple but frustrating stuff in time. I've implemented more complex snippets before this but I just can't figure the problem out.
Can anyone out there tell me what I could be doing wrongly? I wrote a simple stored procedure called 'proc_insert_webuser' in a database that has been moved to a remote server called LAGOS-NTS3. I executed it from SQL Query Analyser and it worked. When, however, I executed the same stored procedure from ASP.Net it gives an impression that a new record has been inserted and even displays the newly generated ID column to me. On checking the base table I found only blank columns with only the ID column having the generated IDs!
I'm perplexed. I checked the code and can't possibly spot the error. Can anyone help. Found below are snippets used.
CREATE PROCEDURE proc_insert_webuser
(
@applicantidnumeric(18) output,
@firstnamevarchar(18),
@midname varchar(18),
@lastname varchar(50),
@secretcodevarchar(18),
@my_errnumeric(18) output
)
AS
IF (SELECT COUNT(ApplicantID) FROM tbl_webuser WHERE Firstname = @firstname AND Midname = @midname AND Lastname = @lastname) = 0
BEGIN
begin transaction
INSERT INTO tbl_webuser (Firstname, Midname, Lastname, SecretCode)
VALUES (@Firstname, @Midname, @Lastname, @SecretCode)
commit transaction
SET @applicantid = @@IDENTITY
END
ELSE
BEGIN
RAISERROR('Error! Execution aborted. A matching user profile exist.', 16, 1)
SELECT @my_err = @@ERROR
END
RETURN
Then the ASP.Net stuff:
Sub PostData()
'connection string production
Dim cstring As String = "User ID = sa; Password = ; database = E-Recruitment; Server = LAGOS-NTS3; Connect Timeout = 60"
'connection instantiation
Dim cnx As SqlConnection = New SqlConnection(cstring)
Try
'open connection
cnx.Open()
'instantiates and execute a command object
Dim cmd As SqlCommand = New SqlCommand
With cmd
.Connection = cnx
.CommandText = "proc_insert_webuser"
.CommandType = CommandType.StoredProcedure
Dim param1 As New SqlClient.SqlParameter("@applicantid", 0)
param1.DbType = DbType.Single
param1.Direction = ParameterDirection.Output
.Parameters.Add(param1)
Dim param2 As New SqlClient.SqlParameter("@firstname", txtFirstNM.Text)
param2.DbType = DbType.String
param2.Direction = ParameterDirection.Input
.Parameters.Add(param2)
Dim param3 As New SqlClient.SqlParameter("@midname", txtMidNM.Text)
param3.DbType = DbType.String
param3.Direction = ParameterDirection.Input
.Parameters.Add(param3)
Dim param4 As New SqlClient.SqlParameter("@lastname", txtLastNM.Text)
param4.DbType = DbType.String
param4.Direction = ParameterDirection.Input
.Parameters.Add(param4)
Dim param5 As New SqlClient.SqlParameter("@secretcode", txtPWD.Text)
param5.DbType = DbType.String
param5.Direction = ParameterDirection.Input
.Parameters.Add(param5)
Dim param6 As New SqlClient.SqlParameter("@my_err", 0)
param6.DbType = DbType.Single
param6.Direction = ParameterDirection.Output
.Parameters.Add(param6)
.ExecuteNonQuery()
'pick the auto-number
Dim strAppID = .Parameters("@applicantid").Value
'notifies the user of record success
lblMessage.Text = "Profile created. Your ApplicantID is : " & strAppID & " Kindly take note of your UserID. You'll need it for future logins."
End With
Catch ex As Exception
'notifies the user of any error(s)
lblMessage.Text = Err.Description & " originating from " & Err.Source
Finally
cnx.Close()
End Try
End Sub
==================
I'm calling this procedure from the click event of a command button. What could be happening to the code? I have done similar things and got positive results! Any hint from anyone out there?
View 6 Replies
View Related
Mar 10, 2000
A table was created in version 6.5 that has 2 blank spaces and a slash '/' in it's name (i.e. Item Type w/Groups). This was done in error and now the table cannot be dropped or renamed and dropped.
Does anyone know how I can delete this table? Please let me know.
View 1 Replies
View Related
Mar 11, 2008
Hello,
In my .NET.ASP application I generate a random five digit password, like so:
string password = Membership.GeneratePassword(5, 1);
The result could be a password like "0l$IE" and I the save this password to an SQL server. The problem is however that with the generated password five blanks are also added. Instead of "0l$IE" I get "0l$IE ". I hope you understand what I mean? Is there an easy way to correct this?
I appreciate any help!
View 6 Replies
View Related
Mar 9, 2006
When I enter a data in a table, SQL Server automatically completes the data with blanks up to length of column.
this happens in a web form also in Management Studiıo,
Whan am I doing wrong ?
does Collation have anything with it ?
SQL Server 2005 / Developer Edition
View 2 Replies
View Related
May 1, 2014
I am try to see if there are any blanks in a node of a table that has xml data in one of the columns. The query I use is returning zero results.
Select COUNT(*)from ENTITY
Where CONVERT(XML, Ent_root_xml, 0 ).value('(//UD_PQ_FLAG/node())[1]', 'VARCHAR(50)')= ''
View 1 Replies
View Related
Apr 3, 2008
I have a comma delimited data file TestData.txt of the form:
123,asss,aweqrrr ,ssdsff , ,2wwwrwrr
434,sff,dgfdgd ,sffsfsfete ,sd ,dff
I want to load this data into SQL Server tables so that the data will be imported without the trailing spaces(blanks) after each data element .
Thus I will like to trim the data of the extra spaces (blanks) before import.
Desired data to be loaded :
123,asss,aweqrrr,ssdsff,,2wwwrwrr
434,sff,dgfdgd,sffsfsfete,sd,dff
Any help will be most welcomed
View 4 Replies
View Related
Aug 15, 2013
Aim- the following columns āCancel_Dateā, āFirst_Post_Dateā,āLast_Post_Dateā can either be populated with figures or have a null value.
I want all Null values to be removed and be replaced with a blank
Final table is
Select *
from #test
left join #SF_AccountBuild on #test.FDMSAccountNo = #SF_AccountBuild.[External FDMSaccountno]
Results
[FDMSAccountNo],
External_ID ,
Parentsfid,
[DBA Name],
[Legal Name],
Street,
[MM3-DBA-ADDR2],
[Code] .....
View 5 Replies
View Related
Jan 8, 2015
I am looking for a query which will populate one of my columns that contains some blanks with a defined value based on a value in another column
For example in the below table I would like the blanks in serial number to be populated with "SN09"
Product_NameSerial_Number
PRODUCT1
PRODUCT1
PRODUCT2SN10
PRODUCT3SN11
PRODUCT3SN11
PRODUCT1
So the query I am looking for would basically add some values to the results where they meet the right criteria.
I am running the query through a view so I dont actually want to add the values to the physical tables only to the view of the results if you understand what I mean?
View 12 Replies
View Related
Jul 20, 2005
I use Blanks in ColumnNames ( I know that this isn“t very good, but alot of code and querys had to be changed if I would remove all blanksin all columnnames).When I link this tables with ODBC in my ACC97 - project, some of thetables causes an ODBC-Error.Are there possibilities to workaround this error?Thanks, Andreas Lauffer, easySoft. GmbH, Germany
View 1 Replies
View Related
Feb 5, 2008
Hi,
I've an int field and when this field has a null, I want to show empty/blanks instead of 0. How can this be achieved?
I've tried the below:
declare @a as int
set @a= null
select isnull(@a,'')
o/p: 0
select cast(isnull(@a,'') as varchar)
o/p: 0
select case convert(varchar,@a)
when null then ''
else @a
end
o/p: null
Thanks for your help!
Subha
View 14 Replies
View Related