Data File Blanks To Nulls
Oct 19, 2007
Hi,
With DTS, the default behavior is to import blanks ("") in a data file as nulls in the table.
However, in SSIS, the default behavior is to import blanks as blanks.
I need to ensure that the packages I'm migrating import the data in the exact same way. I am wondering what is the easiest way to convert blanks to nulls?
Right now I'm using a script to convert the blanks to nulls before they are written to the table, but just wondering if there's an easier way.
Thanks
View 30 Replies
ADVERTISEMENT
Jul 26, 2006
I wrote a simple data flow with an OLE DB source and destination. I do a direct mapping of the columns ( colA - > colA) with no transformations needed. I found that colA on the destinatin does not allow NULLS (required by the program that accesses that database) while colA on the source supports and has NULLs, Is there any accomodaiton for handling NULLS (like mapping them to blanks) in the direct copy approach or do I need to read each row and test colA_ISNull?
TIA,
barkingdog
View 3 Replies
View Related
Dec 6, 2004
Hi,
I'm trying to create a view from a table in which I want to concatenate to columns to a new column. The concatenation part works fine when data "supports" calculations, i.e. no nulls. The data however is "infected" in two ways I have Null-values and I have blanks alongside with some real information. I concatenate via a CASE statement but I can't figure out how to take the 3 situations into account (nulls, blanks and data) and create one meaningfull column in my view.
The result should be
column1 + "-" if column2 is null or blank
column1 + "-" + column2 if column2 has meaningfull data
Both the involved columns are text (varchar).
If anyone has been in this situation before could you please provide som code that handles the situation.
Regards
Kim Hansen
View 2 Replies
View Related
Feb 27, 2008
I have an Excel spreadsheet with a field that may contain blanks (empty). When I filter on that field to show only the blanks, I get 4000 records returned. But if I import the spreadsheet into SQL Server, the test in there for blanks ('') returns nothing. However, if I test for Nulls in that field, I get 5000 records returned. I don't know which one (Excel or SQL Server) is giving me the right answer. Can anyone help me understand what's going on?
Thanks!
View 1 Replies
View Related
Jul 3, 2007
I'm working on a new project using SSIS in SQL Server 2005 and have an issue that I need resolved.
I'm loading an XML file into SSIS using the XML Source Adapter. This maps to an OLE DB destination which reads the data into a SQL table (please see below for the table structure). In between this stage is a data conversion to convert the Unicode characters from XML into non-Unicode characters.
CREATE TABLE [dbo].[XMLTest](
[Forename] [varchar](50) COLLATE Latin1_General_CI_AS NULL,
[Surname] [varchar](50) COLLATE Latin1_General_CI_AS NULL,
[PostCode] [varchar](50) COLLATE Latin1_General_CI_AS NOT NULL
)
The problem I have is with blank strings in xml are being inserted into the SQL table as NULL values and not blanks (see the xml below).
<Root>
<People>
<Forename>Duncan3</Forename>
<Surname></Surname>
<PostCode>ME16 0WH</PostCode>
</People>
</Root>
Correct me if I'm wrong but a null value in xml means the node does not exist (See the xml below)?
<Root>
<People>
<Forename>Duncan3</Forename>
<PostCode>ME16 0WH</PostCode>
</People>
</Root>
Please also find the schema attached for reference.
<?xml version="1.0"?>
<xschema attributeFormDefault="unqualified" elementFormDefault="qualified" xmlns:xs="http://www.w3.org/2001/XMLSchema">
<xs:element name="Root">
<xs:complexType>
<xsequence>
<xs:element minOccurs="0" name="People">
<xs:complexType>
<xsequence>
<xs:element minOccurs="1" name="Forename" type="xstring" />
<xs:element minOccurs="0" name="Surname" type="xstring" />
<xs:element minOccurs="1" name="PostCode" type="xstring" />
</xsequence>
</xs:complexType>
</xs:element>
</xsequence>
</xs:complexType>
</xs:element>
</xschema>
View 5 Replies
View Related
Nov 30, 2006
Hi,
Is it possible using derived column transform to change all blank values in a flat file to say a "0"
Basically convert "" to "0"
Thanks for any help,
Slash.
View 3 Replies
View Related
May 1, 2014
I am try to see if there are any blanks in a node of a table that has xml data in one of the columns. The query I use is returning zero results.
Select COUNT(*)from ENTITY
Where CONVERT(XML, Ent_root_xml, 0 ).value('(//UD_PQ_FLAG/node())[1]', 'VARCHAR(50)')= ''
View 1 Replies
View Related
Apr 3, 2008
I have a comma delimited data file TestData.txt of the form:
123,asss,aweqrrr ,ssdsff , ,2wwwrwrr
434,sff,dgfdgd ,sffsfsfete ,sd ,dff
I want to load this data into SQL Server tables so that the data will be imported without the trailing spaces(blanks) after each data element .
Thus I will like to trim the data of the extra spaces (blanks) before import.
Desired data to be loaded :
123,asss,aweqrrr,ssdsff,,2wwwrwrr
434,sff,dgfdgd,sffsfsfete,sd,dff
Any help will be most welcomed
View 4 Replies
View Related
May 8, 2015
I have a small question here is below how my data look
K_ID ColA ColB
ColC ColD
1 X12134 ABCD
' ' 1/1/2015
2 X12134 ABCD
' ' 2/1/2015
3 X12134 ABCD
DE02635 3/1/2015
I want the output like this
K_ID ColA ColB
ColC ColD
1 X12134 ABCD
DE02635 1/1/2015
2 X12134 ABCD
DE02635 2/1/2015
3 X12134 ABCD
DE02635 3/1/2015
I tried using Over by, Lag functions.. But some where i am missing the logic in achieving the Output...
View 28 Replies
View Related
Apr 25, 2007
I am importing a flat file with fixed width fields and some of the values may be empty so I wanted to insert a NULL for those columns. I am getting errors in the Data Conversion step when trying to convert to TINYINT for example. Is there an easy way to have the process set the value to NULL if the data is blank? The only way I know how to do it is with a Derived Column and an Expression like: LEN(TRIM(ColumnName)) == 0 ? NULL(DT_UI1) : (DT_UI1)ColumnName Is there a better way? Because I need to do this for a couple hudreded columns.
-Ryan
View 3 Replies
View Related
Jan 23, 2007
I have a text file I am trying to import into SQL Server using OLEDB connection.
It's a fixed field text file, ragged right format. One of my columns maps to a numeric column in the DB. In some spots in the file, it is blank, in others there is actual numeric data.
I can't get it to import. If I set the text file column to numeric, I get an error "That value could not be converted because of a potential loss of data." If I set the text file column to string, I get a similar error from the OLE DB provider, "Invalid character value for cast specification"
I have tried telling it to retain nulls in the data flow and the other way as well. Can someone tell me what I am doing wrong?
View 1 Replies
View Related
Jul 23, 2005
We are using the bcp utility (via APIs) to export data from a SQL tablein a fixed format text file. BCP is inserting spaces for a field ifthe field contains a NULL. This is fine with us except at the end ofthe line, there are no spaces for that field just the end-of-rowterminator prematurely, so it looks like that field is not present andmesses up another piece of software we pump the text file into downstream.Example -- The last row illustrates the problem.123-49-890 Mary Smith Raleigh NC
999-88-123 Henry Ax Boston MA
456-99-123 Sue Kite WA
789-88-126 Andy Yates Philadelphia
We have thought about using a SQL query to convert the NULL dataexplicitly to spaces, but were wondering is there a switch or somethingin our format file to get around this.Thanks.
View 2 Replies
View Related
Dec 12, 2006
More SSIS woes. DTS was so much easier.
I have a flat file. It's fixed-with with CRLF record delimiters (a.k.a. Ragged Right format).
Some fields are null, and represented by the text NULL.
I'm trying to import the file into SQL via an OLE DB connection. The target table is a SQL 2000 data table. Two of the fields in the target database are of type smallint.
When I run PREVIEW on the data source (Flat File), everything looks good & correct. I added the convert columns task to convert my strings to smallint. This is where things go haywire.
After linking everything up, the conversion gives me a "Cannot convert because of a possible loss of data." All of my numbers are < 50, so I know this isn't the case. Another SSIS bogus error
My first instinct is the SSIS doesn't understand that NULL means null. I edited the file and replaced all instances of NULL with 4 emtpy string chars. Still no good. It seems to be having a hard time parsing the file now.
I dropped the convert task and tried editing the data source, and set the two smallint fields to smallint instead of string (SSIS formats). I get the same conversion error.
Changing the NULL values to 0 fixed the problem, but they're not 0. They're null.
Short of creating another script that converts all zeros to NULL using the aforementioned hack, I'm out of ideas.
I'm I missing something or is SSIS just incapable of handling nulls in fixed-width flat file formats?
TIA
View 7 Replies
View Related
May 16, 2006
When i do a select on my emplee table for rows with null idCompany i dont get any records
I then try to modify the table to not allow a null idCompany and i get this error message:
'Employee (aMgmt)' table
- Unable to modify table.
Cannot insert the value NULL into column 'idCompany', table 'D2.aMgmt.Tmp_Employee'; column does not allow nulls. INSERT fails.
The statement has been terminated.
This sux
View 4 Replies
View Related
Dec 1, 2014
I have a table called "alert" and in this table, for each columns there are data as 1 , or 0 or NULL. how do I write a query to just give me data's with "1" in them not the NULLS or 0?
View 3 Replies
View Related
Mar 15, 2006
Hello..I was wondering if anyone out there could tell me how they deal withNULL values in a data warehouse? I am looking to implement a warehousein SQL 2005 and have some fields which will have NULL values and Iwould like some further ideas on how to deal with them. At my last jobin dealing with Oracle we were just going to leave the fields NULL, butin SQL how would you best recommend cleaning the data? I greatlyappreicate your help and look forward to your reponses.Thank you
View 2 Replies
View Related
Dec 5, 2006
Hi!
I have data in the following manner:
period course1 course2
4 NULL Eng
4 NULL Math
4 Phys NULL
4 Chem NULL
Is there any way I can show this data this way (the order is irrelevant - row1 can be joined with row3 or row4)?
period course1 course2
4 Phys Eng
4 Chem Math
4 Phys Eng
4 Chem Math
Basically, I want to be able to remove nulls.
Please let me know if any of you have come across such an issue.
Thanks so much!
-Parul
View 3 Replies
View Related
Feb 17, 2008
is there an elegant way to use one equals sign in a where clause that returns true when both arguments are null, and returns true when neither is null but both are equal and returns false when only one is null?
View 4 Replies
View Related
Feb 20, 2008
Hello,
I have a datareader and whenever the value that it is suppose to display in the Label is empty because there is no Data Entered Yet I gives me and error and I have the event set ONLOAD so the page with no data to disply does not load.
So the Ranking.text = a select statement i have, but when I dont have any data into the SQL database there is nothing to SELECT so the program gives of an error.
Can someone help me with this. Thank you.
Here is the code: Dim objReader As SqlDataReader
conn.Open()
objReader = comm.ExecuteReader()
objReader.Read()lblRanking.Text = objReader("Rating")
objReader.Close()
View 7 Replies
View Related
Sep 25, 2013
Our database stores vehicle data in one table, but 3 different types of data are stored in the one table. The table contains all the columns for all 3 types so when you query the table you get at least 3 rows back with null values for all the columns that don't apply to that record. The data is imported to the table when it's updates so there's a possibility that they're updated at different times so they have a different BATCH like:
BATCH TYPE ID RATING INSURANCE SAFETY
300 SAFE 123 NULL NULL A
300 INS 123 NULL YES NULL
250 RATE 123 A NULL NULL
What I'd like returned is:
ID, RATING, INSURANCE, SAFETY
123 A YES A
I'm trying to do a case statement to pull the data down, but I keep ending up with multiple rows because of all the nulls. I tried doing a SUM of the case statement with an ISNULL(SAFETY,0) but I can't SUM char values. I can probably do this with 3 temp tables to load the data that I want for each TYPE into them and then select and join them together, but is there a better way to do this?
View 2 Replies
View Related
Sep 26, 2013
I'm inserting data from a c# webservice into a table via a stored procedure, but I get a Column does not allow nulls on the @alert_id column/field. It is set as int and allow nulls is not ticked.
Here's the sql:
USE [aren]
GO
SET ANSI_NULLS ON
GO
SET QUOTED_IDENTIFIER ON
GO
ALTER PROCEDURE [aren1002].[ArenAlertInsert]
[Code] ....
View 7 Replies
View Related
Feb 21, 2008
In OO programming, data references, such as pointers in C++ or objects in C#. If the reference is null, it points to no object, or to 0x000000, thus the "referenced" object is non existent and the "object" takes up no RAM.
I know pointers are not the same as Database values, but what I'm interested in knowing is, on the bits and bytes level, what is a "NULL" and how much space does it take up, as much as is the field? Or just one byte? and when a NULL is replaced does that cause a shifting in the data of the database page or does the new data simply replace the null?
View 4 Replies
View Related
Dec 12, 2007
I'll preface this by saying I'm pretty new to SSIS; I'm coming over from Coldfusion and don't have much DTS experience to draw from either. That said....
I've got a package that I run to migrate data from a bunch of older databases into a "flat" new schema. The new schema is not identical to the old, in other words, so it's not a simple mapping of existing columns. All the data flow tasks have finally gotten to a working state, with much trial-and-error. Now, suddenly one of the tasks is throwing this error:
"...Cannot insert the value NULL into column 'the_id', table 'the_table'; column does not allow nulls. INSERT fails."
The column is an identity column in the new table; it is NOT NULL as it is the primary key. I've triple-checked that identity is on. Basically it's generated anew each time this package is run. In the data flow task, mappings are set to ignore for this column; also, Keep identity and Keep nulls are on (although since this column is not in the source I can't see how this affects anything.)
(***For anyone wondering why in the heck I'd need this column at all, offhand I can't recall if later tasks use it or not...I'm actually wondering if it's even needed in this read-only table if it's not used as a foreign key somewhere else...however, I'd like to figure out this issue regardless... )
I've had a hard time finding anyone with the same problem out there...usually people with NULL issues simply are trying to insert into a NOT NULL column. The big difference here is that the column is identity.
Thanks,
Rick
View 3 Replies
View Related
May 15, 2008
I have two SSIS packages that import from the same flat file into the same SQL 2005 table. I have one flat file connection (to a comma delimited file) and one OLE DB connection (to a SQL 2005 Database). Both packages use these same two Connection Managers. The SQL table allows NULL values for all fields. The flat file has "empty values" (i.e., ,"", ) for certain columns.
The first package uses the Data Flow Task with the "Keep nulls" property of the OLE DB Destination Editor unchecked. The columns in the source and destination are identically named thus the mapping is automatically assigned and is mapped based on ordinal position (which is equivalent to the mapping using Bulk Insert). When this task is executed no null values are inserted into the SQL table for the "empty values" from the flat file. Empty string values are inserted instead of NULL.
The second package uses the Bulk Insert Task with the "KeepNulls" property for the task (shown in the Properties pane when the task in selected in the Control Flow window) set to "False". When the task is executed NULL values are inserted into the SQL table for the "empty values" from the flat file.
So using the Data Flow Task " " (i.e., blank) is inserted. Using the Bulk Insert Task NULL is inserted (i.e., nothing is inserted, the field is skipped, the value for the record is omitted).
I want to have the exact same behavior on my data in the Bulk Insert Task as I do with the Data Flow Task.
Using the Bulk Insert Task, what must I do to have the Empty String values inserted into the SQL table where there is an "empty value" in the flat file? Why & how does this occur automatically in the Data Flow Task?
From a SQL Profile Trace comparison of the two methods I do not see where the syntax of the insert command nor the statements for the preceeding captured steps has dictated this change in the behavior of the inserted "" value for the recordset. Please help me understand what is going on here and how to accomplish this using the Bulk Insert Task.
View 2 Replies
View Related
Oct 27, 2015
I have two tables one list changes of hospital ward and one lists changes of consultant doctor. These can change independently ie a ward change can occur without a consultant change and vice versa. I want to summarise these changes to give the status at each date_serial value.
create table #temp_ward_stay(date_serial bigint,ward_id varchar(10))
;
insert into #temp_ward_stay
values
(201501021200,'W1'),(201501031201,'W2'),(201501091200,'W3'),(201501161200,'W4'),(201501161201,'W5')(201501271200,'W3'),(201502101200,'W5')
;
create table #temp_consultant_episode(
date_serial bigint,
consultant_id varchar(10))
;
insert into #temp_consultant_episode values
(201501021200,'C1'),(201501031200,'C2'),(201501031201,'C3'),(201501091201,'C4'),
(201501121200,'C3'),(201501301200,'C6'),(201502111200,'C6'),(201502111201,'C7')
;
If I use:
select date_serial,consultant_id,null as ward_id from #temp_consultant_episode
union
select date_serial,null as consultant_id,ward_id from #temp_ward_stay
I get:
date_serial ward_id consultant_id
-------------------- ---------- -------------
201501021200 NULL C1
201501021200 W1 NULL
201501031200 NULL C2
201501031201 NULL C3
201501031201 W2 NULL
201501091200 W3 NULL
201501091201 NULL C4
201501121200 NULL C3
201501161200 W4 NULL
[code].....
whereas what I actually want is:
date_serialward_idconsultant_id
201501021200W1C1
201501031200W1C2
201501031201W2C3
201501091200W3C3
201501091201W3C4
201501121200W3C3
201501161200W4C3
201501161201W5C3
[code].....
I can see how I could remove the nulls where the date_serial values are the same:
select ce.date_serial,ward_id,consultant_id from #temp_consultant_episode ce
join #temp_ward_stay ws
on ce.date_serial=ws.date_serial
but I can't see how to move forward from here.
View 9 Replies
View Related
May 8, 2008
Hello,
I have a simple question. Is it at all possible to replace columns which has nulls with blank spaces for a float data type column.
The columns has null values( written)) in it in some rows and has numbers in other rows . I want to remove nulls before copying it to another file.
Thanks
View 7 Replies
View Related
Aug 31, 2005
Nevil Mascarenhas writes "Hi SQL team,
I am just a beginer in SQL
Here is sample output of a SQL query
Alarm No Site Name Startdate Starttime
7767 ABC 20-08-05 00:00
7765 XYZ 20-08-05 00:00
7762 ASD
5453 QWE 22-08-05 01:00
In this above example, I would like to replace blank fields in Startdate and Starttime coloumn with XXX
Output required is
Alarm No Site Name Startdate Starttime
7767 ABC 20-08-05 00:00
7765 XYZ 20-08-05 00:00
7762 ASD XXX XXX
5453 QWE 22-08-05 01:00
Can you please guide me."
View 1 Replies
View Related
Jan 18, 2006
Quick Question I hope.
Does having a lot of blanks in a column cause errors ?
In 2 or 3 packages where I get 95% of the data in the Error Colomn I can find nothing wrong with the data at all ? it all either seems perfect or there is no data in the column in question usually I would have maybe 800 rows where data would have been inserted but in the other 40, 000 rows the column is blank
Using the "Retain null values from the source as null values in the destination" doesnt seem to make a differance.
This is the error description
The data value cannot be converted for reasons other than sign mismatch or data overflow.
Anyone know of a solution / reason why this keeps happening
Thanks
View 12 Replies
View Related
Mar 5, 2004
I posted this scenario earlier and ndinakar said my code worked on (I guess) his own machine but has not worked on mine! Does that not sound funny? I need to ge through with this simple but frustrating stuff in time. I've implemented more complex snippets before this but I just can't figure the problem out.
Can anyone out there tell me what I could be doing wrongly? I wrote a simple stored procedure called 'proc_insert_webuser' in a database that has been moved to a remote server called LAGOS-NTS3. I executed it from SQL Query Analyser and it worked. When, however, I executed the same stored procedure from ASP.Net it gives an impression that a new record has been inserted and even displays the newly generated ID column to me. On checking the base table I found only blank columns with only the ID column having the generated IDs!
I'm perplexed. I checked the code and can't possibly spot the error. Can anyone help. Found below are snippets used.
CREATE PROCEDURE proc_insert_webuser
(
@applicantidnumeric(18) output,
@firstnamevarchar(18),
@midname varchar(18),
@lastname varchar(50),
@secretcodevarchar(18),
@my_errnumeric(18) output
)
AS
IF (SELECT COUNT(ApplicantID) FROM tbl_webuser WHERE Firstname = @firstname AND Midname = @midname AND Lastname = @lastname) = 0
BEGIN
begin transaction
INSERT INTO tbl_webuser (Firstname, Midname, Lastname, SecretCode)
VALUES (@Firstname, @Midname, @Lastname, @SecretCode)
commit transaction
SET @applicantid = @@IDENTITY
END
ELSE
BEGIN
RAISERROR('Error! Execution aborted. A matching user profile exist.', 16, 1)
SELECT @my_err = @@ERROR
END
RETURN
Then the ASP.Net stuff:
Sub PostData()
'connection string production
Dim cstring As String = "User ID = sa; Password = ; database = E-Recruitment; Server = LAGOS-NTS3; Connect Timeout = 60"
'connection instantiation
Dim cnx As SqlConnection = New SqlConnection(cstring)
Try
'open connection
cnx.Open()
'instantiates and execute a command object
Dim cmd As SqlCommand = New SqlCommand
With cmd
.Connection = cnx
.CommandText = "proc_insert_webuser"
.CommandType = CommandType.StoredProcedure
Dim param1 As New SqlClient.SqlParameter("@applicantid", 0)
param1.DbType = DbType.Single
param1.Direction = ParameterDirection.Output
.Parameters.Add(param1)
Dim param2 As New SqlClient.SqlParameter("@firstname", txtFirstNM.Text)
param2.DbType = DbType.String
param2.Direction = ParameterDirection.Input
.Parameters.Add(param2)
Dim param3 As New SqlClient.SqlParameter("@midname", txtMidNM.Text)
param3.DbType = DbType.String
param3.Direction = ParameterDirection.Input
.Parameters.Add(param3)
Dim param4 As New SqlClient.SqlParameter("@lastname", txtLastNM.Text)
param4.DbType = DbType.String
param4.Direction = ParameterDirection.Input
.Parameters.Add(param4)
Dim param5 As New SqlClient.SqlParameter("@secretcode", txtPWD.Text)
param5.DbType = DbType.String
param5.Direction = ParameterDirection.Input
.Parameters.Add(param5)
Dim param6 As New SqlClient.SqlParameter("@my_err", 0)
param6.DbType = DbType.Single
param6.Direction = ParameterDirection.Output
.Parameters.Add(param6)
.ExecuteNonQuery()
'pick the auto-number
Dim strAppID = .Parameters("@applicantid").Value
'notifies the user of record success
lblMessage.Text = "Profile created. Your ApplicantID is : " & strAppID & " Kindly take note of your UserID. You'll need it for future logins."
End With
Catch ex As Exception
'notifies the user of any error(s)
lblMessage.Text = Err.Description & " originating from " & Err.Source
Finally
cnx.Close()
End Try
End Sub
==================
I'm calling this procedure from the click event of a command button. What could be happening to the code? I have done similar things and got positive results! Any hint from anyone out there?
View 6 Replies
View Related
Mar 10, 2000
A table was created in version 6.5 that has 2 blank spaces and a slash '/' in it's name (i.e. Item Type w/Groups). This was done in error and now the table cannot be dropped or renamed and dropped.
Does anyone know how I can delete this table? Please let me know.
View 1 Replies
View Related
Mar 11, 2008
Hello,
In my .NET.ASP application I generate a random five digit password, like so:
string password = Membership.GeneratePassword(5, 1);
The result could be a password like "0l$IE" and I the save this password to an SQL server. The problem is however that with the generated password five blanks are also added. Instead of "0l$IE" I get "0l$IE ". I hope you understand what I mean? Is there an easy way to correct this?
I appreciate any help!
View 6 Replies
View Related
Mar 9, 2006
When I enter a data in a table, SQL Server automatically completes the data with blanks up to length of column.
this happens in a web form also in Management Studiıo,
Whan am I doing wrong ?
does Collation have anything with it ?
SQL Server 2005 / Developer Edition
View 2 Replies
View Related
Aug 15, 2013
Aim- the following columns “Cancel_Date”, “First_Post_Date”,“Last_Post_Date” can either be populated with figures or have a null value.
I want all Null values to be removed and be replaced with a blank
Final table is
Select *
from #test
left join #SF_AccountBuild on #test.FDMSAccountNo = #SF_AccountBuild.[External FDMSaccountno]
Results
[FDMSAccountNo],
External_ID ,
Parentsfid,
[DBA Name],
[Legal Name],
Street,
[MM3-DBA-ADDR2],
[Code] .....
View 5 Replies
View Related
