REPLACE In Derived Column Transform Causing Repeat Data In Rows That Should Be Blank
Jul 25, 2006
W2k3 server, SQL 2005.
@@version = Microsoft SQL Server 2005 - 9.00.1399.06 (Intel X86)
Standard Edition on Windows NT 5.2
(Build 3790: Service Pack 1)
I have my first SSIS package almost working, but I'm having an odd problem and can't find any information to help resolve it.
I'm importing from a flat file (csv) to an existing table (append). I've got a Derived Column transformation in the middle to do some data cleanup. It's all working except for one little problem...
One of the transformations is 'REPLACE([Column 3],"^","; ")', output to a new column. (The input file has a field that uses carets as delimiters between an unknown number of items; I'm changing that to semicolons for easier reading.) Not all rows have data in this column, some will have one item, some will have multiple items.
The REPLACE works except that it fills in repeated data for all the blank rows.
Example:
Incoming data is:
1 Smith,Jane^Jones,Jane
2 Brown,John
3
4 Adams,James^Adams,Jim
5
6 White,Debra
Data inserted into the table is:
1 Smith,Jane; Jones,Jane
2 Brown,John
3 Brown,John
4 Adams,James; Adams,Jim
5 Adams,James; Adams,Jim
6 White,Debra
I've tried to use a Conditional to skip the empty rows, but I can't get that working at all (get syntax errors no matter what I put in).
Any suggestions on how to fix this would be most appreciated!
Thank you.
View 5 Replies
ADVERTISEMENT
Jul 30, 2015
I have flat file source from which data is imported to a Sql table.The target column is int and input column is string .The column has some numeric values and some blank values.when I tried to convert into int values it fails.
View 7 Replies
View Related
Mar 7, 2008
i have too many DTS packages to migrate to SSIS, and while examining a DTS package in BIDS (converted with the migration utility) i tried to edit the resulting migrated package, which opened the DTS interface with the two connection icons joined by the big fat arrow with a gear on it...not exactly what i had in mind, iow, it looks like SSIS on the outside, but its still DTS on the inside.
So I stripped out a series of components from a more complex package hoping that simplifying it would reveal the contents of old DTS Transformations tab at least partially set up in a Derived Column transformation.
Can i get there from here, or must i recreate every stinking definition in a derived column manually from the ground up?
thanks very much for your help
View 2 Replies
View Related
Apr 12, 2006
HI, I was wondering if there is a possibility to use a confitional if like this:
IF(ISNULL(mycolumn value, "new value if null", mycolumnvalue)
into a derived column transform to infer a value to a null column value. I do know I can do it using a script component by it would be simpler to do by using an expression.
Thank you,
Ccote
View 3 Replies
View Related
Oct 30, 2007
Hello,
I understand that it is not possible to use PATINDEX in a Derived Column transform. I'm trying to eliminate leading zeros from a column where the string is always 14 characters long, but the first non zero character could occur any number of characters from the left of the string. The following achieves what I need:
substring([Account No w/0s], patindex('%[^0]%', [Account No w/0s]), 14) AS ExtractedAcctNo
Does anyone happen to know how I might express this differently in a Derived Column transform? Should I consider a calling a function through an OLE DB Command transform on each row instead? Maybe I should have a SQL Task that runs an UPDATE statement against the column.
Thank you for your help!
cdun2
View 6 Replies
View Related
Nov 30, 2006
Hi,
Is it possible using derived column transform to change all blank values in a flat file to say a "0"
Basically convert "" to "0"
Thanks for any help,
Slash.
View 3 Replies
View Related
Feb 16, 2007
When data is imported from our legacy system, the same functions need to be applied to several columns on different tables. I want to build a kind of "Function Library", so that the functions I define can be re-used for columns in several packages.
The "Derived Column" transform seems ideal, if only I could add my list of user-defined functions to it. Basically I want to inherit from it, and add my own list of functions for the users to select.
Is this possible ?
What other approaches could I take to building about 30 re-usable functions?
View 7 Replies
View Related
May 15, 2008
I want to replace the contents of a value column with itself but rounded to 2 decimal places.
The current column is a double and I have tried to perform this using the following expression but it fails to work.
Code Snippet
Round(cc_vl,2)
How should I achieve this?
View 7 Replies
View Related
May 14, 2007
How do you replace quotes in an expression
I mean for example if I needed to replace xx in a string with empty string then the following works: REPLACE(SelectedString, "xx","")
But the example I have needs to actually replace quote marks in a string with an empty string and REPLACE(SelectedString, " " ","") doesn't work. I tried guessing a few option like "E or &QTE or something...
Any ideas ?
Thanks
Richard
View 1 Replies
View Related
Mar 22, 2007
Hello All.
Hopefully someone out there will have an idea as this isdriving me nuts.
Ihave some sample problem. I want to use function replace() on Derived Column.
For example.
when strDate = 2007/03/22
I used ==> replace(strDate, "/", "") ==> 20060322
But If strTest = "123.10" ====>> 123.10
How can i do to replace ( " )double qoute ?
by function replace()
what is a statement for replace (") in Derived Column ?
please tell me for this event.
any suggesstion appreciated
Thank you very much.
Chonnathan
View 11 Replies
View Related
Apr 30, 2015
I have a query that gives me a result with a column value for example 4.
I now want to repeat this row 4 times with a new column that calculated from 1 - 4.
Or when column value is 3 I want to repeat row 3 times with new column name 1-3
View 7 Replies
View Related
Apr 26, 2006
I'm importing a csv-file delimited with semicolons. Firstly I LTRIM the columns "in place" and the data imports fine. All the numbers in right columns in the target table. Then I add another Derived Colum Transformation to replace decimal character comma (,) to a dot (.) in order to convert the string/varchar value to numeric. But here I run into trouble. Running the task ends in success but the result in the target table (same as above) is not. All the commas are now dots as expected but what is worse is that SSIS have added values in cells that should not be there. I get values in cells that shoud be empty!
Shortly: Only LTRIM([Column1]) as expression and "Derived Column" as Replace 'Column1' works OK.
But adding REPLACE-expression (i.e REPLACE(LTRIM([Column1]) , "," , ".") to this breaks things up
I'm aware that I could do this with SQL but this is not the point...
Any ideas?
BR Jompe
View 6 Replies
View Related
Nov 7, 2006
Ok.. so I have a fixed position data feed. I read the file in as just whole rows initially, process a specific position and evaluate a conditional split to determine direction of the file for proper processing (file contains multiple recors with different layouts). This all works fine. I then use the derived column feature to process all the columns.
Most of the columns are as simple as SUBSTRING(RecordData,1,10) for example to get the first column. This all works for string data. I then have a column that is a date field. The problem occurs that the code SUBSTRING(RecordData,20,10) returns either a date or empty set of data if no date was in the original file. When this gets sent to the OLEDB connection (SQL Server 2005) into the date field it fails. If the record has a date it works, but if it is empty it fails the insert.
I tried to replace empty strings with NULLs with this code. REPLACE(TRIM(SUBSTRING(RecordData,20,10)),"",NULL(DT_WSTR,10)). This does not work. So my question is how do I bring a date field from a fixed flat file into a SQL datetime field using a derived column? More specifically is how do I set it to NULL if its empty data? When I use the above code it inserts all the rows from the file, but it sets all rows to NULL not just the empty ones.
Thanks.
View 6 Replies
View Related
Feb 6, 2008
Hi guys,
I am looking for some syntax to help me with a traditional search/replace style requirement. I am wanting to examine the contents of one column and populate another.
similar to this SQL case statement
CASE ProductLine
WHEN 'R1' THEN 'Road'
WHEN 'M1' THEN 'Mountain'
WHEN 'T1' THEN 'Touring'
WHEN 'S2' THEN 'Other sale items'
ELSE 'Not for sale'
END,
the twist is that R1, M1 etc. can appear anywhere in the ProductLine column.
thanks for any assistance you can provide.
regards,
Chris
View 1 Replies
View Related
May 8, 2008
Hello,
I have a simple question. Is it at all possible to replace columns which has nulls with blank spaces for a float data type column.
The columns has null values( written)) in it in some rows and has numbers in other rows . I want to remove nulls before copying it to another file.
Thanks
View 7 Replies
View Related
Jun 28, 2006
Hi,
I have a common requirement in numerous SSIS processes to take my main input data set and to remove all rows from it that match a second input data set on a given key and output this as the main output. I also want to output (as a second output) all the rows from the main input data set that did match on the given key. However, I don't want to merge in data from the second input, nor am I interested in rows from the second input data set that have no match in the main input.
E.g. If I have the following data:
Main input:
Key Name
--- ----
1 Steve
2 Jamie
3 Donald
Second Input
Key DontCareAboutThisField1
--- -----------------------
1 ...
3 ...
4 ...
Then I would like the following output:
Main Output
Key Name
--- ----
2 Jamie
Second Output
Key Name
--- ----
1 Steve
3 Donald
Can I do this with a standard transform, or will I have to write my own? Any help on this would be greatly appreciated!
Thanks in advance,
Lawrie
View 1 Replies
View Related
Feb 13, 2008
I have a table. In that table I have a list by student number that lists the entry dates into a particular grade. When trying to list only the first time entered, there is no unique way to identify one row from another other than the date. Is there a way to use max or min to only pull one date per student number? I have done a series of case when statements and I am able to get it down to 1 to 2 entries per student number, but I need to get it down to only 1 date per student number.
Thank you for your help
SELECT DISTINCT mx.stu_num,
CASE WHEN er.STU_NUM = ep.STU_NUM
THEN er.enterdater
ELSE CASE WHEN ea.STU_NUM = ep.STU_NUM
THEN ea.enterdatea
ELSE CASE WHEN eb.STU_NUM = ep.STU_NUM
THEN ep.enterdatep
ELSE eb.enterdateb
END
END
END AS entrydate
FROM dbo.mx AS mx LEFT OUTER JOIN
dbo.v_EntryDate9_R AS er ON mx.stu_num = er.stu_num LEFT OUTER JOIN
dbo.v_EntryDate9_P AS ep ON mx.stu_num = ep.stu_num LEFT OUTER JOIN
dbo.v_EntryDate9_A AS ea ON mx.stu_num = ea.stu_num LEFT OUTER JOIN
dbo.v_EntryDate9_B AS eb ON mx.stu_num = eb.stu_num
View 1 Replies
View Related
Oct 5, 2015
I need to convert a a string column to integer. Before converting, I need to check if it has blank values then convert it to NULL. Someone told me that its easier to convert it to NULL before converting to integer.
View 5 Replies
View Related
Jul 22, 2015
how to declare multiple derived columns in SSIS Derived Column Task in one attempt.as i have around 150 columns coming from Flat file. I had created the required Expression in Excel and now i want add those in derived column task but its allowing only 1 expression at a time.
View 4 Replies
View Related
Jun 4, 2015
there is a way to write an if statement for a derived column to count rows?
for example:
Row 35: PRI 7010
Row 100: PRI 7011
formula that when it gets to row 100. it will go back and look at 35 and use that data if the 7010 = 7010, if not use 7011,,,
View 2 Replies
View Related
Sep 21, 2007
Hello all,
I'm running into an issue with a report we have, I was hoping you may be able to help...
We have a report that contains a rectangle which has conditional visiblity. When the expression evaluates to true, the contents of the rectangle are indeed "hidden", but we are seeing a blank page.
The rectangle does NOT have page breaks selected before or after it. (and neither does the objects before or after that rectangle.)
Any ideas?
View 1 Replies
View Related
Mar 14, 2006
I am having problems with a lookup transformation. I have a row in my lookup table for blank ('') source data. If I test the join using SQL the match is made, but the Lookup transform doesn't consider it a match and sends it to error output. Is there a property that I don't have set correctly or something else I am forgetting?
View 5 Replies
View Related
Jan 8, 2014
I have data like this in the table :
IntRecpieID strName intMealtypeID Total
100 ‘A’ 1 20
101 'B' 2 30
100 'A' 3 40
Desired Output required:
IntRecpieID StrName 1 2 3
100 'A' 20 Null 40
101 'B' Null 30 Null
View 5 Replies
View Related
Jan 5, 2006
I need to know what a table's max row Identity is part way thru a data flow. I can't get it at the beginning of the data flow. I need to either (1) add it to the data buffer part way thru or (2) set it into a package variable and then reference the var in a script component.
I've not found a way to add a database column to the data buffer without doing a lookup for each row (too slow and not appropriate here) or some goofy oledb source and then merge join into the data buffer on a contrived join.
I've read questions about referencing package vars in scripts but I can't get that to work. DTS.Variables("varname").Value isn't recognised when I code it up.
Anyone have an idea or solution for either one of these? If you're gonna explain the script code, please include the entire snipet including the INCLUDEs, etc.
View 8 Replies
View Related
Oct 3, 2006
i have a field with blank spaces.
i wanna replace the spaces with just one spaces. ihave 500 fields in 500 tables.
any input will be appreacited.
i have something like this but its not working.
declare @field varchar(50)
declare @minVoter int
declare @maxVoter int
declare @tableName varchar(20)
set @tablename = '00001'
-- select ad_str1 from [00170]
select @minVoter = min(id_voter),
@maxVoter = max(id_voter) from quotename(@tablename)
while (@minVoter <= @maxVoter)
begin
select @field = ad_str1
from quotename(@tablename)
where id_voter = @MinVoter
update [00170]
set ad_str1 = replace(@field, ' ', ' ')
select @minVoter = min(id_voter)
from quotename(@tablename)
where id_voter > @minvoter
end
View 13 Replies
View Related
Feb 27, 2008
I am adding a new column with the derrived column data flow Transformation and am having a problem with it coming through as a deciaml format.
The destination column is set as "numeric(18, 9)" but no matter what i change the data type to in the transfomation editor it will only bring through the calculation as a whole number (IE it rounds the number up) The destination table shows the zero's after the "."
What do i need to do to get this coming through as a number with decimals
Heres the expression i am using
(PROG_CREDITS / (PROG_DURATION / 12)) / 120
Cheers
View 3 Replies
View Related
Apr 17, 2007
I have a numeric column with the following sample values in a source flat file:
240
6
48
310
55
I would like to dump them in a table (destination) as string with the length only 3 and in the following format "xxx" .
Data in the destination column will look like this after the transformation:
240
006
048
310
055
Thanks for your help!
View 3 Replies
View Related
Aug 14, 2014
I'm moving data from one database to another (INSERT INTO ... SELECT ... FROM ....) and am encountering this error:
Msg 8114, Level 16, State 5, Line 6
Error converting data type varchar to numeric.
My problem is that Line 6 is:
set @brn_pk = '0D4BDE66347C440F'
so that is obviously not the problem and my query has almost 200 columns. I can go through one by one and compare what column is int in my destination table and what is varchar in my source tables, but that could take quite a while. How I can work out what column is causing the problem?
View 3 Replies
View Related
Jul 31, 2007
Hi all of you,
I think that I've done a big mess on my work... I've got plain file which must be loaded into a sql table. Up to there no problem, I use Derived Column due to columns needed be transformed with NULL, RIGHT, LEN, and so on...
But in this last package I've done half of work using Data Conversion but I've got five columns which would need be transformed but I don't know how can I do such thing. I can't connect a Derived Column from Flat File Source task, of course, it's already data conversion...
Let me know if you need further details.
Previously I think, silly idea, that it could be unified or better, that Data Conversion task allows me make transformations...
Thanks a lot for your ideas and thoughs,
View 3 Replies
View Related
Sep 4, 2015
I need to create a function that replaces the data in a column with an 'X' based on the LEN of the data in the column. I created one that does a replacement, but it fills the column based on the max data length, and not the current length of the string or integer. An example of what I'm trying to accomplish.
Original data in a varchar(30) column:
thisisavalue
thisisanothervalue
thisisanothervalueagain
shortval
replaced with
xxxxxxxxxx
xxxxxxxxxxxxxxxx
xxxxxxxxxxxxxxxxxxxx
xxxxxxx
My current function is replacing the data like this:
xxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
xxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
xxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
xxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
View 4 Replies
View Related
Aug 1, 2006
Hi all--I've got a derived column transformation where I am adding a field called Import_Date. I'm telling it to add as a new column and use the function "GetDate()" to populate the field. When I run the package, it returns NULL as the data value for all rows. Any idea why this might be happening?
View 5 Replies
View Related
Jun 4, 2015
I have one column CandidatePlaced (Data type Boolean).
Using data conversion i changed data type to DT_WSTR and then i used derived column to manipulate the data. Ex. 1 = "Yes" and 0 = "No"
[Candidate Placed ?] == "1" ? "Yes" : "No"
But at end of result i got all the columns as No. Some should be Yes.
View 7 Replies
View Related
Feb 29, 2008
Hi, I was wondering if it is posible as in derived columns to replace an input column with the output one in Data Conversion?
What I can see that when a data conversion is done, only we can do it in a new column and not in the same one we are using as input.
Do you know if this it is posible?}
Thanks for your help!
Beli
View 3 Replies
View Related