Hi SSIS Gurus,
Coild anybody halp me, how to achive search and replace in the Expressions (Transformation Editor) or in SQL Query box?
If this functionality isn't presented, what is preffered woraround to achieve it?
I have a lot of transformations (about 100) in the derived column task and its too difficult to find transformation what i need.
Greetings, I am attempting to create a flat file delimited by |. I am using (ISNULL(LIN1_OPT_ADDR) ? "" : LIN1_OPT_ADDR + "| ") to replace the blank address column with the pipe delimiter. So that a row that would consist of:
Customer Number,Name,Address Line1,City,State
12345,ACE HARDWARE INC. ,801 Rockefeller St.,New York, New York
56789,BUILDING SUPPLY INC., ,Wichita, Kansas
Should end up as:
12345|ACE HARDWARE INC.|801 Rockefeller St.|NEW YORK|NEW YORK
56789|BUILDING SUPPLY INC.||Wichita|Kansas
When I run the data flow to create the flat file the file contains the following:
12345|ACE HARDWARE INC.|801 Rockefeller St.|NEW YORK|NEW YORK
56789|BUILDING SUPPLY INC.| | |Wichita|Kansas
Can anyone tell me what I am doing wrong?
Thanks.
Howdy!
I am reading in a deliminated file. In the Script Transformation Editor, if the UPC does not past the checksum test, I want to throw the row out right then. I am not sure how to do that...but it is probably really simple.]
Thanks,
Linda
Here is my script:
' Microsoft SQL Server Integration Services user script component
' This is your new script component in Microsoft Visual Basic .NET
' ScriptMain is the entrypoint class for script components
'Option Strict Off
Imports System
Imports System.Data
Imports System.Math
Imports Microsoft.SqlServer.Dts.Pipeline.Wrapper
Imports Microsoft.SqlServer.Dts.Runtime.Wrapper
Public Class ScriptMain
Inherits UserComponent
Private Function DoubleTest(ByVal Value As String) As Boolean
Dim d As Double
If Not Double.TryParse(Value, d) Then
'Windows.Forms.MessageBox.Show(Value + " is not numeric")
Return False
End If
If Double.IsNaN(d) Then
'Windows.Forms.MessageBox.Show(Value + " is NaN")
Return False
End If
'Windows.Forms.MessageBox.Show(Value + " = " + d.ToString())
Return True
End Function
Public Overrides Sub Input0_ProcessInputRow(ByVal Row As Input0Buffer)
Dim d As Double
Dim CheckDigit As Integer = 0
Dim CheckOdd As Integer
Dim CheckEven As Integer
' Copy each source column to the destination column
Row.WholesalerCode = Trim(Row.WholesalerCode)
If Row.UPCNumber.Length = 12 Then
' 12 Digit Checksum
CheckOdd = Convert.ToInt16(Row.UPCNumber.Substring(0, 1), 10) + Convert.ToInt16(Row.UPCNumber.Substring(2, 1), 10) + Convert.ToInt16(Row.UPCNumber.Substring(4, 1), 10) + Convert.ToInt16(Row.UPCNumber.Substring(6, 1), 10) + Convert.ToInt16(Row.UPCNumber.Substring(8, 1), 10) + Convert.ToInt16(Row.UPCNumber.Substring(10, 1), 10) * 3
CheckEven = Convert.ToInt16(Row.UPCNumber.Substring(1, 1), 10) + Convert.ToInt16(Row.UPCNumber.Substring(3, 1), 10) + Convert.ToInt16(Row.UPCNumber.Substring(5, 1), 10) + Convert.ToInt16(Row.UPCNumber.Substring(7, 1), 10) + Convert.ToInt16(Row.UPCNumber.Substring(9, 1), 10)
If ((CheckOdd + CheckEven) + 1.0) / 10.0 = Round(((CheckOdd + CheckEven) + 1.0) / 10.0, 0) Then CheckDigit = 1
If ((CheckOdd + CheckEven) + 2.0) / 10.0 = Round(((CheckOdd + CheckEven) + 2.0) / 10.0, 0) Then CheckDigit = 2
If ((CheckOdd + CheckEven) + 3.0) / 10.0 = Round(((CheckOdd + CheckEven) + 3.0) / 10.0, 0) Then CheckDigit = 3
If ((CheckOdd + CheckEven) + 4.0) / 10.0 = Round(((CheckOdd + CheckEven) + 4.0) / 10.0, 0) Then CheckDigit = 4
If ((CheckOdd + CheckEven) + 5.0) / 10.0 = Round(((CheckOdd + CheckEven) + 5.0) / 10.0, 0) Then CheckDigit = 5
If ((CheckOdd + CheckEven) + 6.0) / 10.0 = Round(((CheckOdd + CheckEven) + 6.0) / 10.0, 0) Then CheckDigit = 6
If ((CheckOdd + CheckEven) + 7.0) / 10.0 = Round(((CheckOdd + CheckEven) + 7.0) / 10.0, 0) Then CheckDigit = 7
If ((CheckOdd + CheckEven) + 8.0) / 10.0 = Round(((CheckOdd + CheckEven) + 8.0) / 10.0, 0) Then CheckDigit = 8
If ((CheckOdd + CheckEven) + 9.0) / 10.0 = Round(((CheckOdd + CheckEven) + 9.0) / 10.0, 0) Then CheckDigit = 9
If CheckDigit = Convert.ToInt16(Row.UPCNumber.Substring(11, 1), 10) Then
Row.UPCNumber = String.Concat("00", Row.UPCNumber)
Else
'Throw out row because checksum did not match. <=== what do i do here???????????????
End If
ElseIf Row.UPCNumber.Length = 14 Then
' 14 Digit Checksum
Else
' Throw out row because checksum did not match. <=== what do i do here???????????????
End If
If Not DoubleTest(Row.RetailPrice) Then
Row.RetailPrice_IsNull = True
'Row.RetailPrice = String.Empty
End If
End Sub
End Class
Help...
I'm having trouble coming up with a valid expression in my derived column transformation editor that tests the input column for NULL and responds something like this:
if[message] isNull then "NA" else [message]
where [message] is the input column.
Thanks!
Can I add Output Columns to the Script Transformation Editor using code? I have to execute a SQL Statement to determine the number of years we have the data for for an item and then create the columns for the months in those years and populate them with the quantities. So my question is can I create output columns to the Script Transformation Editor on the fly that is as it is being executed?
Any input will be good.
Thanks,
MShah
Help...
I'm having trouble coming up with a valid expression in my derived column transformation editor that tests the input column for NULL and responds something like this:
if[message] isNull then "NA" else [message]
where [message] is the input column.
Thanks!
Hi,
In a nut shell I want to be able to instruction some Data Analysts on how to modify SSIS packages using the simpliest solutions possible. This is because there are many different data sources and some of these data sources have a huge number of fields, and yes you guessed it these data sources are subject to change on a regular basis.
A very common task they will need to do is to modify an SSIS package to do a to transform of a source date string format of "YYYYMMDD" into a date data type field within a table.
Similar threads have advised the use of the Data Flow Transformations->Derived Column for this sort of thing.
So within the Expression Text box I have inserted the following SSIS compatible SQL to convert the above string into a british format date data type; -
Code Snippet
(SUBSTRING(DOB_SRC,8,2) + "/" + SUBSTRING(DOB_SRC,5,2) + "/" + SUBSTRING(DOB_SRC,1,4))
But really what I want to be able to do is to instruct the Data Analysts to do is something like; -
ConvertTextToDate(DOB_SRC)
Where I previously defined that behaviour of ConvertTextToDate as a public VB.NET function.
Can someone please help. I'm pretty certain I'm not the only one with this type of requirement.
Thanks in advance,
Kieran.
I am trying to put the following as an expression in the SSIS Derived Column Transformation Editor.
DATEADD(DAY, DATEDIFF(DAY, 0, GETDATE()), 0)
It is not allowing it. This works fine in a regular SQL statement.
Does anyone know how I can get this to work?
Dear friends, can any one tell me how to assign null to the expression value in derived column transfromation editor?
thanks,
From SQL Server 2014, using SQL Server Data Tools for Visual Studio - BI, I'm trying to edit a Script Component within an SSIS Data Flow Task. The 'Edit Script...' button is enabled and turns a nice shade of blue when moused over, but a click has no effect. Perhaps I'm missing a component of VSTA? Everything else seems to work correctly. What might I be missing?
View 2 Replies View RelatedHi
I'm a relative SQL Server newbee and have developed a function that converts mm/dd/yyyy to yyy/mm/dd for use as in a DT_DBDATE format for insert into a column with smalldatatime.
I receive the following erros when using the function in the Derived Column Transformation Editor. First, the function, then the error when using it as the expression Derived Column Transformation Editor.
Can anyone explain how I can do this transformation work in this context or suggest a way either do the transformation easier or avoid it altogerher?
Thanks for the look see...
******************************
ALTER FUNCTION [dbo].[convdate]
(
@indate nvarchar(10)
)
RETURNS nvarchar(10)
AS
BEGIN
-- Declare the return variable here
DECLARE @outdate nvarchar(10)
set @outdate =
substring(@indate,patindex('%[1,2][0-9][0-9][0-9]%',@indate),4)+'/'+
substring(@indate,patindex('%[-,1][0-9][/]%',@indate),2)+'/'+
substring(@indate,patindex('%[2,3][0,1,8,9][/]%',@indate),2)
RETURN @outdate
END
********************************
And the error...
expression "lipper.dbo.convdate(eomdate)" failed. The token "." at line number "1", character number "11" was not recognized. The expression cannot be parsed because it contains invalid elements at the location specified.
Error at Data Flow Task [Derived Column [111]]: Cannot parse the expression "lipper.dbo.convdate(eomdate)". The expression was not valid, or there is an out-of-memory error.
Error at Data Flow Task [Derived Column [111]]: The expression "lipper.dbo.convdate(eomdate)" on "input column "eomdate" (165)" is not valid.
Error at Data Flow Task [Derived Column [111]]: Failed to set property "Expression" on "input column "eomdate" (165)".
(Microsoft Visual Studio)
===================================
Exception from HRESULT: 0xC0204006 (Microsoft.SqlServer.DTSPipelineWrap)
------------------------------
Program Location:
at Microsoft.SqlServer.Dts.Pipeline.Wrapper.CManagedComponentWrapperClass.SetInputColumnProperty(Int32 lInputID, Int32 lInputColumnID, String PropertyName, Object vValue)
at Microsoft.DataTransformationServices.Design.DtsDerivedColumnComponentUI.SaveColumns(ColumnInfo[] colNames, String[] inputColumnNames, String[] expressions, String[] dataTypes, String[] lengths, String[] precisions, String[] scales, String[] codePages)
at Microsoft.DataTransformationServices.Design.DtsDerivedColumnFrameForm.SaveAll()
I'm importing a csv-file delimited with semicolons. Firstly I LTRIM the columns "in place" and the data imports fine. All the numbers in right columns in the target table. Then I add another Derived Colum Transformation to replace decimal character comma (,) to a dot (.) in order to convert the string/varchar value to numeric. But here I run into trouble. Running the task ends in success but the result in the target table (same as above) is not. All the commas are now dots as expected but what is worse is that SSIS have added values in cells that should not be there. I get values in cells that shoud be empty!
Shortly: Only LTRIM([Column1]) as expression and "Derived Column" as Replace 'Column1' works OK.
But adding REPLACE-expression (i.e REPLACE(LTRIM([Column1]) , "," , ".") to this breaks things up
I'm aware that I could do this with SQL but this is not the point...
Any ideas?
BR Jompe
i like to write a code that street_name field look up in table one for 'road' and replace it for 'rd'.
any approaches ?
table1
abr description
---- ------------
ln lane
rd road
ave avenue
table2
street_name
-------------
apple road
i have data in a table which is in format #12345;#22211;#12112; and so on... its a long string.
i want to replace only the numbers using nchar() function.
in above example i want to replace 12345, 22211, 12112 with nchar(12345), nchar(22211), nchar(12112).
i have managed to do this using loop and substring function. but now i have performance issue.
63K records will take many hours to update. i have to do following steps:
1. read record from table.
2. replace numbers with nchar() function.
3. update the same table with new text.
i have to do this using a stored procedure.
i am looking for some quick way to do this. any idea?
I have a notes table in which I have 10,000 rows. Unfortunately some of the notes are compressed together as one long string. I can fix the issue if I can do a search and replace on ';' and replace with '; ' is there a way to do this?
View 3 Replies View Related
Hi,
I'm trying to do a search over a table, the problem is that the default collate applied by this transofrmation is case sensitive... i need a non case sensitive collate. If i edit the SQL Statement to set another Case sensitive collate manually inside the statement, the problem is that if a field (wich is involved in search transformation ) has a null value, the transformation doesnt find any record (it doesnt happens if a dont introduce manually the collate in the statement, null values dont rise any error, and the transofrmation works fine).
Sorry for my english lol.
I have a question about the best method of going about doing this.
I have a records table which stores visitor information from IISlogs (dbo.records)
I have added IP tables to try and resolve where the visitors are from. They are labeled 'ip4_##' where ## is the first section of the ip.. so if my IP was 24.150.66.80 I would need to look in table ip4_24 and inside there look for 150 in column B and then 66 in column C. From that it will say which country is attached, that way I can update the countrycode in the dbo.records.
I am not sure how to do this search, so any ideas would be great.
Thanks in advance
* Table structre for the ip_## tables are
COLUMN 'b' int (represents the 2nd section of an IP)
COLUMN 'c' int (represents the third section of an IP)
COLUMN 'country' int
I'm looking for a stored procedure (or query) to search an entiredatabase for a specific string value and replace it with another. I'msure I saw an SP for this a while back by someone, but cannot find itagain. The SP took the search string and replace string as parametersand did the rest. Any ideas where I can find this ?Bear in mind, the idea is that this can be re-used and run on anydatabase, so it would have to find all tables and search through those.TaRyan
View 2 Replies View RelatedI am developing a complex database-driven application with SQL Server 2000. My database has dozens of stored procedures, and whenever I want to rename a database field, I have to go through my stored procedures, finding where that field is used. This is a laborious and error-prone process, even when I look up depenencies.
Is there an easier way to work stored procedure code – some tool to search/replace the text perhaps?
I found a search-and-replace program for SQL server that works GREAT, but I have a list of about 200 words to search for, with corresponding replacements. Rather than editing the code below for *each* word and running it 200 separate times, I'd like to iterate through the list, but my MS-SQL programming skills are...light (to say the least). Anyone have any ideas how I can create a list or hash-type variable and use the code to loop through and do the replacements all at once?
/*
*
* Search & Replace
*
* Use Ctrl+Shift+M to replace template values
*
*/
set xact_abort on
begin tran
declare @otxt varchar(1000)
set @otxt = '<string1, text, text to be replaced>'
declare @ntxt varchar(1000)
set @ntxt = '<string2, text, replacing text>'
declare @txtlen int
set @txtlen = len(@otxt)
declare @ptr binary(16)
declare @pos int
declare @id int
declare curs cursor local fast_forward
for
select
id,
textptr(<field_name, sysname, target text field>),
charindex(@otxt, <field_name, sysname, target text field>)-1
from
<table_name, sysname, target table>
where
<field_name, sysname, target text field>
like
'%' + @otxt +'%'
open curs
fetch next from curs into @id, @ptr, @pos
while @@fetch_status = 0
begin
print 'Text found in row id=' + cast(@id as varchar) + ' at pos=' + cast(@pos as varchar)
updatetext <table_name, sysname, target table> .<field_name, sysname, target text field> @ptr @pos @txtlen @ntxt
fetch next from curs into @id, @ptr, @pos
end
close curs
deallocate curs
commit tran
Hello Everyone,
Is there a way to search all the tables in a database for a value that is found in a specific column? Another problem is that although this specific column will be found in most of the tables in the database, it has a different two or three digit prefix in the column name for each table.
I think the logic will go something like this...
As the script jumps from table to table, it should first look for a column whose name ends with ***ITM. If it does not find a column with ***ITM it should move on to the next table. If it does, then search the ***ITM column for my value. If it does not find my value, then move to the next table. But if it does, change my value to a new value.
Any help will be greatly appreciated.
Thank you all...
Hi All,
I am experiencing problem to select text wich has carriage return in my search functionality.
I have two tables called @searchwordTable and @DataTable.
@searchWordTable will have search criteria words(data type is varchar) and @DataTable will titles(data type is ntext) need to be searched. Some of titles have carriage return,line feed and tab characters. I am preseting here script to reproduce my problem.
DECLARE @searchwordTable TABLE
(
searchword VARCHAR(MAX)
)
INSERT INTO @searchwordTable (searchword) VALUES('carriage long description')
DECLARE @DataTable TABLE
(
title ntext
)
INSERT INTO @DataTable (title) values('carriage long description'+char(13)+char(10)+'carriagelong')
SELECT * FROM @DataTable dTable,@searchwordTable srcWrdTable
WHERE '% ' + REPLACE(CAST(dTable.title AS VARCHAR(MAX)),char(13),'') + ' %' like '% ' + srcWrdTable.searchword + ' %'
I am expecting the above select statement should select title from @DataTable but not getting .I am not understanding what is going wrong with above select.
Thanks in Advance.
Hey guys,
In my DB i have hundreds on queries setup for all different reporting purposes..
We have just changed they way our system handles costing and are moving from a average cost function to a standard cost function.. This in turns, means that i need to replace any reports where i use the average cost field and replace it with the standard cost field.
Is it possible, that i can do a full search and replace over all my queries, looking for a specific field name and replacing it with something else.
I am dreading the idea of opening each individual queries and checking if it exists..
If anyone knows of software or someway to write a SP to accomadate this, please let me know. I would be most grateful.
Thanks
Scotty
Hi all,
Does anyone know how to do the search and replace in multiple stored procedures using SQL 2005 Management Studio?
Any help will be appreciated.
Thanks!
I am attempting to find quotes (") in a column and replace with the string '--THIS-WAS-QUOTES--'. Right now my script only converts the first quote it finds in the description column, converts to the string and moves to the next row leaving the other quotes as they were. Below is my query script
DECLARE @find varchar(8000),
@replace varchar(8000),
@patfind varchar(8000)
SELECT @find = '"',
@replace = '--THIS-WAS-QUOTES--'
SELECT @patfind = '%' + @find + '%'
UPDATE Incident
SET description = STUFF(convert( varchar(8000), description ),
PATINDEX( @patfind, description ),
DATALENGTH( @find ),
@replace )
WHERE description LIKE @patfind
I am attempting to find quotes (") in a column and replace with the string '--THIS-WAS-QUOTES--'. Right now my script only converts the first quote it finds in the description column, converts to the string and moves to the next row leaving the other quotes as they were. Below is my query script
DECLARE @find varchar(8000),
@replace varchar(8000),
@patfind varchar(8000)
SELECT @find = '"',
@replace = '--THIS-WAS-QUOTES--'
SELECT @patfind = '%' + @find + '%'
UPDATE Incident
SET description = STUFF(convert( varchar(8000), description ),
PATINDEX( @patfind, description ),
DATALENGTH( @find ),
@replace )
WHERE description LIKE @patfind
I've got an nvarchar(max) column that I need to transform with some simple text processing: insert some markup at the very beginning, and insert some markup just before a particular regular expression is matched (or at the end, if no match is found).
Since the SSIS expression language doesn't support anything like this, is a Script Component the only way to go? Does Visual Basic .NET provide regular expression matching?
Thanks!
#1 This stored procedure can be used to search and replace substring in the char, nchar, varchar and nvarchar columns in all tables in the current database. You should pass the text value to search and the text value to replace. So, to replace all char, nchar, varchar and nvarchar columns which contain the substring 'John' with the substring 'Bill', you can use the following (in comparison with the SetTbColValues stored procedure, this stored procedure replace only substring, not the entire column's value):
EXEC replace_substring @search_value = 'John', @replace_value = 'Bill'
Hi guys,
I am looking for some syntax to help me with a traditional search/replace style requirement. I am wanting to examine the contents of one column and populate another.
similar to this SQL case statement
CASE ProductLine
WHEN 'R1' THEN 'Road'
WHEN 'M1' THEN 'Mountain'
WHEN 'T1' THEN 'Touring'
WHEN 'S2' THEN 'Other sale items'
ELSE 'Not for sale'
END,
the twist is that R1, M1 etc. can appear anywhere in the ProductLine column.
thanks for any assistance you can provide.
regards,
Chris
Hi,
If you have two synchronous transformation components and the input of the second is connected to the output of the first, does the first transformation process (loop through) all rows in the buffer before outputting these rows to the second transformation? Or does the first transformation output each individual row to the second transormation as soon as it has finished processing it?
Thanks in advance,
Lawrie.
tell me the difference between Audit transformation and rowcount transformation.
Because audit and rowcount transformation will provide the environment variables.
Only difference i am finding is rowcount returns the count of rows its updating .
Apart from these is there any other difference?
Tell me the scenario where i need to use the audit transformation.
I'm just wonder if this is a bug in MS Search or am I doing something wrong.
I have a query below
declare @search_clause varchar(255)
set @Search_Clause = ' "hepatitis b" and "hepatocellular carcinoma"'
select * from results
where contains(finding,@search_clause)
I don't get the correct result at all.
If I change my search_clause to "hepatitis" and "hepatocellular carcinoma -- without the "b"
then i get the correct result.
It seems MS Search doesn't like the phrase contain one letter or some sort or is it a know bug?
Anyone know?
Thanks
Hi - I'm short of SQL experience and hacking my way through creating a simple search feature for a personal project. I would be very grateful if anyone could help me out with writing a stored procedure. Problem: I have two tables with three columns indexed for full-text search. So far I have been able to successfully execute the following query returning matching row ids: dbo.Search_Articles @searchText varchar(150) AS SELECT ArticleID FROM articles WHERE CONTAINS(Description, @searchText) OR CONTAINS(Title, @searchText) UNION SELECT ArticleID FROM article_pages WHERE CONTAINS(Text, @searchText); RETURN This returns the ArticleID for any articles or article_pages records where there is a text match. I ultimately need the stored procedure to return all columns from the articles table for matches and not just the StoryID. Seems like maybe I should try using some kind of JOIN on the result of the UNION above and the articles table? But I have so far been unable to figure out how to do this as I can't seem to declare a name for the result table of the UNION above. Perhaps there is another more eloquent solution? Thanks! Peter
View 3 Replies View Related