Hi,
I worked with informatica before and bit new to SSIS,
i have one simple question..
How can i achieve the same functionality as External Procedure Transformation (Informatica) in SSIS?
like..i want to use a DLL in my package which contains some set of business rules.
Please let me know.
Charles
I have huge data and i am loading data from EXCEL to database table, after loading 80 percent data i am getting some error. My package got failed and it has lots of transformation and took around 6 hours to process completely because of that i don't want it to reload from start. if i run it again it should start from next record from where i got the error.
View 3 Replies View Relatedin terms of functionality, are there major differences between the two?
View 7 Replies View RelatedI am runnning a package which loads 3 million records, if my package fails after loading 1 million records, How do I execute my package in order to load the records from 1 million record? My guess is checkpoints will not help in this case. Do we have any other option to recover the session?
View 5 Replies View Related
Hello,
I am using the lookup transformation. I made a change on reference view, but I can't seem to get the transformation to recognize the fact that the underlying table has changed.
Is this possible? Surely you don't have to redo the entire lookup task in order to caputre a new column that is added onto a table / view.
Chris
Hello All,
I am developing a package using SSIS which needs to do the following.
1. Read all flat file from a folder. I am doing this using For Loop task. I know the total number of files in that folder hence I am setting the loop counter = file count.
2. The next step is to import the data from flat file to SQL server destination table using data flow task.
3. Upon successful completion of data flow task there are some other tasks like SQL to do some checks/validation on the data, export it to another tables.
Upon successful completion of step 3 the iteration goes to next file.
I want to achieve the following
IF step 2 has error (for example corrupt file or incomplete data), I want to fail data transfer completely, skip step 3, and go to step 1 for next available file and do rest.
How do I do this in SSIS?
Thanks for your help.
SGK
I need help rewriting an extended stored procedure as a CLR.
What this extended stored procedure does is to return the domain username of the person connected via named pipes.
This is accomplished by using the srv_impersonate_client and GetUserName functions from opends60.lib. I have tried rewriting this in CLR using Microsoft.SQLServer.Server.SQLContext.WindowsIdentity but have been unable to replicate the functionality to return the same values as the srv_impersonate_client.
If anyone knows how I can rewrite this as CLR, let me know, Also I am looking for where I can get a 64bit version of opends60.lib to run on an amd64
Thanks
Darryl
This is how I used to create DTS in SQL SERVER 2000 and then create a job for scheduling.
' Visual Basic ActiveX Script
'************************************************************************
Function Main()
url="http://www.pageName.com/abc.aspx"
set xmlhttp = CreateObject("MSXML2.XMLHTTP")
xmlhttp.open "post", url, false
xmlhttp.send ""
'msgbox xmlhttp.responseText
Main = DTSTaskExecResult_Success
End Function
Now how will I do the same in the SQl SERVER 2005.
Please help as it is very critical to my work.
Thanks in advance.Manpreet
Can you put is a call withing a Data Flow that will call an External Application and pass a parameter to that application IE say a command line app and then take the output and assign it to back to the flow as a "column" or whatever for that row... IE I want to take a value push it to an external app and then the output from that app I want to insert it into another field for that row in the new table I am moving the data to.
View 4 Replies View Relatedi have a problem connecting to sql server. I want to connect to sql server using informatica repository server.But i am unable to connect because i couldn't find the connectstring for sqlserver from informatica repositoryserver.
View 2 Replies View RelatedHi all,
I am trying to reference the following stored procedure. I was just wondering if it can be done.
I get the following error when i try to save it.
Error: The Column prefix 'CRMBackOffice.dbo' does not match with the table name or alias name used in the query
SELECT BRN_CustCode, BRN_Name1, BRN_Name2, BRN_Status
FROM [VENUS_II].[dbo].[Branches]
WHERE BRN_CustCode NOT IN ([CRMBackOffice].[dbo].[csp_GetCRMIIICustomerList])
Thanks in advance,
Vini
Hi There
I'm having problems installing a external stored procedure that I created in Visual C++. I've created the .ddl and put it in the correct directory.
I've run the following commands to install it
sp_addextendedproc xp_regex, "D:garthExternalStoredProceduresTecasxp_regex.dll"
I then exec the procedure and get the following:
execute master..xp_regex
ODBC: Msg 0, Level 16, State 1
Cannot load the DLL \tedevsql02garthExternalStoredProceduresTecas, or one of the DLLs it references. Reason: 126(The specified module could not be found.).
The .dll is definately in that directory. I'm not sure what I'm doing wrong. Any tips?
Thanks
SynAck
Hi there,
I'm trying to come up with the best way to build some C# Unit tests for an SSIS package I've built.
My C# code does the following
1. Creates a Transaction Scope using System.Transactions
2. Puts some source data for my package into a table that the package will read
3. Kicks off the Package using System.Diagnostics.ProcessStartInfo
Im getting a Transaction TimeOut execption which I beleive is caused because the Package can't read the source data I've inserted becuase the package has not joined the transaction.
So the question is - can you call a run a package and make it participate in a transaction that you have created outside of the package?
Thanks.
Hi, all here,
I am having a question about the language in which external managed code written can interact with SSIS? Is it only applications wrriten in .NET-compliant language can interact with SQL Server Intergration Services packages?
Thanks a lot in advance for any guidance and advices for that.
With best regards,
Yours sincerely,
I have generated a DLL file in VC++ 2005 by a 'C' file. It works fine when I put in a 32bits machine(32bits Windows Server 2003 + 32 bits SQL Server 2005).
However, when I build it into 64 bits, it doesn't work in a 64 bits machine. I have checked by Dependenct Walker, the DLL generated is linked with KERNEL32.DLL / OPENDS60.DLL / MSVCR80D.DLL, all of these DLL files are on the 64 bits machines and linked correctly.
I used the command
sp_addextendedproc 'abc', 'C:Program FilesMicrosoft SQL ServerMSSQL.1MSSQLBinnabc.dll'
to create a ext. stored procedure. When I run it, the error message shows that
Could not load the DLL C:Program FilesMicrosoft SQL ServerMSSQL.1MSSQLBinnabc.dll, or one of the DLLs it references. Reason: 126(error not found).
I would like to ask what is cause of the problem? Do I need to use CLR instead?
Thank you very much!!~
Hi
I have created a .net application using visual studio .net and sql server destop edition on my pc. I have exported the database tables and stored procedures and imported them into a Sql Server database on a web hosting service. The web host does not allow me to access this database directly through visual studio .net.
My connection string to the external database works ok and I can access my stored procedures through my web pages. I know they accept parameters and that I can receive Return Values from them. However, whenever I try to access any of the tables on the external database through a stored procedure, I get a sqlException saying that the table cannot be found (Invalid object name 'UserList').
I have created a text type command which selects data from one of the tables and this runs through without any errors. I have also managed to Insert a row onto one of the external tables also by using a text type command. My only problem seems to be with commands using stored procedures.
Just in case this is the problem - the owner of the table/procedures on my desktop is shown as dbo but on the external database the owner of the tables is shown as [domainname].co.uk_dbuser while the stored procedures owner is still dbo.
Example of stored procedure on external database:
/****** Object: Stored Procedure dbo.AddUser Script Date: 01/03/2005 21:10:06 ******/
CREATE PROCEDURE dbo.AddUser
(
@Username Varchar(20),
@Password Varchar(20)
)
AS
(
Select User_ID From UserList
Where User_Username = @Username
)
GO
Have tried changing dbo.AddUser to [domain].co.uk_dbuser.Adduser but this would not save because there were too many full-stops!
Any help would be greatly appreciated as I am completely stuck.
John
Hi All,
I am trying to create a CLR based stored procedure in C#. When i tried printing simple "Hello" from it, it works fine.
Now requirement is to run an exe file from it. For that i use process.start. But when i try to execute the procedure i get all the security execptions. Can someone please help. Following is the code snippet.
-------------------------------------------------------------------------------------------------------------------------------------------------------------
public partial class StoredProcedures
{
[Microsoft.SqlServer.Server.SqlProcedure]
public static void RunProc(string arg)
{
SqlPipe pipe = SqlContext.Pipe;
pipe.Send("Hello");
Process.Start("E: est.exe");
}
}
CREATE ASSEMBLY [RunProcess]
FROM 'RunProcess.dll'
CREATE PROCEDURE dbo.sqlclr_RunProc
(
@arg nvarchar(1024)
)
AS EXTERNAL NAME [RunProcess].[StoredProcedures].[RunProc]
--------------------------------------------------------------------------------------------------------------------------------------------------------------
Thanks
Sid
I have begun using SSIS and I am a little taken aback by the complexity of it especially since I just want to do a simple data transformation such as in DTS.
Are there any tutorials for data transformation for SSIS on the web/this forum and what if I want to do a simple transformation from Access to SQL Server?
Hey all - got a problem that seems like it would be simple (and probably is : )
I'm importing a csv file into a SQL 2005 table and would like to add 2 columns that exist in the table but not in the csv file. I need these 2 columns to contain the current month and year (columns are named CM and CY respectively). How do I go about adding this data to each row during the transformation? A derived column task? Script task? None of these seem to be able to do this for me.
Here's a portion of the transformation script I was using to accomplish this when we were using SQL 2000 DTS jobs:
'**********************************************************************
' Visual Basic Transformation Script
'************************************************************************
' Copy each source column to the destination column
Function Main()
DTSDestination("CM") = Month(Now)
DTSDestination("CY") = Year(Now)
DTSDestination("Comments") = DTSSource("Col031")
DTSDestination("Manufacturer") = DTSSource("Col030")
DTSDestination("Model") = DTSSource("Col029")
DTSDestination("Last Check-in Date") = DTSSource("Col028")
Main = DTSTransformStat_OK
End Function
***********************************************************
Hopefully this question isnt answered somewhere else, but I did a quick search and came up with nothing. I've actually tried to utilize the script component and the "Row" object, but the only properties I'm given with that are the ones from the source data.
thanks in advance!
"Lord, beer me strength."
hi all,
My issue is the inner join transformation in SSIS.
See i ll explain my problem clearly now.....
Actually i m just checkin if the inner join performed in business intelligence studio usin the inner join transformation and the inner join performed in the management studio using queries are same. Logically both the resultset should match isn't but in my case it is not so. It is very important for me to figure out where the problem is because i m goin to use lotsa inner join transformations in my current project.
I ll appreciate if someone can help me to figure out this problem. May be you can also tell me the detailed steps in adding the inner join transformation and also how it works.
Thank you,
I have the following query:
SELECT EMPID,EMPNAME from EMPLOYEE
where EMPID = (SELECT MAX(EMPID) from EMPLOYEE group by EMPNAME,insert_date)
Here one can use above query in Dataflow of SSIS and specify SQL to create temporary table and later can use as lookup to join to other table.
Is there any way in SSIS to directly do the MAX of EMPID in lookup and join to the main source table.
Any help is really appreciated.
Thanks.
I have a flatfile source to which different flatfiles will be passed as input,this is connected to an OLEDB destination which changes along with the sourcefile.
But when the new file is given as input, the OLEDB mappings are not getting refreshed.It is showing an error.
Actually this was implemented in DTS, and they have used an activex script for the transformation.
what shd I use in SSIS?
Please help me..
Sharmada
Hi all,
I've got to change values in my source database as follows:
Source: Target:
X 1
Y 1
Z 2
Can I create a lookup table and us a look up task in SSIS to do this or do I need to script it?
Thanks
F
I'm not exactly sure how to ask this question ... but here goes!!
I want to get an idea of how SSIS actually executes transformation tasks.
Do transformation tasks (eg a lookup) complie down to managed code or are the executed as SQL commands in a SQL server database?
Thanks.
Hi
I have migrated a DTS that had some activeX transformation tasks within data pump flow tasks.
Those parts were migrated as "DTS 2000 tasks" .. so activeX transformation tasks aren't possible in SSIS ?
I know ActiveX script tasks are but for transformations ?
1. IF i leave these Encapsulated DTS 2000 tasks in the migrated SSIS package, will it run independently of the original
DTS or does it need the old DTS running to "call" that part from ? (I hope im making sense here)
is it possible to load this functionality internally into the new SSIS ?
2.
How could I (if i can't do ActiveX transformation tasks) achieve this is SSIS ?
can I achive this using the script tasks in SSIS ?
Hello All,
Can someone help me out in providing the STEPS to solve this problem. My scneario is, I've a table which has got 2 fields and 5 default row values have been filled in. Now, using the above, duirng package runtime, it need to dynamically create additional field and has to store values like for.e.g (0001 America). I'm getting the following error while executing the ssis package.
1. [DTS.Pipeline] Warning: Component "Derived Column" (1170) has been removed from the Data Flow task because its output is not used and its inputs have no side effects. If the component is required, then the HasSideEffects property on at least one of its inputs should be set to true, or its output should be connected to something.
2. [DTS.Pipeline] Warning: Source "OLE DB Source Output" (87) will not be read because none of its data ever becomes visible outside the Data Flow Task.
Please suggest with your valuable solution at the earliest.
Thanks
Vaiydeyanathan.V.S
I have a Custom .net assembly which retrieves some data, basically just a set of data containing, ID, value,
I need my stored procedure to return this data as a table.
My question what/how is fastest way to do this, my assembly can return anything you want, a datatable, a hashtable, list of(class) etc..
Is there way to rename parameters Param_0, Param_1 in OLEDBCommand transformation? I am trying to create table driven packages using BIML. I am using OLEDBCommand Transformation to update rows. But since, I will not be sure of how many parameters and order of the parameters, I was planning to rename the parameter programmatically, so that accordingly I can build the update statement and add filter condition.
View 1 Replies View Related
HI,
How do I loop through all columns in a row using a script
transformation? For example if I want trim all columns.
If I want to trim one column this is a simple script:
Public Class ScriptMain
Inherits UserComponent
Public Overrides Sub MyAddressInput_ProcessInputRow(ByVal Row As
MyAddressInputBuffer)
Row.City = Trim(Row.City)
End Sub
End Class
But what if I want to do that for all columns? I don't want to name
them all like this:
Public Class ScriptMain
Inherits UserComponent
Public Overrides Sub MyAddressInput_ProcessInputRow(ByVal Row As
MyAddressInputBuffer)
Row.Column1 = Trim(Row.Column1)
Row.Column2 = Trim(Row.Column2)
Row.Column3 = Trim(Row.Column3)
...
...
Row.Column997 = Trim(Row.Column997)
Row.Column998 = Trim(Row.Column998)
Row.Column999 = Trim(Row.Column999)
End Sub
End Class
Is there a simple foreach column in Row.columns option?
-- Joost (Atos Origin)
I just ran across an interesting problem, that makes no sense. I
built
an SSIS package that updates a column, using an transformation
script.
Testing in Debug mode everything runs perfectly, but when I have SQL
sever agent run the package it insert null into the new column.
Any thoughts or suggestions would be greatly appreciated.
r/ Anthony
First let me say, I really can't believe this chain of events myself--and they are happening to me.
I am upgrading several DTS packages to SSIS on what will be my new production server. These packages create tables, export them to a flat file, and ftp them off to other locations.
What is happening (on the SSIS side) is that the OLE DB Source is reordering some of the columns on its own (moving them to the end of the table/file. Then when my pickup/load routines run, the data is out of place and they fail.
Can anyone please explain what is happening here with the mapping. I have evaluated the table and the columns are in the order that I expect. When I preview the source table in the OLE DB Source Editor the columns are in the correct order/alignment, but when them in the OLE DB Source Editor --Columns section within BIDS the order is changed arbitrarily.
I have been somewhat successful (2 out of 3) in being able to re-map the data, but this last table just doesn't want to change.
Thanks in advance for any help and/or information you can provide
Dear all,
I€™ve made a SSIS package which might take source columns from a plain text file and copy them to the Sql table.
A long time ago, when you did the process I did by dts and that stuff included a pump task which had ActiveX Script transform column with VbScript stuff inside so that, how do I for to do the same with SSIS??
I€™ve got a couple of tasks: Flat File Source and OleDb Source Destination but it€™s useless at all for that goal.
Does anyone have ever used or faced this hended?
Thanks for any input,
How would you do the following in SSIS?
SELECT a.TestID,
a.TestCode
FROM TableA a
WHERE UPPER(RTRIM(a.TestCode)) IN SELECT (SELECT UPPER(RTRIM(b.TestCode)) FROM TableB b)
Of course the above query is missing a few things but with ETL the where clause UPPER(RTRIM does not appear to be something that has an object or property that I can use in the Lookup.
Please correct and educate me.