Is there documentation somewhere about multiple execution paths in SSIS control flow? I didn't find documentation anywhere. I have a situation where I have two tasks that take considerable time, but could be executed in parallel (to speed up things) and I was wondering whether SSIS supports parallelism.
To illustrate the issues in simultaneous execution, I created a test SSIS package. In the package, I have five tasks, let's call them T1, T2, T3, T4 and T5. The taks are connected with "green arrows" like this:
T1->T2
T1->T3
T2->T4
T3->T4
T5 is not connected. The tasks can be e.g. Send Mail tasks, that's not relevant to this issue. I put a breakpoint in each task and execute the package.
When I execute package, T1 and T5 become active, i.e. the arrow that displays where the package execution currently is, is in two tasks simultaneously. Now F10 (step over) doesn't seem to work "Unable to step. Not implemented". If I press F5 nothing happens. After I press F5 for a second time tasks T1 and T5 and executed. Why don't they execute with the first pressing of F5? I would additionally like to know whether these two tasks are executed in parallel or sequentially, i.e. in the same thread or in two threads? Is there documentation of this?
The execution stops at T2&T3. Again, pressing F5 doesn't do anything, but the second time I press F5 T2 and T3 are executed.
I'm trying to conditionally execute a dataflow based on the presence of a data file. If the data file isn't present, I'd like to execute gracefully without error.
Logic is as follows:
If FileExists Then execute dataflow Else exit w/o error End If
I've got the code ready to go, but I'm not sure how to do this conditional branch logic. Right now, the code calls the Dts.Results.Success / Failure. The problem, however, is Failure is exactly that... which doesn't result in the graceful exit I'm looking for.
I am having a hard time with what appears to be something simple. I want to import an excel spreadsheet into a table on a daily basis from a command line. I created a package from the Import Wizzard in the SQL Management Studio and saved it. Since I want a clean table each day, my process needs to be create a temp table, import from the Excel file into the temp table. If that is successful, delete the original table and rename the temp table the original name. The point of this process is to provide for a fail-safe if there is some unforseen problem downloading the data on a particular day.
When I run the package, the first thing it does is delete the original table. I know this because the process shows the time that it finished is before anything else has started or finished. The time shown for the completion of the data flow task is about 2 minutes after that time.
This is maddening!!! The one thing I do not want to happen I can not seem to prevent. I have my control flow set on success. Why does it do this?
This is a question to the SSIS development team. I would like to know what are the requirements to implement custom SSIS Control Flow task in C++ . There is a documentation describing the process when implementing a managed task, but no such documentation exists for implementing a task in C++.
I want to get the start time of data load and end time after data load and store it in a table which has mapping_id , mapping_name,start_time,end_time.
i use ActiveXScript task to get the start time before data load and store the mapping_id in a global variable,then data flow transformation occurs.
i want to use a global variable to store the mapping id ,so that i can update the end time after data load with that variable.how to do this?
is there any other way, i can get the start and end time of data load (other than the logging information)?
Hi All, Just needed an insight into the IF-ELSE construct w.r.t its implementation in SSIS or a similar methodology that could be adopted in SSIS while executing a Package.
Scenario : I want to Start with importing data from Different sources to SQL Server Destination. For Which, i define 3 different Data Flow Tasks each involved in importing data from an external source to SQL Server Destination.
1] Text File Inbound Task : Source - Flat File Source ; Destination - SQL Server Dest. 2] Excel Inbound Task : Source - Excel Data Source ; Destination - SQL Server Dest. 3] Xml Inbound Task : Source - XML Data Source ; Destination - SQL Server Dest
Finally i want to execute the package with an IF-ELSE Scenario which will Check for the external Source being :
I have a package that loads staging tables from an Oracle source DB. In the data flow tab I have 30+ read table/write table task combinations. When I run the package 3-4 of the read/write combos execute at a time. What I'm trying to control is the priority order of the combo execution. My goal is to minimize to total load time by having the larger table transfers run first and the smaller table transfers fill in until they are all complete. Currently, the largest table (16 million) transfers last (because it was the last combo that I created?).
Has anyone come up/determined a generic way to capture and log indicative information within a data flow in SSIS - e.g., a number of rows selected from the source, transformed, rejected, loaded, various timestamps around these events, etc.? I am trying to avoid having to build a custom solution for each of the packages that I will have (of which there will be dozens). Ideally, I'd like to have some sort of a generic component (such as a custom transformation) that will hide the implementation details and provide a generic interface to the package.
It is not too difficult to achieve something similar on the control flow level, but once you get into data flows things get complicated.
I'm building a package wherein I perform a SQL task(A) if the error log is not empty. This same SQL task(A) is also being used by another data flow task(B). The precedence points from B to A bottom to top. When I execute, all the tasks in the downward direction (precedence pointing downward/sideways) execute but this one doesnt as it points updwards. I can copy and paste task A and make B point to A downward, but I dont want to duplicate A in the same package.
Is there any other approach?
If you dont understand the above, see the flow:
X (SQL task) ----on success-------------> A (SQL task) | ^ | | ...(sequence of steps) | | | | | B(Data Flow Task) ---------failure --------|
Execution flow doesnt move from B to A, even though its a failure condition. Hope this explains the problem.
Why isn't there some documentation on how to do this. This should be really simple and it has taken me 2 weeks and I still haven't gotten an answer. Please Help Does anyone know the answner or some place where there is some documentation!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
I get the following error when I try to substitute the strings in the databasedetails collection with variables: Error: Object reference not set to an instance of an object. StackTrace: at Microsoft.SqlServer.Dts.Tasks.TransferObjectsTask.TransferObjectsTask.CheckLocalandDestinationStatus(Database srcDatabase, DatabaseInfo dbDetail) at Microsoft.SqlServer.Dts.Tasks.TransferObjectsTask.TransferObjectsTask.TransferDatabasesUsingSpAttachDetach()
I created the following variables: strDestinationDB = AirCL2Exp_new3 strDestinationDBPath = C:Program FilesMicrosoft SQL ServerMSSQL.2MSSQLDATAAirCL2Exp_new3_Data.mdf strDestinationLGPath = C:Program FilesMicrosoft SQL ServerMSSQL.2MSSQLDATAAirCL2Exp_new3_Data.ldf strSourceDB = AirCL2Exp strSourceDBPath = C:Program FilesMicrosoft SQL ServerMSSQL.2MSSQLDataDataNewAirCL2Exp_Data.mdf strSourceLGPath = C:Program FilesMicrosoft SQL ServerMSSQL.2MSSQLDataDataNewAirCL2Exp_Log.ldf
I then assigned those variable to DatabaseDetails Collection:
Inaddtion I also assigned the following to the two DatabaseFiles Collection: for 0: DatabaseFileSize = 0 DestinationFilePath = @strDestinationDBPath FileType = DatabaseFile SourceFilePath = @strSourceDBPath SourceSharePath = @strSourceDBPath
I recently upgraded to on 2012 SP1 CU5 and have found the SSDT gui for SSIS to be almost unusable. I can't drag or resize items. Any time i try they either automagically shrink to the tiniest possible size, shoot off to some extreme or just shake uncontrollably I didn't have these problems on previous versions (dont remember what It was).
I need to pass a parameter from control flow to data flow. The data flow will use this parameter to get data from a Oracle source.
I have an Execute SQL task in control flow to assign value to the Parameter, next step is a data flow which will need take a parameter in the SQL statement to query the Oracle source,
The SQL Looks like this:
select * from ccst_acctsys_account
where to_char(LAST_MODIFIED_DATE, 'YYYYMMDD') >?
THe problem is the OLE DB source Edit doesn€™t have anything for mapping parameter.
I have an Execute SQL Task that returns a Full Rowset from a SQL Server table and assigns it to a variable objRecs. I connect that to a foreach container with an ADO enumerator using objRecs variable and Rows in first table mode. I defined variables and mapped them to the columns.
I tested this by placing a Script task inside the foreach container and displaying the variables in a messagebox.
Now, for each row, I want to write a record to an MS Access table and then update a column back in the original SQL Server table where I retreived data in the Execute SQL task (i have the primary key). If I drop a Data Flow Task inside my foreach container, how do I pass the variables as input to an OLE DB Destination on the Data Flow?
Also, how would I update the original source table where source.id = objRects.id?
Thank you for your assistance. I have spent the day trying to figure this out (and thought it would be simple), but I am just not getting SSIS. Sorry if this has been covered.
Dear All! My package has a Data Flow Task. In Data Flow Task, I use a Script Component and a OLE BD Destination to transform data from txt file to database. Within Data Flow Task, I want to call File System Task to move file to a folder or any Task of "Control Flow" Tab. So, Does SSIS support this task? Please show me if any Thanks
I'm currently setting variables at the package level with an ExecuteSQL task. This works fine. However, I'm now starting to think about restartability midway through a package. It would be nice to have the variable(s) needed in a data flow set within the data flow so that I only have to restart that task.
Is there a way to do that using an SQL statement as the source of the value in a data flow?
OR, when using checkpoints will it save variable settings so that they are available when the package is restarted? This would make my issue a moot point.
Hi all! I recently started working with SSIS and one of the things that is puzzling me the most is what's the best way to go:
A small control flow, with large data flow tasks A control flow with more, but smaller, data flow tasksAny help will be greatly appreciated. Thanks, Ricardo
I have multiple data source text files with different names , file extensions and format , i need to bring the data into SQL tables for temporary holding of data. Once i bring the data into table i need to identify some fields using substring and then after cleansing at table level command. Once thats done i need to convert those tables back into text file with comma delimited join those tables and bring a different output.
Problem i am facing is if i have one data flow task then the first source file bring the data into table which is destination , but how to convert the data back using those tables back in to text file.
basically when i create one source file which is text file becomes source of data flow task and when convert those files into tables becomes destination. but when i need to bring the data back into a different format , the tables which were destination needs to become source.
Do i need to create a multiple data flow task or is their any other way i could work out.
Hi there.I've been searching for this error specifically but I haven't found anything yet.I have these two tables (USERS and REQUESTS):USERS ( [LOGIN] [varchar] (10) NOT NULL , [NAME] [varchar] (20) NOT NULL)where LOGIN is the primary key.The problem comes when I try to create the "REQUESTS" table.In these requests there's one user who types the request. After one or two days, there's other user who aproves the request. The problem is that I need two foreign keys referencing the table "USERS".CREATE TABLE REQUESTS ([ID] [numeric](5, 0) NOT NULL ,[DATE] [datetime] NOT NULL ,[NOTES] [varchar] (100) NOT NULL ,[TYPED_BY] [varchar] (10) NOT NULL ,[APROVED BY] [varchar] (10) NULL) ON [PRIMARY]GOALTER TABLE REQUESTS ADD CONSTRAINT [PK__REQUESTS__07DE5BCC] PRIMARY KEY ( [ID]) ON [PRIMARY] GOALTER TABLE REQUESTS ADD CONSTRAINT [FK__REQUESTS__TYP__15702E88] FOREIGN KEY ([TYPED_BY]) REFERENCES [USERS] ([LOGIN]) ON UPDATE CASCADE ,CONSTRAINT [FK__REQUESTS__APR__12742E08] FOREIGN KEY ([APROVED_BY]) REFERENCES [USERS] ([LOGIN]) ON UPDATE CASCADEAnd SQL returns:Introducing FOREIGN KEY constraint 'FK__REQUESTS__APR__12742E08' on table 'REQUESTS' may cause cycles or multiple cascade paths. Specify ON DELETE NO ACTION or ON UPDATE NO ACTION, or modify other FOREIGN KEY constraints.Could not create constraint. See previous errors.Ok, after that, I tried creating a new table to store aprovals (Table with two fields: "REQUEST_ID" and "APROVED_BY").So, I removed "APROVED_BY" field from "REQUESTS" and its FK constraint.The same error comes up.I don't think this structure goes into "cycles" or "multiple cascades".How can I do this?Thanks in advanceRegardsRoland
Hello,There are three tables:OS-GroupOFCompanies (Table1)GoC_GroupOFCompaniesID (PK)OS-Organization (Table 2)Org_OrganizationID (PK)OS-UnitAddress (Table 3)Unit_UnitAddress (PK)Scenario:(1)GoC_GroupOfCompanies has -one to many- relationship with Org_OrganizationID.(2)GoC_GroupOfCompanies has -one to many- relationship with Unit_UnitAddress.(3)Org_OrganizationID has -one to many- relationship with Unit_UnitAddress.Following Error message appeared after trying to save the relationship (3) described above.'OS-Unit-UnitAddress' table saved successfully'OS-Organization' table- Unable to create relationship 'FK_OS-Organization_OS-Unit-UnitAddress'.ODBC error: [Microsoft][ODBC SQL Server Driver][SQL Server]Introducing FOREIGN KEY constraint 'FK_OS-Organization_OS-Unit-UnitAddress' on table 'OS-Organization' may cause cycles or multiple cascade paths. Specify ON DELETE NO ACTION or ON UPDATE NO ACTION, or modify other FOREIGN KEY constraints.[Microsoft][ODBC SQL Server Driver][SQL Server]Could not create constraint. See previous errors.NB: ON UPDATE is much needed.How to manage the situation?Please guide.ThanksSuryaPrakash****************************************** This message was posted via http://www.sqlmonster.com** Report spam or abuse by clicking the following URL:* http://www.sqlmonster.com/Uwe/Abuse...f935efcabb55ee9*****************************************
I have two tables called a and b. a has one row called aId where aId is the PK. b has bId, aId_1 and aId_2 where bId is the PK and aId_1 and aId_2 both allow nulls.
If I have a relationship between aId and aId_1, and another relationship between aId and aId_2, where I set the delete rule for both to SET_NULL, then I get the error -
"may cause cycles or multiple cascade paths"
But its completely reasonable that I might wish to do this.
The funny thing is that under Visual Studio, I can create a Data Set with these tables and using the designer, I can set the both foreign key relationships to Delete Rule Set Null and everything works as expected.
So, I'm unsure now if I just need to set the Delete rules using the Dataset designer and not bother with them in the database itself.
I am new to SSIS can anyone tell me the diff between control flow and dataflow. if all the transformation are done using dataflow than why do we use control flow. Sorry if I am asking you very basic question.
Is it possible to setup a Control Flow at the solution level rather than at the package level? I'd like to setup a Control Flow that truncates multiple tables in a staging database then runs multiple packages that reloads those tables. Each package has a Control Flow tab that seems to be specific to the package. Is it possible to set something up that governs the execution of multiple packages?
Essentially I have an incoming file, each line in the file is a record. The records share the same initial key fields for the first 10 columns, then the field structure varies depending on a rectype and sequence number.
My initial plan was load the keys into fields, and load the remaining data into a long varchar field.
Then the stored procedure would evaluate the Rectype and Seqno of each record and chop up the Varchar accordingly.
So I set up a cursor to read the temporary table, do a fetch into variables, and go to evaluate the variables.
I want to be able to use a CASE statement to evaluate the fields and then perform various logic, but it's giving me fits because it seems like CASE only really works in Select statements, and won't really allow you to do any sort of GOTO logic.
I chopped the following SQL up and put in a rough cut of what I thought I was doing.
FETCH NEXT FROM Transaction_Cursor into @eepssn,@rectype,@seqno,@data
WHILE @@FETCH_STATUS = 0 BEGIN select @companycode = cmpcompanycode, @eeccoid = eeccoid, @eeceeid = eeceeid, @eecempno = eecempno from company,empcomp where eeceeid = (select eepeeid from emppers where eepssn = @eepssn ) and eecemplstatus = 'A' and cmpcoid = eeccoid
CASE @RECTYPE+@SEQNO when '001001' then goto parse_pretax when '002001' then goto parse_LOAN else select @RECTYPE+@SEQNO+' not recognized!' end process_it: insert into foo2 (empno,companycode,amt,pct) values (@eecempno,@companycode,@eeamt,@eepct) FETCH NEXT FROM Transaction_Cursor into @eepssn,@rectype,@seqno,@data END
CLOSE Transaction_Cursor DEALLOCATE Transaction_Cursor
goto bypass
parse_pretax: let @eepct = substring(@data,1,5)
goto process_it
parse_loan: let @eeamt = substring(@data,27,11) goto process_it
bypass:
I could sketch it out a little bit better in Northwind or Pubs, but I think I just need a smack upside the head and a little edification.
Hi, I would like to have a decision maker component in Control Flow to decide which Data Flow to run. For example: if myVar=A then run data flowA if myVar=B then run data flowB ......
We have Conditional Split component available in data flow but not in Control Flow. What component in control flow can be used to do the same job that conditional flow is doing in data flow? Or what's a work around for this.
I have a problem when using nested loops in my Control Flow. The package contains an outer Foreach Loop using the Foreach File Enumerator which in my test case will loop over two files found in a directory. Inside this loop is another Foreach Loop using the Foreach Nodelist Enumerator. Before entering the inner loop a variable, xpath, is set to a value that depends on the current file, i e /file[name = '@CurrentFileName']/content. The Nodelist Enumerator is set to use this variable as its OuterXPATHString. Now, this is what happens:
First Iteration: The first file is found and the value of xpath = /file[name = 'test1.txt']/content. When the inner loop is entered it iterates over the content elements under the file with name test1.txt as expected.
Second Iteration: The second file is found and the value of xpath = /file[name = 'test2.txt']/content. When the inner loop is entered it unexpectedly still iterates over the content elements under the file with name test1.txt.
My question is: Should it not be possible to change the loop condition of an inner loop in an outer loop such that the next time it is entered it will be done based on the new condition? It seems that the xpath variable is read once, the first time, and never again. If that is the case, does anyone know of a workaround?
I recently installed sql server 2005 with integration services. While trying to drag control flow items in control flow pane, I am having below error...
The task with the name 'Execute sql task' and creattion name "Microsoft.sqlserver.dts.tasks.executesqltask, Microsoft.sqlserver.SqlTask,Version=9.0.242.0,culture=neutral,public token=7895cd6666345' is not registered for use on this computer.
Contact Information:
Execute SQL Task
Can anybody please help me why this and how we solve this problem...please
Hello, My package "splits" into a number of work flows by the use of precedent management.
After a number of steps I want to combine the flows back to one and finish the package.
What is the mechanism to combine the flows into a singular, linear process again so that I do not have to multiply define identical tasks to finish each flow leg?
I have run into a problem while creating a simple UDF on SQL Server 2000.
Code Snippet
CREATE FUNCTION [dbo].[GetSectionNum] (@section varchar(4)) RETURNS varchar(2) AS BEGIN DECLARE @sTemp varchar(2),@s char DECLARE @count int DECLARE @length int
set @length = LEN(@section) set @count = 1
WHILE (@count <= @length) BEGIN SET @s = SUBSTRING(@section,@count,1) IF(ISNUMERIC(@s)) BEGIN SET @sTemp = @sTemp + @s END SET @count = @count + 1 END
IF(LEN(@sTemp) = 1) BEGIN SET @sTemp = '0' + @sTemp END
RETURN @sTemp ENDWhen I perform a syntax check I receive and error about "Error 156: incorrect syntax near keyword 'BEGIN'. I have narrowed the problem to the IF statement inside the While block. If I remove the IF statement the syntax check is successful. This is the first UDF I have written so I'm swimming in uncharted water. Thanks ahead of time for your help.
I am using the event handling mechanism to do my custom logging. This works fine. Using the OnPreExecute and OnPostExecute my log tables fill up with the start- and enddates of all the containers and tasks in the complete package hierarchy.
However, what I am missing (e.g. in a system variable) is the source ID of the parent container that started the task or container. In other words, in my logging reports, I would like to build up a tree starting with the topmost package, like the progress indication in the IDE.
The built-in Logging features do not log this information either as far as I can tell.
