I am using SSIS with Transaction, and I met a lot of questions as below:
1.IF DTC doesn't support ADO.NET Connection?
Please check the ADO.net Connection's property, it has a property named like: DTCTrancactionSupport. But it's always set to False and unable to edit.
IF the answer is NO, HOW could I bound all my Tasks into one Transaction? You know, I have some Tasks with SQL connection, and I have also some Script Tasks, witch need the ADO.net connection.
2.IF DTC required to run in both the Destination Server and the Source Server?
I have a Dataflow Task, and take some data from a table in a Source Server to a Destination Server. The DTC doesn't run on the Source Server. I found the Dataflow Task hangs when I use the SSIS Transaction(just set the Task's transactionOpion to Required, Only one Task in the Package).
3.IF DTC doesn't support RetainSameConnection?
Many people say that. Really?
4.When a DTC transaction is running, if the table in it could not be modified by process outside of the transaction? For example, a package with a DTC transaction, and there is a SQL Task inside, when it is running, could I modified the same table's data manually(in the Sql Sever Management Studio)?
I have a design a SSIS Package for ETL Process. In my package i have to read the data from the tables and then insert into the another table of same structure.
for reading the data i have write the Dynamic TSQL based on some condition and based on that it is using 25 different function to populate the data into different 25 column. Tsql returning correct data and is working fine in Enterprise manager. But in my SSIS package it show me time out ERROR.
I have increase and decrease the time to catch the error but it is still there i have tried to set 0 for commandout Properties.
if i'm using the 0 for commandtime out then i'm getting the Distributed transaction completed. Either enlist this session in a new transaction or the NULL transaction.
and
Failed to open a fastload rowset for "[dbo].[P@@#$%$%%%]". Check that the object exists in the database.
1) Can one find the location of the 'Transaction Log' at: <HardDisk>Program FilesMicrosoft SQL ServerMSSQLDataMyDb_Log.ldf?2) Is it safe to delete it, as SQL will create a new Transaction Log when itrealises none already exist?3) When trying to do 2) I get error message that file is in use, even thoughSQL Server is closed. Any suggestions?4) Can anyone shed light on the error message 'Write on ... <my pathhere>... failed, status = 112. BACKUP LOG is terminating abnormally'? I getthis when trying to manually backup the Transaction Log 'All Tasks - Backup'Thanks for any replies to my ignorance.
I'm receiving the below error when trying to implement Execute SQL Task.
"The ROLLBACK TRANSACTION request has no corresponding BEGIN TRANSACTION." This error also happens on COMMIT as well and there is a preceding Execute SQL Task with BEGIN TRANSACTION tranname WITH MARK 'tran'
I know I can change the transaction option property from "supported" to "required" however I want to mark the transaction. I was copying the way Import/Export Wizard does it however I'm unable to figure out why it works and why mine doesn't work.
How do I make use of begin transaction and commit transaction in SSIS.
As am not able to commit changes due to certain update commands I want to explicitly write begin and commit statements. but when i make use of begin and commit in OLEDB commnad stage it throws an error as follows:
Hresult:0x80004005
descriptionyntax error or access violation.
its definately not an syntax error as i executed it in sql server. also when i use it in execute sql task out side the dataflow container it doesnt throw any error but still this task doesnt serve my purpose of saving/ commiting update chanages in the database.
I am having some problem with SSIS transaction. Eventhought I tried to imitate the concept that Jamie presented at http://www.sqlservercentral.com/columnists/jthomson/transactionsinsqlserver2005integrationservices.asp
. My workflow is as followed
********************************* For Each ADO.Record in Oracle (transaction=not supported)
If (Certain_Field_Value = 'A')
Lookup Data in SQL DB with values from Oracle (transaction=not supported)
DO Sequence A (Start a Transaction , transaction=required)
INSERT/UPDATE some records in SQLDB(transaction=supported) Finish Sequence A ( transaction should stop here) UPDATE Oracle DB ( Execute SQLTask, transaction=not supported) If (Certain_Field_Value = 'B')
Lookup Data in SQL DB with values from Oracle (transaction=not supported)
DO Sequence B (Start a Transaction , transaction = required)
INSERT/UPDATE some records in SQLDB (transaction=supported) Finish Sequence A ( transaction should stop here) UPDATE Oracle DB ( Execute SQLTask, transaction=not supported) If (Certain_Field_Value = 'C')
------------ ------------ End ForEach Loop ************************************* My requirements are that I want separate transaction for each Sequence A, B, C, etc... If Sequence A transaction fails, the other should still be continuing with another transaction. But I am getting an error regarding the OLEDB Error in next Task (e.g in Certain_Field_Value = 'B') "Lookup Data in SQL DB with values from Oracle ", the error message is ".......Distributed transaction completed. Either enlist this session in a new transaction or the NULL transaction. ". What is it that I am doing wrong? Regards KyawAM
I am using SAS Dataset and loading in to SQL server 2005.Now i want to delete columns which have 0 values and null values.Can anyone guide me how to do this?thank you.
I was wondering if anyone know how to do Dynamic Properties Tasks in SSIS? And, does anyone know how to create a global database in SSIS so I won't have to change all of the database names in my package? I'm sorry, I'm new to SSIS and I'm just really frustrated. Thank you.
I realize that I am confused about SSIS and security.
In BIDS, I work on and modify my packages. That part I understand. Then, I want to build my project, then deploy to SQL Server 2005. I know how to do that too (for the most part, please see below):
My confusion arises around the "Protection Level" options in the package properties. Right now I have everything set to "EncryptSensitiveWithUserKey". My understanding is that I need to change this in order to run my packages from SQL Server jobs, because only the creator of the package can currently run the job.
So my question is, since I want to deploy to SQL Server, don't I want to change the "Protection Level" to "ServerStorage"?
However, it will not let me change the protection level to ServerStorage. It says "The protection level, ServerStorage, cannot be used when saving to this destination"... presumably because it is attempting to save to the file system.
However, even when I built out my packages and saved to SQL Server, I could not change the protection level either.
Which is why I am really confused... Once you save your packages to SQL Server, how do you make changes to the packages, so that the changes are reflected in the packages stored on SQL Server?
There is some concept I am not understanding here.
Please write YES/NO against point number in your answer i.e. for example:
1) YES, 2) YES, 3) NO, 4) YES etc.
We have source SQL Server 200 databases in one instance and target SQL Server 2005 in another instance both being on different windows 2000 server systems.
Target OLAP DB design is derived from source OLTP.
Most master & transaction tables are as is but some target tables are a result of merging source tables.
We need to do SSIS packages based migration using data flow task based on column mapping & queries using Business Intelligence Development Studio (BIDS). There are no transformations required.
1) Can I group packages under project into sub-groups i.e. super packages?
2) Can I execute a set of packages together which are logically/physically grouped?
3) Can we execute set of ordered packages/individual packages from command line and even stored procedure?
4) Can we maintain transaction integrity across the entire set of ordered packages we execute?
5) Can we send a mail to mail id(s) in case of events like OnError etc.
6) Can we schedule the packages?
7) Does SSIS provide via BIDS a way to compare data in source with target to decide what got inserted/deleted/updated and based on that do the necessary?
8) Can SSIS packages be developed including code for data encryption?
9) Can we pass parameters to packages that get called from stored procedure?
10) Can we call stored procedure/function in SSIS package?
I have been working with SSIS and find myself stopped on what seem like very basic tasks.
Am I right in thinking that, if you want to import a flat file into a SQL datatable, in most PC based systems, that you have to manually cast EVERY SINGLE COLUMN to unicode string, if you don't want to get the 'cannot convert between unicode and non-unicode string data types'? I mean, this is surely a VERY COMMON task, taking a CR/LF delimited ASCII file and dumping it into a database tables? What if you have 50 fields to import?
Am I the only one who finds it very difficult to cast a string from a textfile to a date format? At this point, I always just dump the date to a 8 character string and save it that way, and use a SQL view to cast it to a date. At least SQL Server can do 'basic date conversions'. Is there any way to freeze execution and find out what data was provided to a derived column, if you're trying to do some substring operations and hodge together a date? I couldn't find any. I usually put a extra column in a datatable and plunk some data there to find out 'what it looked like', how else can you debug an SSIS program?
Is there a planned service pack or update to SSIS that will make this more usable? Does Microsoft ever intend to implement an UPSERT type module? Or a File Move operation that handles directories that fill up with log files, each of which have to be added to a datatable and then moved to an archive directory (surely, another very common task - I figured out how to accomplish this but it is far from intuitive, the way it currently works). Sorry for grumbling, but honestly, I find turnaround time to develop a very simple import program is RIDICULOUS. It's quicker to write a COBOL program to issue a bunch of formatted SQL UPDATE statements than using this application.
Hello all, this is my first post, I have a couple of basic questions regarding the use of SSIS. I have used SSIS for a couple of years for on off table loads but never implemented production jobs with it so please bear with these questions.
1. When making scripts that connect to one or more databases, is there a way to store the userid and password somewhere besides embedded within the package, such as a web.config file, this would make routine password changes much more manageble.
I guess it was only one question but any help would be greatly appreciated.
The following is a list of questions that I have not been able to obtain concrete answers. I am probably missing something: 1) ReadWriteVariables -- can the updated value for a ReadWriteVariable be accessed within the same data flow? It appears not as I think the PostExecute() fires at the completion of the data flow not the end of the Script Component. Secondarily, the Script Component is a non-blocking transformation so the component does not "see" the end of the pipeline prior to sending data down stream.
2) Record Count -- Because of #1 above, How could you calculate a record count for a data stream? It does not appear that one can calculate the number of records for a data stream within a data flow and then access the count from within the same data flow.
3) FinishOutputs() -- Is the concept of FinishOutputs() applicable to Script Component Destinations? Asked another way, is FinishOutputs() executed at the end of the data stream regardless of whether there are "real" outputs for the component? I can create a "Dummy" output to create FinishOutputs() but is this ok?
4) Script Component -- It appears that the Script Component Source, Transformation or Destination are really defined based on the columns defined in "Inputs and Outputs". Can you convert an Source script component to a transformation script component by simply adding an Output?
Sorry for these basic questions but I am not getting it completely. As you can tell...
Hi, I am a newbie in SSIS. I'd like to hear your recommendation to build the whole SSIS package as one transaction. In other words, how we can rollback all DML operations (insert/update/delete) that are applied in different parts of a package if for any reason the package failed to complete?. Something like rolling back a proc if the proc failed.
Hi all, I'm having an issue with transactions in SSIS.
I have a Sequence Container, that contains 5 Tasks.
1) Data Flow Task using an OLEDB Connection to access an Oracle RDB database via a linked server. then insert into SQL Server 2005.
2) Execute Sql Task on Sql Server 2005 (UPDATE Statement)
3) Execute Sql Task on Sql Server 2005 (DELETE Statement)
4) Data Flow Task using an OLEDB Connection to insert data into a SQL Server 2005 DB
5) Execute Sql Task performing an UPDATE statement on an Oracle RDB database using an OLEDB Connection to to access the database via a linked server
If the Sequence container and all contained tasks are set to Transaction: Supported (No transaction will be created I believe), then the package runs successfully.
If I change the Sequence container to Transaction: Required, and leave all contained tasks as Transaction: Supported, then Task 1 fails with the following error:
[SELECT From RDB Change Table [1]] Error: SSIS Error Code DTS_E_CANNOTACQUIRECONNECTIONFROMCONNECTIONMANAGER. The AcquireConnection method call to the connection manager "SSISPOC" failed with error code 0xC0202009. There may be error messages posted before this with more information on why the AcquireConnection method call failed.
Any Ideas would be much appreciated, as I'm stumped!
very simple task... I want to copy some data from a DB2 table to SQL Server. So I want to delete the records in the SQL Server table and copy all data from DB2 to the SQL Server. That's easy...
The only thing I want to make sure is that in case of an error I still have a valid dataset in the table. So my idea was to use a transaction covering the delete and the data flow.
So I moved the two tasks in a container, switched transactions to "required"...
What I see is that now a distributed transaction is started for DB2 and the SQL Server. But that's not what I want (and it doesn't work also...). I don't need the transaction on the DB2 side, I just want it on the SQL Server to cover the delete and the insert.
I'm getting this when executing the code below. Going from W2K/SQL2k SP4 to XP/SQL2k SP4 over a dial-up link.
If I take away the begin tran and commit it works, but of course, if one statement fails I want a rollback. I'm executing this from a Delphi app, but I get the same from Qry Analyser.
I've tried both with and without the Set XACT . . ., and also tried with Set Implicit_Transactions off.
set XACT_ABORT ON Begin distributed Tran update OPENDATASOURCE('SQLOLEDB','Data Source=10.10.10.171;User ID=*****;Password=****').TRANSFERSTN.TSADMIN.TRANSACTIONMAIN set REPFLAG = 0 where REPFLAG = 1 update TSADMIN.TRANSACTIONMAIN set REPFLAG = 0 where REPFLAG = 1 and DONE = 1 update OPENDATASOURCE('SQLOLEDB','Data Source=10.10.10.171;User ID=*****;Password=****').TRANSFERSTN.TSADMIN.WBENTRY set REPFLAG = 0 where REPFLAG = 1 update TSADMIN.WBENTRY set REPFLAG = 0 where REPFLAG = 1 update OPENDATASOURCE('SQLOLEDB','Data Source=10.10.10.171;User ID=*****;Password=****').TRANSFERSTN.TSADMIN.FIXED set REPFLAG = 0 where REPFLAG = 1 update TSADMIN.FIXED set REPFLAG = 0 where REPFLAG = 1 update OPENDATASOURCE('SQLOLEDB','Data Source=10.10.10.171;User ID=*****;Password=****').TRANSFERSTN.TSADMIN.ALTCHARGE set REPFLAG = 0 where REPFLAG = 1 update TSADMIN.ALTCHARGE set REPFLAG = 0 where REPFLAG = 1 update OPENDATASOURCE('SQLOLEDB','Data Source=10.10.10.171;User ID=*****;Password=****').TRANSFERSTN.TSADMIN.TSAUDIT set REPFLAG = 0 where REPFLAG = 1 update TSADMIN.TSAUDIT set REPFLAG = 0 where REPFLAG = 1 COMMIT TRAN
It's got me stumped, so any ideas gratefully received.Thx
I am running a package in which all tasks participate in a transaction. One task is a ForEach Loop container that executes 1 SQL INSERT statement per loop. The Loop runs well over 100 times in order to normalize a very, very, very wide table.
It seems that at a certain point, the Loop task will fail. I've tested a few different scenarios and they all fail at the same point... the 98th loop.
The error message is below:
[Execute SQL Task] Error: Failed to acquire connection "ADO.NET Connection". Connection may not be configured correctly or you may not have the right permissions on this connection.
I'm guessing that there is a limit to the number of connections that can be started at any one time. But I figured SSIS wouldn't start a new connection for every SQL command.
Any insight on this issue? Anyone have an idea for a work around?
I have a situation where i need to insert or update the data from a flat file to a sql server database. The flat file contains nearly one lakh records.
I am using transactions. If all the rows are inserted or updated successfully i am commiting. If there is any error i am rollbacking the transaction.
when rollbacking the transaction it is taking more than 3 to 4 hours.
What I have set out to do is to try and truncate the transaction logs on my server instances after a nightly backup. I went ahead and used the shrink database option in the maintenance plan wizard. Unfortunately I have found this will only truncate the logs for databases using the simple database recovery model, and I wish to truncate the whole lot to save space...
I went ahead and put together an SSIS package that does the following:
1. First thing I do is build a fresh copy of a database listing in a temp table using the following SQL statement
USE Master; Go SELECT name, DATABASEPROPERTYEX(name, 'Recovery') AS RecoveryModel, DATABASEPROPERTYEX(name, 'Status') AS DBStatus INTO TempTables.dbo.Recovery FROM sysdatabases ORDER BY name GO
2. Next I read this dataset back into memory using a SQL statement as follows:
USE TempTables GO Select name from dbo.recovery Where name <> 'master' and name <> 'model' and name <> 'msdb' and name <> 'tempdb'
I map the results to a results set called User::TableName (which is of an object data type)
3. Next the package goes to a ForEach Loop container and I use the Foreach ADO Enumerator Enumerator to read the data I just selected into the look. I select the User::TableName variable and use the enumeration mode 'Rows in first table'. I have used a variable mapping of Variable: User::TableName with an Index 0.
4. I then use the following SQL statement (which sits in the For Each Loop) to try and alter the recovery model:
ALTER DATABASE @TableName SET RECOVERY SIMPLE; GO This is where the package falls over with the following error message
SSIS package "mnt_TransLog.dtsx" starting. Error: 0x0 at Simple Mode: Incorrect syntax near '@TableName'. Error: 0xC002F210 at Simple Mode, Execute SQL Task: Executing the query "ALTER DATABASE @TableName SET RECOVERY SIMPLE; " failed with the following error: "'RECOVERY' is not a recognized SET option.". Possible failure reasons: Problems with the query, "ResultSet" property not set correctly, parameters not set correctly, or connection not established correctly. Task failed: Simple Mode Warning: 0x80019002 at For Each Table Set to Simple: SSIS Warning Code DTS_W_MAXIMUMERRORCOUNTREACHED. The Execution method succeeded, but the number of errors raised (2) reached the maximum allowed (1); resulting in failure. This occurs when the number of errors reaches the number specified in MaximumErrorCount. Change the MaximumErrorCount or fix the errors. Warning: 0x80019002 at mnt_TransLog: SSIS Warning Code DTS_W_MAXIMUMERRORCOUNTREACHED. The Execution method succeeded, but the number of errors raised (2) reached the maximum allowed (1); resulting in failure. This occurs when the number of errors reaches the number specified in MaximumErrorCount. Change the MaximumErrorCount or fix the errors. SSIS package "mnt_TransLog.dtsx" finished: Failure.
I think the problem here is either I am not passing the variable across correctly (i.e. not declaring it or something..!) or I am passing the wrong type of variable across.
Once the recovery model is changed I can truncate the transaction logs successfully but I can€™t seem to until this is done! Any help in finding a solution to this problem or a better way of approaching this problem would be appreciated!
Thanks
Marek Kluczynski Management Information Officer Investors in People UK 7-10 Chandos Street London W1G 9DQ Tel: 020 7467 1956
I am getting this error :Distributed transaction completed. Either enlist this session in a new transaction or the NULL transaction. Description: An unhandled exception occurred during the execution of the current web request. Please review the stack trace for more information about the error and where it originated in the code. Exception Details: System.Data.OleDb.OleDbException: Distributed transaction completed. Either enlist this session in a new transaction or the NULL transaction.have anybody idea?!
i have a sequence container in my my sequence container i have a script task for drop the existing tables. This seq. container connected to another seq. container. all these are in for each loop container when i run the package it's work fine for 1st looop but it gives me error for second execution.
Message is like this:
Distributed transaction completed. Either enlist this session in a new transaction or the NULL transaction.
