SSIS Package Progress And Testing Using C#.net
Nov 22, 2007
How to perform testing on SSIS packages?
I need to find out how the package progress and the re-entry takes place through c#.net.
Please provide me an appropriate code.
ADVERTISEMENT
Can We Pass Form Object Like Progress Bar To An Ssis Package
Dec 11, 2006Hi friends,
The problem that i am facing right now is that I have to show progress bar in my vb front end Application code which call my package using the
Application.LoadPackage(pakage Nothing).
What i am intending is that , I pass my progress bar instance to Package object and as the package executes, I
am changing the progress bar value to reflect the progress of execution.
Please Help ME.. with ur valuable responses
Regards
Maheswar
Retrieving Native Progress Events For A 2000 Package From SSIS. Is It Possible?
Sep 28, 2006Dear all,
AFAIK I don't think so. Let me know if possible and how.
I'm talking about a SSIS package which owns a Execute DTS 2000 Package Task and from there it calls a DTS 2000. Programatically, from our VB .Net front-end app we're seeing only SSIS events (when validate and execute methods are called) and any kind of information is provided from RCW proxy (I suppose that internally it gonna be created in order to send messages between managed and unmanaged code).
When a task inside our DTS 2000 fails we're awaring by means a "generic message" such this:
Error: System.Runtime.InteropServices.COMException (0x80040427): Execution was canceled by user. at DTS.PackageClass.Execute() at Microsoft.SqlServer.Dts.Tasks.Exec80PackageTask.Exec80PackageTask.ExecuteThread()
Thanks a lot for your time and comments,
Package Execution Progress
Nov 13, 2007
I have a pacakage wich loads data from a .csv file into my database. When I run the package from Visual Studio there is a Progress/Execution Results tab where I see information about the progress. I would like to insert some of these messages into my database. I tried to handle the OnProgress event with an Execute SQL Task and insert the Progress descriptions, but that doesn't seem to give all the information I see on the above mentioned tab (moreover, I get different numbers of events running the package from VS and running it programmatically from c#, why is that?).
What I really would like to get are the messages from the Progress tab which look like these:
[Agreement File [11773]] Information: The processing of file "D:****files*********.csv" has started.
[Agreement File [11773]] Information: The total number of data rows processed for file "D:****files*********.csv" is 201.
[Agreement File [11773]] Information: The processing of file "D:****files*********.csv" has ended.
[DTS.Pipeline] Information: "component "OLE DB Destination" (12598)" wrote 200 rows.
Any help would be appreciated.
How Do You Code The Package Execution Progress Window In C#
Mar 25, 2008
I need to write a front end to execute our .dtsx package due to it being encrypted and not wanting to expose the password at all.
I have successfully written code to run it but I want to show something like the Package Execution Progess window and I don't know how.
This window shows up when you just double click on the .dtsx file and Execute it using the interface provided.
Can anyone help?
Here's my code so far.
using System;
using System.Collections.Generic;
using System.ComponentModel;
using System.Data;
using System.Drawing;
using System.Text;
using System.Windows.Forms;
using Microsoft.SqlServer.Dts.Runtime;
namespace RunMigration
{
public partial class Form1 : Form
{
public Form1()
{
InitializeComponent();
}
private void button1_Click(object sender, EventArgs e)
{
DTSExecResult pkgResults_Sql;
try
{
Cursor.Current = Cursors.WaitCursor;
Microsoft.SqlServer.Dts.Runtime.Application app = new Microsoft.SqlServer.Dts.Runtime.Application();
app.PackagePassword = "SomePassword";
Microsoft.SqlServer.Dts.Runtime.Package pkg = new Microsoft.SqlServer.Dts.Runtime.Package();
pkg = app.LoadPackage("T:\amber\encrypt\EncryptedPHDMigration.dtsx", null);
string mypath = this.textBox1.Text.ToString();
pkg.ImportConfigurationFile(mypath);
pkgResults_Sql = pkg.Execute();
MessageBox.Show(pkgResults_Sql.ToString());
MessageBox.Show(pkg.ExecutionStatus.ToString());
foreach (Microsoft.SqlServer.Dts.Runtime.DtsError error in pkg.Errors)
{
Console.WriteLine("{0}", error.Description);
this.richOutput.Text = error.Description;
}
}
catch (System.Exception f)
{
MessageBox.Show(f.Message);
}
finally
{
Cursor.Current = Cursors.Default;
}
}
private void btnConnect_Click(object sender, EventArgs e)
{
OpenFileDialog open = new OpenFileDialog();
open.Filter = ("SSID config files|*.dtsConfig");
if (open.ShowDialog() == DialogResult.OK)
{
this.textBox1.Text = open.FileName.ToString();
}
}
}
}
SSIS Error Or Progress Log
Feb 21, 2008
I have just finished building an SSIS package which does a couple of things including using variables, calling child packages as well as starting off replication.
I plan to kick-off the package via SQL Server Agent.
Is there any way I can monitor the progress of this report by creating a web front-end that refreshes every few minutes to report on errors and progress?
Any ideas are welcome.
Thanks
Progress Indication From Ssis App In VB
Nov 11, 2006Hi
am calling an ssis app from my vb programme. It takes about 5 minutes to run. What is the best way to provide progress feedback to my vb user.
Cheers
Ants
I
How To Launch Package From Client Machine, Display Progress And Finally Results?
Oct 4, 2006Please help. I'm completely new to SSIS, and I was hoping someone could point me in the right direction(s).
I am developing a winforms client app, and I need to be able to provide the user with the ability to import data in CSV format into our application's database.
I'd like to use SSIS if possible, as long as what I am trying to do isn't near impossible.
I'm thinking of a UI where the user can specify a flat file (CSV) to upload and be processed by an SSIS package on a remote database server.
This package will be responsible for validating the CSV file and inserting data into the database as appropriate.
Is there a way I can:
launch an SSIS package remotely, from a winforms app on the client machine that is not running SQL server (preferably asynchronously) -- and pass it some parameters, including the import file itself
(optionally) provide progress feedback to the client pc to let them know it is being processed
display a nice SSRS report on the client upon completion that tells them exactly what the success/failure of the import was (how many rows processed, which ones failed and why, etc)
Any helpful examples, links, etc would be most appreciated.
Thanks in advance!
Package Setup, Configurations And Testing
Aug 1, 2007Hi,
I'm looing for advice on how to organize my SSIS packages. I'll quickly describe my requiremenst, and then outline how I see the solution.
1. Should work for development in BI Studio. Design time experience should be good, so static schema information should be available in all steps. Packages should run in debug mode inside BI Studio.
2. Should be able to run automated tests (which I gather is difficult. See proposed solution below). Tests should run both on a test machine, in batch mode, and from BI Studio.
3. Should work for deployment.
The packages should be fine-grained. Automated tests will be implemented as SSIS packages that call the packages to be tested via an Execute Package Task. There will be one test package per testcase. A test will look like: [Create Schema] -> [Load test data]->[Execute Package to be tested]->[Verify expected results]->[Teardown].
There will be one master test package that executes all the tests.
For this I need a) some way to vary the connections for the packages depending on whether they are run from a testcase or from the normal control flow. I'm considering making the connection strings configurations based on a parent package variable. Other configuration types strike me as not suitable since developers will want to run packages both as part of normal flow and as testcases on their machines. Is this a viable strategy? Any drawbacks?
b) need to organize testcases some way. I tried putting tests in a subfolder of the project to minimize the clutter, but BI studio shows them at the root node. I tried putting a new project (for containing the tests) in the same folder, and adding the packages (via add existing item), but BI Studio insists of adding copies (!) of the package files, rather than reusing the existing files. Any suggestions?
I welcome any feedback, suggestions of alternate strategies, etc.
/Anders
Problem: SSIS, ODBC And Progress Database
Dec 15, 2005Hi,
The project I´m working needs to import data from a Progress Database. I had configured my ODBC (MERANT 3.60 32-BIT Progress SQL92 v9.1D) driver and everything seems to be ok.
But when I tried to create a DataRearder Source pointing to my ODBC Data Source, and open the next tab I received this error message:
€œError at Data Flow Task [DataReader Soucer [135]]: Cannot acquire a managed connection from the run-time connection manager€?
Anyone can help ?
Error Testing Transformations In SQL7 DTS Package
Mar 8, 2001In SQL 7 DTS I'm creating a Data Driven Query Tasks. When I get to writing the transformations in VB Script and try using the Test button to test the script, I get the following message:
Error Source: Microsoft Data Transformation Services Flat File Rowset
Provider
Error opening datafile. The system cannot find the file specified.
The documentation says:
[The transformation is tested] by executing it against a part of the
source data and copying the results to a temporary text file, for
preview purposes.
I can find no place to specify what temporary file is used for the test result and, because it is temporary, have assumed the system creates and
deletes it as needed.
Is this true?
I get this test error for every transformation. even those that work correctly when the DTS package is actually executed.
What can I do to get the transformations to test properly?
Any help or hints will be greatly appreciated.
===============================
William J Brown
Systems Analyst
College of Human Medicine
A114 East Fee Hall
Michigan State University
East Lansing, MI 48824
Voice: (517) 432-7490
Fax: (517) 355-0342
Email: brownwj@msu.edu
Integration Services :: Connect SSIS 2005 To Progress Database?
Sep 18, 2015how can I connect SSIS 2005 to Progress database?
View 4 Replies View RelatedTesting SSIS
Jan 16, 2008hi every one
can any one please explain me how to test manually SSIS packages
thanks in advance
Load Testing With SSIS
Sep 14, 2006Has anyone attempted to perform a load test with SSIS? I am trying to saturate(100% processor utilization) a HP integrity rx4640 server with 60GB of ram and 4 processors, windows 2003 and Sql server 2005 in order to to build a sizing document?
Testing For IsNumeric In SSIS
Dec 18, 2007Hello,
I have searched the forum, and have discovered that the DTS method using IsNumeric to check for numierc values (ActiveX) is not valid in SSIS. Most of what I have seen prescribes using the script component to handle this.
So formerly, I checked to see if a column was numeric. If it was, then I needed to use the numeric value as is, or in some cases, I needed to perform a calculation on the value and use the result. If the value was not numeric, then whatever the value was needed to be changed to zero.
Here is an example of how I would use the current value, or set the value to zero:
If IsNumeric(DTSSource("Col003")) Then DTSDestination("ADepTrnx") = CLng(DTSSource("Col003")) Else DTSDestination("ADepTrnx") = 0 End If
This is an example of how I would use the current value in a calculation, or set the value to zero:
If IsNumeric(DTSSource("Col012")) Then DTSDestination("AlliStdFee") = CLng(DTSSource("Col012"))/100 Else DTSDestination("AlliStdFee") = 0 End If
Does anyone have an example of how I would handle both situations in a script component?
Thank you for your help!
cdun2
Testing Strategies For SSIS?
Aug 23, 2007I have been somewhat dismayed by the lack of information available relative to testing SSIS solutions. At a time when both managed code and db code are getting tooling (through VSTS) to help drive an automated testing mentality into developers everywhere - there seems to be little out there for those using SSIS. I have seen people talk about having SSIS packages that test the actual packages - there is always the manual way to write the tests in something like C# and then programmatically invoke the package and manually validate the results. Anyone out there have approaches that are working for them?
View 1 Replies View RelatedSSIS Foreach Loop Stress Testing Problem.
Mar 2, 2006I am currently working on a project which needs to load over a 1000 xml files. The files are stored across 10 subfolders. I am using a foreach loop with a file enumerator, which is configured at the top of the folder structure and traverses the subfolders. This loops through the files, load the data and then moves the file to another folder. The package executes fine for a few 100 files but then hangs; this happens after a different number of processed files each time the package is run. While trying to resolve the problem we ran performance counters and noticed that the number open handles increased significantly just about the time Dtexec looked like it had hanged and DTexec also then started taking a lot of the cpu processing time. Has one else come across similar situations?
View 9 Replies View RelatedJob Running SSIS Package Keeps Failing But The SSIS Package By Itself Runs Perfectly Fine
Aug 30, 2006Hey, I've a few jobs which call SSIS packages. If I run the SSIS package, it runs fine but if I try to run the job which calls this package, it fails. Can someone help me troubleshoot this issue? None of my jobs that call an SSIS package work. All of them fail.
Thank you
Tej
Error While Executing SSIS Package From Other SSIS Package
Jan 10, 2007Hi,
In our project we have two SSIS package.
And there is a task (Execute SSIS package) in First package that calls the execution of second package.
I m continuously receiving an error "Failed to decrypt protected XML node "PackagePassword" with error 0x8009000B "Key not valid for use in specified state.". You may not be authorized to access this information. This error occurs when there is a cryptographic error. Verify that the correct key is available."
As we are running first package by job, job runs successfully logging above error
The protection level of second package is set to "EncryptSensitiveWithUserKey"
Can anybody please suggest how to handle it?
Passing Execute DTS Package Result (success/failure) To Calling SSIS Package
Mar 6, 2008I have a SSIS job, one of the last steps it performs is to execute a SQL 2000 DTS package. This has to be done as a SQL 2000 DTS package as it is performing rebuilds of SQL 2000 Analysis Services dimensions and cubes. We've found that when the DTS fails the SSIS job is happily completing showing as a success, we would prefer to know it went wrong.
As far as I'm aware SSIS merely starts the DTS off and doesn't care about it's result. I've taken a look in to turning on the logging for the execute DTS package and thought that the ExecuteDTS80PackageTaskTaskResult would give me the answer I need...but is merely written to the log not available as an event-handler. It also looks like it is not safe to put a SQL task in as the next item to go look at the SQL 2000 system tables to look at the log for the DTS package as the SSIS documentation warns that the DTS package can continue to run after the execute DTS package task has ended.
Ideally I want any error raised within the DTS package to cascade up to be an error in the SSIS job, I can then handle it appropriately. I cannot find a way to do this. Is there a way?
If not, can anyone suggest how in the remainder of the SSIS tasks I can be sure that the DTS has completed before I start any other tasks that will check for the SQL 2000 log of its execution?
Error Stating Package Failure While Executing SSIS Package In Standard Edition
Feb 2, 2007Hi,
I have developed an SSIS package for ETL purpose. I am invoking the SSIS package through .Net console application by referencing the ManagedDTS Assembly. I am able to execute the package in Sql Server 2005 Developer Edition and it runs fine till completion.
But when i try to execute the packahe in Sql Server 2005 Standard edition, by invoking the package through .Net console application the status of the package is failure.
Can any one help me how to over come this problem.
Several Different Errors When Creating And Running SSIS Package. Package Runs Successfuly Sometimes.
May 2, 2008Hi All,
I am in the process of moving from a 32-bit SQL Server 2005 Enterprise (9.0.3054) to a 64-bit SQL Server 2005 Enterprise (9.0.3054 with 4 CPUs and 8GB of memory on Win 2003 SP2) and the process has been very frustrating to say the least. I am having a problem with packages that I created on my 64-bit SQL Server. I am importing a few tables from the 32-SQL Server into the 64-bit SQL Server using the Task --> Import to create the package.
Sometimes when I am creating a package I get the following error in a message box:
SQL Server Import and Export Wizard
The SSIS Runtime object could not be created. Verify that DTS.dll is available and registered. The wizard cannot continue and it will terminate.
Additional information: Attempted to read or write protected memory. This is often an indication that other memory is corrupt. (System.Windows.Forms)
Other times when I run a package that has run successfully before I get the following error:
Faulting application dtexecui.exe, version 9.0.3042.0, stamp 45cd726d, faulting module unknown, version 0.0.0.0, stamp 00000000, debug? 0, fault address 0x025d23f0.
Other times I get this error message:
.NET Runtime version 2.0.50727.1433 - Fatal Execution Engine Error (79FFEE24) (80131506)
And still other times
The package appears to hang when running. By this I mean that the Package Execution Progress shows progress up to a point then it just stops. (The package takes about 17 seconds to run normally) CPU usage is at 1% and the package cannot be stopped.
I have deleted and re-created the package several times and I have also re-installed the service pack on the SQL Server (9.0.3054) but that did not help.
Does anyone have any other suggestions to try?
Thanks.
Is It Possible To Change Or Replace The Default Package Template Named New SSIS Package?
Feb 21, 2008I would like to standardize SSIS development so that developers all start with the same basic template. I have set it up so it is an available template ( http://support.microsoft.com/kb/908018 ) but I would like it to be the default when a new project or package is created. Is this an option?
View 4 Replies View RelatedSQL 2012 :: How To Capture Data Flow Component Name Dynamically While Package SSIS Package Is Executing
Jun 3, 2014I would like to fetch the data flow component name while package is executing. Since system variable named [System::SourceName] only fetches name of the control flow tasks? Is there a way to capture them?
View 5 Replies View RelatedProblem When Running A SSIS Master-package-child Package On Non Default Sql-server Instance
Dec 6, 2007Hi there
We have a SSIS run which runs as follows
The master package has a configuration file, specifying the connect strings
The master package passes these connect-strings to the child packages in a variable
Both master package and child packages have connection managers, setup to use localhost. This is done deliberately to be able to test the packages on individual development pc€™s.
We do not want to change anything inside the packages when deploying to test, and from test to production. All differences will be in the config files (which are pretty fixed, they very seldom change). That way we can be sure that we can deploy to production without any changes at all.
The package is run from the file system, through a job-schedule.
We experience the following when running on a not default sql-server instance (called dkms5253uedw)
Case 1:
The master package starts by executing three sql-scripts (drop foreign key€™s, truncate tables, create foreign key€™s). This works fine.
The master package then executes the first child package. We then in the sysdtslog get:
Error - €œcannot connect to database xxx€?
Info - €œpackage is preparing to get connection string from parent €¦€?
The child package then executes OK, does all it€™s work, and finish. Because there has been an error, the master package then stops with an error.
Case 2:
When we run exactly the same, but with the connection strings in the config file pointing to the default instance (dkms5253), the everything works fine.
Case 3:
When we run exactly the same, again against the dkms5253uedw instance, but now with the exact same databases defined in the default instance, it also works perfect.
Case 4:
When we then stop the sql-server on the default instance, the package faults again, this time with
Error - €œtimeout when connect to database xxx€?
Info - €œpackage is preparing to get connection string from parent €¦€?
And the continues as in the first case
From all this we conclude, that the child package tries to connect to the database before it knows the connection string it gets passed in the variable from the master package. It therefore tries to connect to the default instance, and this only works if the default instance is running and has the same databases defined. As far as we can see, the child package does no work against the default instance (no logging etc.).
We have tried delayed validation in the packages and in the connection managers, but with the same results (error).
So we are desperately hoping that someone can help us solve this problem.
Thanx,
/Nils M - Copenhagen
Passing Value From A Child Package To The Parent Package That Calls It In Ssis
May 21, 2007hi,
I am interested in Passing value from a child Package variable to the Parent package that calls it in ssis.
I am able to call the Child package using the execute package task and use Configurations to pass values from the parent variable to the child, but I am not able to pass the value from the child to the parent.
I have a variable called datasetId in both the parent and child. it gets computed in the child and needs to be passed to the parent...
Any suggestions?
Thanks for any help in advance..
smathew
Report With SSIS Package Having Indirect Package Configuration Setup
Sep 10, 2007Deployed Report having SSIS package as source do not work when Indirect Package configuration is used in ETL package. It seems ETL package when called/executed from Report manager does not recognize environment variable to pick up the dtsconfig file.
The Report works when Direct package configuration is used to same dtsconfig file.
What could be the reason? Any solution for this? This will cause our build/deployment to QA and Prod very difficult.
How To Compose The Connection String Of A SSIS Package That Execute Another Package?
Jul 6, 2006Dear All,
I now have two SSIS package, "TESTING" and "LOADING". The "TESTING" package have an execute package task that call the "LOADING" package. When I want to execute the TESTING package, how can I setup the connection string so that I can edit the password of the database connected by the "LOADING" package?
Regards,
Strike
Unit Testing For SSIS - To Test Or Not To Test?
Oct 17, 2006Now that we have a good programming model in SSIS - the question is whether to write automated unit tests for your packages, and would it generally be a good idea for packages?
Also - if yes to write tests - then where to find more informations regarding How to accomplish that?
SSIS Parent Package Can Not Find The Child Package
Oct 13, 2007I have two SSIS packages in a project, one calling the other. The parent package works fine in my local mechine. After they are deployed to the production, I schedeul jobs to run the packages in the SqlServer. The child package works fine if I run it alone, but the parent package could not find its child package if I run the parent package . As I checked, all xml config files and the connection string pointing to the child package were set correctly. It seems the parent package did not use the xml config file. Can someone help me? Thanks in advance.
View 9 Replies View RelatedSchedule A SSIS Package Which Execute DTS 2000 Package
Mar 25, 2008
I have successfully created a SSIS package which execute a DTS 2000 package and with no problem to execute the task. But I failed to schedule this package. I was not success in setting the logging. When running the package in command line:
dtexec file "C:Documents and SettingslyangMy DocumentsVisual Studio 2005ProjectsTraingDTSTraingDTSDTSTraining.dtsx"
Error: 2008-03-24 08:03:24.36
Code: 0xC0012024
Source: Execute DTS 2000 Package Task
Description: The task "Execute DTS 2000 Package Task" cannot run on this edit
ion of Integration Services. It requires a higher level edition.
End Error
Warning: 2008-03-24 08:03:24.38
Code: 0x80019002
Source: DTSTraining
Description: The Execution method succeeded, but the number of errors raised
(2) reached the maximum allowed (1); resulting in failure. This occurs when the
number of errors reaches the number specified in MaximumErrorCount. Change the M
aximumErrorCount or fix the errors.
End Warning
DTExec: The package execution returned DTSER_FAILURE (1).
Any help will be greatly appreciated.
(32 bit machine, standartd edition of SQL 2005)
Problem Setting Package Variables In SSIS Package
Sep 8, 2006Hi,
I am making use of the DtUtil tool to deploy my package to SQL Server.
Following is my configuration:
32-bit machine and 32-bit named instance of Yukon.
I have some package variables which need to be set in the code.
Previously I did it as follows:
Set the package variables in the code. For example:
pkgFile.Variables["User::DestinationServerName"].Value = <myvalue>
Deploy the package as follows:
applnObj.SaveToSqlServer(pkgFile, null,
destinationServer, null, null);
Here the package was successfully deployed and when i open those packages using BIDS, I am able to see that the variables are set to the values as doen in teh code.
Because of oen problem I am not using SaveToSQLServer method. So I switched to DTUtil tool.
Now I am doing it this way:
Set the package variables as before.
Deploy the package to SQL Server using DTUtil tool.
Now is the problem:
The package is successfully deployed. But the variables are not set to the value that I have specified in the code.
I also tried DTexec utility to set the package variable. Even that does n't work.
Can anyone help me out? Is there any alternate method to set package variables?
Thanks,
Sandhya
SSIS Package Failing As A Job, But Getting Successful When Run As Package
Nov 6, 2007when i run the job using network service account credentials job is failing. But when i run the package individually, it is tasting success. when it runs as job, this is the error message i am getting
SSIS Error Code DTS_E_CANNOTACQUIRECONNECTIONFROMCONNECTIONMANAGER. The AcquireConnection method call to the connection manager "Excel Connection Manager" failed with error code 0xC0202009. There may be error messages posted before this with more information on why the AcquireConnection method call failed.
I have changed to 32-Bit run time and ran the excel package, even then it is failing...i tried to use my credentials (i am admin on the box), even then it is failing...please suggest