Lately, I have been experimenting with SSIS and I created a generic custom error logging component that saves all offending data on data flow component failure. However...
Instead of re-directing rows at the data flow level and handling/logging the data at that level, is it possible to catch all of this information at the package level and handle/process it there?
If so how would you do this?
Thanks!
Tony
Hi All,
I am in the process of moving from a 32-bit SQL Server 2005 Enterprise (9.0.3054) to a 64-bit SQL Server 2005 Enterprise (9.0.3054 with 4 CPUs and 8GB of memory on Win 2003 SP2) and the process has been very frustrating to say the least. I am having a problem with packages that I created on my 64-bit SQL Server. I am importing a few tables from the 32-SQL Server into the 64-bit SQL Server using the Task --> Import to create the package.
Sometimes when I am creating a package I get the following error in a message box:
SQL Server Import and Export Wizard
The SSIS Runtime object could not be created. Verify that DTS.dll is available and registered. The wizard cannot continue and it will terminate.
Additional information: Attempted to read or write protected memory. This is often an indication that other memory is corrupt. (System.Windows.Forms)
Other times when I run a package that has run successfully before I get the following error:
Faulting application dtexecui.exe, version 9.0.3042.0, stamp 45cd726d, faulting module unknown, version 0.0.0.0, stamp 00000000, debug? 0, fault address 0x025d23f0.
Other times I get this error message:
.NET Runtime version 2.0.50727.1433 - Fatal Execution Engine Error (79FFEE24) (80131506)
And still other times
The package appears to hang when running. By this I mean that the Package Execution Progress shows progress up to a point then it just stops. (The package takes about 17 seconds to run normally) CPU usage is at 1% and the package cannot be stopped.
I have deleted and re-created the package several times and I have also re-installed the service pack on the SQL Server (9.0.3054) but that did not help.
Does anyone have any other suggestions to try?
Thanks.
So now ive got a ssis project that works and does what i need it to. How do I save the package into sql server ?
View 4 Replies View RelatedI am creating SSIS packages on the fly using C#.net. I need to save SSIS packages on a SQL Server using windows authentication. I have found a way to save the package using SQL authentication Microsoft.SqlServer.Dts.Runtime.Application .SaveToSqlServer(Package pkg,IDTSEvents evts,string serverName,string serverUserName,string serverPassword)
Can you give me a solution for saving SSIS packages using windows authentication, as it is a must in my project.
Hi
Can I limit an SSIS package that should run only for specific time, if it exceeds then should stop.
Ex. mySSIS.dtsx should run only 5 min, if exceeds 5 min it should stop running and exit.
Thanks,
Madhu
For those of you who have had a hard time like me trying to figure out using Protection level for an SSIS package whilst deploying the package via the SQL Server Agent, here is a piece of advice:
Firstly the protection level is set by Default to - "EncryptSensitivewithUserKey".
The encryption actually takes place only if you have things like - passwords etc..
From my experience - using both - "EncryptSensitivewithUserKey" and "EncryptSensitivewithPassword" Security features have turned out to be unreliable when deploying through SQL Server Agent (even while using a Proxy account having all previliges).
This is it seems because of issues with the user who created the package being different from the user who deployed the package. (which is really ridiculous).
So I used the ProtectionLevel - "DontSaveSensitive" - which means it is not going to encrypt anything in the package and so ur sensitive information would be blank. You would have to then supply your password etc using a configuration XML file. - using SSIS "Package configuation" in your menu....
This has been the most reliable way of solving the whole problem with encryption.
bear in mind that you might want to put the XML file in a secure location to which no one else has access to.
Thanks
Pramod
When attempting to save an SSIS package in Visual Studion I receive the following error message detailed below. If I attempt to "Save As" to another location, I then receive an insufficient storage error. The development machine has over 1.5 GB of available physical memory and several GB of disk space availabe to save my 16 MB package. I have checked the event log and have found no related messages in the Application or Server logs.
Any suggestions on how to determine the cause or resolution of this error message would be greatly appreciated.
Failure saving package. (Microsoft Visual Studio)
Insufficient memory to continue the execution of the program. (Microsoft.SqlServer.ManagedDTS)
Advanced Error Message Details
Failure saving package. (Microsoft Visual Studio)
------------------------------
Program Location:
at Microsoft.DataTransformationServices.Design.Serialization.DtrDesignerSerializer.SerializePackage(IDesignerSerializationManager manager, Package package, TextWriter textWriter)
at Microsoft.DataTransformationServices.Design.Serialization.DtrDesignerSerializer.SerializeComponent(IDesignerSerializationManager manager, IComponent component, Object serializationStream)
at Microsoft.DataWarehouse.Serialization.DesignerComponentSerializer.Serialize(IDesignerSerializationManager manager, Object value)
at Microsoft.DataWarehouse.VsIntegration.Designer.Serialization.DataWarehouseDesignerLoader.Serialize()
at Microsoft.DataWarehouse.VsIntegration.Designer.Serialization.BaseDesignerLoader.Flush(Boolean forceful)
at Microsoft.DataWarehouse.VsIntegration.Designer.Serialization.BaseDesignerLoader.Flush()
at Microsoft.DataWarehouse.VsIntegration.Designer.Serialization.DataWarehouseContainerManager.OnBeforeSave(UInt32 docCookie)
===================================
Insufficient memory to continue the execution of the program. (Microsoft.SqlServer.ManagedDTS)
------------------------------
Program Location:
at Microsoft.SqlServer.Dts.Runtime.Package.SaveToXML(String& packageXml, IDTSEvents events)
at Microsoft.DataTransformationServices.Design.Serialization.DtrDesignerSerializer.SerializePackage(IDesignerSerializationManager manager, Package package, TextWriter textWriter)
I got an SSIS package
that first "truncate the product table" then populate the table with new rows....if there is an error then i will invoke the send email task. However in the "truncate the product table" task, the sql i put "truncate dbo.product" instead of "truncate table dbo.product" but this error is not captured in the text file that I m going to send an email to the appropriate personnel.
Under logging, and when i go into "truncate product table" tasks , "Details", i saw "OnError", "OnWarning" and other error handlers...which should i tick in order for the text file to ONLY show errors I encounteered during the running of the SSIS package?
Hi folks,
I have a simple SSIS package I built to migrate data from a SQL db on one server to another SQL db on another server. All it does is: Execute SQL task to disable some triggers on the target table, Data Flow Task to pull the data from the source (a simple sql query) into the destination table (using OLEDB), then another Execute SQL task to re-enable the triggers.
This package ran fine several times during my testing. However, this morning it's not working anymore. When I run it and check its progress, the "Validation has started" is the only thing runs, then nothing else happens... no error messages, no timeouts, nothing in the output window besides "DTS.Pipeline: Validation phase is beginning".
The only thing that has changed is one change to a date in the where clause of my source query, so I'm baffled on why it no longer runs.
I'm pretty new to SSIS, so I could easily be missing something obvious here.
Anyone have any ideas on what's up, or who can point me in the right direction? I'm currently reading up on how to debug and add more comprehensive error handling, in hopes of finding a better description of the problem.
Thanks in advance,
Geoff
Hi All,
I hope someone can help becuase this problem is issue us several headaches.
We are currently trying to deploy an SSIS package to a production server. The deployment goes fine, the package runs ok when executed manually. The issues start when we try and execute it under the SQL agent.
Having gone back to the drawing board and spent much of the day reading various articles and applying the various options (especially those within the MS KB article 918760), we are still no closer to a resolution.
The SSIS package was created under an Administrator, and the SQL agent runs under a different Domain Admin account.
When we set up the Schedule to read from SQL Server or the SSIS Store the standard "Executed as user: DOMAINUSERNAME. The package execution failed. The step failed" in the history.
We tried to create the package as a file access and now get "Package could not be found" even though you can browse to i in the schedule list. The Domain account as full access to the folder where the package resides.
Has anyone else come across this issue, or have a workable solution?
Many TIA.
My project currently has task which have their own individual event handlers that get called onError (setup event messages). I also have a package level event handler that performs a generic task (sending events to the windows eventviewer) In the package level event handler there is a script task that decides on a boolean variable whether to "Success" or "Failure" to different task. When I fail one task of the main control flow, the task level event handler runs, then the package level event handler runs, and then it also runs again for some unknown reason. The second time it runs it picks up the value of a variable set in the variables window. However, I change this value at runtime to the value from a database. I can't understand why it would run the second time, and if it did run why it would have the value from the variables window and not the value that is set in memory. It's like the event handler runs with the value from memory and then runs and picks the values back out of the variables window, replacing the db values and re-runs.
Maybe the package itself is failing all together and then re-runing the package level event handler?
Any help would be greatly appreciated.
-Chris
Hi,
Can anyone explain to me why my ssis packages will not work when DontSaveSensitive protection level is selected? My package configurations are set as SQL server configuration type, and I have a table in a database that contains all the sensitive information (passwords and such). If I select "EncryptSensitiveWithUserKey" everything works, but I will be the only one able to execute the packages (not good, I need others to be able to execute them as well).
The error I'm getting tells me that the connection is not configured correctly or I may not have the right permissions on the connection.
My guess is that the DontSaveSensitive drops the passwords, but when I edit the data source and re-enter the password, it still does not work. Also, the database table I use that contains the sensitive data is not affected, all data remains.
What the fudge is going on here? I'm a newbie at this, can someone help me out?
I have an SSIS package that fires an OnTaskFailed event whenever any of my tasks fail.
I would like to put any SSIS task failure message into a user defined variable.
Any idea how to do this?
Any help appreciated.
Regards,
Paul.
I get errors when I check out an SSIS package from source control (both Source Safe and TFS) relating to the connection objects where I was not the original developer who checked it in. Is there a solution to this other than altering the connection loginpassword for every connection object in the package before deploying?
View 4 Replies View RelatedWhile executing SSIS package from command prompt as follows:
dtexec.exe /DTS "MSDBetl-hyundai-package" /SERVER tvmwindev02 /CONNECTION "TVMWINDEV02.HISNA_POC";"Data Source=TVMWINDEV02;In
itial Catalog=HISNA_POC;Provider=SQLOLEDB.1;Integrated Security=SSPI;Auto Translate=False;" /MAXCONCURRENT " -1 " /CHECKPOINTING OFF /REPORTING E /CONSOLELOGO
while executing the package following error message is received as :
Error: 2006-07-28 15:12:36.60
Code: 0xC00470FE
Source: Data Flow Task DTS.Pipeline
Description: The product level is insufficient for component "Data Conversion" (202).
End Error
and at the end as :
DTExec: The package execution returned DTSER_FAILURE (1).
Same error appers while executed from Integration Services - ->stored packages - - >name of the package -> mouse right button, run package.
But the same executes perfectly from visual studio, where it is developed.
what could be the reason?
I've deployed an ssis package to the database server through the manifest file over the network..
The package runs great when testing through vs.net on my client.
I've added the package to a job step in the sql server job agent.
When I test the new job, the package step fails, but there is no error or log information in the job history.
Wwhere can I get error information? And/or How can make SSIS error information more verbose in the job history?
Thanks for any help or information.
Hi
I have created a package which executes every 10 mins. Last week end for maintenance purpose, I shutdown my database. Now as an initial execution process, my package does the default validation steps on which the database connection validation step fails. As this is the default functionality of SSIS I am not able to capture this error. Is there anyway to capture this error inside SSIS Package?
Thanks in advance.
Gnan
I have a data flow task in which I have a ADO NET source and OLE DB Destination. I have in the ADO NET source a sql command which pulls all the columns in a table. My requirement is to ignore a particular column,say column99. I opened advanced editor and deleted the mapping between the external and output columns for column99. I had also set the Error and Truncation to "Ignore Failure" for column99. I had also mapped the destination column to <Ignore> in OLD DB destination.
But this still throws the error-
Description: The ADO NET Source was unable to process the data. Field table-column99 missing an escape character for a quote.Unable to update PK WHERE clause.Error processing data batch.
How do I solve this?
I have imported a package using SSIS into the Maintenance Plan folder from another server.
It imports fine, but when i Modify the package then click save I get the following error : 'Guid should contain 32 digits with 4 dashes'.
This has been reported on other forums without resolution.
Pete
I have an SSIS and SSAS project in the same solution. Â I need to debug the SSIS package regardless if there is an error or two in the SSAS project. Â Is there a way to ignore the SSAS project while I debug the SSIS package?
View 2 Replies View RelatedI can't seem to upload images so here's the link:Â [URL] ....
I need to get the last value of # Orders week avg in the current context (the highlighted value). The calendar dimension here is a week level one so I don't think I can use LASTNONBLANK. The measure is from the facts_sales table which is linked to a calendar dimension via a date key.Â
I have a parent package that calls child packages inside a For Each container. When I debug/run the parent package (from VS), I get the following error message: Warning: The Execution method succeeded, but the number of errors raised (3) reached the maximum allowed (1); resulting in failure. This occurs when the number of errors reaches the number specified in MaximumErrorCount. Change the MaximumErrorCount or fix the errors.
It appears to be failing while executing the child package. However, the logs (via the "progress" tab) for both the parent package and the child package show no errors other than the one listed above (and that shows in the parent package log). The child package appears to validate completely without error (all components are green and no error messages in the log). I turned on SSIS logging to a text file and see nothing in there either.
If I bump up the MaximumErrorCount in the parent package and in the Execute Package Task that calls the child package to 4 (to go one above the error count indicated in the message above), the whole thing executes sucessfully. I don't want to leave the Max Error Count set like this. Is there something I am missing? For example are there errors that do not get logged by default? I get some warnings, do a certain number of warnings equal an error?
Thanks,
Lee
When I create/alter a store procedure in SQL Server 2005, SQL server always checks for syntax errors first and won't let me save the change if it detects any error. Is there a way we can force the SQL server to save the store procedure that fails the syntax check?
I know SQL server will allow such invalid store procedures if you detach & re-attach the entire database from one SQL server to another server. However, if I try to manually create the same store procedure from one server on a different server with a script, then it won€™t let you save the store procedure if the linked server (or the table) can€™t be accessed from the new sql server.
How do you get around this?
Thanks
One question I haven't found to date is concerning context. First of all my setup is importing 250 fox tables into sql 2005. I have one main package with 4 child packages. These child packages have in turn on average 3 packages which in turn contain several tables each. This setup is due to resource limitation. Currently it takes about 12 hours, mainly due to my transformations, table size and resource limitations.
Anyway, my question is if i simply changed the defaultbuffermaxrows from default 10,000 in one of my tables to say 100,000, then executing this table import ad-hoc to analyse import time would not be a true reflection of package performance?
What I mean is if I analyse this table stand alone and due to change above time is reduced by say 1 minute, i might think great. But overall, I reckon increasing defaultbuffermaxrows to 100,000 will probably reduce resource allocation to other tables in package and thus could have a detriment to package, so the only way to accurately compare package performance is to make changes at table level and the run main package? - which of course taking 12 hours is not very practical......
Thanks in advance
Background:
I am executing a package programmatically from ASP.NET using C#. The package is being loaded from SQL Server at runtime and then it is executed from a custom class written in C#.
Problem:
When a task fails, I'd like information on why it failed. I read about implemeting the IDTSEvents interface as a way to have my package call back to code. This works fine (I have code in the OnTaskFailed event handler), but I'm not sure how to find out why a task failed. Since my task is loading a flat file to SQL Server, failure will likely come in one of two flavors: either the file format will be wrong or the data will violate a database constraint. In etiher case, I'd like that feedback. I am looking that the TaskHost object that gets passed to my event handler and I haven't figured out how to access this information. Am I going about this the right way? If yes, how do I access the error information I am looking for?
Hi There
I am trying to understand how to predict what user context a package will execute under, more specifically when it comes to operating system access.
For example we have a package that creates a flat file, during developement it was simply created on the C: drive of the users pc.
Now it must create the files on a remote server.
How do permissions to folders work is this case?
When you execute it in BIDS debug does it use the window account you logged onto the server as to connect to the remote folder ?
If so what happens when deployed to sql server and scheduled in a job? Does it use the domain account that the sql server agent account runs under ? Or will it use the credentials of the proxy account?
I am unclear in this scenario ? Obviously we need to grant the correct permissions on the remote server shared folder to the correct user, but i am not sure under what user account the package will be trying to create files on the remote server as ?
In a nutshell i need clarity on what user a package is executed as when running in a sql job ?
Anyone have a good link or the time to clarify this for me please.
Thanx
We get this error when we add IP addresses to the Windows system when SQL 2005
database activity is on-going:
Database error: A transport-level error has occurred when receiving results
from the server. (provider: TCP Provider, error: 0 - The semaphore timeout
period has expired.)
.NET application can be running for weeks without error, but after adding a
new IP address, application gets 5-16 'transport-level errors' before
correcting itself.
Error occurs on Windows XP computer in our case. SQL Server, running on Windows server 2003, doesn't seem to pick up on the newly added IP address.
This is my error text... how do I get this nonsense to stop?!
===================================
Exception of type 'System.OutOfMemoryException' was thrown. (Microsoft Visual Studio)
------------------------------
Program Location:
at System.String.GetStringForStringBuilder(String value, Int32 startIndex, Int32 length, Int32 capacity)
at System.Text.StringBuilder.GetNewString(String currentString, Int32 requiredLength)
at System.Text.StringBuilder.Append(Char value)
at Microsoft.DataWarehouse.VsIntegration.Designer.Serialization.BufferTextWriter.Write(Char ch)
at System.IO.TextWriter.Write(Char[] buffer, Int32 index, Int32 count)
at System.IO.TextWriter.Write(Char[] buffer)
at System.IO.TextWriter.Write(String value)
at Microsoft.DataTransformationServices.Design.Serialization.DtrDesignerSerializer.SerializePackage(IDesignerSerializationManager manager, Package package, TextWriter textWriter)
at Microsoft.DataTransformationServices.Design.Serialization.DtrDesignerSerializer.SerializeComponent(IDesignerSerializationManager manager, IComponent component, Object serializationStream)
at Microsoft.DataWarehouse.Serialization.DesignerComponentSerializer.Serialize(IDesignerSerializationManager manager, Object value)
at Microsoft.DataWarehouse.VsIntegration.Designer.Serialization.DataWarehouseDesignerLoader.Serialize()
at Microsoft.DataWarehouse.VsIntegration.Designer.Serialization.BaseDesignerLoader.Flush(Boolean forceful)
at Microsoft.DataWarehouse.VsIntegration.Designer.Serialization.BaseDesignerLoader.Flush()
at Microsoft.DataWarehouse.VsIntegration.Designer.Serialization.DataWarehouseContainerManager.OnBeforeSave(UInt32 docCookie)
I had just installed SQL 2005 dev on my laptop and got an error message when I tried to create a package using the BI IDE. I received the same error using VS2005 IDE. But the project was created regardless without any packages. When I tried to create a new package in the project, I received the same error again, but with an option to view the error details.
Following is the text of the error details:
TITLE: Microsoft Visual Studio
------------------------------
Error creating package
------------------------------
ADDITIONAL INFORMATION:
Failed to save package file "C:Documents and SettingsmyoungbloodLocal SettingsTemp mpD4B.tmp" with error 0x80040154 "Class not registered".
------------------------------
Failed to save package file "C:Documents and SettingsmyoungbloodLocal SettingsTemp mpD4B.tmp" with error 0x80040154 "Class not registered".
------------------------------
BUTTONS:
OK
------------------------------
I found a similar post that suggested that they try the following:
regsvr32 msxml3.dll
regsvr32 msxml4.dll
regsvr32 msxml6.dll
But msxml6.dll could not be found.
I did get a copy of this file from a coworker, and after copying it and registering it, I was able to add a package to the project.
My concern now is what is the likelyhood of this being the only file missing from the installation.
I'm wondering if I should reinstall, or (if it exists) do a repair on the installation.
Thanks.
Marshall
I have a SSIS package developed by a different user which does a lot of DML. This package sits on the server.
The package needs to be executed on a regular basis.
I have given RO access for a regular user on production DB, he is executing the package from his client desktop.
I was expecting this execution fail, since the package is doing lot of INSERTS , where the user has ONLY RO access.
I understand from the above experience that there is an “execution context� for SSIS execution. Can someone tell me how can I define the execution context for SSIS?
------------------------
I think, therefore I am - Rene Descartes
When I deploy the cube which is sitting on my PC (local) the following 4 errors come up:
Error 1 The datasource , 'AdventureWorksDW', contains an ImpersonationMode that that is not supported for processing operations. 0 0
Error 2 Errors in the high-level relational engine. A connection could not be made to the data source with the DataSourceID of 'Adventure Works DW', Name of 'AdventureWorksDW'. 0 0
Error 3 Errors in the OLAP storage engine: An error occurred while the dimension, with the ID of 'Customer', Name of 'Customer' was being processed. 0 0
Error 4 Errors in the OLAP storage engine: An error occurred while the 'Customer Alternate Key' attribute of the 'Customer' dimension from the 'Analysis Services Tutorial' database was being processed. 0 0
Hey, I've a few jobs which call SSIS packages. If I run the SSIS package, it runs fine but if I try to run the job which calls this package, it fails. Can someone help me troubleshoot this issue? None of my jobs that call an SSIS package work. All of them fail.
Thank you
Tej
Hi all,
When I try to save a DTS package on SQL Server(local),I have a error message "refused acces" ,T don't know what's the problem?
Any help will be appreciate?
thanks in advance