Difficult Times Deploying A Few Packages To SQL Server And Running As A Job
Jan 4, 2007
Hi Guys!
I have created a big list of packages, some calling others. They all work fine from my computer using Visual Studio.
When I try to deploy them (building them with deployment turned on and running them either directly from Management Studio or as a job) I get the errors with the password of connection strings. From what I read so far its the encryption process that kills it.
I have tried to add a password to some packages, but it still didnt work (only when run directly on my computer in management studio after deploying to SQL Server, but not as a job).
I have tried to change ProtectionLevel to SecurityStorage, wouldnt let me save in Visual Studio (I understand it is ot allowed in VS because you are saving to File System, how the hell am I supposed to save it to anything else? why is it showing there if its not even valid?).
If anyone can please give me the steps to doing it properly, that would be awesome. I simply need to run the packages from SQL Server! thats all! I have no idea why it has to be soooo difficult :/
View 8 Replies
ADVERTISEMENT
Oct 23, 2006
Hello,
We have been conducting some testing regarding package deployment and SQL Server based configuration. It seems there is a problem that was documented in an MS Feedback entry (
https://connect.microsoft.com/SQLServer/feedback/ViewFeedback.aspx?FeedbackID=126797
)
I searched for a viable answer, work-around, or hotfix that would address this issue but found none.
The jist is that a package that uses SQL Server based configuration must have an xml-based configuration entry to "re-point" the SQL Server based configuration connection manager to the deployment target server. We cannot use environment variable or registry as this is against internal policy. The problem is that even though the xml based config is specified first in the list of package configurations it does not get applied first at run-time. So, when the package runs from a SQL Server Agent job the package's connection manager for the configuration entries is not updated.
The package runs correctly through BIDS. You can change the connection string in the .dtsConfig file and the SQL Server based package configuration is obtained from the correct source.
Environment is SQL Server Enterprise Edition 64-bit w/ SP1, Windows Server 2003 Enterprise Ed. 64-bit.
Does anyone have any experience with this issue? Know any hotfixes, other work-arounds?
Thanks in advance!
View 9 Replies
View Related
Apr 10, 2006
Can someone show me, or direct me, to a source, that shows me how, and what to change, when deploying a website from a development server running Sql Ex to a production server running Sql server 2005. I can’t get the sites to run under Sql server 2005.
They work in Sql Ex. what must I change? The connection string, to what format? and what else? I attached the dB to Sql 2005 and browsed the content in the Sql manager. But can’t get the aspx pages to work on the server.
Help please
View 6 Replies
View Related
May 1, 2006
Error: 0xC0202009 at Package, Connection manager "Presup Dev sql_prov": An OLE DB error has occurred. Error code: 0x80040E4D.
An OLE DB record is available. Source: "Microsoft SQL Native Client" Hresult: 0x80040E4D Description: "Communication link failure".
An OLE DB record is available. Source: "Microsoft SQL Native Client" Hresult: 0x80040E4D Description: "TCP Provider: Se ha forzado la interrupción de una conexión existente por el host remoto.
WTF????????? ·%$·&$%%& i`m sure my connections are ok,, cause i prevoiusly checked them, the problem occurs when deploying the package.. it seems to be an error when connecting to the source but whyy?????, and what about the package configurations ?? should i use them,, and how????
View 4 Replies
View Related
Jan 31, 2008
Hi
I am fairly new to SSIS and I have a number of questions concerning deploying SSIS packages and Configuration Files
The SSIS packages will be deployed to three environments (DEV, TEST,PROD - this is a clustered environment) and will be executed by SQL Server Agent Jobs. There will be a need for different configuration values for each environment (file import directory, database server connectivity) , the configurations will be reused by mulitple packages.
I have decided to deploy the packages and configuration files to a file directory in the format
<<DRIVE LETTER>>SSIS PackagesPackages
<<DRIVE LETTER>>SSIS PackagesConfiguration Files
The questions are:
1) I storing the packages/config files to a file directory the best approach, or should I be deploying the packages to SQL or file ( C:Program FilesMicrosoft SQL Server 90DTSPackages)
2) Is it best to have different configuration files for each environment (dDBConn_DEV.dtsconfig, dDBConn_TEST) or have the same config file and then change the values during deployment (via scripts).
3) What is the best way to deploy the packages and config files to the different environments (rather do it via scripts than the deployment utility)
4) Where is the best place to store the config files (I have one VS 2005 project per package, the confi files are used by multiple packages), TFS is our source control software
5) Does any one know of a good website to look at for best practice when deploying packages and config files
Regards
Tim
View 5 Replies
View Related
Apr 27, 2008
Hi
I need to move my packages from development to production.
In the development the database name is dev and in producton it is prod.
The username passwords are also different .
When i deploy my packages in production i dont think so it will work any pointers on how to make these things configurable like when the person deploying this package runs it he is prompted for the connection string and username and password
regards
Hrishy
View 2 Replies
View Related
Jan 25, 2008
I am currently in the process of trying to migrate from DTS to SSIS. SSIS is totally new to me and I am trying to put together a mechanism for easily moving packages between our Development, Test, and Production environments. I have done some investigating into using Configurations to accomplish this. What I basically need to do it update my Connection objects when moving between environments without having to recompile. The optimal solution is that the Connection objects dynamically update, but I'm not sure this is possible. I think some of this could be done with Configurations, but one catch is that I need to have the packages running on both environments.I have been able to put this all together to figure out a good solution for this and I was hoping some of you on the forums could give me some ideas.
View 5 Replies
View Related
Aug 24, 2007
We have a system here where we develop SSIS packages on a development server. I am trying to figure out the cleanest way to promote these changes to a production server where stored procedures/tables that are used in the package are not deployed yet.
When I switch the connection in the package to the production server there are alot of objects in the package that are "invalidated" because they are trying to verify existence of tables/columns. One example is outputing the results of a query to a text file. The text file destination gets a red X on it because it cant grab the columns from the source query (because that stored procedure doesnt exist yet)
Is there a best practices or something on how to deploy packages to a production system? I have tried turning off "ValidateExternalMetaData" with no success.
Thanks!
View 3 Replies
View Related
Jan 31, 2008
Hi
Just a brief question,
is there a way of displaying a success or not success message when running (testing) an SSIS package on SQL 2005 ?
Because i have to go through the steps to see if there was an error or not and for large SSIS packages this is a pain.
With DTS 2000 after it ran you got a message saying successfull or not, is there a way to activate this in SQL 2005 ?
I suppose i could just run the job ... but I prefer the other way as sometimes a job fail is a permissions problem, not a SQL/datapump task problem...
View 1 Replies
View Related
Jan 18, 2006
I am trying to deploy a SSIS package which includes a FTP task. It works fine on the machine it was developed . When deploying the package on a production server, all other config changes work, but the FTP task is failing, with authentication error.
any help on how to persist the data would be appreciated
thanks
View 1 Replies
View Related
Sep 25, 2007
It is clear to me that in order to be able to use certain SSIS components (for example the Excel jet provider) I must launch my packages using the 32bit DTEXEC located at Program Files (x86)Microsoft SQL Server90DTSBinn. However, when I do this it seems that there are other components of the package that no longer work as expected.
To test this I have created a simple package with two tasks (Run64BitRuntime is set to False):
1. Data flow task importing data from Excel
2. Execute SQL Task which does a simple select (select 1) from a Native OLE DB SQL data source (same SQL Server on which packages are stored). This task contains no input or output parameter.
When I try to execute the package using the 64bit DTEXEC, task 1 fails with the following error (as expected):
Code: 0xC0202009
Source: connection1 Connection manager "SourceConnectionExcel"
Description: SSIS Error Code DTS_E_OLEDBERROR. An OLE DB error has occurred. Error code: 0x80040154.
An OLE DB record is available. Source: "Microsoft OLE DB Service Components" Hresult: 0x80040154 Description: "Class not registered".
When I execute the package with the 32bit DTEXEC, task 2 fails with the following error
Code: 0xC002F210
Source: Execute SQL Task Execute SQL Task
Description: Executing the query "" failed with the following error: "Attempted to read or write protected memory. This is often an indication that other memory is corrupt.". Possible failure reasons: Problems with the query, "ResultSet" property not set correctly, parameters not set correctly, or connection not established correctly.
Now here is the confusing part: When I change task 2 to use the .Net provider instead of the OLE DB the package works fine. According to the MS documentation, both of these providers are supported on 32 and 64 bit so am I missing something? One more thing to note: before I was able to use the 32 bit DTEXEC I had to re-register it as described in this KB article: http://support.microsoft.com/kb/919224
View 5 Replies
View Related
Feb 13, 2006
Is there any information around what the SSIS packages are doing in the first 5-10 seconds of execution, and ways to speed this process up?
View 3 Replies
View Related
May 28, 2008
Hello,
I see the following problem regarding a SQL function that returns a value.
We have the following query
SELECT c.Id_Customer, c.Name
FROM t_Customers c
WHERE c.Id_Status = fn_GetParameter('ID_Status.Active')
The idea is not hardcoding the status and other values on each query, and, since SQL Server does not support the definition of constants, we have a table with many parameters and we search them.
We defined the function "WITH SCHEMABINDING" and the SQL Server recognizes it as DETERMINISTIC, so I do not understand why it executes the function as many times as records in the t_Customers table, since every time it is executed it returns the same value.
We could define a variable, assign the value returned by this function to this variable and then use it on the SELECT, but this approach is useless if we use SQL instead of stored procedures (for example, in reports from reporting / BI tools).
Any explanation about why SQL chooses to execute the function many times, and any hint regarding how to make SQL Server execute only once the function will be very appreciated.
Thanks in advance,
Lisandro.
View 3 Replies
View Related
Mar 2, 2005
Hi,
I was wondering if this can be done...
I have a complex query which has to do a few calculations. I'm using subqueries to do the calcs, but most of the calcs have to use a value gotten from the first subquery. I don't want to have to type the subquery out each time, so is there a way of assigning it to a variable or putting it in a UDF or SP?
E.g.
I have a table with 2 cols - amount, date.
SELECT total_amount, closing_amount,
FROM table1
GROUP BY month(date)
Total amount is the SUM(amount) for the month.
Closing amount is the Total Amount plus the amounts for the current month with a few extra calcs.
As I have to use SUM(amount) in the second subquery, is there a way I can do it without having to type hte subquery out again?
This is only a basic example, what I'm trying to do will invovle a lot more calcultions.
Hope someone can help,
Thanks,
Stuart
View 5 Replies
View Related
Jun 25, 2007
I’m binding the distinct values from each of 9 columns to 9 drop-down-lists using a stored procedure. The SP accepts two parameters, one of which is the column name. I’m using the code below, which is opening and closing the database connection 9 times. Is there a more efficient way of doing this?
newSqlCommand = New SqlCommand("getDistinctValues", newConn)newSqlCommand.CommandType = CommandType.StoredProcedure
Dim ownrParam As New SqlParameter("@owner_id", SqlDbType.Int)Dim colParam As New SqlParameter("@column_name", SqlDbType.VarChar)newSqlCommand.Parameters.Add(ownrParam)newSqlCommand.Parameters.Add(colParam)
ownrParam.Value = OwnerID
colParam.Value = "Make"newConn.Open()ddlMake.DataSource = newSqlCommand.ExecuteReader()ddlMake.DataTextField = "distinct_result"ddlMake.DataBind()newConn.Close()
colParam.Value = "Model"newConn.Open()ddlModel.DataSource = newSqlCommand.ExecuteReader()ddlModel.DataTextField = "distinct_result"ddlModel.DataBind()newConn.Close()
and so on for 9 columns…
View 7 Replies
View Related
Jan 22, 2008
I have multiple SQLDatasources on multiple pages. When I call the Select command for the datasource the query is run multiple times. I was wondering if anyone had any problems like this. The data is being read into listboxes. If you need any more info or have any specific questions, feel free to ask.
View 6 Replies
View Related
Jul 26, 2007
I created an SSIS Package and once complete there is 4 instances of the same data. I checked the source and it only has 1 instance. Why is it pulling the data 4 times? It will first show 3 instances and then another will show up before the package finishes running. Is there a bug in the program or is there a place I'm not aware of where it sets how many times the package should run.
Thank you,
Cindy
View 3 Replies
View Related
Jun 12, 2015
I want  the below query to run 24 hours ..once the insert is complete, run again , so on for 24 hours .
there is a way to run every second in as job but i want to run only after run complete ..is there a way to run the query after every complete run ? and keep in jobÂ
Â
INSERT INTO [dbo].[Audit_Active]
([SPID],[LoginName],[HostName],[ProgramName],[Command],[LastQuery],[DBName],[ServerName])
 SELECT  Â
--DISTINCT Â
p.SPID,
p.LogiName,
p.HostName, Â Â
[Code] ....
View 8 Replies
View Related
Mar 26, 2008
Is it possible to run through a buffer multiple times in an asynchronous script?
Let say I have a data set and I want to get the max value and then compare/subtract each row in the data set to that max value and add that as a new column - is that possible in the asynchronous script?
Basically I would need to run through the buffer once and pull out the value for the max, and then go through the buffer again pushing to the output buffer the row with the new column "DiffFromMax".
I already tried adding an asynchronous script to pull the max and put into a variable and downstream add a derived column which subtract from the variable but it doesn't work as the variable cannot be assigned till the postexecute() so its always too late.
I've tried having an asynchronous script that has 2 output, one containing the max and the other the rest of the data, however there is no way to subtract without spoofing a cross join which is really slow becuase of the sort required (I still can't believe msft rejected my request for adding a cross join, it had lots of votes and it should be easy to add... I'd code it myself if they let me have a script transformation with multiple inputs)
View 5 Replies
View Related
Jan 11, 2008
Hi,
I have stumbled on a problem with running a large number of SSIS packages in parallel, using the €œdtexec€? command from inside an SQL Server job.
I€™ve described the environment, the goal and the problem below. Sorry if it€™s a bit too long, but I tried to be as clear as possible.
The environment:
Windows Server 2003 Enterprise x64 Edition, SQL Server 2005 32bit Enterprise Edition SP2.
The goal:
We have a large number of text files that we€™re loading into a staging area of a data warehouse (based on SQL Server 2k5, as said above).
We have one €œmain€? SSIS package that takes a list of files to load from an XML file, loops through that list and for each file in the list starts an SSIS package by using €œdtexec€? command. The command is started asynchronously by using system.diagnostics.process.start() method. This means that a large number of SSIS packages are started in parallel. These packages perform the actual loading (with BULK insert).
I have successfully run the loading process from the command prompt (using the dtexec command to start the main package) a number of times.
In order to move the loading to a production environment and schedule it, we have set up an SQL Server Agent job. We€™ve created a proxy user with the necessary rights (the same user that runs the job from command prompt), created an the SQL Agent job (there is one step of type €œcmdexec€? that runs the €œmain€? SSIS package with the €œdtexec€? command).
If the input XML file for the main package contains a small number of files (for example 10), the SQL Server Agent job works fine €“ the SSIS packages are started in parallel and they finish work successfully.
The problem:
When the number of the concurrently started SSIS packages gets too big, the packages start to fail. When a large number of SSIS package executions are already taking place, the new dtexec commands fail after 0 seconds of work with an empty error message.
Please bear in mind that the same loading still works perfectly from command prompt on the same server with the same user. It only fails when run from the SQL Agent Job.
I€™ve tried to understand the limit, when do the packages start to fail, and I believe that the threshold is 80 parallel executions (I understand that it might not be desirable to start so many SSIS packages at once, but I€™d like to do it despite this).
Additional information:
The dtexec utility provides an error message where the package variables are shown and the fact that the package ran 0 seconds, but the €œMessage€? is empty (€œMessage: €œ).
Turning the logging on in all the packages does not provide an error message either, just a lot of run-time information.
The try-catch block around the process.start() script in the main package€™s script task also does not reveal any errors.
I€™ve increased the €œmax worker threads€? number for the cmdexec subsystem in the msdb.dbo.syssubsystems table to a safely high number and restarted the SQL Server, but this had no effect either.
The request:
Can anyone give ideas what could be the cause of the problem?
If you have any ideas about how to further debug the problem, they are also very welcome.
Thanks in advance!
Eero Ringmäe
View 2 Replies
View Related
Oct 18, 2005
Hi, I'm building a portal for supervisors can view their agents errors. We have a team that have been using Access to import data and QC it. I want to get these records into SQL but at the same time, the agents need to import their own work. I've create a DTS package that will import the data into the SQL table from a folder the agents have access to. I tried to create the strored proc to be run from the ASP page however it's not working and I'm having a hard time figuring out why. Here is the procCREATE PROCEDURE DTS
AS
exec master..xp_cmdshell 'dtsrun /KEMTSQL02 /ADJUSTMENTS /E'GODoes anyone know what I'm doing wrong? Any help would be great. Thanks!
View 1 Replies
View Related
Apr 4, 2007
on the server i am planning to run my packages at runtime using sql server agent...i remove the text files from exisiting location(for eg C to different location(D.when ever i receive a file at c:
I want my package to run.
How can i set this up in the sql server agent?????how will the sql server agent know that this package had received a file.
or is there anyway i can run my packages automatically other than sql server agent
please help
View 6 Replies
View Related
Mar 29, 2007
Hi,
when call a sub-package from the parent package with the executeoutofprocess=true I have about 5-10 sec delay before sub-package starts running. ( this is a big delay for me as the sub_package is in a foreach loop)
the sub_package itself it's very simple and contains just one script task with a small script in in it.
even if this delay is for validation I still can't understand why it is taking 5-10 ses.
cheers
View 3 Replies
View Related
Mar 31, 2006
Hi everybody,
I have to create a package that executes other packages I've already created... Every one of these packages run with the same Configuration File, but if I try to execute the main one, including the path of the file, I get errors from the other packages because they can't find it... How can I manage to pass this file and its content to the other packages??
Here a little explanation of te process:
Main Package needs configuration file X.dtsConfig, and calls: package1, which needs configuration file X.dtsConfig; package2, .......
I hope everything is clear...
View 9 Replies
View Related
Jul 16, 2007
hi all!
i have two packages, one that adds data to a Fact Table (package1) and one that updates the dimension tables (package2). i want package1 to run all day, as it waits for incoming csv files and i want it to stop every night at 11pm and run package2, which updates dimension tables. then package2 should stop at say, 1am and package1 would run again. i tried setting the Daily Frequency schedule of package1 to occur every 24hours starting at 3pm and ending at 3:15pm, just to test if it will stop at 3:15pm, but it didnt. how can i create a job schedule that will perform the behavior? thanks a lot!
View 1 Replies
View Related
Sep 20, 2006
Is there any way to check from SQL code if a specific SSIS stored package is running like a SQL system view or stored procedure? Management studio option works fine but not if you need to determine if a specific package already is running or not?
View 3 Replies
View Related
Dec 21, 2006
I need to execute a long running package (it takes about 16 hours to finish) to load a data warehouse for the first time with all historical data. This package it's a master package and execute other packages; I log the start time and the finish time of the package in a table to manage future incremental loads.
I executed the package on sql server where it is saved, but after 8 hours it was running, a new package was started automatically. Then two more packages started .. each every two hours.
I set the MaxConcorrentExecutable = 4, this could affect this strange behavoir ?
Anyone could imagine wath happened ?
Thanks
Cosimo
View 3 Replies
View Related
Nov 7, 2007
Before I decided to post this thread, I searched the forum and saw that I'm not the only one going crazy with running SSIS packages in a job agent.
My package has some warnings ('truncation may occur due to inserting data from data flow column 'xxx' with a length of 50 to a database column with a length of 20') but it runs correctly in VS (development).
If I run it from dtexecui, it runs correctly...
If I configure my job as a SQL Server Integration Services Package I get this error in History
Started: 3:24:12 PM Progress: 2007-11-07 15:24:13.19 Source: Package1 Validating: 0% complete End Progress Progress: 2007-11-07 15:24:13.19 Source: Load T_TITLE Validating: 2% complete End Progress Progress: 2007-11-07 15:24:13.20 Source: Load T_TITLE Validating: 4% complete End Progress Progress: 2007-11-07 15:24:13.20 Source: Load T_TITLE Validating: 6% complete End Progress Progress: 2007-11-07 15:24:13.22 Source: Load T_TITLE Validating: 8% complete End Progress Progress: 2007-11-07 15:24:13.23 Source: Load T_TITLE Validating: 10% complete End Progress Progress: 2007-11-07 15:24:13.23 Source: Load T_TITLE Vali... The package execution fa... The step failed.
If I change to Operating System (CmdExec) type this is my error output:
Started: 3:31:20 PM Progress: 2007-11-07 15:31:22.04 Source: Load T_TITLE Validating: 0% complete End Progress Progress: 2007-11-07 15:31:22.05 Source: Load T_TITLE Validating: 2% complete End Progress Progress: 2007-11-07 15:31:22.07 Source: Load T_TITLE Validating: 4% complete End Progress Progress: 2007-11-07 15:31:22.07 Source: Load T_TITLE Validating: 6% complete End Progress Progress: 2007-11-07 15:31:22.08 Source: Load T_TITLE Validating: 8% complete End Progress Progress: 2007-11-07 15:31:22.68 Source: Load T_TITLE Validating: 10% complete End Progress Progress: 2007-11-07 15:31:22.68 Source: Load T_TITLE Validating:... Process Exit Code 1. The step failed.
Is there a way to see a more meaningful output?
SSIS really frustrates me in the last couple of days...
View 15 Replies
View Related
Jun 18, 2007
Hello,
I have a sql 2005 cluster. I recently moved a new server in place of an old server we had
Both the servers now have w2k3 sp2 sql 2005 sp1 both have 2000 dts designer package.
My problem is that i have a dts package that runs on sql01inst01 without any issues but the same package when i try to run it from sql02inst02 just hangs ... digging deeper i found that - when i try to parse a query / build query / try giving parameters in a Execute SQL task it says
"The selected connection does not provide sql support"
The connection it reference is a postgres connection looking up info from a udl file.
I am not sure what is missing.
Any help is appreciated as soon as possible...
View 3 Replies
View Related
Apr 5, 2007
Hi All
I have a meaty job that currently runs as a set of sql scripts under sql agent and I'm transferring this job to run as a SSIS package. I have managed to set it up so that it runs as a sql job (cheers guys!), and I'm watching the job run now, maxing out the CPU usage as it tries to run 10 different tasks at the same time . great stuff!
However, I've been thinking about how to view the progress of this job/package (the job only has one step; "run SSIS package"). With the current job as single scripts, in SQL agent you can view which step of the job you are at.... is it possible to somehow view which package task you're at (ideally with nice shiny yellow green and red updates like in debugging), or do you have to output each step result into a log file?
Oh, and another question! If i have a checkpoint in a package, and the package falls over, when the job is rerun through SQL agent, will it start at the checkpoint, or run from the start?
Thanks!
michal
View 10 Replies
View Related
Sep 5, 2006
When I run an SSIS package from the MSDB in SQL Server Management Studio it never shows up under 'Running Packages'.
I have confirmed that the package is indeed running.
I've also tried to check the packages running on the server programatically by using the GetRunningPackages() method. This too returns nothing when clearly packages are running.
Any ideas?
Malcolm Barclay
Disney Consumer Products UK
View 4 Replies
View Related
Mar 19, 2007
I have created a master controller package which runs as follows
deletes all the log files -> deletes few flat files on different drives -> preprocess task(execute package task) -> c# executable (execute process tasks) -> postprocess tasks (execute package tasks)
i need a create a task just before the preprocess task with an user input asking whether he wants to run a particular batch file before proceeding to preprocess. if the user says yes it should run a batch file followed by preprocess tasks, c# and post process or else it should directly goto preprocess, c# and post process (neglecting batch file task)
can anyone help me how to do it.
View 9 Replies
View Related
Apr 18, 2006
Hi there,
if I have some packages (.dtsx) that are depolyed to Server A and I have SQL Agent installed on Server B - if I run a run a package from SQL Agent, does it run on Server A or Server B?
Thanks in advance.
View 1 Replies
View Related
