Integration Services :: Post To A Web Service Using SSIS
Sep 2, 2015
I have a table is SQL server database A that is my source.
I have another database B which is accessed via webservice call.(its a CRM server basically).
My intention is to transfer data from A to B while B is accessible only via web service. I need to Trasnfer the data from the source database A to the destination Database B by calling the webservice.
Is there a way where I can retrieve whole set of rows in source table in preexecute(), And transfer the data to Database destination B by calling the webservice?
View 18 Replies
ADVERTISEMENT
Oct 6, 2006
Hi,
I'm hoping someone has tried something similar before. I am trying to run an integration from a remote computer using a web service. I have set up the web service and the bog standard launch package routine works fine. What i'd like to be able to do is to launch a package and also specify any parameters associated with it. I have the following code which is identcal to the main launch sub with the exception of an extra argument (jagged array of varible name and value variablePair[4][2]). It also processes the variable array and sets them on the DTSPackage object.
[code]
[WebMethod]
public int LaunchSSISPackageWithVariables(string sourceType, string sourceLocation, string packageName, string[][] variableArray)
{
string packagePath;
Package myPackage;
Application integrationServices = new Application();
// Combine path and file name.
packagePath = Path.Combine(sourceLocation, packageName);
switch (sourceType)
{
case "file":
// Package is stored as a file.
// Add extension if not present.
if (String.IsNullOrEmpty(Path.GetExtension(packagePath)))
{
packagePath = String.Concat(packagePath, ".dtsx");
}
if (File.Exists(packagePath))
{
myPackage = integrationServices.LoadPackage(packagePath, null);
}
else
{
throw new ApplicationException("Invalid file location: " + packagePath);
}
break;
case "sql":
// Package is stored in MSDB.
// Combine logical path and package name.
//if (integrationServices.ExistsOnSqlServer(packagePath, ".", String.Empty, String.Empty))
if (integrationServices.ExistsOnSqlServer(packagePath, ".", "executeSSIS", "p4ssw0rd"))
{
//myPackage = integrationServices.LoadFromSqlServer(packageName, "(local)", String.Empty, String.Empty, null);
myPackage = integrationServices.LoadFromSqlServer(packageName, "(local)", "executeSSIS", "p4ssw0rd", null);
}
else
{
throw new ApplicationException("Invalid package name or location: " + packagePath);
}
break;
case "dts":
// Package is managed by SSIS Package Store.
// Default logical paths are File System and MSDB.
if (integrationServices.ExistsOnDtsServer(packagePath, "."))
{
myPackage = integrationServices.LoadFromDtsServer(packagePath, "localhost", null);
}
else
{
throw new ApplicationException("Invalid package name or location: " + packagePath);
}
break;
default:
throw new ApplicationException("Invalid sourceType argument: valid values are 'file', 'sql', and 'dts'.");
}
//Variables var = myPackage.Variables;
//foreach (string key in variablePairs.Keys)
//{
// var[key].Value = variablePairs[key].ToString();
//}
Variables var = myPackage.Variables;
for (int i = 0; i < variableArray.Length; i++)
{
var[variableArray[0]].Value = variableArray[1].ToString();
}
return (Int32)myPackage.Execute();
}
[/code]
For some reason the above code runs and passes back the value that it succeeded. Yet the package doesn't do what it should. Does anyone have any ideas as to why this might be the case or even a better way to pass the variables.
Many thanks,
Grant
View 5 Replies
View Related
Feb 20, 2007
Hi,
A coworker of mine is experiencing some problems with SQL Server Integration Services (SSIS) and long running Web Service calls. Any feedback on the problem would be greatly appreciated.
The problem is as follow:
He has a large SSIS-package that, among other things, contain a few Web Service Tasks. All of the tasks are executing as expected for small amounts of data. All the Web Service Methods have in common that they have a long running time. The running time depends on the amount of data.
During requests with large amount of data the Web Service task fails with the error message €œThe operation has times out€?, but on the server they see that the service call completes as expected (after quite some time, approx 200sec).
The Web Service tasks are using an €œHTTP Connection Manager€?, created in the SSIS package. The timeout-value is set to 300 sec (which is the largest value for the property). My coworker expected that this value was the same as setting the timeout value for a Web Service proxy object for any other project.
After testing the Web Service from a Console application, with 300 sec timeout, the app executed as expected with no timeout exception.
After doing some testing, he€™s quite sure that the SSIS task times out after 100 seconds, which is the default .NET 2.0 timeout value for a WS proxy, even though the timeout property in SSIS is set to 300 seconds.
As a work around he have created a console application that does the web service call, and then use the €œexecute process task€? in SSIS. This is off course a €œdirty€? hack, and he want€™s to use the €œHTTP Connection Manager€? task instead.
So the question is: Have anyone experienced this problem before? Is there a limitation on the HTTP Connection Manager task? Is the timeout value on the HTTP Connection Manager task the same as setting the timeout property on a WS proxy? Why isn€™t this €œtimeout€? value picked up my SSIS?
Again, thanks for any help!
View 5 Replies
View Related
May 11, 2012
So I have a Web Service that works just fine when I enter the parameter values myself. I get results and can parse it out, storing the results into a table. I even have it working in a "For Each Loop" for one of my variables.  The issue I have is that the Web Service accepts 3 variables, the first being an Array.  How can I populate the array variable for web service?
View 5 Replies
View Related
Oct 9, 2015
I want to achieve the following in (SSIS/SSDT for SQL 2012) -Â
I have a generic SSIS package which simply sends out email notifications using SMTP email task (this package is within its own project, and has project level input parameters).
I need to be able to call this package in the Event handler section of every package (numbering in about less than 60) that we have. These packages are within their own respective projects.
I thought I could use the "execute package task", but it turns out , using this, I cannot call a package that is part of some other project. I also cannot call a package that is stored in the CATALOG. Is there any way I can do this ?
When I call the child package , I should be able to send in parameters like - error information and package name of the Parent package.
View 8 Replies
View Related
Mar 19, 2008
We encountered an error the other day that was causing us major head-aches trying to resolve.
Error: 1053 -- Could not start the SQL Server Integration Services service
In addition to SSIS not starting up, we had errors trying to launch Management Studio.
The above errors started happening after we deployed a small release that included editing the machine.config file.
To make a long story short, make sure the machine.config file is well formatted and is not missing any double quotes (") or < /> .
Hopefully this information can help someone out.
View 1 Replies
View Related
Sep 16, 2015
We may need to change the account presently used to run the Windows Service "SQL Server Integration Services".What are the implications of making such a change?
View 5 Replies
View Related
Jun 5, 2015
I have to access a web service where I pass in a product number and it produces an XML result with an /image section where it has a long string (base64?) Â Â I am trying to call the web service and save the image to the OS.
I am trying to get the result into a variable and save it to disk with  the itemno.jpg as the name
This is a sample file I get back..
<NewDataSet>
 <Table>
  <ITEMNO>2065</ITEMNO>
  <Image>/9j/4QAYRXhpZgAASUkqAAgAAAAAAAAAAAAAAP/sABFEdWNr ...  removed since too long for post  ...</Image>
  <ImageDate>2015-04-15T00:00:00+00:00</ImageDate>
 </Table>
</NewDataSet>
View 9 Replies
View Related
Oct 5, 2011
When you pass a complex type (the one represented by class) to a web service the BIDS UI allows you to enter values for every field of that type as constants. But what if you want to pass a variable? Once again the UI allows you to specify a variable for that complex type parameter. But how to make this variable in SSIS?I understand it should have the type of Object. But how to specify what the runtime type of this object is? And how to assign all fields to that object?
View 6 Replies
View Related
Jan 16, 2008
I need to read in general web pages (not a web service) from a typical web site using SSIS and make it available for other SSIS transformations (Script Component). I tried using the XMLSource data source but this appears to require well formed XML, and will not accept HTML which is what I am likely to be getting from the web pages.
I tried a HTTP Connection Manager with a DataReader Source, but seems to only accomodate web services.
Can this be done? If someone has an example (tutorial) of how to accomplish this I would greatly appreciate a copy.
James
View 1 Replies
View Related
May 2, 2006
Please help me with editing a XML file through script tack in SSIS (Integration Service) 2005
Thank you
View 1 Replies
View Related
Jul 25, 2007
Hi,
I want to insert datas from a txt-file into a sql-table.
Therefor i would use a xml-file for the structure!
How can i refer this xml-file to a measurement insertion task?
Tanks for your help and sorry for my bad english :rolleyes:
View 1 Replies
View Related
Oct 23, 2015
I want to use UDF in SSIS where function accept 4 parameter value and return 1 value.
How can i achieve that. I need workflow.
View 13 Replies
View Related
Apr 30, 2015
How to achieve the below condition in ssis
if person_id is like '123%' or '124%'
then details='xyz'
else details='yxz'
View 7 Replies
View Related
Oct 20, 2015
if there is any way to accurately size a single server using SSIS. The server will be a virtual machine. The data being loaded will be approximately 200 MB per load with loading to a 150 GB database on a separate server.
View 3 Replies
View Related
Aug 31, 2015
I have scheduled SSIS package through Sql Agent and when I right click on job start job as step package runs successfully but when I schedule job it dosent run.
View 12 Replies
View Related
Nov 14, 2007
Can anyone help regarding the SQL server integration Services(SSIS), ETL
We have requirement like this:
We have Live Database( LIVE_DB ) and Reports Database (REP_DB)
I want to trasfer the few tables data from LIVE_DB into the REP_DB for end of the day using SSIS
If any new records are added, updated or deleted in LIVE_DB, these should reflect in the REP_DB, Our requirement is not to delete the old data, we should append or delete or insert the new transaction data in REP_DB.
Thanks in advance, if anyone help me in resolving this issue.
Regards,
Bhushanam.
View 3 Replies
View Related
Apr 20, 2015
I have a maintenance plan which consist db full backup and log backup ( in two subplans), I execute both on SQL agent, and both failed.The DB Log Backup : DB FULL BACKUP LOG:
View 14 Replies
View Related
Feb 21, 2007
After upgrading to SP2 straight from from SP1, I get the message that my current reporting server instance cannot be configured for Sharepoint Integration when I click on the Sharepoint integration tab. Also, I do not get the option of choosing Sharepoint integration if I try to create a new database in the Database Setup tab. I've read that the new Reporting Services Configuration Manager is not compatible with an older instance of Reporting Services, and realize that might be the problem. But I haven't found how to get around it! Do I need to uninstall SSRS and start over? I'd really like to switch to Sharepoint integration, if at all possible.
Another funny thing: I thought I might try re-installing SP2 to rectify the above problem, and it sees that SSRS is the only thing that needs upgrading. When you continue, it says SSRS was successfully updated, but nothing's changed. If you re-run SP2, SSRS is chosen to be upgraded again! I've installed SP2 3 times now, and each time it says SSRS is needing upgrading.
Has anyone else seen this problem?
Thanks,
Matt
View 16 Replies
View Related
Oct 8, 2007
We implemented a service broker application last October, 2006 and until recently everything has worked flawlessly. In September, 2007, we applied SQL SP2 and shortly thereafter we've experienced periodic disruptions!?
In the past, when issues surfaced within the SB plumbing, it would typically break (disactivate) the que, whereas the activated stored procedure could not successfully process the message. Now however, there appears to be an issue within the sys.transmission_queue, as items begin to periodically build up here (several thousand at a time) on both ends, in our remote SQL server as well as local SQL server. This time though, the queue remain enabled, activation enabled, etc., however messages don't get delivered into the target que, and hence never processed - and now we receive the following entries within SQL profiler:
This message was dropped because it could not be dispatched on time. State: 2
This message could not be delivered because the conversation endpoint has already been closed.
The only way we can seem to get this resolved is to restart the actual SQL service, which obviously poses significant issues within our production database. There doesnt seem to be any pattern we can detect, happens randomly during the week, the sys.transmission_queue simply starts filling up and transactions wait to be processed until the restart. I've had to temporarily put a listener on the transmission_queue to alert me when transactions get backed up so I may manually reset. The hardware on both ends is 64bit windows 2003 enterprise, SQL 2005 64bit SP2, 8 CPUs and 8gb of RAM.
Again, things worked flawlessly prior to SP2; now this issue surfaces about 4-6 times per week requiring a restart of the SQL service to recover. I have struggled to find others experiencing similar issues and would greatly appreciate any assistance you can lend to help me get this resolved.
Regards,
Steve
View 3 Replies
View Related
Oct 25, 2007
Remus - please help! Per your instructions on post: http://forums.microsoft.com/msdn/ShowPost.aspx?postid=2243476&siteid=1 I've applied CU3 in hopes it would solve our issue. While it delayed the occurance of the issue, it did not resolved it!? I applied CU3 on both servers and the issue continues to surface every few days, vs. multiple times a day.
Brief background (full description is on previous post) - our SB framework has been working flawlessly for over 1 year, after applying SP2 we began having transmission issues between servers. The queues themselves are not breaking, activation is enabled on both sides, yet the sys.transmission_queue on both sides periodically begins to backup. The following errors are surfaced using SQL Profiler: This message could not be delivered because it is a duplicate
The only way we have found to resolve the issue is to restart the SQL service on either end, obviously disrupting our production environment. I've been struggling with this for over a month now, after applying SP2 in early september. Can you please help me shed some light on this, or give me a referral of someone I can contact about this - I desperately need to get it resolved!!
Once the service is restarted, these messages are logged in the profiler:
1.) This message could not be delivered because the conversation endpoint has already been closed
2.) This message was dropped because it could not be dispatched on time. State: 1.
Since several days go by in between occurances, is there somekind of logging I should enable to capture traffic that may help to diagnose the issue?? Are there any SB tables I should be checking that may indicate trouble? Its just odd to me that things ran for so long without disruption, and literally within a day of applying SP2 things began breaking.
Looking forward,
Steve
View 1 Replies
View Related
Jun 17, 2015
I have a ssis package with an oledb connection using windows authentication. i want to understand when i promote this package to the server and add it to a job, then a user login to the server with sql server authentication and run this job. Which/what  windows authentication this package gonna to use to connect to the server ?
View 3 Replies
View Related
Oct 7, 2015
converting seconds to HH:MM:SS using SSIS. I know how to do it in T-SQL.
View 13 Replies
View Related
Jun 8, 2015
I have installed VS community edition 2013 on a widows 7 PC. When I try to create a ssis package the ssis tool bar comes up empty.It just says "loading" on the panel .
View 2 Replies
View Related
Nov 21, 2015
Win 7 SP1 x64 PC. I installed SQL Server 2014 Dev Edition + Visual Studio 2015.
I'd like to create some basic ETL SSIS packages, and I worked very comfortably in 2008R2.
For 2014, I started with this tutorial:[URL]However, it says to go to Start->All Programs->Microsoft SQL Server->SQL Server Data Tools.Â
I did explicitly install SSDT when I installed VS2015. I also installed it separately. I see SSDT listed in Programs, and SSIS is running according to SQL Server Config Manager, and Services. Half of Microsoft's docs seem to be 2012 era, which is a shame because 2014 is out and it's nearly 2016...
how do I get to the GUI where I can design ETL packages?Â
View 7 Replies
View Related
Sep 25, 2015
We'll be using 2014 enterprise to populate some excel spreadsheets. We may not have the option of using ssrs so here is the question.
The excel spreadsheets have some pretty fancy heading structures, multiple tabs and in a few cases graphs that i'm sure are dependent on data sitting somewhere in the spreadsheet. The heading have variable info in them (eg FYxx where xx is a parameterized fiscal year, month names etc). Sometimes those headings have sub headings (with variable info) and so on.
Generally speaking how (if at all) do folks deal with this kind of excel challenge when limited to using ssis? I don't know if templates are going to be a good idea. I don't know if i'll be looking at a fair amount of c# etc code behind the scenes. I can be more specific depending on the feedback.
This is part of a conversion from 2 of the larger BI and statistical tools out there to the MS sql stack. Obviously with some shortcuts that we may wish we didn't take. Â
View 5 Replies
View Related
Apr 19, 2015
 we have a table with xml column. This column has a large xml data . I am trying to use ssis to import xml from sql column (table a) to destination (another table).
steps which i did in ssis:
1. Â execute sql task:
  fetch the xml column by query and store "full result set" into an object variable.
2. foreach loop:
select Ado enumerator option and select variable which has reset set of execute sql task. In variable mapping selected a new variable of type string.
when I run package I get below error:
"Error: ForEach Variable Mapping number 1 to variable "User::variable" cannot be applied".
View 14 Replies
View Related
Jun 23, 2015
Have Visual Studio 2008 R2 with SP 2 installed. Due to a merger we now have a MySQL database that we need to update from SSIS. Everything works except for the table insert or update. Would upgrading to SP 3 or SP 4 maybe useful with that?Â
We have installed the latest driver from MySQL. Have tried the ADO.Net and ODBC drivers with similar results when we try to update the database.
View 7 Replies
View Related
Apr 29, 2015
I have a ETL ( SSIS ) Process in which i am loading around 150 tables in each run. ( Truncate and Insert ). I have four packages each from different sources. ( Each package loads different tables and different numbers )These are run on weekly basis one after the other. Each package is taking around 60 to 90 minutes each. Now i want to track the progress of the ETL on my front End application.Â
We want this in two ways.Â
First Way : I need to show the user how much percent of ETL Process is completedÂ
Second Way : I need to show the No of tables completed and how many rows have been completed in the ongoing table ( which is in process )
how to design the table and ssis process.
View 3 Replies
View Related
Oct 5, 2015
Im newto SSIS. I want to develop package for data validation.
FirstName
1. Mandatory  field checking: if Null, reject the record
 2. If field length > 50, then reject the record
SSN
1. Â If field length > 12, then reject the record
2. If SSN is not in valid format, issue warning and process rhe record  without SSN value.
3. Valid format: 9 digit numeric values should present after striping off  all non-numeric characters.
4. Only send 9 digits to MDM
Like these i have 30 rules. And I have to shop the error msg if the validation fails like "Mandatory feild is missing".
View 2 Replies
View Related
Jul 15, 2015
How to use derived columns in SSIS
string to datetime
Input value : (string)
14/03/2014
NULL
15/04/2015
Note : Having null or blank value
View 4 Replies
View Related
Jun 4, 2015
I'm unable to find a solution to this truncation error on google.This happens only on one field which has comments. The offending Excel row/column has text that was entered in two lines i.e they entered the data and pressed "enter" and wrote a new line in the same row.Im using an Excel file source in SSIS and an OLEDB Destination (SQL Server) but one column keeps erroring out and I have tried to do the following:
1) Change output column width in advanced editor (still errors)
2) Data conversion tool between the source and destination (still errors)
View 4 Replies
View Related
Jun 23, 2015
I have a below JSON script. I need to run the below script, fetch the json files and parse them in SSIS.
var client = new Keen({
 projectId: "<PROJECT_ID>",
 readKey: "<READ_KEY>"
[code]....
I tried using script component, but unable to find how to capture these JSON.
View 5 Replies
View Related
