I notice that when deploying a package that uses data sources behind its connection managers, the deployed package has the connection string built within the package that gets deployed.
View 6 RepliesI had posted this in the following pre-existing thread :
http://forums.microsoft.com/MSDN/ShowPost.aspx?PostID=89737&SiteID=1
Also wanted to start a thread on this to see if I can get some suggestions. Any suggestions on how to do this are appreciated.
--------------------------------------------------------------------
Books-Online quote:
>> In Integration Services, data sources are design-time objects that you use when you build packages in SSIS Designer.
This is indeed the case but why? I would like to see it changed or provide some way of including the data source in some kind of a "solution/project configuration", the same way one can create a "package configuration" right now.
If one can build a project, and create a deployment utility to deploy the entire project (all packages), it is incoceivable why one shouldn't be able to set configurations at a "project" level.
Right now I am having to create package configurations to configure my connection setting for dev/test/prod etc environments. But the trouble is, if I have 10 packages each involving 3 connections in my project, I have only 3 data sources in my project, but come deployment time, I'm having to create 10 package configurations, and set my ConnectionString (which is different for dev/test/prod etc) 30 times during deployment.
If there is a better way to do it, I would like to know. Otherwise, this is what I would like to see in the next SP.
------------------------------------
thanks,
RV
We have an SSAS project that we want to auto deploy. I am not sure how to handle the external data sources in the project. This one in particular has a single external data source defined to Microsoft SQL Server.
I would like to be able to change the data source based on the environment. In SSIS projects I can do this by setting up environments in the SSISDB and linking them to project parameters in the SSIS project but SSAS projects don't seem to have a similar mechanism.
How to handle this? I would like to be able to have the build/deployment agent pass in server / database information to the data source based on environment (dev, QA, production).
The only way to automate this that I've discovered is to have an intermediary process that executes after the build that updates the generated .asdatabase and .configsettings files in the bin folder replacing the connection string information.
HI,
I have a query regarding the order of deployed packages in SSIS (MSDB)
I have four projects in my solution (SSIS) and i need to deploy and test them frequently, whenever i deploy them, they are deployed in no particular order what so ever. Even i created some folder in MSDB (for each project) but coudnt able to deploy project in its corresponding project . I have read something that you can do in it XML configuration manually but this does not solve my problem as I have to deploy them very frequently (it will be a hard task to change XML every time I deploy packages).
Is there ne solution of it? or MS has left this alone like this?
MS!!! it shud be, atleast if not more, in alphabetical order. Did I say ne thing wrong? or i dont hv sufficient knowledge about it?
regards
Hi, all experts here,
Thank you very much for your kind attention.
I am having a question on the deployment of the packages to integration services server. Why after I have deployed the whole solution, but my packages are not actually deployed to the integration services server?
Also, after we add the package manually to the integration services server, to users outside of local host, how could we add permissions for them to execute the packages?
Thanks a lot in advance for your help and I am looking forward to hearing from you shortly.
With best regards,
Yours sincerely,
I have 2 servers. DWTEST and DWSQL1. I developed my packages connecting to DWTEST in the design time. I have 32 packages, one of them is the master one and the others are child packages.
package PM calls P1, P2, P3,.......P31
i have 2 data source in my project. RDS and S0, both setup to connect DWTEST in the design. i defined rds_connection_string and s0_connection_string variables in my master package and created an XML config file and put those variables into that config file.
C:PROJECTSDEVELOPMENTSSIS ConfigurationsSERVER.dtsConfig
<DTSConfiguration>
- <DTSConfigurationHeading>
<DTSConfigurationFileInfo GeneratedBy="TC sql" GeneratedFromPackageName="PM" GeneratedFromPackageID="{CBE946D0-279F-44F6-80A3-83D056C1E36C}" GeneratedDate="11/10/2006 10:50:44 AM" />
</DTSConfigurationHeading>
- <Configuration ConfiguredType="Property" Path="Package.Variables[User::rds_connection_str].Properties[Value]" ValueType="String">
<ConfiguredValue>Data Source=DWTEST;Initial Catalog=RDS;Provider=SQLNCLI.1;Integrated Security=SSPI;</ConfiguredValue>
</Configuration>
- <Configuration ConfiguredType="Property" Path="Package.Variables[User::s0_connection_str].Properties[Value]" ValueType="String">
<ConfiguredValue>Data Source=DWTEST;Initial Catalog=S0;Provider=SQLNCLI.1;Integrated Security=SSPI;</ConfiguredValue>
</Configuration>
</DTSConfiguration>
built the project into directory C:PROJECTSDEVELOPMENTS1_REUSABLE_DATA_STORES1_REUSABLE_DATA_STOREinDeployment
when i deploy my packages into DWTEST and run them as a SQL Server Agent job they work because data sources are signing to DWTEST already.
when i am deploying them(using the same manifest file in the deployment directory i wrote above) into the server DWSQL1, i select config file source as:
C:PROJECTSTESTINGSSIS ConfigurationsSERVER.dtsConfig
and update data source strings in it to DWSQL1:
- <DTSConfiguration>
- <DTSConfigurationHeading>
<DTSConfigurationFileInfo GeneratedBy="TC sql" GeneratedFromPackageName="PM" GeneratedFromPackageID="{CBE946D0-279F-44F6-80A3-83D056C1E36C}" GeneratedDate="11/10/2006 10:50:44 AM" />
</DTSConfigurationHeading>
- <Configuration ConfiguredType="Property" Path="Package.Variables[User::rds_connection_str].Properties[Value]" ValueType="String">
<ConfiguredValue>Data Source=DWSQL1;Initial Catalog=RDS;Provider=SQLNCLI.1;Integrated Security=SSPI;</ConfiguredValue>
</Configuration>
- <Configuration ConfiguredType="Property" Path="Package.Variables[User::s0_connection_str].Properties[Value]" ValueType="String">
<ConfiguredValue>Data Source=DWSQL1;Initial Catalog=S0;Provider=SQLNCLI.1;Integrated Security=SSPI;</ConfiguredValue>
</Configuration>
</DTSConfiguration>
When i run SQL Server Agent Job i defined in DWSQL1(forcing it to use DWSQL1 data source in step definition), my master package PM connects to DWSQL1, and my child packages don't use my config definitions and connect to DWTEST which is defined in the design time.
How can i force all my packages to connect DWSQL1 server using the same built files in deployment directory WITHOUT editing my packages(recreating config files or changing the definition of Data Sources and then built the project again) in Visual Studio?
Thanks to you for reading my long question...
Hi:
I am trying to utilize DTUtil command to automate deployment of SSIS Package from the Build Server to Dev Server. I am getting an invalid option error, when I try to run the following command:
dtutil /FILE C:Program FilesABCABC ABC CADETL PackagesLoad_Staging_FromWCF.dtsx /MOVE FILE;\serverAC$Program FilesABCABC ABC CADETL PackagesLoad_Staging_FromWCF.dtsx /QUIET
Currently I have a .CMD file, in which I have the above command, and I am trying to run the CMD file and it returns an error. Can someone help me the DTUtil switches. My goal is to move a DTSX file from Server A(Build Server) to Server B(Dev Server). Also is there a switch to specify the server name, if UNC path is not supported. Also I am using File System Deployment, and running the file as an Administrator. I have admin privs on both Build and Dev Server. Thanks and I appreciate some help on this.
Hi ,
I'm deploying the packages to SQL Server using the deployment utility. It goes through. However i wish to organize the packges into folders on the SSIS Server -> MSDB.
I'm able to create folder under the MSDB node. By default "Maintenance Plans" exists. But while deploying the package to the server i'm unable to specify the folder into which the package to be deployed.
Is this a known gap in the deployment wizard or am i missing something?
MTIA.
Hi All,
I have created a solution which contains only 2 packages say Package1.dtsx and Pakage2.dtsx. I want to create a deployment utility to deploy onto other developers machince. I changed the project properties "CreateDeploymentUtility" to TRUE. When I do the build it is not creating the files in "Deployment" folder. It is saying Rebuild All Failed but the error is not showing.
For more information the 2 packages have 4 indirect configurations from environment variables which are storing the actual config file path.
Am I missing something here?
Thanks.
Venkat.
I need to periodically import a (HUGE) table of data from an external data source (not SQL Server) into SQL Server, with the following scenarios:
Some of the records in the external data source may not exist in SQL.Some of the records in the external data source may have a different value at different imports, but this records are identified univocally by the same primary key in the external datasource and in SQL Server.Some of the records in the external data source may be the same in SQL.
Due to the massive volume of the import, I would like to import only the records which are different from what I have in SQL Server (cases 1 and 2 above). In fact case 2 is the most critical.
I thought of making a query with a left outer join between the data in the external data source table (SOURCE) and the data in the SQL Server table (DESTIN). The join is done on the respective primary keys (composed keys of up to 10 columns) and one of the WHERE conditions will be that the value in SOURCE is different from the value in DESTIN.
The result of this query would be exactly what I need to import.
How to do this in SSIS??? I couldn't figure out how to join tables in different data sources yet.
In fact I cannot write a stored procedure to do that, since one of the sources is in a datasources not SQL Server.
I have seen the Lookup transformation in this article http://www.sqlis.com/default.aspx?311 but this is not exacltly what I want to do.
Another possibility is to use the merge join, but due to the sorting I believe its performances would be terrible!
Thanks in advance for your suggestions!
Hi,
I am pretty new to SSIS. I am trying to create a package which can accept data in any of several formats. i.e. CSV, Excel, a SQL Server database/table and import the data into my destination database.
So far i've managed to get this working OK. However I am now TOTALLY stuck. I'm currently trying to just concentrate on the data sources being a CSV (using a Flat File Data Source) and/or an Excel Spreadsheet.
I can get the data in and to my destination using a UNION ALL component and mapping the data sources to it so long as both the CSV file and the Excel spreadsheet exist.
My problem is that I need my package to handle the possibility that only the CSV file might exist and there is no Excel spreadsheet. In which case i'd like the package to ignore the Excel datasource completely. Currently either of my data sources do not exist I get errors and the package terminates.
Is there any way in SSIS that I can check all my data sources to see which ones exist (i.e. are valid). If they exist I want to use them. If it doesn't exist i'd like to disgard it (without error - as long as there is a single datasource the package should run)
I've tried using the AcquireConnection method in a script task on each of my connections, hoping that it would error if the file/datasource did not exist. It doesn't though (in the case of an Excel datasource it just creates a empty excel file for me).
The only other option I can come up with are to have seperate packages depending on the type of data we want to import and then run a particular package depending on the format of the source data. This seems a bit long winded. I am pretty sure I must be able to do what I want to achieve but I can't work out how.
I'll be grateful to anyone who can send me any tips/hints/links on how I can achieve this.
Many thanks
Rob Gibson
Hi, i'm wondering which is the best way to search data in a SQL Server.
I reach data using Data Sources and Data Views and also with OLE DB Source with a Data access mode: Named query.
I have to write the data into a Flat File. So, does any one knows which is the best practice for this? Or any one of the two are good choices?
Thanks for your help.
Beli
Hi,
In my project i want a report. In that report data is getting from more than one data sources(systems). While creating data source view i used named query for both primary and secondary data source. But at the time of crating "Report Model" i am getting below error.
An error occurred while executing a command.
Message: Invalid object name 'Table2'.
Command:
SELECT COUNT(*) FROM (SELECT SerialNum, ModelNum AS com_model
FROM Table2) t
Is there any way to create a report with multiple data sources?
Hi all :)
I'm wondering if SSIS will be the solution to the problem I'm working on.
Some of our customers give us an Excel sheet with data they want to insert or update in the database.
I've created a package that will take an Excel sheet, do some data conversion so the data types match up and after that I use a Slowly Changing Data component to create the insert/update commands.
This works great. If a customer adds a new row to the Excel sheet or updates an existing row changes are nicely reflected in the database.
But now I€™ve got the following problem. The column names and the order of the columns in the Excel sheet are not standard and in the future it could happen a customer doesn't even use an Excel sheet but something totally different.
Can I use SSIS for this? Is it possible to let the user set the mappings trough some sort of user interface? I€™ve looked at programmatically creating the package but I€™ve got to say that€™s quit hard to do€¦ It would be easier to write the whole thing myself than to create the package trough code ;)
If not I thought about transforming the data in code before I pass it on to the SSIS package in something like XML. That way I can use standard column names and data types.
So how should I solve this problem? Use SSIS or not?
Thnx :)
Wouter de Kort
Hi i have data on a Server in a different database which i like to access from within my ssis job.
I just need to look up information from one table on this database so i can references it. Is there a way of doing this is SSIS. Rather then me having to load the data from one database to another as the data may change.
i tried having 2 sources of data feed into a look up but that does not work..
Hi there,
On my home page I have several different folders to reports which require different data sources. the problem is that within these folders there are multiple copies of the same datasource. is it possible to store all of the datasources in one folder, one location? it would certainly be easier when changes to usernames and passwords need to be modified!
Thanks,
Rhonda
I searched and read about Data Sources and I'm seeing that there is no advantage in using it, which is what I found from playing around with it.
I expected that you would set a global connection in Data Sources and somehow link this to the things in your Connection Manager, giving you one place to switch from one environment to another. But reading the discussions here and playing around with it, this is not the case.
So, why is it there?
Next question.... another thing I gathered so far is something called "Configurations" that will do what I was describing above. Where do I do this?
Hi,
how can i use var for Data Sources ,
my project have to be suitable for the QA system and the production system and i want to connect them by var or something else.
if you have another idea about this subject i'll happy to read.
thanks,
gil.
I'm trying to combine two reports I've created into a form letter. The goal is to have a letter that indicates the students maximum UBSCT score for Math, Reading and Writing. Also in the letter we want to include the student's total number of credits awarded and overall GPA. I not sure if I need to try and combined the two queries or use two different data sources in Reporting Services. I've tried creating one sql statement but the grouping are giving me fits. I fairly new to Reporting Services.
Report 1:
704 Dixie High 11 311880 Student_1 UBSCT Math 176
704 Dixie High 11 311880 Student_1 UBSCT Reading 182
704 Dixie High 11 311880 Student_1 UBSCT Writing 173
704 Dixie High 11 311881 Student_2 UBSCT Math 168
704 Dixie High 11 311881 Student_2 UBSCT Reading 172
704 Dixie High 11 311881 Student_2 UBSCT Writing 165
SELECT
track.schoolc as School_Code,
school.schname as School,
stustat.graden as Grade,
studemo.ident as Ident,
RTRIM(studemo.lastname) + ', ' + RTRIM(studemo.firstname) AS [Student],
stutscors1.testc as Test_Code,
left(zsubtest1.descript,20) as [Subject],
max(stutscors1.testscore) as Score
FROM
stutscors1 INNER JOIN
stutests1 ON stutscors1.ststuniq = stutests1.ststuniq INNER JOIN
studemo ON stutests1.suniq = studemo.suniq INNER JOIN
stustat ON studemo.stuuniq = stustat.stuuniq INNER JOIN
track ON stustat.trkuniq = track.trkuniq INNER JOIN
school ON track.schoolc = school.schoolc INNER JOIN
testdef1 ON stutests1.testuniq = testdef1.testuniq INNER JOIN
zsubtest1 ON stutscors1.subtestc = zsubtest1.subtestc and stutscors1.testc = zsubtest1.testc
WHERE
stutscors1.testscore <> ' ' and
stutscors1.testscore <> '0' and
school.schname = 'Dixie High'and
stutscors1.testc = 'UBSCT' and
stustat.graden = 11
GROUP BY
track.schoolc,
school.schname,
studemo.ident,
RTRIM(studemo.lastname) + ', ' + RTRIM(studemo.firstname),
stustat.graden,
stutscors1.testc,
zsubtest1.descript
---------------------------------------------------------------------------------
Report 2:
704 Dixie High 11 Student_1 311880 23.2500 23.2500 87.50300000 3.763569
704 Dixie High 11 Student_2 311881 20.2500 20.2500 76.84300000 3.794716
select
stugrp_active.schoolc as School_Code,
school.schname as School,
stugrp_active.graden as Grade,
rtrim(stugrp_active.lastname) + ', ' + rtrim(stugrp_active.firstname) as Student,
trnscrpt.suniq as Ident,
sum(trnscrpt.gpacratt) as CreditAtt,
sum(trnscrpt.gradcrawd) as CreditAwd,
round(sum(case when Trnscrpt.GpaCrAtt is null then 0 else Trnscrpt.GpaCrAtt end * gpamarks.gpavallvl0),3) AS TrmGpaPts,
(round(sum(case when Trnscrpt.GpaCrAtt is null then 0 else Trnscrpt.GpaCrAtt end * gpamarks.gpavallvl0),3))/(sum(trnscrpt.gradcrawd)) as GPA
from
dbo.trnscrpt
inner join dbo.stugrp_active on (trnscrpt.suniq = stugrp_active.suniq) INNER JOIN school ON stugrp_active.schoolc = school.schoolc INNER JOIN
gpamarks ON trnscrpt.marksetc1 = gpamarks.marksetc AND trnscrpt.markawd1 = gpamarks.mark
where
school.schname = 'Dixie High' and
stugrp_active.graden = 11 and
trnscrpt.graden >= 9
group by
stugrp_active.schoolc,
school.schname,
stugrp_active.graden,
rtrim(stugrp_active.lastname) + ', ' + rtrim(stugrp_active.firstname),
trnscrpt.suniq
Hi guys,
I manage to get the SSIS working. Now I would need to do these tasks.
I first want to get data from 2 different sql servers. What would be the best method to accomplish this? Both are in Sql Server 2005.
Secondly I want to make sure if any of the servers couldn't be found on the network or in any case the getting data task failed for any one of them the package won't continue and an email should be send to an email address.
Thirdly If everything is ok then I should combine both and generate one sequence no for them and save them on to another location and then generate a file with modified values.
Can anyone help me regarding these tasks?
Thank you
Gemma
In brief, here is my quandary… I have a GridView on a page with some radio buttons, a couple of check boxes, some drop downs etc. And depending on what the user selects the Grid View needs to show different columns from the database…
The easiest way I have found to do this is to have a separate Data Object for each “view�, i.e. each variation of the checkboxes and radio buttons (which determine which dropdowns to use for the variables) and then bind the GridView to the specific Data Source at runtime…
It seemed like a good idea at the time, but now I have over 7 different data connections cluttering up the design view, and more to come and I feel like I am missing a better way to achieve the same results…
Does anyone have some advice for a major novice?
Here’s the beginning of a large set of nested ifs, if this is the way it’s done that’s fine too, I’ll just keep creating data sources, I just get the feeling there must be an easier way to do this..
protected void Button1_Click1(object sender, EventArgs e)
{
if (RadioButtonList1.SelectedValue == "Vendor") {
if (CheckBox1.Checked == true) {
GridView2.DataSourceID = "ObjectDataSource4";
GridView2.DataBind();
TextBox1.Text = "Search";
} else {
GridView2.DataSourceID = "ObjectDataSource1";
GridView2.DataBind();
TextBox1.Text = "Search";
}
} else if (RadioButtonList1.SelectedValue == "Store Number")
{ GridView2.DataSourceID = "ObjectDataSource2";
GridView2.DataBind();}
else {
GridView2.DataSourceID = "ObjectDataSource3";
GridView2.DataBind();
TextBox1.Text = "Search";
}
}
Any advice for a major novice would be greatly appreciated! Thanks!
I'm just wondering if it is bad programming practice to have many data sources on one page. I need a "Matrix" type of page that grabs data from many tables and so far I have 5 data sources. Is this common or is there something I should be doing to make it more efficient?
View 1 Replies View RelatedIs it possible to create a recordset from two different datasources?
Such as
SELECT * FROM DataSource1.TableName WHERE ColumnName IN (SELECT Column FROM Datasource2.TableName)
Thanks.
Is it possible to create a recordset from two different datasources?
Such as
SELECT * FROM DataSource1.TableName WHERE ColumnName IN (SELECT Column FROM Datasource2.TableName)
Thanks.
Can I import in my query a file from other sources (in this case it's a job that has elaborated server database data, but I could be in the need of using for instance Excel files or others) and compare its fields with the tables in my query?
Thank you.
Anna - Verona (Italy)
Not sure if this is possible, here is the scope:
I have a vb program that calls a sp. My vb program connects to DataSource1, Catalog1 and to DataSource2, Catalog2. My sp resides in Catalog1 using a table from catalog1 and a table from catalog2.
Is this possible to do?
If I were to change a data source that many packages use (say, from box "SQL-2" to "SQL-1"), would I have to re-upload the packages that used that data source? Or would the packages already on the server receive that change somehow?
(currently realizing that most of the packages use package-local data connections instead of project-wide data sources...)
Thanks!
-Rob
Hey all,
I'm coming from ASP and I used to have a global connection string accessible to all of my ASP pages that I'd use for all of my data access. This was really convenient because I could easily switch to a backup or local data source for testing/debug by changing the connection string in one place.
Now, I'm using ASP.NET with Web Matrix and I love the drag and drop functionality but its dropping my connection string all over the place. How can I do this and keep my connection string in one spot so I can have the same convenience of switching data sources.
Thanks in advance!
Larry
I am using SQL Server 2012, and I have a query that gets data from 4 different sources and sums them all up into one field to get a total, and right now I am using 4 queries for each data source and using UNION ALLs to combine them and I was wondering if there is a different way to accomplish this without using UNION statements.Here is a sample of my code.
SELECT
Resv01 = ISNULL(SUM(Resv01),0)
FROM
(
SELECT
Resv01 = SUM(Reservations)
[code]...
For credentials, I have a Shared Data Source that "uses a specific user name and password".
When I open the data source's RDS file, I observed that the credentials are not stored in the RDS file.
Where are the credentials for the data sources stored?
It has to be stored somewhere. When I close and reopen the solution, the credentials for the data source are still intact.
Hi,
I have to get all the customer records from one table and compare this records with another table and finally if there is same record I have to perform an update but the record does not exist I have to make an insert.
How can I do it with SSIS Except writing sql or tsql scripts ?
thanks,
J
I want to configure a Data Source and then Deploy the Data Source and Data Source view along with several packages that use connection managers based on the Data Source/Data Source Views.
Can someone explain this process?
Thanks,
Michael
Is it possible to tie a linked report to a different data source that the base report?
I have a set of base reports that point to Data Source A.
I have a set of linked reports that I want to point to Data Source B. These linked reports live in a different folder altogether.
Is there a simple way to accomplish this?
Thanks In Advance...