Dynamicly Load Tables For DataSources/DataDestinations
Apr 1, 2006
Hi everybody,
The names of the tables that sould be transfered from the production system to the DWH are stored in a table in the production system. Yet I haven't found a proper way to dynamicly define these tables as DataSources and DataDestinations within an Integration Services project.
I have the following code and I'm trying to set an event dynamicly in the foreach statement... SqlConnection conn = new SqlConnection(ConfigurationManager.AppSettings["BvtQueueConnectString2"]); SqlCommand select = new SqlCommand("select top 100 b.jobid as [Job ID], bq.timesubmitted as [Time Submitted], b.timereceived as [Time Received], bq.requestid as [Request ID], bq.timecompleted as [Time Completed], bq.buginfoid as [Bug Info ID], e.eventid as [Event ID], bq.closingeventid as [Closing Event ID], et.eventtype as [Event Type], e.eventtime as [State Start Time], l.twolettercode as [Language], p.projectname as [Project Name], p.packagedesignation as [Package Designation] from bvtjobs b inner join bvtrequestsnew bq on b.jobid = bq.jobid inner join eventlog e on bq.requestid = e.bvtrequestid inner join eventtypes et on e.eventtype = et.eventtypeid inner join languages l on bq.targetlanguage = l.langid inner join projects p on bq.targetplatform = p.projectid order by b.jobid asc", conn); SqlDataAdapter da = new SqlDataAdapter(select); DataSet ds = new DataSet(); conn.Open(); da.Fill(ds); conn.Close(); Table table = new Table();
foreach (DataRow dr in ds.Tables[0].Rows) { TableRow tr = new TableRow(); for (int i = 0; i < dr.ItemArray.Length; i++) { TableCell tc = new TableCell(); tc.Text = dr.ItemArray.GetValue(i).ToString(); tc.BackColor = Color.White; if (dr.ItemArray.GetValue(8).ToString() == "COMPLETE") { tc.BackColor = Color.BlueViolet; } tr.Cells.Add(tc); } table.Rows.Add(tr); } foreach (TableRow tablerow in table.Rows) { TableCell newtc = new TableCell(); Button mybutton = new Button(); newtc.Controls.Add(mybutton); mybutton.Text = "details"; //set the event here tablerow.Cells.Add(newtc); } Panel1.Controls.Add(table); DataBind();
I want to get which column that are sorted dynamicly when the report is shown. I want to use this informastion in some formulas (Like for example: Iif(<sortcolumn>(3);True;False)).
I dont't want to use parameters indicate which column to sort.
I have the follow, i get the right amout of checkboxes but they all have the same value(System.Data.Common.DbDataRecord) Dim objconn As New SqlConnection(connstring_MPR)Dim objcmd As SqlCommand = New SqlCommand("SELECT [Parts Master Table].COMMD_CODE as comcode FROM [Parts Master Table] INNER JOIN [Warehouse balance table] ON [Parts Master Table].PART_NUMBER = [Warehouse balance table].PART INNER JOIN POREPORT ON [Parts Master Table].PART_NUMBER = POREPORT.[Part Number] INNER JOIN [DEMAND TABLE] ON [Parts Master Table].PART_NUMBER = [DEMAND TABLE].PART WHERE POREPORT.[PO Bal] > 0 OR [DEMAND TABLE].QTY > 0 or [Warehouse balance table].ONHAND > 0 and [Parts Master Table].M_B = 1 AND [Warehouse balance table].WHSE = 'sgr' AND ([Parts Master Table].FAMILY NOT LIKE 'lam%' or [Parts Master Table].FAMILY NOT IN ('ULTCH', 'REMOT', 'MKSIN')) GROUP BY [Parts Master Table].COMMD_CODE ORDER BY [Parts Master Table].COMMD_CODE", objconn) objconn.Open() chkComCode.DataSource = objcmd.ExecuteReader(CommandBehavior.CloseConnection) chkComCode.DataBind() objconn.Close()
I have a stored proc that I am trying to use for sorting and paging below is a portion of it. When I hard code "Stan" into the Where clause I get all the records back that where posted by "Stan" but when I try to pass in "Stan" as a variable it tells me something about an invalid column name.
Here is the working version. When I hard code "Stan" into the Where Clause it works. I am using 2 single quotes on each side. See Where clause.
SET @sql ='SELECT [ThreadName],[PostID],[PostTypeID],[LanguageID],[PostAccessID],[UserID],[ThreadID],[PostParentID],[VoteSummaryID],
I have SQL Server 2003 Standard and am attempting to use DTS for a data load/transformation and I€™m not sure if I am using the right tool for the job. I have a somewhat denormalized Access database that has to be loaded into a normalized SQL Server database. Values from one row in any of the source tables generally need to be separated and inserted into several destination (SQL Server) tables. There are no unique ids in the source data since it is coming from a 3rd party and the tables are not related to others. I€™ve created a DTS Package and have the beginnings of several Transform Data Tasks. Each destination table has an Identity id column, which is calculated automatically. The roadblock I€™ve run into is that I can€™t figure out how to take each newly created ID and insert it into a new row in another table as a foreign key. Basically, I have to move data from a single input row to new rows in multiple destination tables and create ids (PK, FK) that tie these tables together. The Transform Data Task only allows me to reference one source and one destination, not multiple destinations. I hope this makes sense. Any suggestions would be appreciated.
Here€™s a simple example that may help illustrate the problem. My database is much more complex.
Input table is called parcels and each row has three columns: Address, Owner, and Legal_Description.
Output database has two tables that will receive this data: Parcel table will have the Address, Owner and Parcel_ID (auto calculated). Legal table will have Legal_Description, Legal_Desc_ID (auto calculated), and Parcel_ID.
When the row is inserted into the Parcel table, the newly auto calculated Parcel_ID has to be captured. Next create a row into the Legal table and insert the Parcel_ID so the two rows are related.
How can I do this through DTS? Thanks for any suggestions, code snippets, or references.
Hi, My problem is I only can put the database table to another database.I use example to ********. For example. My database name is Career. The user has a database name called Online. The Online has already some tables in the database they created before. I just need load my database career tables to the Online database. My database career has seven tables. These tables has a reltionship for each other. I just need to load these table to the Online database, But don't build relation to the tables that already exist in the Online database. Besides, I want to know how load the stored procedure from my database to another database.
We need to load a "master" flat file to SQL Server tables. The file is a dump from mainframe. Based on a field called "record_type", each record in the file has different columns. I would use the following as an example (the real file is much more complicated than this, but you get the idea):
If "M", the fields are "age", "gender", "birthdate", "state", "salary"
If "F", the fields are "age", "gender", "birthdate", "state", "company", "salary"
We need to load the file (only one file) into two different tables, M_table, and F_table. But I have researched and discovered in DTS the source (TEXT file) can not be queried against to filter on the gender field.
Since each record may have different number of fields, I cannot really load the flat file into a "staging" table.
Does anyone has any idea on how to achieve this? Thanks in advance!!!
I have around 140 text files in a folder where all the file entries are delimited by tabs. Currently i am copying the data and then pasting it in to the tables and it is perfectly working fine. I have already created tables for those 140 files. But now i want to load those files in to the table at a single shot. How can i do it? I am using SQLServer 2005.
I am new to SSIS and am looking to load XML files (with a DTD definition) into tables via a SSIS package. I have created a XML task and am able to load the XML and output it to file. I have also stripped out the DTD definition and am able through a dataflow using XML source an OLE db Destination load and map the XML to table sin my DB. But have no idea how to get the data in when it has a DTD definition included. I either want to put each file into a row in a table then query it. Or from the SSIS package input the relevant info into a set of staging tables or the real tables.
i have a fact table which loads through package,when i m trying to load this table by running the package,i m truncating the fact table and loading the fresh data,instaed of this without truncating the fact table i have to implement the incremental logic in this.
For this i can use SCD or Conditional split,but problem here is i have many source tables to load this fact table,so its very difficult to trace the changes in different source tables.
Hi , I am loading the Data into the Tables with the constraints on and redirecting the error rows into a seperate table is there a way to capture the error rows from a execute sql task by directly loading data without constraints and later adding them with the execute sql task and redirecting them to error table as this approach would make the loads quicker. the approach now that i am using is on a row by row basis ..... and if i drop constraints and load data and then add constraints will this deposit the same error rows as in case of the current approach please send me ur suggestions
To setup some data in some new tables in t-sql 2012, I need to create a relationship between 2 tables were data is obtained from an inventory file. The inventory file will be loaded to sql server 2012 from an excel spreadsheet.
1. Columns that will be loaded to the first table are:
1. Inventory category number, 2. Inventory date, 3. Inventory Category total amount, and 4. Inventory category quantity on hand.
This table will have a column called Item number. This will need to be the value of item number that is in the second table.
2. Columns that will be loaded to the second table are:
Thus can you show me the sql that will load the table into both tables? This sql would also need to show how to obtain the values of item number from the second table and place that value into the first table.
I have situation where I get data from SRC Flat file and have to load Dimensional table and also fact table, using same data flow(have no other choice since I have to unpivot some src data). Since I have to load both tables in same data flow, I have to have a way to put load ordering constraint (I know informatica allows that). Does any one have any idea on how this can be done in SSIS?
So i have about 80 different SSIS PAckages that load different text files into tables. each package has a config file that contains password, server name and user id. I want to forward these packages to a use who will execute them. So instead of giving the user 80 packages to execute manually, how do i put these packages into one parent package so that the user has to only execute this parent package instead of executing 80 different packages NB. package exec is manually. (we will automate later) so this is a temp measure.
I need to load images from folders into a SQL Server table and I have done it successfully for individual images, however I need to load all the names of the folders and sub folders names in separate columns + load all images.
So the folders look like as in the screenshot and the final result of the table in SQL Server should look like the second screenshot.
I have one small requirement.. I want to load the different types of files(.txt, .csv, .tsv, .xlsx).
Using one forearch loop container how can I load the files to database and I shouldn't use the script task to split the filenames. Is there any other way to load all the files using forearch loop container, exesql task..
I am trying to implement some powerview report in excel 2013 itself (not in sharepoint).
I have implemented a SSAS tabular model and deployed to server. Now my aim is to load this tables in the SSAS tabular database into powerpivot so that i can use this data for creating powerview reports. However when i tried to import data from SSAS, it asks database and MDX query. My requirement is to load the dimension and fact tables to power pivot as it is.
I have 2 different excel files file1 and file2. file1 should be loaded to table1 and file2 should be loaded to table2. Both of the files will have 1 sheet inside. Do I need to create separate excel source for file1 and file2? I mean file1 in one excel source and that will be connecting to 1 execute sql task. file2 in other excel source and that will be connecting to another execute sql task. Is this the way I should proceed or is there any looping should be done? I need to schedule this activity to run every week. So, I'll get new files every week with the same file names and sheet names. Do I need to consider anything for this requirement also?
I'm planning to do truncate and reload not an incremental load.
I am trying to load the data from one data base to another using the Task->Generate Script->tables (data only). Â So I saved the script of the data and when tried to run the script to the destination DB, getting the below error when ran the script.
Error: Visual Studio has encountered an exception. This may be caused by an extension.You can get more information by running the application together with the /log parameter on the command line, and then examining the file'C:UsersKK67AppDataRoamingMicrosoftAppEnv10.0ActivityLog.xm'. When tried opening the folder using the above path... don't see anything for AppData folder from that path.
My Requirement ,In Source Database 5 tables are there ( Emp,Loc,dept,Time,Product ), Destination is Single Excel file.But Dynamically how to load each table information to load into each sheet wise through SSIS Package?
We are using SQL Server 2014 and SSDT-BI 2013. We have a reporting environment where business users create objects which need to be persisted for fiscal year reporting. Let's say for instance SQLSERVER1SRVR1 they create table objects like below in the reporting environment.
Accounting2014, Accounting2015 in AccountingDB;Â Sales2014, Sales2015 in SalesDB;Â Products2014, Products2015 in ProductsDB;Â Inventory2014, Inventory2015 in InventoryDB etc....
These tables are persisted for auditing in a different environment SQLSERVER2SRVR2 for finance & audit folks.We would want to automate this process using SSIS to create tables in corresponding database and load data. I tried using For Each Loop container but the catch is I could loop the source or destination but how do we loop on Source & Destination at the same time (i.e when source is in AccountingDB destination to be AccountingDB, source SalesDB then destination SalesDB so on etc....
One being a query which sums up claims for a group of members. Col A beign the memberID and B and C being amounts.
Now I set the output to Isosrted = true and set the output column A sorted property to 1.
The other OLE source is a table. Col A being member and b,c,d.. being name, address... I set the same properties for column a and I am trying to use a merge join.
The first two sources return 2245 rows and after the merge I end up with 648.
What would cause the merge to strip some records out?
I have several SSIS packages which I have setup using a datasource to connect to the database. I should be able to switch between databases by just changing the server and database on the datasource but for some reason the ssis package keeps on using the old database. I have the connection managers setup to reference the datasource and can see the connection manager pointing to the new database but when I run a task by right clicking on it and choosing execute task it calls the stored procedure in the old database. I know it is doing this because I renamed the stored procedure in the old database and the task errored out saying it could not find the stored procedure. Can someone please tell me what I am doing wrong?
I need to set a Repeater.Datasource to one of three stored procedures depending on what button the user selects. How do I set it up please?<asp:SqlDataSource ID="SqlDataSource5" runat="server" ConnectionString="<%$ ConnectionStrings:ABCConnectionString %>"SelectCommand="???????????????" SelectCommandType="StoredProcedure"><SelectParameters><asp:ControlParameter ControlID="???????????????????" Name="?????????????" PropertyName="Text" Type="string" /></SelectParameters></asp:SqlDataSource> protected void btnKeyWordSearch_Click(object sender, ImageClickEventArgs e){ mode = 1; StoredProcdureName = "KWSearch"; KWords = tbxKWordText.Text;}protected void btnCompanyNameSearch_Click(object sender, ImageClickEventArgs e){ mode = 1; StoredProcdureName = "NAMESearch"; CoName = tbxCoNameText.Text;}protected void btnBrandNameSearch_Click(object sender, ImageClickEventArgs e){ mode = 1; StoredProcdureName = "BRANDSearch"; Brand = tbxBrandText.Text;}
I am currently working with SQL 2005 Reporting Services and MS Visual studio 2005.
In my SQL server I have data about organizations, so that each organization's data is stored in an individual database. In addition, the number of organizations/databases is variable. I know the number of organizations and the name of each one (that it agrees with the name of its database) because there is a database (organizationsDB) with a table (organizationsTB) containing the list of the organizations.
So, It would be possible to create a report containing a summary of each organization? .... for example a table in which each row contains data of each organization.
I have discovered some shortcomings in the way inline table valued function parameters are treated in the OLEDB datasource. You can select the user designed function ine the Generic Query Builder and test it with the required parameters. However when you attempt to set up the parameters for the result ing SQL Command Text you get and error message to the effect that the parameters cannot be retrieved from the datasource. Once again this is disappointing because Report Services seems to deal with the parameters perfectly well.
Get this when selecting parameter value in Report Manager
Report is working fine in preview
Up to me, it has nothing to do with credentials
I have some analysis server reports.
I have a single value report parameter, the report runs fine with the default value, when I select another value than the default one, I get above message immediately.
I got the error since I rebuild the report with SP2 CTP.
Strange thing is that the other single value parameter of the report works fine (year)
And that I get the error also with other reports, and on the same dimension (Organisation)
And that I do not have the error when I change the parameter to multi-value (but as the report is not written for multiple values...)
MDX queries are as follows:
SELECT NON EMPTY { KPIValue("KPINewBiz"), KPIGoal("KPINewBiz"), KPIStatus("KPINewBiz"), KPITrend("KPINewBiz") } ON COLUMNS,
NONEMPTY
([Reps].[Organisation].[Organisation].ALLMEMBERS
*[Reps].[Rep].[Rep].ALLMEMBERS
*[Date].[Year].[Year].ALLMEMBERS
*[Date].[Month].[Month].ALLMEMBERS
,[Measures].[Gross Sales]
)
DIMENSION PROPERTIES MEMBER_CAPTION, MEMBER_UNIQUE_NAME ON ROWS
FROM ( SELECT ( STRTOSET(@ByYear, CONSTRAINED) ) ON COLUMNS FROM ( SELECT ( STRTOSET(@ByOrganisation, CONSTRAINED) ) ON COLUMNS FROM [Sales])) CELL PROPERTIES VALUE, BACK_COLOR, FORE_COLOR, FORMATTED_VALUE, FORMAT_STRING, FONT_NAME, FONT_SIZE, FONT_FLAGS
WITH MEMBER [Measures].[ParameterCaption] AS '[Reps].[Organisation].CURRENTMEMBER.MEMBER_CAPTION' MEMBER [Measures].[ParameterValue] AS '[Reps].[Organisation].CURRENTMEMBER.UNIQUENAME' MEMBER [Measures].[ParameterLevel] AS '[Reps].[Organisation].CURRENTMEMBER.LEVEL.ORDINAL' SELECT {[Measures].[ParameterCaption], [Measures].[ParameterValue], [Measures].[ParameterLevel]} ON COLUMNS ,
NONEMPTY([Reps].[Organisation].[Organisation].ALLMEMBERS,[Measures].[Gross Sales]) ON ROWS FROM [Sales]
WITH MEMBER [Measures].[ParameterCaption] AS '[Date].[Year].CURRENTMEMBER.MEMBER_CAPTION' MEMBER [Measures].[ParameterValue] AS '[Date].[Year].CURRENTMEMBER.UNIQUENAME' MEMBER [Measures].[ParameterLevel] AS '[Date].[Year].CURRENTMEMBER.LEVEL.ORDINAL' SELECT {[Measures].[ParameterCaption], [Measures].[ParameterValue], [Measures].[ParameterLevel]} ON COLUMNS , NONEMPTY([Date].[Year].[Year].ALLMEMBERS,[Measures].[Gross Sales]) ON ROWS FROM [Sales]
Discovered something else that is very strange, you change the value of the parameter, you get the error, you hit the back button, and you can run the report with the new parameter
Did one more test, changed the order of the parameters, is now going wrong on the other parameter, Year.
So, it is always on the first parameter.
Correction, I have put a multi-value parameter as first, I still have the problem, it is always on the first single value parameter
I have the same problem.
Up to me, it has nothing to do with credentials
I have some analysis server reports.
I have a single value report parameter, the report runs fine with the default value, when I select another value than the default one, I get above message immediately.
I got the error since I rebuild the report with SP2 CTP.
Strange thing is that the other single value parameter of the report works fine (year)
And that I get the error also with other reports, and on the same dimension (Organisation)
And that I do not have the error when I change the parameter to multi-value (but as the report is not written for multiple values...)
MDX queries are as follows:
SELECT NON EMPTY { KPIValue("KPINewBiz"), KPIGoal("KPINewBiz"), KPIStatus("KPINewBiz"), KPITrend("KPINewBiz") } ON COLUMNS,
NONEMPTY
([Reps].[Organisation].[Organisation].ALLMEMBERS
*[Reps].[Rep].[Rep].ALLMEMBERS
*[Date].[Year].[Year].ALLMEMBERS
*[Date].[Month].[Month].ALLMEMBERS
,[Measures].[Gross Sales]
)
DIMENSION PROPERTIES MEMBER_CAPTION, MEMBER_UNIQUE_NAME ON ROWS
FROM ( SELECT ( STRTOSET(@ByYear, CONSTRAINED) ) ON COLUMNS FROM ( SELECT ( STRTOSET(@ByOrganisation, CONSTRAINED) ) ON COLUMNS FROM [Sales])) CELL PROPERTIES VALUE, BACK_COLOR, FORE_COLOR, FORMATTED_VALUE, FORMAT_STRING, FONT_NAME, FONT_SIZE, FONT_FLAGS
WITH MEMBER [Measures].[ParameterCaption] AS '[Reps].[Organisation].CURRENTMEMBER.MEMBER_CAPTION' MEMBER [Measures].[ParameterValue] AS '[Reps].[Organisation].CURRENTMEMBER.UNIQUENAME' MEMBER [Measures].[ParameterLevel] AS '[Reps].[Organisation].CURRENTMEMBER.LEVEL.ORDINAL' SELECT {[Measures].[ParameterCaption], [Measures].[ParameterValue], [Measures].[ParameterLevel]} ON COLUMNS ,
NONEMPTY([Reps].[Organisation].[Organisation].ALLMEMBERS,[Measures].[Gross Sales]) ON ROWS FROM [Sales]
WITH MEMBER [Measures].[ParameterCaption] AS '[Date].[Year].CURRENTMEMBER.MEMBER_CAPTION' MEMBER [Measures].[ParameterValue] AS '[Date].[Year].CURRENTMEMBER.UNIQUENAME' MEMBER [Measures].[ParameterLevel] AS '[Date].[Year].CURRENTMEMBER.LEVEL.ORDINAL' SELECT {[Measures].[ParameterCaption], [Measures].[ParameterValue], [Measures].[ParameterLevel]} ON COLUMNS , NONEMPTY([Date].[Year].[Year].ALLMEMBERS,[Measures].[Gross Sales]) ON ROWS FROM [Sales]
Discovered something else that is very strange, you change the value of the parameter, you get the error, you hit the back button, and you can run the report with the new parameter
is there a way to bring in oracle and sql tables into the same report and join them together on common fields? also, if this is possible, i am trying to join them in the native UI and not the generic query designer mode.
i am hoping that through fully qualifying the tables as you bring them in, that this may be possible. any suggesions would be greatly appreciated...