Hi,
My procedure to implement a task is like this
I will be using execute SQL task to fetch the records from source,after this wanna use For each loop to access each record one at a time,perform some trnsformations and insert that record into destination.
Help me in accessing the data stored in the Variable(SQL task) in Dataflow task of foreach loop.
Hey Guys.
I am new to using SSIS and need some assistance. I am trying to create a package that will loop through a folder of DBF files and import them into SQL server. Here is what I did so far:
- Created a Foreach Loop Container with a Data Flow Task within
- Created a string variable to store the file names
- Created the OLE DB Source and Destinations
- Created the connection manager with dBASE III Extended Property pointed to the folder where the files are.
When I try to set the OLE Source to use the file name variable, I get an error stating that there is no Destination table defined. I get the same error when I try to setup the OLE Destination. How do I resolve this issue? I want it to import each file into its own table. Is there any way to reuse this file name variable as the destination table name?
Any help would be greatly appreciated.
Thanks!
-Michael
I have a csv file in which I am reading into a recordset destination and want to than use a foreach loop to cycle through those records and do some things. The problem I am having is after defining the variable name of the records set as results, and than going into the foreach loop, choosing collection, using the foreach ado enumerator, i dont see anything in the dropdown under ado object source varable?????? I am new to SSIS but I basically want to parse through this file, change some columns in each line and than either update or insert data in a sql table.
View 4 Replies View RelatedI need to essentially do 2 loops. One loops through each record and then inside each record row, I want to perform an insert on each column.
Something like this maybe using a cursor or something else:
For each record in my table (I'll just use the cursor)
For each column in current record for cursor
perform some sql based on the current column value
Next
Next
So below, all I need to do is figure out how to loop through each column for the current record in the cursor
AS
DECLARE Create_Final_Table CURSOR FOR
SELECT FieldName, AcctNumber, Screen, CaseNumber, BKYChapter, FileDate, DispositionCode, BKUDA1, RMSADD2, RMSCHPNAME_1, RMSADDR_1,
RMSCITY_1, RMSSTATECD_1, RMSZIPCODE_1, RMSWORKKPHN, BKYMEETDTE, RMSCMPNAME_2, RMSADDR1_2, RMSCITY_2, RMSSTATECD_2,
RMSZIPCODE_2, RMSHOMEPHN, BARDATE, RMSCMPNAME_3, RMSADD1_2, RMSADD2_3, RMSCITY_3, RMSZIPCODE_3, RMSWORKPHN_2
FROM EBN_TEMP1
OPEN Create_Final_Table
FETCH FROM Create_Final_EBN_Table INTO @FieldName, @AcctNumber, @Screen, @CaseNumber, @BKYChapter, @FileDate, @DispositionCode, @BKUDA1, @RMSADD2, @RMSCHPNAME_1, @RMSADDR_1,
@RMSCITY_1, @RMSSTATECD_1, @RMSZIPCODE_1, @RMSWORKKPHN, @BKYMEETDTE, @RMSCMPNAME_2, @RMSADDR1_2, @RMSCITY_2, @RMSSTATECD_2,
@RMSZIPCODE_2, @RMSHOMEPHN, @BARDATE, @RMSCMPNAME_3, @RMSADD1_2, @RMSADD2_3, @RMSCITY_3, @RMSZIPCODE_3, @RMSWORKPHN_2
WHILE @@FETCH_STATUS = 0
BEGIN
@Chapter = chapter for this record
For each column in current record <---- not sure how to code this part is what I'm referring to
do some stuff here using sql for the column I'm on for this row
Next
Case @Chapter
Case 7
Insert RecoverCodeRecord
Insert Status Code Record
Insert Attorney Code Record
Case 13
Insert Record
Insert Record
Insert Record
Case 11
Insert Record
Insert Record
Insert Record
Case 12
Insert Record
Insert Record
Insert Record
END
close Create_Final_Table
deallocate Create_Final_Table
Hi,
I have read only permission in the source (OLTP) database. The source database is running in SQL Server 2000 and the size is more than 200 GB. I need to pull data from source and load target which is running in SQL server 2005.
Following are the objectives I want to achieve.
Data should be loaded on incremental basis.
Whatever changes take place (Update/Delete) in source, that should be replicated to already uploaded data.
Here I want to mention that, the source database does not have any identification key or timestamp column like Updated_Date by which I can filter the data which are recently inserted or updated into the source and upload the same. The source does not maintain any history data also. So I do not have any track of deleted record also.
I dont have any scope to change the schema in the source. In this scenario can anybody suggest me the best approach to achieve the above mentioned objectives?
Can I retrieve only the recent updated or inserted date form transaction log back up. Can log shipping solve the give the solution?
One more question. Say I have a table and I am exporting/importing all the data from/to my target table using SSIS or DTS. In this scenario does using query or using directly the table affects the performance?
Regards
Sudripta Rakshit
I'm curious to know how other people are handling the Recordset Destination in to be processed by a Foreach Loop Container. It seems a little odd to me that you can only map the parameters by knowing the index of the columns of the Recordset, however, the order that you built the recordset destination doesnt stay the same. I've been debugging for a while to find out that after I saved my recordset destination the order of the fields changed. To some order without a clear logic. I'm going to guess it might be the lineage id.
The bigger problem was this was a really large record set with 60 or so columns. To try and debug the problem of finding the indexes, I had added a Multicast tranform and saved the output to an Excel Destination. Of course, the order I setup the Excel file was the order I got the fields. Why would this not be the case with the Recordset Destination?
Hi,
I want to do the following with a ssis package:
INPUT:
A table contains 2 columns with data i need. column A=Filename and column B=FileContent
PROCESS:
I need to loop through ea record in the table and retrieve columns A and B. Then for ea column i need to write the Content hold in column B into File hold in column A.
I so far found out, that i need a Execute SQL Task in Control Flow querying the table and get columns A and B into 2 variables, plus a 3rd var holding the object. Then the output goes into a Foreach Loop Container. From this point i don't know how to continue. I tried to put a Data Flow Task inside the Foreach Loop, but couldn't find out how i now get the 2 variables to the Data Flow Task and use them to for the file to be written and the content to be placed in the file.
Is there any example similiar to that so i could learn how to start on that?
Thanks
Danny
I have a client who needs to copy an existing sale. The problem isthe Sale is made up of three tables: Sale, SaleEquipment, SaleParts.Each sale can have multiple pieces of equipment with correspondingparts, or parts without equipment. My problem in copying is when I goto copy the parts, how do I get the NEW sale equipment ids updatedcorrectly on their corresponding parts?I can provide more information if necessary.Thank you!!Maria
View 6 Replies View RelatedHi
I have a table with a user column and other columns. User column id the primary key.
I want to create a copy of the record where the user="user1" and insert that copy in the same table in a new created record. But I want the new record to have a value of "user2" in the user column instead of "user1" since it's a primary key
Thanks.
Below is a stored procedure that designed to populate a drop down menu system on a website. It works fine as long as the 'id's in the first select start at 1 and are sequential. It fails to grab all the sub tables if the ids are not sequential. So, how do I structure the loop so that the WHERE clause uses not the loop iterator, but rather, the ids from the first Select statement.
Alternatively, is there a more elgant approach that will return the same set of recordsets?
Any help would be much appreciatedThanks
ALTER PROCEDURE dbo.OPA_GetMenuItemsASDeclare @i tinyint ,@tc tinyintSet @i = 1
/* Select for top level menu items*/
SELECT id, label, url, sortFROM mainNavORDER BY sort
Set @tc = @@rowcount
while @i <= @tc
beginSet @i = (@i + 1)
/* Select for submenu itemsSELECT id, label, url, sort, mainNavIdFROM SubNavWHERE (mainNavId = @i)ORDER BY mainNavId, sortend
RETURN
hi all,
I want to execute a store procedure but my query returns more than one value. how can I asign value for that. Here is the code. Please help me out.
problem is @aaa where i cannot set the value.
Thanks
Regards
Sudadg
CREATE PROCEDURE movetable
@id int
AS
declare @VType varchar(10)
declare @count int
declare @aaa varchar(50)
set nocount on
create table #temp1(id int identity(1,1),Ac_Code varchar(50))
select count(*) from Journal_Table where Voucher_No=@id
set @count=1
set @count=@@rowcount
while @count <>0
begin
set @aaa=(select Ac_Code from Journal_Table where Voucher_No =@id)
insert into #temp1(Ac_Code) values(@aaa)
set @count=@count-1
end
select * from #temp1
GO
Below is a stored procedure that designed to populate a drop down menu system on a website. It works fine as long as the 'id's in the first select start at 1 and are sequential. It fails to grab all the sub tables if the ids are not sequential. So, how do I structure the loop so that the WHERE clause uses not the loop iterator, but rather, the ids from the first Select statement.
Alternatively, is there a more elgant approach that will return the same set of recordsets?
Any help would be much appreciated
Thanks
ALTER PROCEDURE dbo.OPA_GetMenuItems
AS
Declare @i tinyint ,
@tc tinyint
Set @i = 1
/* Select for top level menu items*/
SELECT id, label, url, sort
FROM mainNav
ORDER BY sort
Set @tc = @@rowcount
while @i <= @tc
begin
Set @i = (@i + 1)
/* Select for submenu items*/
SELECT id, label, url, sort, mainNavId
FROM SubNav
WHERE (mainNavId = @i)
ORDER BY mainNavId, sort
end
RETURN
I am new to SQL Server (coming from Oracle background) and have a large table I need to loop thru and assign a number 1 thru 5 on each record.
below is an example of how I would do it in Oracle.
declare
num number := 1;
cursor c_rec is SELECT rowid, t.* FROM temp t order by t.column;
begin
for d_rec in c_rec
loop
update temp set column = to_char(num)
[Code] ....
Ok,
I have a table with the following fields:
ID, Type, Name, Date, Description
ID is the PK. Names all have different values, but are predefined... a visual:
ID, Type, Name, Date, Description
1, monday, mon1234,3-3-08, some text
2, monday, mon1234,3-4-08, some text
3, monday, mon2345, 3-5-08, some text
4, monday, mon2345, 3-6-08, some text
Now What i need to do is get the last report for each unique "Name". How would I do that?
When I try to do a Select Distinct, of course, it only gives me those names that only appear once.
Hi,Can you guys see if there's a solution to this problem?I have a database from which I have to read each record and processthat record. New records are being added all the time, so I need to goback and check for new records and process them.However:-- there is no 'identity' column in the database design (so I cannotkeep track of the last record read by use of a numeric variable)-- I am not allowed to update the database (so I cannot flag therecords I have read).My problem is: how can I know which records I have already read andwhich ones I haven't read yet? I don't want to process records twiceand don't want to miss any records.Is there a known solution to this problem? Any ideas?Thanks.
View 12 Replies View Related
Hi.
I have a table where I need to set some rows to Read only (to protect the rows from being edited). Can I do this with a trigger checking against a column (where dataReadOnly = 1)?
Or is there a better way?
thankyou
Howdy all,
I've seen several posts about reading and writing files that have different record types with varying column metadata. My particular file has 11 record types plus several header types and looks something like:
<Header1>
<Header2>
<Detail01-#1>
<Subdetail02>
<Subdetail03>
...
<Detail01-#2>
<Subdetail02>
<Subdetail03>
...
...
Since i need to get different detail and subdetail records, i can't really use the technique of 3 dest file connection managers found in http://forums.microsoft.com/MSDN/ShowPost.aspx?PostID=87269&SiteID=1
I've tried using an exec sql to get the main detail records and then a forech ADO en umerator that would get the subdetails, but it all seems so kludgy. I'm starting to think that I should just write the bulk of the file creation code in a c# app instead of trying to smush this into SSIS. Opinions? Am I missing some trick in SSIS?
TIA,
-Peter
Hi All,
I am trying to create package something like that..
1- New Customer table as OleDB source component
2- Lookup component - checks customer id with Dimension_Customer table
3- And if same customer exist : I have to update couple fields on Dimension_Customer table
4- if it does not exist then I have insert those records to Dimension_Customer table
I am able to move error output from lookup to Dimension_Customer table using oledb destination
but How can I update the existing ones?
I have tried to use oledb command but somehow it didnt work
my sql was like this : update Dimension_Customer set per_X='Y', per_Y= &Opt(it should come from lookup)
I will be appreciated if you can help me...
Hey guys,,
Well im new at this SSIS stuff and i have something that i am trying to do, but cannot get it to work.. Ill try to explain, and if anyone can help me or point me in the right direction it would be much appriciated..
I have 2 tables, one header table. and one lines table. This is a one - to -many relationship.. ie 1 header, many lines.. This is a Order Header, and Order Lines table setup.. Order header has Order numbers and and email address field that link to the lines table by order number. I also have a view which links all this info together.
I would like to export a excel file (preferable named from the order number column - but that can come later) for each order number in the header table. The excel file will contain the details from the View that was created. I want this to loop through all the header records in the header table and create a excel file for each one..
Down the track i want to add a send mail task to this and pass the email address to a variable so i can use it in the send mail task.. But ill get the main part working first..
Anyhelp would be more than helpful.. I tried to set this up, but i am stuck on the enumerator part..
thanks again, scotty
Hello,
Is there any way to implement sequence data read.
Note:
source is .csv or flat file file
i want process the records one by one.
please give me the solution ASAP.
Thanks
Thiru
Greetings from a SSIS newbie still on the learning curve...
I have a SQL query OLE DB Source that yields a result set that I'd like to put into a SQL Server Destination, but only those records that don't already exist in the destination.
Is there a recommended (read: easy) way to accomplish this? Perhaps a handy transformation?
I have tried to incorporate a subquery in my source query along the lines of:
SELECT fields FROM table1 WHERE keyfield NOT IN (SELECT keyfield from table2)
which works in design time but fails at the server with a cryptic:
"Error: Unable to prepare the SSIS bulk insert for data insertion."
and
"Error: component "<component>" (16) failed the pre-execute phase and returned error code 0xC0202071."
Don't mean for that to cloud the issue, though. I would appreciate any help given.
Thanks!
Paul
Hello,
I have 10 tables (T1Orj,T2Orj,¦T10Orj) and I need to find modified rows from each table and insert them to T1Bak, T2Bak, ¦T10Bak. Although original and bak tables have the common fields , the original tables have more fields than bak tables, and T1Orj, T2Orj, ¦ T10Orj tables have different table structures.
I can go head and write a insert into query for each table, I am just wondering Is there any way I can do this in a loop for all tables?
OK. I give up and need help. Hopefully it's something minor ...
I have a dataflow which returns email addresses to a recordset.
I pass this recordset into a ForEachLoop configuring the enumerator as (Foreach ADO Enumerator). I also map the email address as a variable with index 0.
I then have a Execute SQL task which receives this email address as a varchar variable (parameter 0) which I then use in my SQL command to limit the rows returned. I have commented out the where clause and returned all rows regardless of email address to try to troubleshoot this problem. In either event, I then use a resultset to store the query result of type object and result name 0.
I then pass this resultset into a script variable to start parsing the sql rows returned as type object. ( I assume this is the correct way to do this from other prior posts ...).
The script appears to throw an exception at the following line. I assume it's because I'm either not passing in the values properly or the query doesn't return anything. However, I am certain the query works as it executes just fine at the command prompt.
Try
ds = CType(Dts.Variables("VP_EMAIL_RESULTS_RS").Value, DataSet)
My intent is to email the query results to each email address with the following type of data by passing the parsed data from the script to a send mail task. Email works fine and sends out messages but the content is empty. I pass the parsed data as string values to the messagesource and define the messagesourcetype as a variable in the mail task.
part number leadtime
x 5
y 9
....
Does anyone have any idea what I might be doing wrong?
thanks
John
I am trying to migrate database from old structure to new structure usign SSIS.
The table in new db have extra field that i need to assign it using variable. this is because i have few customer that having different variable value. (meaning for 1 customer, the variable will be fix for all the tables in the database)
my question, without using the Execute Sql Task, can i assign the variable into the the old db destination?
eg my data flow Task is : OLE Db Source - Derived Column - OLE DB Destination.
Example data
Old structure (key = txnID)
---------------------
TxnID
ChequeNo
Bank (chq Bank - Bank in Bank)
Amount
New Structure (key = TxnID & CoCode)
-------------------------
TxnID
ChequeNo
ChqBank
BankInBank
Amount
CoCode
TQ.
I am new to integration services. I am trying to a build a data warehouse and need to be able to insert new data as well as update data that has changed. I am getting the data in a flat file and need to import it into SQL 2005. I saw some post on www.sqlis.com/default.aspx?311 but I did the example is for OLE DB component. I am not sure how to achieve this using a text file as source. Any help would be greatly appreicated.
View 3 Replies View Related
Help with Looping in a SSIS Package
Scenario:
We have a web app that lets our existing clients insert new locations into a table (Clients) in a SQL Server DB. This table has an Identity column as the PK. This table also has another ID field HBID that is the PK for another table (HLocation) in another Database system (Sybase). The HBID field is given a Default value of 1 during an insert operation in our (SQL Server) table, Clients. The Sybase database uses a Sequence table for inserting a PK into the table. Whenever our clients insert a new record in Clients (SQL Server), I need to generate a new HBID from the Sequence table in (Sybase), update the HBID in the Clients table (SQL Server) and then finally insert the record with the new HBID into table HLocation (Sybase).
I have devised a SSIS package for this as following: Note all variables are scoped at the Package Level.
Execute SQL Task #1 that drives a For Each Loop with the following properties:
General:
Connection Type: OLE DB
Connection: SQL Server DB
SQL SourceType: Direct Input
SQLStatement:
SELECT COID, HBID, Name, DateCreated
FROM Client
WHERE HBID = '1'
ResultSet: = Full Resultset
Result Set:
Result Name = 0
VariableName = USER::rsClient Variable Type = Object
ForEachLoop with the follwing properties:
Collection:
Foreach ADO Enumerator
ADO Object Source Variable = USER::rsClient
Enumeration Mode = Rows in first table
Variable Mappings:
Variable: Index:
COID 0
HBID 1
Name 2
DateCreated 3
Inside of the ForEachLoop I have another Execute SQL Task to generate a new HBID from Sybase set up as following.
Execute SQL Task #2
Connection Type: OLE DB
Connection: Sybase
SQL SourceType: Direct Input
SQLStatement:
UPDATE Autoinc SET INC_LAST = (INC_LAST+1) WHERE INC_KEY ='HBID';
SELECT INC_LAST AS NewHBID FROM Autoinc WHERE INC_KEY ='HBID'
ResultSet = SingleRow
Result Set:
Result Name = NewHBID
VariableName = USER::NewHBID, VariableType = Int32
Also Inside the ForEachLoop is a Script Task that has all of my variables as ReadWrite = COID, HBID, Name, DateCreated and NewHBID. I concatenate the values in a string a pass the string into a Message Box to make sure they are looping correctly and they are.For example the results might look like the following:
12698, 1, John Doe Trucking, 10/1/2007, 14550
13974, 1, Joe Smith Trucking, 10/1/2007, 14551
10542, 1, Dave Jones Trucking, 10/1/2007, 14552
Etc.
The values 14550 -14552 are the new HBID being generated in the loop.
The problem is that when I try to Update the HBID in the Client table (SQL Server) with another Execute SQL Task I keep getting the same NewHBID number.
In this case 14550 would be updated for every record instead of the next number in the loop.
I have set up Execute SQL Task #3 as follows:
General:
Connection Type: OLE DB
Connection: SQL Server DB
SQL SourceType: Direct Input
SQLStatement:
UPDATE Client
SET HBID = ?
WHERE HBID = '1'(SELECT COID, HBID, Name, DateCreated FROM Client)
ResultSet: = Full ResultSet
Result Set:
Result Name = 0
VariableName = USER::rsNewClient, VariableType = Object
Parameter Mapping:
VariableName USER::NewHBID, Direction = INPUT, DataType = Long
ParameterName = 0
Ive tried putting Execute SQL Task #3 inside of the ForEachLoop, connecting it to the output of the ForEachLoop. Ive tried setting up a dataflow with a Derived Column using the USER::NewHBID as the Expression.
I still get the same results, 14550 added to every row.
Can any one help or shed some light?
Any and all suggestion will be deeply appreciated!
Thanks !!!
I used bulk insert to insert a txt file into a table. It works fine. (see code below) Now, one txt file with column's name at first row and has about 200 columns. There is no table created before. How to code to create a destination table based on first row of the txt file so that bulk insert will work for that txt file?
BULK INSERT #MBRACCT
FROM 'c:order.TXT'
WITH
(
FIELDTERMINATOR = '|',
FIRSTROW = 2,
ROWTERMINATOR = ''
)
Hello,
I've searched around and can't find any references to the problem I'm having. I'd appreciate any ideas or input.
I'm trying to use the OLEDB Destination for an insert at the end of a long data flow. I need to parameterize the input, and for some of the columns I need to use literal values instead of parameters. It seems like this should be the most common thing in the world, but I'm at a loss to get it to work.
I type in the SQL statement just like I would with an OLEDB Command transformation, with the ? character for the appropriate columns in the VALUES clause. However, when I try to use Parse Query I get this error:
"Parameter Information cannot be derived from SQL statements. Set parameter information before preparing command."
OK, so I start searching around for ways to set the parameter information. Nada. On the Mappings tab the parameter list is empty. I check MSDN and it says this:
"If you have entered a parameterized query by using ? as a parameter placeholder in the query text, use the Set Query Parameters dialog box to map query input parameters to package variables."
Set Query Parameters dialog box? I don't see this anywhere. What am I missing?
The options with the SQL Server Destination seem even more limited, as I don't see any way to use a SQL statement or stored procedure.
For the moment I'm going to stub this off with an OLEDB Command transformation with a downstream Trash desintation, but hopefully that's only going to be temporary.
Thanks,
Dan
I have a table with employee references and a startdate.
I want to insert into a new table an entry for each employee for each date since their startdate to today.
Eg
EMPTABLE
empref,startdate
0001,01.01.2012
0002,02.02.2012
What I require is
NEWTABLE
empref, Date
0001,01.01.2012
0001,01.02.2012
0001,01.03.2012
......
0001,08.27.2012
0002,02.02.2012
0002,02.03.2012
......
0002,08.27.2012
Hi everyone
I want to know if it's possible to do a for/while-loop so i can use INSERT
Look:
I've this int [] test = new test[140];
But i need to insert for every value (140) a number
so normally it would be :
INSERT ... (case1, case2, case3 ...) value (test[1],test[2],test[3] ...)
But isn't there a way to it with a loop?
SOmething Like this ?
for( int i = 0 , i< 140, i++) {
INSERT case[i] value test[i]
}
Thanks in advance
Jonas
I wanted to insert values in columns as explained in below ex.
I am having a table that contains Column1,Column2,Column3,......,Column10.
Inside my for loop, i am getting Column1 value then Column2 then Column3 values and so on till Column10.
My requirement is that on each iteration,I wanted to insert value of Column1 in field Column1, value of Column2 in field Column2 and so on.
Hi, Anybody has an example of reading a csv comma delimited file and insert the fields into a datatable?Thanks
View 2 Replies View RelatedHello everyone,
I am having a little problem with a simple package and I do not know if this is a known issue or that I am missing something.
I have a data flow task, a simple one, with an oledb source pulling data, using a select statement, from a sql server 2005 instance, and an ole db destination pointing to a table in a sql server 2000 instance. Both intances are standard edition. The table in the destination has a column which allow null values and has also a default constraint (getdate()), and this column is not present in the source. When I map the columns in the destination, I leave this column as "ignore", not being mapped to any column from the source. The problem is that when I execute the task, SSIS is trying to insert NULL value into this column, so the package fail with the error "can not insert NULL value into column myColumn". I wonder why is it trying to insert NULL value if the column is not mapped to any column from the source.
Is this a known issue or I am nissing something in the settings?
If the destination table has rowversion or identity columns, there is no problem ar all. I ignore those columns in the mapping and SQL Server feeds them as expected.
Thanks in advance,
Alejandro Mesa