I've a SSIS 2008 parent/child package solution to manage data transfers between two different data sources, so we can copy multiple tables and capture how many rows were transferred and duration for each transfer. This solution was working fine up until last week, when I made some changes to allow the package to perform a source count using standard SQL determined by an expression, or SQL provided from configuration tables, I also changed the package to Truncate or not the destination table, again controlled by configuration settings in a table. The child packages which perform the data flows have not changed!
The day after the controlling package promotion to live, I saw the bizarre behaviour of the Package log stating all rows transferred, but the actual table counts were not what the log stated, see attached file. The package solution works ok on other servers and was ok in DEV, but there were less tables and rows transferred.Re-running the package gave the same errors, but on some of the same tables and some different ones.
As it is the child packages doing the transfers and nothing has changed in them. I cannot see how the log would be able to say all rows are transferred and yet not all of the rows are actually moved?
Process output - where you can see counts and log Table transfer controller (as txt not dtsx)
An example of the data transfer child packages (as txt not dtsx)When I set the ExecuteOutOfProcess = True the package worked fine, unfortunately, this is not a good solution as SSIS 2008 does not tidy up the Dtshost.exe processes it starts and I'd be left with a memory issue after a very short time, we transfer hundreds of tables each day. ( I could write a .net script in the controlling package to kill the child processes, but that would still have hundreds of processes running before I could end them, as we have three parallel streams to allow a bit better performance.
I have created a table Table with name as Varchar and id as int. Now i have started inserting the rows like, insert into Table values ('arun',20).Yes i have inserted a row in the table. Now i have got the values " arun's ", 50. insert into Table values('arun's',20) My sqlserver is giving me an error instead of inserting the row. How will you solve this problem?
Table B: Name Data UserA xxx UserB asdasd UserB ewrsad UserC dsafasc UserA sdf UserB dfvr4
I want to count the total entries in Table B for every user in Table A. The output would be:
Name Count UserA 2 UserB 3 UserC 1
I can use a Select Count statement, but I will have to make a SQL call for every user in Table A. Also, Table A is dynamic, so the users are always changing. Can this be incorporated into one SQL call to count the total rows in Table B for each user in Table A?
I need Insert rows in the OrderDetails Table based on values in the Orders Table
In the Orders table i have a columns called OrderID and ISale. In the OrdersDetails i have columns called OrderID and SaleType
For each value in the OrderID Column of the Orders Table, anytime the ISale Column in the Orders table = 1, and the SalesType column in the OrderDetails table is empty, I want to add two rows in the OrderDetails table. One row with the value K and another row with the value KD. That is a row will be added and the value in the SalesType column will be K, also a second row will be added and the value in the SalesType column will be KD
Also for each value in the OrderID Column of the Orders Table, anytime the ISale Column in the Orders table = 0, and the SalesType column in the OrderDetails table is empty, I want to add two rows in the OrderDetails table. One row with the value Q and another row with the value QD That is a row will be added and the value in the SalesType column will be Q, also a second row will be added and the value in the SalesType column will be QD.
I would like to know the best way to count and display the number of rows in a given database table called memberinfo. This should tell me how many members I have right? Thanks
Hi all, I’ve a table with production objects, and another with possible items composition of the object. I need to count how many occurrences of each standard composition appears: Table PROD: PROD_ID COMPONENT TYPE --------- ----------- ----- 1 AAA X1 BBB Y2 AAA X3 BBB Y4 AAA Y5 AAA X5 BBB YTable ITEM_COMPITEM_ID COMPONENT TYPE-------- --------- -----7 AAA X7 BBB Y8 AAA X9 BBB Y The result should be: ITEM_ID OCCURRENCES-------- ----7 28 19 1 Table PROD have millions of rows, my way is too slow (I’ve a loop where each PROD object are separately queried against the ITEM_COMP), some have an idea for a most efficient way?
I have a question regarding the total count of the table rows
Select count (name) from test. Lets say I have got 200 count. And Select count (lastname) from test1.I have got 200.And this counts I should store it in "There are nn name items awaiting your attention and nn pending lastname's awaiting your approval".
I need to run a query and insert the count results and a name of another table into an existing table.
I have 20 jobs that import records from text files into seperate tables. When that completes I want the count to be sent to another table along with the table name that the count was run on. Any ideas? I can't get SELECT INSERT to work.
Hi,i m using sqlexpress 2005 and sql management express studio. I want to know how could i insert multiple records on a single query in a table?i also want to whats wrong with this insert query?DROP TABLE IF EXISTS `tblcountry`;CREATE TABLE `tblcountry` ( `ID` int(3) NOT NULL auto_increment, `LCID` int(4) unsigned default '0', `CountryCode` char(2) default NULL, `Country` varchar(50) default NULL, `CountryInt` varchar(50) default NULL, `Language` varchar(50) default NULL, `Standard` tinyint(1) unsigned default '0', `Active` tinyint(1) unsigned default '0', PRIMARY KEY (`ID`)) ENGINE=MyISAM AUTO_INCREMENT=4 DEFAULT CHARSET=latin1 ROW_FORMAT=DYNAMIC;INSERT INTO `tblcountry` (`ID`,`LCID`,`CountryCode`,`Country`,`CountryInt`,`Language`,`Standard`,`Active`) VALUES (1,1030,'DK','Danmark','Denmark','Dansk',0,1), (2,2057,'GB','England','England','English',1,1), (3,1031,'DE','Deutschland','Germany','Deutsch',0,1);finally how could i extract the database structure and data from the sql express management studio?so that i can copy it and re use it some other computer.Thanks.Jack.
I am trying to insert multiple rows into a table getting data from a web form.
I have 2 fields being passed from a web form to the below stored procedure @FormBidlistID Sample Data --> 500 @CheckBoxListContractors Sample Data --> 124,125,145,154,156,
The below DELETE function is working great. However for some reason or another the INSERT INTO sql command is not putting seperating the string and adding rows to the database? I am not sure where the error is occuring.
In the end I need the data to go into the database columns like so
CREATE PROCEDURE dbo.UpdateBidlistContractors @FormBidlistID int, @CheckBoxListContractors varchar(3999) AS DELETE FROM BidlistContractors WHERE BidlistID = @FormBidlistID DECLARE @ContractorID nvarchar(10) DECLARE @BidlistID nvarchar(10) DECLARE @startPosition int DECLARE @commaPosition int SET @startPosition =1 SET @commaPosition = 0 WHILE (@startPosition < LEN(@ContractorID)) BEGIN SET @commaPosition = CHARINDEX(' , ' , @ContractorID, @startPosition) SET @ContractorID= SUBSTRING(@ContractorID, @startPosition, @commaPosition - @startPosition) INSERT INTO BidlistContractors (BidlistID, ContractorID) VALUES (@BidlistID, @ContractorID) SET @startPosition = @startPosition + LEN(@ContractorID) + 1 END GO
Where am I going wrong? Thank you in advance for any help. :-)
I have a table that stores a structure. With the main material and component. The components are numbered.
Now I want to insert two components in the end of all components per main material in the table. These two new components are similar for all materials. How do I do this the best way?
I use Ftp to import a comma delim file.Problem:Day 1 file has 5 records. During the course of Day 1 ,users makechanges to a field in the five records.Day 2 has same 5 records as they were at the beginning of Day 1(withoutchanges made during Day 1) plus 1 new record.I need a statement that will only add the 1 new record to the mastertable and leave the other 5 fields alone.I have a composite key that prevents duplicate addition.My original idea was to have a master table(Day 1) and a dailytable(Day 2) and do some kind of join that would give we only therecords in Day 2 that do not appear in Day 1 using the Composite key.This has not worked.Any ideas?
When i try to insert new rows using SSIS into an emty table in my database, SSIS freezes. The same happens when i use a lookup on an empty table.
The only solution i have found is to manualy insert a row and remove it afterwards. When i have done this SSIS works without a glitch.
I start off by taking a script of a existing database and applying this script on a new database. So that i create an empty duplicate of my current database.
Any ideas why SSIS freezes? Do i have to initialize something first in my database? It is really annoying that i have to insert a row and delete a row into every table i want to use.
i have temp table name "#TempResult" with column names Memberid,Month,Year. Consider this temp table alredy has some rows from previuos query. I have one more table name "Rebate" which also has columns MemberID,Month, Year and some more columns. Now i wanted to insert rows from "Rebate" Table into Temp Table where MemberID.Month and Year DOES NOT exist in Temp table. MemberID + Month + Year should ne unique in Temp table
Searched around for a sample, maybe I missed one?I have an insert statement....INSERT INTO objScores (objID, studentId, courseId) VALUES ( @objID, @userId, @course) This works fine, but problem is it reads only the first value from the tableDECLARE c_oId CURSOR FOR SELECT objID FROM objStructure FOR READ ONLYThe userID and course will always be the same.But, the objID, there will be many in the table objStructure such as Lesson1, Lesson2..... Lesson19I need it to read all of them, 1 - 19 or whatever the count, could be 3 and insert all of them into the table.So, the insert needs to input Lesson1, userID, course ----- Lesson2, userId, course ----- Lesson3, userID, course ---- and so on.It must read the objID from the objStructure table and just tranfer all that is there that = the course.Links? Suggestions?Thanks all,Zath
In a t-sql 2012 sql script, I have the following script, that only works for a few records since the value of TST.dbo.LockCombination.seq only contains the value or 1 in most cases. Basically for every join listed below, there should be 5 records where each record has a distinct seq value of 1, 2, 3, 4, and 5. Thus my goal is to determine how to add the missing rows to the TST.dbo.LockCombination where there are no rows for seq values of between 2 to 5. I would like to know how to insert the missing rows and then do the following update statement. Thus can you show me the sql on how to add the rows for at least one of the missing sequence numbers?
UDATE LKC SET LKC.combo = lockCombo2 FROM [LockerPopulation] A JOIN TST.dbo.School SCH ON A.schoolnumber = SCH.type JOIN TST.dbo.Locker LKR ON SCH.schoolID = LKR.schoolID AND A.lockerNumber = LKR.number JOIN TST.dbo.Lock LK ON LKR.lockID = LK.lockID JOIN TST.dbo.LockCombination LKC ON LK.lockID = LKC.lockID WHERE LKC.seq = 2
A normal select statement looks like the following:
select * from TST.dbo.Locker LKR JOIN TST.dbo.Lock LK ON LKR.lockID = LK.lockID JOIN TST.dbo.LockCombination LKC ON LK.lockID = LKC.lockID where LKR.number in (000,001,1237)
In case you need the ddl statements for the tables affected here are the ddl statements:
CREATE TABLE [dbo].[Locker]( [lockerID] [int] IDENTITY(1,1) NOT FOR REPLICATION NOT NULL, [schoolID] [int] NOT NULL, [number] [varchar](10) NOT NULL, [serialNumber] [varchar](20) NULL, [type] [varchar](3) NULL, [locationID] [int] NULL,
I need to create a SQL statement to read the string, split it at the "," character, and insert it into individual rows. I also need to insert an ID (the same for all split values).
I have to create a script to install a database, and one of the tableshas about 200 rows of static data... I dont want to have to manuallytype in 200 insert statements, so is there a better way to do this? Ithought about maybe exporting the data into a CSV file and using somesort of procedure to insert the records that way... Any advise?
Hello, I have a problem. I am trying to pull data out of one system and bring it into a SQL Server database for faster retrieval. The original table does not have an identity column and has a composite primary key. The table I am inserting the data into matches the original table exactly except I have an Identity column that I need for the removal of duplicates. The task gets to the final commit and then fails telling me that it can't insert a NULL into an identity field. Why is it trying to insert a NULL, the field should be auto-populating. I did add the Identity field to the table after the SSIS package was already built, but I did go into the destination and fix the column mappings. For the Identity column I just selected the "skip" option or whatever it was. Why are the identities not being auto-inserted and why is SSIS throwing this error?
I am finding it difficult to find an example that allows for insertion of additional rows into a table, without dropping the table I'm inserting into. Or inserting specific values. Like this example..
[URL] ....
I have 6 table I am formatting the data to conform to the final table as I'm inserting it into, but none of these examples gives me the example needed. I am using SQL 2012.
<code> SELECT CONVERT(VARCHAR(50),[FName]) + ' ' + CONVERT(VARCHAR(50),[LName]) AS [CustName] ,CAST('ALARMCOM' as nvarchar(8)) as VendorName ,CONVERT(VARCHAR(25),[CUSTOMER_CS_ACCOUNT_NUMBER]) AS [Cust_ID] ,CONVERT(VARCHAR(40),[Charge_Description])as [ChargeType] ,CASE
There is a valuable script out there that will take the rows from a table and display INSERT STATEMENTS.
Good thing is this script converts the data to HEXADECIMAL ( or some other ) and we don't have to worry about dealing with apostrophies embedded in varchar fields.
Im creatting an SSIS project that uses an Data Flow OLE DB Source to read data from an SQL Table and import it into a Destination table using Data Flow OLE DB Destination. but now everytime I run the project it appends all the rows not the new data rows only. How can I make the application so that it appends only the new data from a source table to a destination table. Is there maybe another Data Flow Control that can copy source table to destination and the next time it runs it only copy new rows. or any other way to do this using SSIS.
I am writing a query to return some production data. Basically i need to insert either 1 or 2 rows into a Table variable based on a decision as to does the production part make 1 or 2 items ( The Raw data does not allow for this it comes from a look up in my database)
I can retrieve all the source data i need easily but when i come to insert it into the table variable i need to insert 1 record if its a single part or 2 records if its a twin part. I know could use a cursor but im sure there has to be an easier way !
Below is the code i have at the moment
declare @startdate as datetime declare @enddate as datetime declare @Line as Integer DECLARE @count INT
set @startdate = '2015-01-01' set @enddate = '2015-01-31'
I have a form to assign JOB SITES to previously created PROJECT. The JOB SITES appear in the DataList as it varies based on customer. It can be 3 to 50 JOB SITES per PROJECT. I have "PROJECT" table with all necessary fields for project information and "JOBSITES" table for job sites. I also created a new table called "PROJECTSITES" which has only 2 columns: "ProjectId" and "SiteId". What I am trying to do is to insert multiple rows into that "PROJECTSITES" table based on which checkbox was checked. The checkbox is located next to each site and I want to be able to select only the ones I need. Btw the Datalist is located inside of a formview and has it's own datasource which already distincts which JOBSITES to display. Sample: ProjectId - SiteId 1 - 5 1 - 9 1 - 16 1 - 18 1 - 20 1 - 27 1 - 31 ProjectId stays the same, only values for SiteId are being different. I hope I explaining it right. Do I have to use some sort of loop to go through the automatically populated DataList records and how do I make a multiple inserts to database table? We use SQL Server 2005 and VB for code behind. Please ask if I missed on some information. Thank you in advance.