Loading Tables With An Identity Field Using DTS Packages
Aug 2, 2001
Is there a way ,(if so what is the syntax?), to set up a DTS package
that loads a table that has an identity column. I am trying to load the data from another table, (leaving the identity field unmapped), and de-selecting the "enable identity insert" from the advanced tab of the Data Transformation Properties window. I keep getting errors due to the table not allowing null values. I tried using the set_identity command, but this still did not work.
Any help would be appreciated.
TB
View 1 Replies
ADVERTISEMENT
Jul 17, 2006
We're having a performance problem with a package since an error occurred. The original error came from the Job Manager whicj was unable to start a thread. Our understanding is that this is a memory related issue and we'll deal with that.
What is realy odd is what happened after that. The package executes from a SQL Agent job that includes 3 other packages. Each package is stored on the file system. Package execution time for the affected package changed from less than a minute to over 5 minutes. The other packages continued to execute normally.
In checking the logs there is large time gap between the start of the SQL Agent step and the first pre-validation message that accounts for 4 minutes, as if there is some issue in loading the package. The issue is ongoing. Does anyone know what happens between the start of the SQL Agent job and the first prevalidate message? Is this some type of caching issue?
SQL Agent step starts at 4:05:27
First Prevalidate message: 4:09:24
Package Execution start: 4:09:57
End Execution 4:10:59
thanks
Peter
View 1 Replies
View Related
Jul 9, 2006
While I have learned a lot from this thread I am still basically confused about the issues involved.
.I wanted to INSERT a record in a parent table, get the Identity back and use it in a child table. Seems simple.
To my knowledge, mine would be the only process running that would update these tables. I was told that there is no guarantee, because the OLEDB provider could write the second destination row before the first, that the proper parent-child relationship would be generated as expected. It was recommended that I create my own variable in memory to hold the Identity value and use that in my SSIS package.
1. A simple example SSIS .dts example illustrating the approach of using a variable for identity would be helpful.
2. Suppose I actually had two processes updating these tables, running at the same time. Then it seems the "variable" method will also have its problems. Is there a final solution other than locking the tables involved prior to updating them or doing something crazy like using a GUID for the primary key!
3. We have done the type of parent-child inserts I originally described from t-sql for years without any apparent problems. (Maybe we were just lucky.) Is the entire issue simply a t-sql one or does SSIS add a layer of complexity beyond t-sql that needs to be addressed?
TIA,
Barkingdog
View 10 Replies
View Related
Oct 30, 2006
I have a project with 48 packages. When I press the green button for Start Debug, it seems like Integration Svcs. is re-loading or some type of validation occurs (screen flashes with each package being highlighted in solution manager) for all packages in the project before executing the specific package.
Also, If I close all the packages except one that is to be executed and press the green start debug button, all the packages are loaded before executing the specific package. There is always a delay before the actual package is executed when using the green start debug button.
If I right click on the specific package in solution manager and execute it, the package is executed immeadiately without any delay.
I did not have this issue until I tried to build and deploy the package.
Thanks.
View 3 Replies
View Related
Jun 30, 2006
I want to insert a new record into a table with an Identity field and return the new Identify field value back to the data stream (for later insertion as a foreign key in another table).
What is the most direct way to do this in SSIS?
TIA,
barkingdog
P.S. Or should I pass the identity value back in a variable and not make it part of the data stream?
View 12 Replies
View Related
Apr 26, 2015
We've recently upgraded to SQL Server 2014, and are now using SSIS integrated with Visual Studio. We have a SSIS project which contains about 20 packages which are nested in Sequence Containers and executed concurrently. These packages have been set up as project references.
The problem is that when I press the start button to run the packages, they all light up green reporting completion before the data has finished loading into the SQL database. If I press the stop button without waiting a sufficient length of time, then not all of the data gets loaded. i.e. a certain number of rows will be missing from some of the SQL tables.
If I click through to the individual package items and check the data flow progress while running, some of the data flows appear to hang at a certain number of rows without ever reaching completion. The number of rows indicated in the data flow is incorrect - i.e. it will count up to ~150,000 and stay there indefinitely in the running state, when in actual fact there are ~500,000 rows to load.
To clarify, the main package will show all items green and display the "Finished: Success" message in the log window, however when I drill through to certain packages in the set, they'll be stuck in the yellow running state, with no way of knowing whether they've actually completed or not.
My current workaround is to just wait a certain length of time before pressing the stop button. This bug doesn't seem to inhibit rows being loaded - it just incorrectly identifies the point when the load finishes, causing people to terminate the load prematurely.
This issue only occurs if I run the project from the main package container. If I execute the child packages individually, they correctly report the number of rows being loaded and light up green once complete.
View 2 Replies
View Related
Sep 14, 2006
I have a table with an integer field (contains test values like 2, 7,8,9,12,..) that I want to convert to an Identity field. How can this be done in t-sql?
TIA,
Barkingdog
View 1 Replies
View Related
Aug 10, 2001
I am attempting to import data from a Lotus Notes database using DTS. The SQL table I am importing to has an identifying auto-number. I can't insert directly into it because the SQL server should, however I get an error if I ignore it in the DTS package. Is there any way to get around this?
View 4 Replies
View Related
Aug 20, 2007
I have a table named PERSON and a field named PERSON_ID. How can I set this field to Autonumber? I know I have to use the IDENTITY command, but when I view this field in "design" view, all the IDENTITY options are grayed out.
How can I set this field with the IDENTITY properties?
Thanks
View 8 Replies
View Related
Mar 19, 2001
We are trying to load flat text files with upwards of 7 million records into a table on SQL. The table has a clustered index on 3 fields. We are sometimes able to complete smaller tables (500,000-750,000 records) and build the indexes prior to importing the data, however when we try the larger tables an error occurs :
Error at Destination for row number 6785496. Errors encountered so far in this task: 1
Location: somerge.c:1573
Expression: mrP->mrStatus!=MERGERUN::NONE
SPID: 11
Process ID: 173
None of the recods end up importing. The row number it gives is always the total number of records that was in the text file I was trying to import. I tried to import the text files first and then build the clustered indexes but a table with only 300,000 records ran for nearly 4 days without completing before we killed it.
Please help me on where to look......:(
Thanks,
Cheri
View 3 Replies
View Related
Mar 2, 2008
Does anyone have a script that can drop the Identity columns from all the tables in a database? Thanks
View 1 Replies
View Related
Dec 1, 2007
Hi,
I have got an xml file with size more than 2 GB. I have to load this file into tables. With 32 bit platform, I am unable to load this file using SSIS. Ram is 8 GB, but it is still bombing out. As I know it uses XML DOM Parser and tries to shred the file in memory and because of memory limition, it fails. Although I have already written code in C# using XmlTextReading object(implemetation of SAX Parser) to load data in tables, but I want to keep this loading process within the limits of DBAs.
I am stuck. Can someone guide me through the situation?
Thanks for your help!
Navnish
View 8 Replies
View Related
May 30, 2015
I am using Sql Server 2012. I have a table which has a field as Datetime (it is a table in Dynamics CRM 2011 so I have no control of the data type). Say this field is called BisStartDate. If I run this query in management studio.
select
BisStartDate, BisStartDateutc
from myTable
where _bisnumber=10375
I will get:
BisStartDate BisStartDateutc
2014-07-29 00:00:00.000 2014-07-29 05:00:00.000
*in CRM, datetime is saved in 2 fields, one is the current time, the other one is the utc time.
You can see the offset between the datetime and utc is 5 hours.
However when the same statement was running inside a SSIS package on the server, the result returned is:
BisStartDate BisStartDateutc
2014-07-28 23:00:00.0000000 2014-07-29 05:00:00.000
And if I do
datediff(MINUTE,CRMAF_BisSection.ttc_section_startdatetimeutc,CRMAF_BisSection.ttc_section_startdatetime)
I will get -5 if I run it in ManagementStudio and -6 is running on server package(running inside VisualStudio will be -5, same as running a query in ManagementStudio).
I think when the record was saved, “date” is 5 hours offset to UTC time but now the system use the current utc offset which is 6 hours. I just want to use the BisStartDate as it is. How do I let the SSIS turn off the conversion.
The same datatime is saved in another system then we compare them to check the data entry. Now because of this one hour difference, sometime the Day will be different.
View 8 Replies
View Related
Jun 23, 2004
I'm having a very irritating time trying to migrate data from a COBOL system to SQL Server.
One of the A/R Master files has approx. 200 columns.
I can export this file any number of ways that will (sometimes) load partially into my database, but always when the load succeeds, columns 75 through N simply contain NULL, even though there is data in the file. When the load fails in DTS, the error is always missing column delimiter. Using BULK INSERT the error is always something like data too long at column 75.
Putting all this together, I have deduced that something isn't working if I try
to load a staging table with more than 74 columns of data. This seems like a way-too-low threshold for a robust database, especially since SQL Server is supposed to be able to handle up to 1,024 columns per table.
Has anyone ever encountered this problem?
Thanks in advance for any help
View 2 Replies
View Related
Aug 31, 2006
Hi,
I am having trouble loading tables (within the same data flow) that have a foriegn key relationship defined between them. For instance:
Table A is a parent (one side of the relationship) to Table B (many side of the relationship).
I am trying to load Table A first within the data flow and then Table B after, but I get the following error:
[OCMD EntityRole Insert [2666]] Error: An OLE DB error has occurred. Error code: 0x80040E2F. An OLE DB record is available. Source: "Microsoft SQL Native Client" Hresult: 0x80040E2F Description: "The statement has been terminated.". An OLE DB record is available. Source: "Microsoft SQL Native Client" Hresult: 0x80040E2F Description: "The INSERT statement conflicted with the FOREIGN KEY constraint "FK_EntityRole_Entity". The conflict occurred in database "ODS", table "dbo.Entity", column 'EntityGuid'.".
I am currently using OLE DB commands to perform the inserts, I load table A and move on to then load Table B, I can see the records in Table A before trying to load Table B but for some reason Table B load still fails.
I was thinking maybe this has something to do with the transaction setting or Isolation level but have played with this to no avail (currently everything is the default - supported/serializable). Also, I am thinking maybe because the OLE DB commands are creating two seperate connections (they are using the same connection manager) the second one is unable to see the transactions from the other (first) connection (Table A)?
Is there a way around this without dropping (disabling) forigen keys before the load and adding them back in after? Would like to avoid this?
I would also like to avoid reading the data source multiple times. Everything I need is in the one source so I would like to populate multiple tables from the one source data stream instead of reading the same data 2,3 or 4 times etc.
Seems to me there must be a simple explanation/solution for this but I'm stuck at this point?
P.S.
I was intially using OLE DB destinations (because they are much faster) and was having the same issue, which made sense because the OLE DB destinations do not let you pass the data stream on so I had to multi cast to the destinations so they were loading at the same time. I would rather use the OLE DB destinations so if you have any ideas around how I could do this using those components that would be appreciated too!
Thanks!
View 3 Replies
View Related
Sep 16, 2007
I am trying to do here is to load different flat files to different tables:
For example, if the file name starts with "enrollment", then it goes to table "enrollment" table;
if the file name starts with "student", then it goes to "student" table.
For now, I created a foreach loop container for the each different files. So it ended up using several foreach loop containers. I am wondering if there is a way just to use one foreach loop containters to process the loading.
Anyone shed some light on this?? Thank you very much for your help!
View 1 Replies
View Related
Jul 14, 2004
Hi I have a question how to load data to tables linked by Foreign Keys in MSDE/SQL server. Example:
If I have 2 tables linked (by Foreign Key):
One table:
ITEM idITEM NAMEITEM CATEGORY (FK)
1cheese 2
And another:
Category IDCATEGORY NAME
1 household
2 food
3 general
How do I enter the load of data
Do I have to enter it as
1cheese2
or is there some way of entering it as
1cheesefood
TDS wizard does not allow me to transfer to views/querries what I thought would be a normal way as I would enter data to view(relevant to Access's form) and it would update related tables . When I wrote sql to do it it said I can not update my view table as too many tables would be affected(I had lookup tables empty then though)
I am doing it by number using TDS wizard to transfer it directly to the main table but there must be a better way
View 1 Replies
View Related
Apr 6, 2015
Currently we are trying to load the xml files into sqlserver tables by using ssis 2012,We are getting xml files as a column in source table ,so we have to push these xml files into destination tables.
I'm following the below way to perform this activity
[URL]
But We have standard XSD structure for all the xml files ,and if xml file matches the XSD structure then only we have to load ,else it should skip to next xml file.
View 1 Replies
View Related
May 6, 2008
Hi Guys,
I have to work with a poorly designed table :(, that has columns
IDINT,
thisIDvarchar(50) null,
parentIDvarchar(50) null,
Titlevarchar(255) null,
Description varchar(8000) null,
ProductTypevarchar(255) null,
The reason it is poorly designed is the table is used to hold questions and answers, all with a 1:1 relationship. Instead of having ID, ProductType, Question, Answer they have unfortunately adopted the approach of the above i.e:
id 1
thisID 3
parentid nuLL
DESCRIPTION: this is a question
id 20
thisID 3_1
parentID 3
DESCRIPTION: this is the answer to the question above
So I am writing a sproc that does this using a temp table. I got this far:
SET ANSI_NULLS ON
GO
SET QUOTED_IDENTIFIER ON
GO
-- =============================================
-- Author:Spencer
-- =============================================
ALTER PROCEDURE [dbo].[GetFAQs]
-- Add the parameters for the stored procedure here
@ProductType varchar(255)
AS
BEGIN
SET NOCOUNT ON;
-- Insert statements for procedure here
CREATE TABLE TEMP
(
IDINT,
thisIDvarchar(50) null,
parentIDvarchar(50) null,
Titlevarchar(255) null,
DescriptionQvarchar(8000) null,
DescriptionAvarchar(8000) null,
ProductTypevarchar(255) null,
)
SELECT
ID,
thisID,
parentID,
Title,
DescriptionQ,
DescriptionA,
ProductType
FROM
A2Z
WHERE
ProductType = @ProductType AND parentID IS NULL
END
GO
This gets all my questions for that product type.
What I need to do is load the questions into my temp table and then run through the a2z table again gaining the answers to the questions (the parentid holds the question ID). The answers then will also get loaded into the temp table.
Any bright sparks out there that can help me?
Cheers
View 9 Replies
View Related
Nov 21, 2007
Hi All,
I have problem in loading multiple excel sheets data in to according to that excelsheets tables in a DB.
All the excel sheets are in a folder,from that folder i have to acces all excel sheets.
For this i am unsing script task and one dataflow task.
But the error is coming in script task i am not able to put the path in the script..
Is this the correct way to do like this? Or any other way?
Can u please tell me the solution for this..Thanks in advance who are responding to this mail...
Maruthi..
View 9 Replies
View Related
Apr 6, 2006
Hi All,
I am facing a peculier problem. Problem definition goes like this,
I have one staging DB in which all the tables resides in Primary file and one production DB in which tables resides in 2 secondary files.
Now when iam trying to load the data from the table A in staging which is on primary file to the table A1 in production DB which in secondary file, all the data are going to error log instead of table A1.
Can you please tell me, where am i going wrong.
Regards,
Chetan
View 2 Replies
View Related
Jul 10, 2007
Hello Everybody,
Can anybody please let me know the procedure for loading data into Dimensions & Facts? And what is the sequence of loading?
Thanks in Advance
Rajesh
View 1 Replies
View Related
Aug 18, 2007
Data_Staging:
Unique_id
Gender
Ethnicity
Race
MCP_key
Admission_Dt
Discharge_Date
Enrollment_key
Reason
Disability
Income
Employment
I need to load the data from this table to three different tables all have foreign key relationship
Registration Table:
Registration_key ( Indetity) -PK
Unique_id
Gender
Ethnicity
Race
Episode:
Episode_Key(Identity)- PK
Registration_key (FK)
MCP_key
Admission_Dt
Discharge_Date
Assessment Table:
Assessment_Key(Identity) PK
Registraion_Key(FK)
Episode_Key(FK)
Enrollment_key
Reason
Disability
Income
Employment
View 1 Replies
View Related
Aug 17, 2007
when i use the query designer and add tables it takes 15-30 mins to load a table....so if i need 6 tables in my query thats like 6 times 30 minute load times...i'm using oracle as my data source
when i switch the source to oledb it adds tables within seconds the only thing is i need to use named parameters
any ideas?
View 14 Replies
View Related
Jul 2, 2006
hi friends,i have an identity column in my table.(e.x : id )and it's identity increment is 1.when i insert a row the id field is 1, and in next record the field is 2.....now , i delete second record(id=2)and now when i insert a record again , the id column is 3.i want to record be 2 instead 3.plz help me.thanks
View 2 Replies
View Related
Nov 7, 2007
Hi:
I created a small SQL Express database/ASP.net/C# application and in the learning process. Before I implement it I would like to re-set autonumber / identity field back to 1. Also, I need to start with the blank database. I am not sure how to approach that?
Can you assist?
Thanks
View 1 Replies
View Related
Mar 10, 2008
Hello friends,
I had created a web application and uploaded the application. Now the problem is with the DB. I had created the DB on the server (using script). But the fields that have identity field yes is not been set Now how I can set the fields identity field to yes. Fields are already there. Only I want to set there identity field to yes.Let me know how this can be done.
Thanks & RegardsGirish Nehte
View 2 Replies
View Related
Nov 5, 2005
Please, How can I get the value of the identity field of the register that I was including in the data base. I am using a stored procedure in SQLSERVER in a asp .net application and I need to show that for the user, it´s like the number of the reclamation.
View 2 Replies
View Related
Apr 2, 2001
If my table has an IDENTITY field, say, the table schema is:
CREATE TABLE BBB(
id int NOT NULL IDENTITY(1,1),
name varchar(20),
job varchar(40))
My data file, which does not carry the IDENTITY field
and its field terminator. The data file looks like this:
debbie cao,programmer
John Doe,engineer
Mary Smith,consultant
I tried to use a format file to bulk copy data from the
data file to the table. Never had any luck.
On the other hand, if I put a comma before the name field,
say, the data file looks like the following:
,debbie cao,programmer
,John Doe,engineer
,Mary Smith,consultant
then, bcp works fine.
But, my data file is automatically generated. It does not
suppose to have the leading comma. Without the leading comma,
I have no idea how may I make bcp work. The SQLBOL says it
can be done. Does anybody have an example to show me?
Please help, thanks!
View 2 Replies
View Related
Jul 12, 1999
Dear Fellow Administrators,
We are experiencing an identity problem on a table with 3.7 millions rows of data in the table. The identity field is not auto numbering any more and are wondering if anyone has a suggestion on how to get the auto number field working again without doing a bcp and restoring the table back to the database from bcp. Any suggestions?
Thanks in Advance,
Daimon
View 1 Replies
View Related
Jul 9, 1998
Greetings !
Having an unusual problem - have created several (20) tables in a database.
All of these tables have an identity field in them (defined as FIELDNAME int IDENTITY(1,1) NOT NULL ). In most of the tables this works as expected, but in 2, so far, when I do the first load (an insert based on a different database) the identity fields are all zero.
Have recreated the tables, changed the field name and location, etc.
Anyone have any clues ?
TIA
Eric
View 3 Replies
View Related
Apr 19, 1999
I need to update an identity field that I have on a table. How can
I go about doing this?
Thanks!
Toni
View 1 Replies
View Related
Mar 8, 2004
Folks
I am inserting some values into a table with the following stmt
Insert into table(number,name) values ('12','name')
In the table I have one more identity column ID. I know that I cannot insert a value in that column and the value is automatically increased once I insert a record. After this insert statment, I need to get the value
of the ID (the most recent one) in the next select statement.
ie Select @@identity from table (any condition????)
How do I get the most recent ID value? Actually I m inserting the records in a loop and the ID is increased for every insert.
Thanks for the help,
View 3 Replies
View Related
