Need Help Running 600,000 Lines Of Insert Statements
Aug 24, 2007
I have a file with about 600,000 lines of insert statement given to me by a developer.
There are basically 5 inserts into different tables for each Product or Item. Each Insert MUST run in the order specified and must complete before the next insert runs,
To complicate things further, there are triggers that fire on every insert and each trigger must complete its transaction before the next insert starts.
What is the best and most efficient way to run the inserts, while ensuring that each statement completes before the next.
Tried using Serialization but appears some of the transactions overlap and generate errors.
Tried disabling the triggers but run into other problems.
At this point I am tempted to run each statement manually
hello friends my one insert code lines is below :) what does int32 mean ? AND WHAT IS DIFFERENT BETWEEN ONE CODE LINES AND SECOND CODE LINES :)Dim conn As New SqlConnection(ConfigurationManager.ConnectionStrings("ConnectionString").ConnectionString) Dim cmd As New SqlCommand("Insert into table1 (UserId) VALUES (@UserId)", conn) 'you should use sproc instead cmd.Parameters.AddWithValue("@UserId", textbox1.text) 'your value Try conn.Open()Dim rows As Int32 = cmd.ExecuteNonQuery() conn.Close()Trace.Write(String.Format("You have {0} rows inserted successfully!", rows.ToString())) Catch sex As SqlExceptionThrow sex Finally If conn.State <> Data.ConnectionState.Closed Then conn.Close() End If End Try MY SECOND INSERT CODE LINES IS BELOWDim SglDataSource2, yeni As New SqlDataSource() SglDataSource2.ConnectionString = ConfigurationManager.ConnectionStrings("ConnectionString").ToString SglDataSource2.InsertCommandType = SqlDataSourceCommandType.Text SglDataSource2.InsertCommand = "INSERT INTO urunlistesi2 (kategori1) VALUES (@kategori1)" SglDataSource2.InsertParameters.Add("kategori1", kategoril1.Text)Dim rowsaffected As Integer = 0 Try rowsaffected = SglDataSource2.Insert()Catch ex As Exception Server.Transfer("yardim.aspx") Finally SglDataSource2 = Nothing End Try If rowsaffected <> 1 ThenServer.Transfer("yardim.aspx") ElseServer.Transfer("urunsat.aspx") End If
Does anyone know of a way to make BULK INSERT or bcp ignore blank lines in the file? I am having trouble with a bunch of data files coming back with 1 or 2 blank lines at the end, and it causes the entire bcp to fail.
I suppose I could write a utility to trim the files but that seems a bit overkill. Any thoughts?
<p>Hi , I would like to know anyway to insert two strings into a row with 2 lines created?My codes are as below If NodeName = "subProductPrice" Then If xmlrder.NodeType = Xml.XmlNodeType.Text Then SubPrPriceList.Add(xmlrder.Value) For i = 0 To SubPrPriceList.Count - 1 SubPrPrice = CStr(SubPrPriceList.Item(i)) PriceBrkDownStr &= SubPrPrice Next PricePerDay = SvcDate & PriceBrkDownStr dr("dailyPrice") = PricePerDay End If End If Although both the SvcDate and PriceBrkDownStr are inserted into the same row but they are displayed into one line such as below: <u>dailyPrice </u>02/03/2007 03/03/2007 120 230 Any idea how to make the date and price separate into two rows in the same table grid row? thanks =)</p>
Does an UPDATE statment lock the entire table or just the rows that will be affected by the UPDATE?
I ask because -
Can I run UPDATE statements in parallel on the same table on the same column. The need for doing this is because the table is a large fact table. I plan to execute the same UPDATE statements on different time sections of the data to expedite the processing.
If the UPDATE statment lock the entire table then I cannot run an UPDATE in parallel. If the UPDATE statement just locks the rows that will be affected then maybe I can because rows affected will be different for each UPDATE.
My goal is to run a bunch of select statements from different tables in one database and have them all insert to the same columns/table in the new database. Do I need a new data source for each statement, or is there a way to run all the statements in one set seeing as they all have the same destination. I keep receiving the SQL statement improperly ended error when trying.
Hi guys! I have these commands that insert into two tables, if condition 1 is met, it will insert into the first table, if the second condition is met, it will insert into the second table. Is there a way for the insert statements to be merged so that I won't be executing two statements? Dim update_phase_before As New SqlCommand("INSERT INTO TE_shounin_todokesho_jizen (syain_No,date_kyou,time_kyou) SELECT syain_No,date_kyou,time_kyou FROM TE_todokesho WHERE TE_todokesho.b_a='before'", cnn) Dim update_phase_after As New SqlCommand("INSERT INTO TE_shounin_todokesho_jigo (syain_No,date_kyou,time_kyou) SELECT syain_No,date_kyou,time_kyou FROM TE_todokesho WHERE TE_todokesho.b_a='after'", cnn)
i'm quite new with sql and i have an question about het insert statements. Is the order of the insert statements from any importance? and why is/isn't it?
I need to write some insert statements, 1 per table, ~100 tables, all having the approximante form:
Select Into TableA Select * From TableB
Except that I need explicit statements:
Select Into TableA Col1, Col2, Col3, ... ColN Values ...
The reason is that I need to preserve the current identity values (it's a replication setup scenario). I can set Identity_Insert On, but then it wants the explicit column names and values.
Is there a wizard or utility that will generate the statements for me? With 100 tables in the db, I'm not looking forward to writing it all :-)
TIA, Arthur
PS. Given that it's Easter weekend, if you have an answer could you please e me directly? Thanks!
I am creating my companys' database and I have a small problem that must be solved.
I have a pictures table: PicturesTable ------------- ProductID int ForeignKey Picture nvarchar(30) ...
(A product can have many pictures & the ProductID is unique for any product, but not for the table of pictures).
What I want to do is to somehow do a procedure to: 1) Check if any images (for a productID) exists in the table 2) if they do not exist then add the appropriate images into the table 3) if the images exist, then update the images with the new one that I have.
What I thought was to just delete all the images from the table for the specific product:
DELETE FROM PicturesTable WHERE ProductID = '10-11'
and then add the appropriate images: INSERT INTO PicturesTable (ProductID, Picture) VALUES ('10-11', 'Dir1/Pic1.gif') INSERT INTO PicturesTable (ProductID, Picture) VALUES ('10-11', 'Dir1/Pic2.gif') INSERT INTO PicturesTable (ProductID, Picture) VALUES ('10-11', 'Dir1/Pic3.gif')
but I do not like a lot this idea because if a user tries to read the pictures for that product (at the same time I was deleting them) s/he would get nothing. Is any other way that I can do it please?
Hi Friends, I have the following set of Insert Statements that calculates sums for various criteria and inserts a row at a time onto my table. I have a row for every month starting from January with sums for 4 severity levels. So for 12 months that would be 48 Insert Statements and if I want to do this for 4 different types of [EName] that would be 48 * 4 = 192 Insert Statements. Is there a better way to write this. Thanks for your help
INSERT INTO dbo.tbl_Ticket ([EName], TrendMonth, [Severity Level], [Count]) SELECT 'OVERALL' AS [EName], DATENAME(MONTH, '1/1/06') AS TrendMonth, 1 , Sum([Count]) FROM dbo.tbl_Ticket WHERE (TrendMonth LIKE 'January' and [Severity Level] = 1)
INSERT INTO dbo.tbl_Ticket ([EName], TrendMonth, [Severity Level], [Count]) SELECT 'OVERALL' AS [EName], DATENAME(MONTH, '1/1/06') AS TrendMonth, 2 , Sum([Count]) FROM dbo.tbl_Ticket WHERE (TrendMonth LIKE 'January' and [Severity Level] = 2)
INSERT INTO dbo.tbl_Ticket ([EName], TrendMonth, [Severity Level], [Count]) SELECT 'OVERALL' AS [EName], DATENAME(MONTH, '1/1/06') AS TrendMonth, 3 , Sum([Count]) FROM dbo.tbl_Ticket WHERE (TrendMonth LIKE 'January' and [Severity Level] = 3)
INSERT INTO dbo.tbl_Ticket ([EName], TrendMonth, [Severity Level], [Count]) SELECT 'OVERALL' AS [EName], DATENAME(MONTH, '1/1/06') AS TrendMonth, 4 , Sum([Count]) FROM dbo.tbl_Ticket WHERE (TrendMonth LIKE 'January' and [Severity Level] = 4)
INSERT INTO dbo.tbl_Ticket ([EName], TrendMonth, [Severity Level], [Count]) SELECT 'OVERALL' AS [EName], DATENAME(MONTH, '2/1/06') AS TrendMonth, 1 , Sum([Count]) FROM dbo.tbl_Ticket WHERE (TrendMonth LIKE 'February' and [Severity Level] = 1)
INSERT INTO dbo.tbl_Ticket ([EName], TrendMonth, [Severity Level], [Count]) SELECT 'OVERALL' AS [EName], DATENAME(MONTH, '2/1/06') AS TrendMonth, 2 , Sum([Count]) FROM dbo.tbl_Ticket WHERE (TrendMonth LIKE 'February' and [Severity Level] = 2)
INSERT INTO dbo.tbl_Ticket ([EName], TrendMonth, [Severity Level], [Count]) SELECT 'OVERALL' AS [EName], DATENAME(MONTH, '2/1/06') AS TrendMonth, 3 , Sum([Count]) FROM dbo.tbl_Ticket WHERE (TrendMonth LIKE 'February' and [Severity Level] = 3)
INSERT INTO dbo.tbl_Ticket ([EName], TrendMonth, [Severity Level], [Count]) SELECT 'OVERALL' AS [EName], DATENAME(MONTH, '2/1/06') AS TrendMonth, 4 , Sum([Count]) FROM dbo.tbl_Ticket WHERE (TrendMonth LIKE 'February' and [Severity Level] = 4)
I would like to get opinions about the code below and what I can do to improve it. I don't know if I am using the best techniques in this code. I am not running into any problems, but I am assuming there has to be cleaner ways of doing this.
Also I am trying to figure out why the INSERT INTO statement in the query is giving me the error: "The column prefix '#ttsku' does not match with a table name or alias name used in the query."
The purpose of the code is to select all the item records from a linked server (Non MS SQL) and bring it into a temp table. I did this because I can't create a cursor to the other DB.
Using this temp table, look if the SKU table has the item in it, if it does then update the record, otherwise I need to create the record and fill in the values from the temp table.
Finally I need to delete any records that don't exist in the temp table.
Here is the code:
SELECT pt_mstr.pt_part as part, pt_mstr.pt_desc1 as desc1, dbo.udf_GetEntry(cd_det.cd_cmmt,1,';') as custdesc, dbo.udf_GetEntry(cd_det.cd_cmmt,2,';') as caformat, dbo.udf_GetEntry(cd_det.cd_cmmt,3,';') as plformat, dbo.udf_GetEntry(cd_det.cd_cmmt,6,';') as eaformat, (pt_mstr.pt__qad24 * pt_mstr.pt__qad25) as pallqty, case when um_mstr.um_conv is not null then round((pt_net_wt / (cast(pt_drwg_loc as numeric) /um_conv)),0) else round((pt_net_wt / cast(pt_drwg_loc as numeric)),0) end as packqty, pt_mstr.pt_drwg_loc as packsize, pt_mstr.pt_drwg_size as packum, dbo.udf_GetEntry(cd_det.cd_cmmt,4,';') as dist1, dbo.udf_GetEntry(cd_det.cd_cmmt,5,';') as dist2, pt_mstr.pt_user2 as brand, pt_mstr.pt__qad24 as ti, pt_mstr.pt__qad25 as hi, pt_mstr.pt_status as status INTO #ttsku FROM mfgprod..pub.pt_mstr pt_mstr left join mfgprod..pub.cd_det cd_det on (cd_det.cd_ref = pt_mstr.pt_part and cd_det.cd_type = 'MK' and cd_det.cd_lang = 'US' and cd_det.cd_seq = 0) left join mfgprod..pub.um_mstr um_mstr on (um_mstr.um_um = pt_mstr.pt_net_wt_um and um_mstr.um_alt_um = pt_drwg_size and um_mstr.um_part = '') WHERE pt_part_type = 'FG' and (pt_status = 'A' or pt_status = 'AMTO') and (pt_drwg_loc <> '' and pt_drwg_loc <> '0')
declare c_part cursor for select * from #ttsku
open c_part
fetch next from c_part
while @@fetch_status = 0 begin
if exists (select * from sku where sku.part = #ttsku.part) BEGIN update sku set sku.Desc1 = #ttsku.desc1, sku.CustDesc = ##ttsku.custdesc where sku.part = #ttsku.part END ELSE BEGIN insert into sku (sku.part, sku.desc1) select #ttsku.part, #ttsku.desc1 END
fetch next from c_part
END -- while
close c_part deallocate c_part
DELETE FROM sku WHERE not exists (select * from #ttsku where #ttsku.part = sku.part)
Hello all. Got bit of a long winded question here...........so here we go lol.
OK.......ive got data on an Excel spreadsheet. Ive set the spreadsheet up as a linked server and i'm creating a set of insert statements from it by using the following code:
For most records this generates a correct insert statement.........for example:
INSERT INTO TRAINREC (EMPLOY_REF, COURSE_NAME) VALUES ('153', 'NMA Panel');
However.........my problems start when the value for course name is containes an ' character. If it does the insert statement generated is incorrect. For example:
INSERT INTO TRAINREC (EMPLOY_REF, COURSE_NAME) VALUES ('139', 'Annual Accounting in Lloyd's Market');
can anyone suggest any ideas on how to get round this? Also if i havent explained it clearly enough just let me know and i can try and expand on it.
I'm using version 3.5.5386.0 of SqlServerCompact Edition for windows mobile.
My first tests indicates that an insert statement with '?' ist 20 % faster than with '@p1' parameters:
"INSERT INTO Itest(PKey,value1,value2,value3,Date1) VALUES (?,?,?,?,?)" "INSERT INTO Itest(PKey,value1,value2,value3,Date1) VALUES (@P1,@2,@3,@P4,@P5)"
I've no explanation for that. I reported some problems with the '?' parameters in my other task http://forums.microsoft.com/MSDN/ShowPost.aspx?PostID=2715685&SiteID=1 So I'm a little bit afraid to use this version.
Is there a recommandation which version to use for Inserts?
I want to prepare insert statements for the data of all tables in my database. I want this to create seed data. Just by using those scripts i can insert the data in another db. Please suggest ho wto prepare in easy way.
Is it possible to export data from tables to SQL insert statements? Got SQL 2005 developer, visual studio 2005, and Visio enterprise architect, if that makes any difference. I already found this tool but it costs money.
I rememeber they used to have this option to generate data script for the table in SQL 2000, but I can not find it in SQL 2005. I need to move one table from one database to another, but I need to generate SQL Insert Statements...
I'm using the following code to add some data to a table: Dim rand As Random = New Random Dim num As Int32 = rand.Next(10000000) Dim strConn as string = "......." Dim sql as string = "INSERT INTO tblitemid (itemid, userid, datetime, supplier, comment, commenttype, uniqueid) VALUES ('" & label1.Text & "', '" & user.identity.name & "', '"& System.DateTime.Now & "','3763' ,'" & textbox1.text & "' , 'C' ,'" & num & "')" Dim conn as New SQLConnection(strConn) Dim Cmd as New SQLCommand(sql, conn) Try conn.Open() Catch SQLExp as SQLException Response.Write ("An SQL Server Error Occurred: " & e.toString()) Finally cmd.ExecuteNonQuery conn.Close() End Try
As far as I can tell the code works fine. But for some odd reason I click the button, the code execute and the page closes as it should, but the data is never inserted into the database. I cant really seem to pick up on any paterns for why this would be happening. As a rough guess I'd say it doesnt insert the data 1 out of every 5 times or so it seems. Anyone every have any experience with this? Any comments would be helpful, cuz I'm at a loss. If youd like to see more code let me know.... Thanks, Scott
Is this a limitation of SQL server. I am running a quite complex sp that I wrote which uses exec to execute an SQL string. Running the SP produces the desired results but if I try to use this sp with an insert statement then I get an error message that exec cannot be nested in an insert statement.....any help would be appreciated
Our existing DW's ETL was written in a very complex fashion by the previous team. They use DTS package lookups to read a row in the Source SQL Server database see if that row exists in the taget SQL Server database. If the row does not exist, they use ActiveX scripts to INSERT the row in the target SQL Server database. If it exists, they update the row on the target side. How would you do this in SSIS? Apologize if this sounds like a basic question, however, I would have done this via Stored Procedures or SQL Scripts especially since it involves SQL Servers alone. Appreciate any help.
I am looking to transfer a database to another db but I just need to copy the data (not drop the tables and recreate them). Is this possible?
I was thinking of backing up the database in a set of insert statements in a file and execute this file to get the desired result. Or alternatively, to use a data flow task with multiple tables if possible.
first of all is it possible? if so, what am i doing wrong with this
INSERT into TB2
(
ClientCode, EngagementCode, EngagementDescription
)
SELECT (SELECT dbo.tarCustomer.CustID FROM dbo.tPA00175 INNER JOIN dbo.tarCustomer ON dbo.tPA00175.CustKey = dbo.tarCustomer.CustKey INNER JOIN dbo.tPA00007 ON dbo.tPA00175.intJobKey = dbo.tPA00007.intJobKey),
NULL,
SELECT (SELECT dbo.tPA00175.chrJobNumber FROM dbo.tPA00175 INNER JOIN dbo.tarCustomer ON dbo.tPA00175.CustKey = dbo.tarCustomer.CustKey INNER JOIN dbo.tPA00007 ON dbo.tPA00175.intJobKey = dbo.tPA00007.intJobKey)
the first select statement for works fine, but the second one and all after i get a syntax error near 'select'.
this is just a shortened version of the statement. how would i run select statements for a table to be inserted into with different column names. also with items that are hard coded like the 'null'. thanks
Hello!I have a developer that is playing around with some SQL statementsusing VB.NET. He has a test table in a SQL 2000 database, and he hasabout 2000 generated INSERT statements.When the 2000 INSERT statements are run in SQL query analyzer, all2000 rows are added to the table. When he tries to send the 2000statements to SQL Server through his app., a random number ofstatements do not get executed. But, SQL Profiler shows that each ofthe 2000 statements are getting sent to the server.I suggested that he add a "GO" statement at the end of the INSERTblock, but the statement fails when that is sent to the server.I know that this is not the ideal manner to insert bulk data to thesystem, but now we are all just curious as to why SQL server doesn'texecute each individual INSERT.Any thoughts?
I'd I have a problem I'd like to post CREATE TABLE and INSERT statementsthat will create my table and insert data into the table.I can use the scripting feature in Enterprise Manager to generate CREATETABLE scripts.Is there a script I can run that will generate INSERT statements so I caninclude sample data.Thanks
How are you guys doing? Hope all is well. This is my problem...I accidentally deleted a bunch of my SQL scripts and would like to generate the INSERT statements from Management Studio. I am currently using SQLExpress. Is there a way I can accomprish my task? Kindly advise.
Hi, I just want you to know that I am very young in ASP.NET world so please bear with me.I have been looking for an answer to my problem, but unfortunately I couldn’t find one. So I created a user here on www.asp.net just for making this post. Before I continue I just want to apologies if there is another post where this question is already answered.
Please watch this Print Screen I just took: � http://www.bewarmaronsi.com/Capture.JPG “ As you can see the “INSERT, UPDATE, and DELETE Statements� are disabled, and that’s exactly my problem. I tried with an MS access database and it works perfect, but when I use a MS SQL database this field gets disabled for some reason. The MDF file is located in the App_data folder and is called ASPNETDB. And when I try to add custom SQL statements, it gives me Syntax error near “=�. Something like that. I bought the Total Training Set1 package and it works perfect in their examples. I just want to thank you for reading my post and I hope that you got some useful information for me. By the way, I’, from Sweden so you have to excuse me if my English is rusty. Thanks! PS: Can it be that I’m running windows Vista?
I have problem in using the SQLDataSource. When in VS 2005 I drag and drop the SQLDataSource onto my page and then add a GridView control.I bind the GridView control to the SQLDataSource control. But the problem is it does not generate the INSERT, UPDATE, and DELETE statements. The dialog box is inactive. The screenshots may help. please help me in this regard. I also tried it for Accesscontrol but the same problem. Sorry for my poor English!. thanks in advance
the screenshot links: http://img139.imagevenue.com/img.php?image=28285_2_122_937lo.JPGhttp://img205.imagevenue.com/img.php?image=27550_1_122_203lo.JPG
There is a valuable script out there that will take the rows from a table and display INSERT STATEMENTS.
Good thing is this script converts the data to HEXADECIMAL ( or some other ) and we don't have to worry about dealing with apostrophies embedded in varchar fields.
Could someone tell me how to get the data from all tables of the database in the form of insert script? We are moving our databse from SQL Server 2000 to SQL Server 2005. The scripts for the Database, Tables, Views , Procedures, Functions have been obtained and it is only the data that is remaining. Some are small tables with 5 to 6 columns but there are some with 50 odd columns. A friend of mine told me about a procedure that returns a dataset with INSERT statements by passing a table name as a parameter. Such procedure would be of great help.
