Large SQL Insert On 3000+ Non Sequential Ids
Apr 25, 2008
im sure there must be function for this but i cantfind it anywhere
essentially i want to do a Large insert on a table around 3000 records
which links to a table using 3000+ non sequential IDS
the content will be the same but bar the ids
roughly along the lines of
Code Snippet
insert into comment
values
('Some comment',date,commentypeid, non sequential List of IDs)
i could do this fairly simply in c# using a List<int>
Code Snippet
foreach (int ID in IDS)
{
insert into comment
values
('Some comment',datetime.now,1234, ID)
}
surely there is a a function
Thanks in advance
Jake
View 2 Replies
ADVERTISEMENT
Nov 14, 2007
I have a web form with a text field that needs to take in as much as the user decides to type and insert it into an nvarchar(max) field in the database behind. I've tried using the new .write() method in my update statement, but it cuts off the text after a while. Is there a way to insert/update in SQL 2005 this without resorting to Bulk Insert? It bloats the transaction log and turning the logging off requires a call to sp_dboptions (or a straight-up ALTER DATABASE), which I'd like to avoid if I can.
View 6 Replies
View Related
Apr 12, 2008
I am writing a sql table, which is used to record a webpage's clicked number at different times.
The table structure is sth like:
click_number_1, time_1, click_number_2, time_2, ... ..., click_number_3000, time_3000
Cos there are too many columns, I feel it's impossible to use CREATE TABLE to add them by hand. I wonder if sql has statements like "for" statement in c++ to add those columns in a looping way.
Thanks!
View 9 Replies
View Related
Jul 6, 2006
Hi there,
I am
currently planning an ASP.NET and MS SQL project to distribute customer codes
to my customers. All the codes will be stored in the database, and then when a
customer registers, the next available code is linked to their account. Does
this sound like a feasible solution? The number of codes (ie. rows) could be
anywhere from 3,000 to 100,000 – but will only contain a couple of fields.
Also, it is
a separate company that is producing the codes (we’re offering them as an
incentive) and I was wondering the best way for them to get the codes to me, so
I can easily import them into MS SQL? CSV / Excel ??
This is the
main problem I need help with.
Thank you
all for your continued support, and all the work you kind guys do here!
Nathair
View 1 Replies
View Related
May 2, 2006
I can't seem to get SQL Server to run on my 64 bit box running Windows XP Pro 64-bit. Is there a fix for this? I really need SQL Server.
View 5 Replies
View Related
Jun 7, 2007
Hi All,
We are developing a system which will have to support more than 3000
subscribers. We will have to support both Transactional replication
and Merge replication.
I checked the following document about SQL 2005 replication <http://
www.microsoft.com/technet/prodtechnol/sql/2005/mergrepl.mspx>. The
document does not clearly specify what is the maximum number of
subscribers supported without a significant performance degradation.
The questions i have are:
1. Given the fact that there will be more than 3000 subscribers, there
will be more than 500-1000 subscribers trying to replicate at the same
time. Will be there be a performance degradtion in such a scenario
2. Has anyone used SQL Server 2005 in a scenario involving more than
3000 subscribers?
3. Will it be better if we develop our own system to perform
replication activity instead of relying on SQL Server 2005?
- Ngm
Mail me atnarasimha (DOT) gm (AT) gmail (DOT) com )
View 6 Replies
View Related
Nov 9, 2005
I have to copy a large (3000) amount of different tables from a Oracle machine into an
SQLServer machine.
I am able to do this using a (VB) script.
I use now several methods:
1) INSERT INTO TABLE1 SELECT * FROM SID1..DB.TABLE1 (SID1 is a linked server)
2) INSERT INTO TABLE1 SELECT * FROM OPENQUERY(SID1,'SELECT * FROM DB.TABLE1')
3) Also used OPENROWSET method (similar to 2)
For small tables this is fine, however for BIG tables (15M Rows/150Cols) the methods above are too slow.
If I compare the same copy action with a simple DTS, the DTS is 3 times faster.
Also, the DTS seems to bulk copy the data directly into the desired database while the
mentioned methods first fill the tempdb, then the transaction log of the desired database and
then finally the desired table (need very much extra space on your filesystem).
The total size of data is about 300GB.
Can anyone supply me with a simple example how to copy data from an Oracle table into a
SQLServer table in script (or SQL) that is as fast as the DTS and not filling my logfiles??
I read the bcp (which I use for import/export files) and bulk insert commands, but
I do not understand how to use them in this question.
View 14 Replies
View Related
May 18, 2006
Hi,
I have a large text which contains special characters and I need to insert it in "ErvTxt" field which is a varchar field.
I get an error:
Error:Line 1: Incorrect syntax near '1'.
Unclosed quotation mark before the character string ', 19 )'.
At line 4467SQL statement: INSERT INTO tbl1 (ZPERS, Status, ErvTxt, VENTS, DNR, ANAME, Fall, Zust, GER, Empf, BET, SSK, GA, JAHR, PZ, LAUFNR, RVLAUFNR, EDAT, ETIME, VDAT, UTLAUFNR, Bew, AINR, ErvNr, PrintStatus, FristNr, HDAT, HTIME, RVExportiert, CrDat, CrPers, GATT, AZ, NR) VALUES ( 'STS', -1, '0123'1'00'0138-33/02(1)'J'F'452'R020311'KL01'ZB'01'C'00001'03'i'000000000'430200001'20030116'0905''000000460'20030116'1100'0242'3'15'00'00000001000'''-'00000000000100''''''''00100û¥00000000000100û¥00000000000100û¥0000000000010000000000015699200000'0000000363û¥00000000001000û¥00000000001000 rrrrrrrrrrrr ssss''RI20030'101012003010'1200301160'0''''''''''''''''''''', '', 0, '0138-33/02(1)', 'J', 'F', 97, 'R020311', 'KL01', 'ZB', 1, 3, 'i', 0, 430200001, '2003-01-16 12:00:00', '1899-12-30 09:05:00', NULL, 460, 0, 2927, 1744, 0, 0, '2003-01-16 12:00:00', '1899-12-30 11:00:00', NULL, '2003-01-16 04:04:04', 'STS', 'C', '00001', 19 )
Can anyone tell me what's wrong?
Thanks.
View 4 Replies
View Related
May 21, 2013
What will be the best way to go to select summary data from a big table with detail records and insert it into another table.
The source table contains approx 100 million detail records for a couple of months. I have a select statement that select a summary of the latest month's data and it average about 15 million records for the month which i want to insert into another table.
In the past i just used a standard insert into statement but not the best way of doing it.
If a view is created with just the last months summary data and i select from the view will the performance be better or will it just add more overhead?
Will a SSIS package work better to insert the summary data?
View 1 Replies
View Related
Apr 9, 2008
how do i insert a large chunk of text into a table column. my project is to build a news website. where people can go and read news articles. the articles are provided by the author in word format, so how do i insert that news article into the table's column? any help would be appreciated
thanks
View 2 Replies
View Related
Feb 9, 2007
Hello!I have a developer that is playing around with some SQL statementsusing VB.NET. He has a test table in a SQL 2000 database, and he hasabout 2000 generated INSERT statements.When the 2000 INSERT statements are run in SQL query analyzer, all2000 rows are added to the table. When he tries to send the 2000statements to SQL Server through his app., a random number ofstatements do not get executed. But, SQL Profiler shows that each ofthe 2000 statements are getting sent to the server.I suggested that he add a "GO" statement at the end of the INSERTblock, but the statement fails when that is sent to the server.I know that this is not the ideal manner to insert bulk data to thesystem, but now we are all just curious as to why SQL server doesn'texecute each individual INSERT.Any thoughts?
View 3 Replies
View Related
Sep 4, 2007
Hello,
I have an SSIS package that basically inserts a large amount of data into a SQL Server table. The table contains sixty five columns, and a single load of data can contain two million records.
The 'loads' are split up into several 'daily' flat files. The package uses a ForEachFile loop to process each of the files. As each file is processed, the data from the files is loaded into a SQL Server table (destination).
Apparently, as the package is running, tempDB begins to consume a lot of disk space. The data file for TempDB on this particular server is configured to grow in 50mb increments with unrestricted file growth. During the last run of the package, the data file grew to 17GB. I ran the following and got the data file size down to 50mb;
USE TempDb
GO
DBCC SHRINKFILE(tempdev, 1)
Should I consider incorporating this code as part of the package, or is there something else I should consider to configure the SSIS package so that I don't run into space problems with TempDB?
Thank you for your help!
cdun2
View 2 Replies
View Related
Dec 19, 2006
Hi,
I'm testing Mirroing.
1) I have dedicated NIC for Mirroring - 100Mb
There is no issue with the network (file of 25MB goes in 2.5 seconds)
2) I'm issuing the next simple command
Select * into dbo.Table2 from dbo.Table1
3) the size of the table is 25MB
in async mode it takes 3-4 sec (as if it runs only local)
in sync mode it takes 25-29 sec !!!!
WHY ? IS THIS NORMAL ?
is there any configuration i can change ?
View 5 Replies
View Related
Oct 21, 2007
HiI have a VB.net web page which generates a datatable of values (3 columns and on average about 1000-3000 rows).What is the best way to get this data table into an SQL Server? I can create a table on SQL Server no problem but I've found simply looping through the datatable and doing 1000-3000 insert statements is slow (a few seconds). I'd like to make this as streamlined as possible so was wondering is there is a native way to insert all records in a batch via ADO.net or something.Any ideas?ThanksEd
View 1 Replies
View Related
Jul 20, 2005
I'm running a resource-intensive stored procedure, which reads a filewith about 50,000 lines with a BULK INSERT into a temp table, thengoes through it and inserts a record for each line into another table.While this procedure is running, SQL server stops accepting any otherrequests coming from the website.Question:Is there a way to make SQL server "listen", or emulate an "interrupt"to other requests while in the middle of a long intensive process?I really appreciate your replies.Thank you,Oleg.
View 5 Replies
View Related
Aug 30, 2007
I have a row that is being used log track plays on our website.
Here's the table:
CREATE TABLE [dbo].[Music_BandTrackPlays](
[ListenDate] [datetime] NOT NULL DEFAULT (getdate()),
[TrackId] [int] NOT NULL,
[IPAddress] [varchar](20)
) ON [PRIMARY]
There's a CLUSTERED INDEX on ListenDate ASC and a NON CLUSTERED INDEX on the TrackId.
I have a TRIGGER on the Music_BandTrackPlays table that looks like the following:
CREATE TRIGGER [trig_Increment_Music_BandTrackPlays_PlayCount]
ON [dbo].[Music_BandTrackPlays] AFTER INSERT
AS
UPDATE
Music_BandTracks
SET
Music_BandTracks.PlayCount = Music_BandTracks.PlayCount + TP.PlayCount
FROM
(SELECT TrackId, COUNT(*) AS PlayCount
FROM inserted
GROUP BY TrackId) AS TP
WHERE
Music_BandTracks.TrackId = TP.TrackId
When a simple INSERT statement is done on the Music_BandTrackPlays table, it can take quite a long time. When I remove the TRIGGER the INSERTs are immediate. The Execution plan for the TRIGGER shows that a 'Inserted Scan' is taking up most of the resources.
How exactly is the pseudo 'inserted' table formed?
For now, I think the easiest thing to do is update my logging page so it performs 2 queries. One to UPDATE the Music_BandTracks table and increment the counter, and perform the INSERT into the Music_BandTrackPlays table seperately.
I'm ok with that solution but I would really like to understand why the TRIGGER is taking so long. The 'inserted' pseudo table will be 1 row 99% of the time. Does SQL Server perform a table scan on all 20 million rows in order to determine what's new and put it in the inserted pseudo table?
Thanks!
View 6 Replies
View Related
Mar 28, 2015
Our system runs a SQL Server 2012 DB, it has a table (table_a) which has over 10M records. Our system have to receive data file from previous system daily which contains approximate 3M updated or new records for table_a. My job is to update table_a with the new data.
The initial solution is:
1 Create a table (table_b) which structur is as the same as table_a
2 Use BCP to import updated records into table_b
3 Remove outdated data in table_a:
delete from table_a inner join table_b on table_a.key_fileds = table_b.key_fields
4 Append updated or new data into table_a:
insert into table_a select * from table_b
As the test result, this solution is very inefficient. Step 3 costs several hours, e.g. How can I improve it?
View 9 Replies
View Related
Jun 1, 2006
Hi ,
How to INsert and Update Large Amount of Records (4 Lacs) into Destination Table Through Business Intelligence Studio Using SSIS Pacakge .How to Achieve this .i tryed with left outer join & conditional split but the problem its not able to insert & update records simultaneously . can any one give me a sample .
Thanks & Regards
Jeyakumar.M
View 3 Replies
View Related
Oct 31, 2007
Hello,I am creating a new database and I was advised to use Sequential Guids.I was reading some information and, as far as I understood, I can use NEWSEQUENTIALID. This can be used when I have a uniqueidentifier column as the key of a clustered index to avoid fragmentation during insert. Ok, so I use NEWSEQUENTIALID instead of NEWID.But I will use LINQ most of the time instead of Stored Procedures.So can I specify in my tables scripts to use Sequential Guids when, for example, a record is created? And am I right when using Sequential Guids?Here is a part of my code:-- Blogs ...
create table dbo.Blogs
(
BlogID uniqueidentifier not null
constraint PK_Blog primary key clustered,
Title nvarchar(400) null,
Description nvarchar(2000) null,
CreatedDate datetime null
)
-- Posts ...
create table dbo.Posts
(
PostID uniqueidentifier not null
constraint PK_Post primary key clustered,
BlogID uniqueidentifier not null,
AuthorID uniqueidentifier not null,
Title nchar(1000) null,
Body nvarchar(max) null,
UpdatedDate datetime not null,
IsPublished bit not null,
constraint FK_Posts_Blogs
foreign key(BlogID)
references dbo.Blogs(BlogID)
on delete cascade,
constraint FK_Posts_Users
foreign key(AuthorID)
references dbo.Users(UserID)
on delete cascade Thanks,Miguel
View 2 Replies
View Related
Apr 8, 2008
In Foxpro I would often run sequential queries on data. Each query would store the results in a cursor, which would be the datasource for the next query. It was an easy way to write several simple queries to accomplish something complex.
Like this:
select <fields> from table1 where <condition> into cursor work1
select <fields> from work1 where <condition>
How do I accomplish this in TSQL?
View 4 Replies
View Related
Jan 25, 2007
Hi everyone I stumbled across this problem recently and have tried to figure out a good solution and have come up with nothing.
Environment: ADO.NET, ASP.NET 2.0, MS SQL 2000, C#
Problem: I have a set of data which I order according to two columns of data in the set. One column is a varchar or a date value (depending on what the user has chosen from GUI controls) and the other is an identity column. This dataset contains lots of data so its not feasible to pull the entire dataset to the client, also everything must be performed via ADO.net (no stored procedures). When the user selects one of the records I load another page and drill down into the record details (think of this as the record details page). This is fine and easy but on this record details page I would like to keep a Next and Previous button so users can move to the next record in the set (remember this set is sorted somehow on the previous page). My question is how can I know what next record should be? It would seem I need to attach a sequential number to the rows of data so I can easily grab the next one in the set. Solutions to this seem to make use of a temp table which I dont think is possible via ado.net.
Is there a decent performing way to do this through ado.net?
Thanks for all insight.
View 4 Replies
View Related
Feb 28, 2007
Hi all,I might be getting this all wrong but bear with me.
I need to create some kind of Unique field in my DB that is
nonsequential. This is because I need it to be difficult to guess
ids if you have an example in front of you.I have looked at
8digit EAN codes which include a check digit system.( I use a base digit of the row_id for these) Can anyone tell me
how many uniques I can get out of this system?For my ID: I have looked at something along the lines of:
Hex(row_id) + "T" + Hex( Trimmed(EAN)
)
The "T" serves to split the numbers for when I am converting back.
So for example:row_id EAN_code Hex(row_id) + "T" + Hex( Trimmed(EAN) )
------------------------------------------------------------------------------------------
3166 00031663 C5ET7BAF
3167 00031673 C5FT7BB9
3168 00031686 C60T7BC6
Is this too easy to guess (once you can tell there are two hex numbers there?)
What do people think?
Thanks,Pete
View 4 Replies
View Related
Apr 4, 2008
I am currently trying to get a dataset of sequential decreasing calculations, but they all end up on one row. I need each of the secondary selects statements to show up as separate rows, any idea?Select
(select Rate * @Premium from ProfitLossTable where rateID =1)
, (select Rate * @Premium - @premium from ProfitLossTable where rateID =1)
,(select @premium - Rate * @Premium from ProfitLossTable where rateID =3)
View 3 Replies
View Related
Apr 27, 1999
I have a query which returns information about transactions similar to this:
Select account, trans_code, quantity
from ledger_table
This returns something like:
acct trans_code quantity
----- ---------- ------------
2 2 1000
2 3 500
2 3 300
3 2 100
3 2 500
etc.
What I need to do is add a lot number for each acct/trans code type. This is merely a sequential number for the transaction. This changes the output as follows:
acct trans_code quantity lot
----- ---------- ------------ ---
2 2 1000 1
2 3 500 1
2 3 300 2
3 2 100 1
3 2 500 2
3 3 1000 3
3 5 200 1
etc.
The lot number is reset for each grouping.
Does anyone have a way to do this outside of a cursor or temp table?
TIA, any help greatly appreciated.
View 1 Replies
View Related
Mar 8, 2006
Hello--I need to create a SQl statement (SQL SERVER 2000) which will do the following:
read table1 sequentially
select every "nth" record
write each selected record to and output table
Your help is greatly appreciated.
thanks,
MB
View 3 Replies
View Related
Jul 27, 2006
I've created a sproc that will provide a recordset for an Access report via a pass-thru query. The report is a production schedule.
Some of the runs on the schedule have a note associated with them. I need
to be able to number these notes, so that they can be displayed in the report
bibliography style. When I populate the data in the sproc, if the "notes"
field in my table contains any data I display a 1, else it's a 0
Like:
SELECT NOTE_FLAG = CASE WHEN ISNULL(SCHED_NOTE,'')<>'',1,0
FROM MASTER_SCHEDULE
The problem is, I really need to display sequential numbers instead of 1's
If I create a base set with this:
CREATE TABLE #TMPRST (
RECID INTEGER,
PRODUCT VARCHAR(10),
QTY FLOAT,
NOTE_FLAG INTEGER)
INSERT INTO #TMPRST
SELECT 1, 'ABC123', 4, 0
UNION ALL
SELECT 4, 'DEF123', 5, 1
UNION ALL
SELECT 5, 'ABC456', 12, 0
UNION ALL
SELECT 13, 'PQR789', 10, 1
How do I go back and convert the note_flag column to read
... 0
... 1
... 0
... 2
View 14 Replies
View Related
Oct 17, 2012
We are integrating all our applications/databases into one application/database. During the transition phase, I need to create a number of views based on the new database that mimic the old tables of the old databases, so the old programs can continue to function until they are gradually replaced.
In one of the views, I need to generate a sequential number. The value is unimportant, as long as it is unique in the dataset; strictly spoken, it even doesn't need to be sequential:
eg:
SELECT * FROM myView
should give
Code:
col1col2...id
lalacar..1
bababike..2
....
zsrdpen..896
ghrtink..897
SELECT * FROM myView ORDER BY col2
should give
Code:
col1col2...id
bababike..1
lalacar..2
..
ghrtink..45
..
zsrdpen..396
....
The view is created based on a number of tables.
View 1 Replies
View Related
Mar 17, 2004
I am trying to automatically insert records into my existing customer table. Is there a way when I insert these new records and assign the customer number that it can sequentially pick the next available unique customer number for each record that is inserted? for example the first record would be customer number 100, the next 101, and so on? Please advise.
View 4 Replies
View Related
Oct 25, 2013
I have a table with 13,000 rows, in one column called Prioirty each row has a value of 1.
Is it possible to use SQL to replace all of these '1' values with a sequential list. Example 1,2,3,4,5,6,7....all the way to 13,000?
View 3 Replies
View Related
Oct 27, 2006
this is a slight change to a fequently asked question around here. Ihave a table which contains a "sortorder" column where a user canspecify some arbitrary order for records to be displayed in. Usershave sometimes ordered records the way we used to number lines in aBASIC program (10,20,30, etc.). I'd like to do an update query and fixthis so that every record is in sequential order. I found an examplein this newsgroup of how to do this.However, I have a slight problem! Sometimes, the users duplicated thesortorders. So for example, I might have two records were thesortorder is 20. The query I found in this newsgroup does not work inthat case. Here is some code so that you can see what I mean.create table sorttest (label char(5),sortorder int)goinsert sorttest values ('joe',20)insert sorttest values ('dan',10)insert sorttest values ('jim',44)insert sorttest values ('tom',20)insert sorttest values ('jan',50)-- data dumpselect label, sortorder from sorttest order by sortorder-- I'd like to fix all of the sortorder fields so that they aresequentialupdate sorttestset sortorder = (select count(*)from sorttest subquerywhere sorttest.sortorder <= subquery.sortorder)-- note that tom and joe BOTH HAVE SORTORDER = 4select label, sortorder from sorttest order by sortorderdrop table sorttestThanks in advance for any help.
View 19 Replies
View Related
Sep 25, 2015
I have to increment sequential values for the following:
Current Data
Col1Â Â Â Â Â Â Â Â Â Â Col2Â Â Â Col3
12.345.678Â 0001Â Â 32
13.456.789Â 0002Â Â 43
Updated Data
Col1Â Â Â Â Â Â Â Â Â Â Col2Â Â Â Col3
12.345.678Â 0001Â Â 32
12.345.678Â 0002Â Â 32
12.345.678Â 0003Â Â 32
13.456.789Â 0002Â Â 43
13.456.789Â 0003Â Â 43
13.456.789Â 0004Â Â 43
What I need is: Increment up to 3 times the values in Col2. Need to identify the first number in Col2 and increase the number up to 3.
View 3 Replies
View Related
Feb 7, 2008
Hi,
I've a column say page#. This has by default 4 pages which I'm numbering as 1,2,3,4. I'll be adding pages to this table and the number needs to start from 5 (since 4 pages are already present)
Page# is not an identity column. I want to know if theres a function which can do the trick.
Thanks for your help!
Subha
View 4 Replies
View Related
Oct 30, 2007
Hi,
I need a sequential primary key to store a document number.
Is IDENTITY(1,1) the best way to achieve this on SQL Server 2000?
If I have a great number of users on my application doing an insert on the DB at the same time, could this lead to any problem like it trying to insert the same PK for some of them? Can this happen, and if so what can I do to prevent this?
I've been reading about NEWSEQUENTIALID() but it's only avaiable on SQL Server 2005 and i'm still using 2000.
Another question.. should I avoid using NEWID() and use IDENTITY instead? I've been reading that newid() lowers the system performance because all values are non sequential.
Thanks!
View 3 Replies
View Related