Large SQL Insert On 3000+ Non Sequential Ids

Apr 25, 2008



im sure there must be function for this but i cantfind it anywhere

essentially i want to do a Large insert on a table around 3000 records
which links to a table using 3000+ non sequential IDS

the content will be the same but bar the ids


roughly along the lines of




Code Snippet
insert into comment
values
('Some comment',date,commentypeid, non sequential List of IDs)






i could do this fairly simply in c# using a List<int>



Code Snippet

foreach (int ID in IDS)
{
insert into comment
values
('Some comment',datetime.now,1234, ID)
}




surely there is a a function

Thanks in advance

Jake

View 2 Replies


ADVERTISEMENT

Can I Insert/Update Large Text Field To Database Without Bulk Insert?

Nov 14, 2007

I have a web form with a text field that needs to take in as much as the user decides to type and insert it into an nvarchar(max) field in the database behind.  I've tried using the new .write() method in my update statement, but it cuts off the text after a while.  Is there a way to insert/update in SQL 2005 this without resorting to Bulk Insert? It bloats the transaction log and turning the logging off requires a call to sp_dboptions (or a straight-up ALTER DATABASE), which I'd like to avoid if I can.

View 6 Replies View Related

How To Add 3000 Columns Into A Sql Table?

Apr 12, 2008



I am writing a sql table, which is used to record a webpage's clicked number at different times.
The table structure is sth like:
click_number_1, time_1, click_number_2, time_2, ... ..., click_number_3000, time_3000
Cos there are too many columns, I feel it's impossible to use CREATE TABLE to add them by hand. I wonder if sql has statements like "for" statement in c++ to add those columns in a looping way.
Thanks!

View 9 Replies View Related

ASP.net And MS SQL - Importing 3000-100000 Rows

Jul 6, 2006

Hi there,

 

I am
currently planning an ASP.NET and MS SQL project to distribute customer codes
to my customers. All the codes will be stored in the database, and then when a
customer registers, the next available code is linked to their account. Does
this sound like a feasible solution? The number of codes (ie. rows) could be
anywhere from 3,000 to 100,000 – but will only contain a couple of fields.

 

Also, it is
a separate company that is producing the codes (we’re offering them as an
incentive) and I was wondering the best way for them to get the codes to me, so
I can easily import them into MS SQL? CSV / Excel ??

This is the
main problem I need help with.

 

 

 

Thank you
all for your continued support, and all the work you kind guys do here!

 

Nathair

View 1 Replies View Related

SQL SERVER 2000 + AMD 3000 64bit BOX

May 2, 2006

I can't seem to get SQL Server to run on my 64 bit box running Windows XP Pro 64-bit. Is there a fix for this? I really need SQL Server.

View 5 Replies View Related

SQL 2005 Replication Performance: More Than 3000 Subscribers

Jun 7, 2007

Hi All,


We are developing a system which will have to support more than 3000
subscribers. We will have to support both Transactional replication
and Merge replication.


I checked the following document about SQL 2005 replication <http://
www.microsoft.com/technet/prodtechnol/sql/2005/mergrepl.mspx>. The
document does not clearly specify what is the maximum number of
subscribers supported without a significant performance degradation.


The questions i have are:
1. Given the fact that there will be more than 3000 subscribers, there
will be more than 500-1000 subscribers trying to replicate at the same
time. Will be there be a performance degradtion in such a scenario


2. Has anyone used SQL Server 2005 in a scenario involving more than
3000 subscribers?


3. Will it be better if we develop our own system to perform
replication activity instead of relying on SQL Server 2005?

- Ngm

Mail me atnarasimha (DOT) gm (AT) gmail (DOT) com )

View 6 Replies View Related

How To Copy The Content Of 3000 Oracle Tables Into SQLServer?

Nov 9, 2005

I have to copy a large (3000) amount of different tables from a Oracle machine into an
SQLServer machine.
I am able to do this using a (VB) script.
I use now several methods:

1) INSERT INTO TABLE1 SELECT * FROM SID1..DB.TABLE1 (SID1 is a linked server)

2) INSERT INTO TABLE1 SELECT * FROM OPENQUERY(SID1,'SELECT * FROM DB.TABLE1')

3) Also used OPENROWSET method (similar to 2)

For small tables this is fine, however for BIG tables (15M Rows/150Cols) the methods above are too slow.
If I compare the same copy action with a simple DTS, the DTS is 3 times faster.
Also, the DTS seems to bulk copy the data directly into the desired database while the
mentioned methods first fill the tempdb, then the transaction log of the desired database and
then finally the desired table (need very much extra space on your filesystem).
The total size of data is about 300GB.

Can anyone supply me with a simple example how to copy data from an Oracle table into a
SQLServer table in script (or SQL) that is as fast as the DTS and not filling my logfiles??
I read the bcp (which I use for import/export files) and bulk insert commands, but
I do not understand how to use them in this question.

View 14 Replies View Related

Large Text Insert

May 18, 2006

Hi,

I have a large text which contains special characters and I need to insert it in "ErvTxt" field which is a varchar field.

I get an error:

Error:Line 1: Incorrect syntax near '1'.
Unclosed quotation mark before the character string ', 19 )'.
At line 4467SQL statement: INSERT INTO tbl1 (ZPERS, Status, ErvTxt, VENTS, DNR, ANAME, Fall, Zust, GER, Empf, BET, SSK, GA, JAHR, PZ, LAUFNR, RVLAUFNR, EDAT, ETIME, VDAT, UTLAUFNR, Bew, AINR, ErvNr, PrintStatus, FristNr, HDAT, HTIME, RVExportiert, CrDat, CrPers, GATT, AZ, NR) VALUES ( 'STS', -1, '0123'1'00'0138-33/02(1)'J'F'452'R020311'KL01'ZB'01'C'00001'03'i'000000000'430200001'20030116'0905''000000460'20030116'1100'0242'3'15'00'00000001000'''-'00000000000100''''''''00100û¥00000000000100û¥00000000000100û¥0000000000010000000000015699200000'0000000363û¥00000000001000û¥00000000001000 rrrrrrrrrrrr ssss''RI20030'101012003010'1200301160'0''''''''''''''''''''', '', 0, '0138-33/02(1)', 'J', 'F', 97, 'R020311', 'KL01', 'ZB', 1, 3, 'i', 0, 430200001, '2003-01-16 12:00:00', '1899-12-30 09:05:00', NULL, 460, 0, 2927, 1744, 0, 0, '2003-01-16 12:00:00', '1899-12-30 11:00:00', NULL, '2003-01-16 04:04:04', 'STS', 'C', '00001', 19 )

Can anyone tell me what's wrong?

Thanks.

View 4 Replies View Related

Insert And Select From Large Table

May 21, 2013

What will be the best way to go to select summary data from a big table with detail records and insert it into another table.

The source table contains approx 100 million detail records for a couple of months. I have a select statement that select a summary of the latest month's data and it average about 15 million records for the month which i want to insert into another table.
In the past i just used a standard insert into statement but not the best way of doing it.

If a view is created with just the last months summary data and i select from the view will the performance be better or will it just add more overhead?

Will a SSIS package work better to insert the summary data?

View 1 Replies View Related

Insert Large Text Files Into Db

Apr 9, 2008

how do i insert a large chunk of text into a table column. my project is to build a news website. where people can go and read news articles. the articles are provided by the author in word format, so how do i insert that news article into the table's column? any help would be appreciated


thanks

View 2 Replies View Related

Large Number Of INSERT Statements - Not All Are Executed

Feb 9, 2007

Hello!I have a developer that is playing around with some SQL statementsusing VB.NET. He has a test table in a SQL 2000 database, and he hasabout 2000 generated INSERT statements.When the 2000 INSERT statements are run in SQL query analyzer, all2000 rows are added to the table. When he tries to send the 2000statements to SQL Server through his app., a random number ofstatements do not get executed. But, SQL Profiler shows that each ofthe 2000 statements are getting sent to the server.I suggested that he add a "GO" statement at the end of the INSERTblock, but the statement fails when that is sent to the server.I know that this is not the ideal manner to insert bulk data to thesystem, but now we are all just curious as to why SQL server doesn'texecute each individual INSERT.Any thoughts?

View 3 Replies View Related

Large Insert Causing Problem With TempDB

Sep 4, 2007

Hello,
I have an SSIS package that basically inserts a large amount of data into a SQL Server table. The table contains sixty five columns, and a single load of data can contain two million records.

The 'loads' are split up into several 'daily' flat files. The package uses a ForEachFile loop to process each of the files. As each file is processed, the data from the files is loaded into a SQL Server table (destination).

Apparently, as the package is running, tempDB begins to consume a lot of disk space. The data file for TempDB on this particular server is configured to grow in 50mb increments with unrestricted file growth. During the last run of the package, the data file grew to 17GB. I ran the following and got the data file size down to 50mb;


USE TempDb

GO

DBCC SHRINKFILE(tempdev, 1)


Should I consider incorporating this code as part of the package, or is there something else I should consider to configure the SSIS package so that I don't run into space problems with TempDB?

Thank you for your help!

cdun2

View 2 Replies View Related

Performance Issue On A Singel Large Insert

Dec 19, 2006



Hi,



I'm testing Mirroing.

1) I have dedicated NIC for Mirroring - 100Mb

There is no issue with the network (file of 25MB goes in 2.5 seconds)

2) I'm issuing the next simple command

Select * into dbo.Table2 from dbo.Table1

3) the size of the table is 25MB



in async mode it takes 3-4 sec (as if it runs only local)

in sync mode it takes 25-29 sec !!!!

WHY ? IS THIS NORMAL ?

is there any configuration i can change ?

View 5 Replies View Related

Best Way To Insert Large Amounts Of Data From A Webform To SQL Server 2005

Oct 21, 2007

HiI have a VB.net web page which generates a datatable of values (3 columns and on average about 1000-3000 rows).What is the best way to get this data table into an SQL Server? I can create a table on SQL Server no problem but I've found simply looping through the datatable and doing 1000-3000 insert statements is slow (a few seconds). I'd like to make this as streamlined as possible so was wondering is there is a native way to insert all records in a batch via ADO.net or something.Any ideas?ThanksEd

View 1 Replies View Related

Interrupt Processing Of A Large Insert Process In SQL Server 2000.

Jul 20, 2005

I'm running a resource-intensive stored procedure, which reads a filewith about 50,000 lines with a BULK INSERT into a temp table, thengoes through it and inserts a record for each line into another table.While this procedure is running, SQL server stops accepting any otherrequests coming from the website.Question:Is there a way to make SQL server "listen", or emulate an "interrupt"to other requests while in the middle of a long intensive process?I really appreciate your replies.Thank you,Oleg.

View 5 Replies View Related

AFTER INSERT Trigger Takes Forever On A Large Table (20 Million Rows)

Aug 30, 2007

I have a row that is being used log track plays on our website.

Here's the table:


CREATE TABLE [dbo].[Music_BandTrackPlays](
[ListenDate] [datetime] NOT NULL DEFAULT (getdate()),
[TrackId] [int] NOT NULL,
[IPAddress] [varchar](20)
) ON [PRIMARY]


There's a CLUSTERED INDEX on ListenDate ASC and a NON CLUSTERED INDEX on the TrackId.

I have a TRIGGER on the Music_BandTrackPlays table that looks like the following:


CREATE TRIGGER [trig_Increment_Music_BandTrackPlays_PlayCount]
ON [dbo].[Music_BandTrackPlays] AFTER INSERT
AS
UPDATE
Music_BandTracks
SET
Music_BandTracks.PlayCount = Music_BandTracks.PlayCount + TP.PlayCount
FROM
(SELECT TrackId, COUNT(*) AS PlayCount
FROM inserted
GROUP BY TrackId) AS TP
WHERE
Music_BandTracks.TrackId = TP.TrackId


When a simple INSERT statement is done on the Music_BandTrackPlays table, it can take quite a long time. When I remove the TRIGGER the INSERTs are immediate. The Execution plan for the TRIGGER shows that a 'Inserted Scan' is taking up most of the resources.

How exactly is the pseudo 'inserted' table formed?

For now, I think the easiest thing to do is update my logging page so it performs 2 queries. One to UPDATE the Music_BandTracks table and increment the counter, and perform the INSERT into the Music_BandTrackPlays table seperately.

I'm ok with that solution but I would really like to understand why the TRIGGER is taking so long. The 'inserted' pseudo table will be 1 row 99% of the time. Does SQL Server perform a table scan on all 20 million rows in order to determine what's new and put it in the inserted pseudo table?

Thanks!

View 6 Replies View Related

SQL Server 2012 :: How To Quickly Update / Insert 3M Records In Large Table

Mar 28, 2015

Our system runs a SQL Server 2012 DB, it has a table (table_a) which has over 10M records. Our system have to receive data file from previous system daily which contains approximate 3M updated or new records for table_a. My job is to update table_a with the new data.

The initial solution is:

1 Create a table (table_b) which structur is as the same as table_a

2 Use BCP to import updated records into table_b

3 Remove outdated data in table_a:
delete from table_a inner join table_b on table_a.key_fileds = table_b.key_fields

4 Append updated or new data into table_a:
insert into table_a select * from table_b

As the test result, this solution is very inefficient. Step 3 costs several hours, e.g. How can I improve it?

View 9 Replies View Related

Need Help For SSIS Package Creation With INSERT,UPDATE Large Amount Of Records Through Business Intelligence Studio

Jun 1, 2006

Hi ,

How to INsert and Update Large Amount of Records (4 Lacs) into Destination Table Through Business Intelligence Studio Using SSIS Pacakge .How to Achieve this .i tryed with left outer join & conditional split but the problem its not able to insert & update records simultaneously . can any one give me a sample .
Thanks & Regards
Jeyakumar.M

View 3 Replies View Related

Sequential Guids

Oct 31, 2007

 Hello,I am creating a new database and I was advised to use Sequential Guids.I was reading some information and, as far as I understood, I can use NEWSEQUENTIALID. This can be used when I have a uniqueidentifier column as the key of a clustered index to avoid fragmentation during insert. Ok, so I use NEWSEQUENTIALID instead of NEWID.But I will use LINQ most of the time instead of Stored Procedures.So can I specify in my tables scripts to use Sequential Guids when, for example, a record is created? And am I right when using Sequential Guids?Here is a part of my code:-- Blogs ...
create table dbo.Blogs
(
BlogID uniqueidentifier not null
constraint PK_Blog primary key clustered,
Title nvarchar(400) null,
Description nvarchar(2000) null,
CreatedDate datetime null
)

-- Posts ...
create table dbo.Posts
(
PostID uniqueidentifier not null
constraint PK_Post primary key clustered,
BlogID uniqueidentifier not null,
AuthorID uniqueidentifier not null,
Title nchar(1000) null,
Body nvarchar(max) null,
UpdatedDate datetime not null,
IsPublished bit not null,
constraint FK_Posts_Blogs
foreign key(BlogID)
references dbo.Blogs(BlogID)
on delete cascade,
constraint FK_Posts_Users
foreign key(AuthorID)
references dbo.Users(UserID)
on delete cascade Thanks,Miguel

View 2 Replies View Related

Sequential Queries

Apr 8, 2008



In Foxpro I would often run sequential queries on data. Each query would store the results in a cursor, which would be the datasource for the next query. It was an easy way to write several simple queries to accomplish something complex.

Like this:

select <fields> from table1 where <condition> into cursor work1
select <fields> from work1 where <condition>

How do I accomplish this in TSQL?

View 4 Replies View Related

T-SQL Sequential Record Navigation

Jan 25, 2007

Hi everyone I stumbled across this problem recently and have tried to figure out a good solution and have come up with nothing.
Environment:  ADO.NET, ASP.NET 2.0, MS SQL 2000, C#
Problem:  I have a set of data which I order according to two columns of data in the set.  One column is a varchar or a date value (depending on what the user has chosen from GUI controls) and the other is an identity column.  This dataset contains lots of data so its not feasible to pull the entire dataset to the client, also everything must be performed via ADO.net (no stored procedures).  When the user selects one of the records I load another page and drill down into the record details (think of this as the record details page).  This is fine and easy but on this record details page I would like to keep a Next and Previous button so users can move to the next record in the set (remember this set is sorted somehow on the previous page).  My question is how can I know what next record should be?   It would seem I need to attach a sequential number to the rows of data so I can easily grab the next one in the set.  Solutions to this seem to make use of a temp table which I dont think is possible via ado.net.
Is there a decent performing way to do this through ado.net?
 Thanks for all insight.
 

View 4 Replies View Related

Create Non-Sequential Unique ID (EAN?)

Feb 28, 2007

Hi all,I might be getting this all wrong but bear with me.
I need to create some kind of Unique field in my DB that is
nonsequential.  This is because I need it to be difficult to guess
ids if you have an example in front of you.I have looked at
8digit EAN codes which include a check digit system.( I use a base digit of the row_id for these)  Can anyone tell me
how many uniques I can get out of this system?For my ID: I have looked at something along the lines of:

Hex(row_id) + "T" + Hex( Trimmed(EAN)
)                                  
The "T" serves to split the numbers for when I am converting back.
So for example:row_id   EAN_code   Hex(row_id) + "T" + Hex( Trimmed(EAN) )
------------------------------------------------------------------------------------------
3166    00031663    C5ET7BAF   
3167    00031673    C5FT7BB9   
3168    00031686    C60T7BC6   

Is this too easy to guess (once you can tell there are two hex numbers there?)
What do people think?
 Thanks,Pete  

View 4 Replies View Related

Getting Sequential Rows From A Calculation

Apr 4, 2008

I am currently trying to get a dataset of sequential decreasing calculations, but they all end up on one row.  I need each of the secondary selects statements to show up as separate rows, any idea?Select

(select Rate * @Premium from ProfitLossTable where rateID =1)
, (select Rate * @Premium - @premium from ProfitLossTable where rateID =1)
,(select @premium - Rate * @Premium from ProfitLossTable where rateID =3)  

View 3 Replies View Related

Query Help - Sequential Numbers

Apr 27, 1999

I have a query which returns information about transactions similar to this:

Select account, trans_code, quantity
from ledger_table

This returns something like:

acct trans_code quantity
----- ---------- ------------
2 2 1000
2 3 500
2 3 300
3 2 100
3 2 500
etc.

What I need to do is add a lot number for each acct/trans code type. This is merely a sequential number for the transaction. This changes the output as follows:

acct trans_code quantity lot
----- ---------- ------------ ---
2 2 1000 1
2 3 500 1
2 3 300 2
3 2 100 1
3 2 500 2
3 3 1000 3
3 5 200 1
etc.

The lot number is reset for each grouping.

Does anyone have a way to do this outside of a cursor or temp table?

TIA, any help greatly appreciated.

View 1 Replies View Related

SQL: Sequential Random Select

Mar 8, 2006

Hello--I need to create a SQl statement (SQL SERVER 2000) which will do the following:

read table1 sequentially
select every "nth" record
write each selected record to and output table

Your help is greatly appreciated.
thanks,
MB

View 3 Replies View Related

Sequential Numbering Problem

Jul 27, 2006

I've created a sproc that will provide a recordset for an Access report via a pass-thru query. The report is a production schedule.
Some of the runs on the schedule have a note associated with them. I need
to be able to number these notes, so that they can be displayed in the report
bibliography style. When I populate the data in the sproc, if the "notes"
field in my table contains any data I display a 1, else it's a 0
Like:

SELECT NOTE_FLAG = CASE WHEN ISNULL(SCHED_NOTE,'')<>'',1,0
FROM MASTER_SCHEDULE


The problem is, I really need to display sequential numbers instead of 1's

If I create a base set with this:

CREATE TABLE #TMPRST (
RECID INTEGER,
PRODUCT VARCHAR(10),
QTY FLOAT,
NOTE_FLAG INTEGER)

INSERT INTO #TMPRST
SELECT 1, 'ABC123', 4, 0
UNION ALL
SELECT 4, 'DEF123', 5, 1
UNION ALL
SELECT 5, 'ABC456', 12, 0
UNION ALL
SELECT 13, 'PQR789', 10, 1



How do I go back and convert the note_flag column to read
... 0
... 1
... 0
... 2

View 14 Replies View Related

Sequential Number In A View?

Oct 17, 2012

We are integrating all our applications/databases into one application/database. During the transition phase, I need to create a number of views based on the new database that mimic the old tables of the old databases, so the old programs can continue to function until they are gradually replaced.

In one of the views, I need to generate a sequential number. The value is unimportant, as long as it is unique in the dataset; strictly spoken, it even doesn't need to be sequential:

eg:
SELECT * FROM myView

should give

Code:
col1col2...id
lalacar..1
bababike..2
....
zsrdpen..896
ghrtink..897
SELECT * FROM myView ORDER BY col2

should give

Code:
col1col2...id
bababike..1
lalacar..2
..
ghrtink..45
..
zsrdpen..396
....

The view is created based on a number of tables.

View 1 Replies View Related

Assign Sequential Numbers

Mar 17, 2004

I am trying to automatically insert records into my existing customer table. Is there a way when I insert these new records and assign the customer number that it can sequentially pick the next available unique customer number for each record that is inserted? for example the first record would be customer number 100, the next 101, and so on? Please advise.

View 4 Replies View Related

Sequential Numbers In Column

Oct 25, 2013

I have a table with 13,000 rows, in one column called Prioirty each row has a value of 1.

Is it possible to use SQL to replace all of these '1' values with a sequential list. Example 1,2,3,4,5,6,7....all the way to 13,000?

View 3 Replies View Related

Sequential Number In An Update

Oct 27, 2006

this is a slight change to a fequently asked question around here. Ihave a table which contains a "sortorder" column where a user canspecify some arbitrary order for records to be displayed in. Usershave sometimes ordered records the way we used to number lines in aBASIC program (10,20,30, etc.). I'd like to do an update query and fixthis so that every record is in sequential order. I found an examplein this newsgroup of how to do this.However, I have a slight problem! Sometimes, the users duplicated thesortorders. So for example, I might have two records were thesortorder is 20. The query I found in this newsgroup does not work inthat case. Here is some code so that you can see what I mean.create table sorttest (label char(5),sortorder int)goinsert sorttest values ('joe',20)insert sorttest values ('dan',10)insert sorttest values ('jim',44)insert sorttest values ('tom',20)insert sorttest values ('jan',50)-- data dumpselect label, sortorder from sorttest order by sortorder-- I'd like to fix all of the sortorder fields so that they aresequentialupdate sorttestset sortorder = (select count(*)from sorttest subquerywhere sorttest.sortorder <= subquery.sortorder)-- note that tom and joe BOTH HAVE SORTORDER = 4select label, sortorder from sorttest order by sortorderdrop table sorttestThanks in advance for any help.

View 19 Replies View Related

Increment Sequential Values

Sep 25, 2015

I have to increment sequential values for the following:

Current Data
Col1           Col2    Col3
12.345.678  0001   32
13.456.789  0002   43

Updated Data
Col1           Col2    Col3
12.345.678  0001   32
12.345.678  0002   32
12.345.678  0003   32
13.456.789  0002   43
13.456.789  0003   43
13.456.789  0004   43

What I need is: Increment up to 3 times the values in Col2. Need to identify the first number in Col2 and increase the number up to 3.

View 3 Replies View Related

Sequential Numbering In Queries

Feb 7, 2008

Hi,

I've a column say page#. This has by default 4 pages which I'm numbering as 1,2,3,4. I'll be adding pages to this table and the number needs to start from 5 (since 4 pages are already present)

Page# is not an identity column. I want to know if theres a function which can do the trick.

Thanks for your help!

Subha

View 4 Replies View Related

SQL Server 2000 + Sequential Primary Key

Oct 30, 2007

Hi,
 I need a sequential primary key to store a document number.
Is IDENTITY(1,1) the best way to achieve this on SQL Server 2000?
If I have a great number of users on my application doing an insert on the DB at the same time, could this lead to any problem like it trying to insert the same PK for some of them? Can this happen, and if so what can I do to prevent this?
I've been reading about NEWSEQUENTIALID() but it's only avaiable on SQL Server 2005 and i'm still using 2000.
 Another question.. should I avoid using NEWID() and use IDENTITY instead? I've been reading that newid() lowers the system performance because all values are non sequential.
 Thanks!

View 3 Replies View Related







Copyrights 2005-15 www.BigResource.com, All rights reserved