i have to import a csv file into a database via ssis and so far everything works fine. but now i have to add a column where a incremental int should be inserted, so every row should have a unique number ... i tried to realize this by using derived col transofrmation, but without success ... has someone an idea how to do this?
View 10 RepliesHi All
I have an existing table of students, the school now has decided to give then a General Record No.The table structure is as follows
Students
++++++++++++++
Name
Address
TelNo.
++++++++++++++
I have modified the table as
Students
++++++++++++++
GRN.
Name
Address
TelNo.
++++++++++++++
The data already exists in the table . I just want to add the GRN No. that would be incremental
E.g.
it should be 20060001 for the 1st record
20060002 for the second
The rest data should be as it is
Plz help
Thanks
I use management studio on the sql server. Each time i want to run scripts over new data, i have to delete the old files in a database and import new ones (from csv files to .dbo). These are the same files everytime except that the data change. Is it possible to make an automated proces for this import?
View 2 Replies View RelatedHi guys,I followed the ASP.net official tutorial to create a DAL & Business Logic Layer (http://www.asp.net/learn/dataaccess/tutorial02cs.aspx). I have a table with a int ID field. I wish to write a function to add a new entry into this table but have the ID field auto-increment.The ID field is set as the Identity Column and has a Identity Increment & Seed of "1". If I manually go to the table and insert a new record leaving the ID value null it automatically increments. But if I create a C# function to add a new entry I get an error saying that the ID field can't be Null. Is there any way to use the Update method as shown on line 14 below to add a new entry but with it automatically incrementing? I did create a function called InsertDevice that simply inserts the other fields using a SQL INSERT and it auto-increments fine, just wondering if there is a way to do it using the DataTable and the Update method? Thanks for any help!!! 1 public bool AddDevice(string make, string model)
2 {
3 //cannot have the same device entered twice!
4 if (Adapter.FillDeviceCountByMakeModel(make, model) == 1)
5 return false;
6
7 RepositoryDataSet.DevicesDataTable devices = new RepositoryDataSet.DevicesDataTable();
8 RepositoryDataSet.DevicesRow device = devices.NewDevicesRow();
9
10 device.make = make;
11 device.model = model;
12
13 devices.AddDevicesRow(device); << Error thrown Here!
14 int rows_affected = Adapter.Update(devices);
15
16 return rows_affected == 1;
17 }
We have implemented a very small reporting database which has a main table that started off small and has now grown to around half a million rows. Initially, there were no indexes on the table apart from a clustered index, but as the data has grown, performance has dropped and so we have added a number of indexes. This has resolved the performance issues.
Before creating the indexes SQL Server had auto created a number of statistic objects (_WA_Sys_000... etc). After creating the indexes, new statistic objects where created for the new indexes. In some cases, there are duplicate statistics (auto and index) for the same columns.Should I go through and drop the duplicate auto statistics? Will having duplicates cause issues at all?
Hello all,I'm using SS2K on W2k.I'v got a table say, humm, "Orders" with two fields in the PK:OrderDate and CustomerID. I would like to add an "ID" column whichwould be auto-increment (and would be the new PK). But, I would reallylike to have orders with the oldest OrderDate having the smallest IDnumber and, for a same OrderDate, I'd to have the smallest CustomerIDfirst. So my question is:How could I add an auto-increment column to a table and make it createits values in a particular order (sort by OrderDate then CustomerIDhere)?In the real situation, the table I want to modify has around 500krecords and the PK has 5 fields and I want to sort on three of them.Thanks for you helpYannick
View 7 Replies View RelatedSince upgrading from SQL Server Management Studio 2008 R2, I've noticed that it no longer autosaves queries that have not been manually saved first. If a file has been manually saved the autorecover files end up in the following directory:
%appdata%MicrosoftSQL Server Management Studio11.0AutoRecoverDatSolution1
However, I have ended up in the situation where I have unsaved queries when my computer has crashed and have not been able to recover them.
I have also found references to .sql files stored in temp files in the following directory, but the files here seem to be very haphazardly caught:
%userprofile%AppDataLocalTemp
I have an MS SQL Server table with a Job Number field I need this field to start at a certain number then auto increment from there. Is there a way to do this programatically or within MSDE?
Thanks, Justin.
Hi guys,
I am trying to get a result like below without using the cursor.
col1 col2 col3 col4
1111 date uniquenumber 6
1111 date uniquenumber 5
1111 date uniquenumber 4
1111 date uniquenumber 3
1111 date uniquenumber 2
1111 date uniquenumber 1
2222 date uniquenumber 4
2222 date uniquenumber 3
2222 date uniquenumber 2
2222 date uniquenumber 1
3333 date uniquenumber 2
3333 date uniquenumber 1
the column that say unique number is unique and is not duplicated and date column might have duplicates
Please advise whether it is possible to write a query without cursor.
Thanks,
Anu
Hi!
I have a request of having a select statement displaying a list of values in descending order and another column that tells what record it is.
something like this:
id | value | no
4 345 1
7 234 2
2 143 3
9 32 4
3 4 5
...
is this possible?
can it be something like this:
select id, value, increment(1,1) as no from tblTable
order by value desc
thanks.
I am looking to create a incremental value based on the resulting insert that I am using. There already is another field being used as an identity field. I have a beginning value that I just want to add the row number to for the insert.
insert into lineitem select substring(group_id,4,len(ltrim(rtrim(group_id)))-3) as co_code,
0,0,(case when enddate < cast(month(getdate()) as varchar(10))+cast(day(getdate()) as varchar(10))
then 'Prior' else 'Current' end ),
left(acct_type,2) as bene_type, convert(smalldatetime,left(ltrim(rtrim(eff_date)), 8)),
0,trans_amt,0,0,convert(smalldatetime,left(ltrim(r trim(sett_date)),8)),
ltrim(rtrim(b.fname)) + ' ' + ltrim(rtrim(b.lname)) as payee,0,0,a.ssn,'Y',999+count(*),
(case when isnull(b.location,'') = '' then '' else b.location end) as location
from mbi_tran_temp a
left join enrollees b on a.ssn = b.ssn and
ltrim(rtrim(a.group_id)) = ltrim(rtrim(b.mbicode))
The '999+count(*)' is where I would like to have the incremental value.
My Question is of understanding Incremental Backup. How does this work,
Does it go by new files added or activate on file size.
Hi,
Is there any way to maintain history of data while incremental load without using the Slowly changing dimension(SCD) concept?
Hi Friends please let me know how can we incrementally load a destination table with source table. bearing in mind that we need to track that there are no duplicates in the destination table. I need to load only changed or new data in the final load. Please give me some examples also. I am tryin this from last 2 days as I am totally new to SSIS.
View 11 Replies View Relatedhi guys,
I'm a newbie in sql.
anybody know how,
if I want to update incrementally using integration service.
say I have 2 server. we want to took the data from server A and put the data to server B.
but we want just the changes in server A that send to server B.
regards,
-dedys
I have done an bulk upload and I would like to start doing incremental uploads. I just want to upload only the new records that have been added to my data source ( free foxpro tables ). Can anybody point me to an example or info to accomplish this.
Thanks,
Sergio
Dear all,I have an SQL table where I am doing insert from an asp.net web project.I have a primary Key, set to be incremented by 1on each insert.I have another column, Col1 that should be incremented on each insert, How can I do this?I need to have 2 incremental columns? do u have any idea on how to do this?I am doing this right now by calling a select statement in the Add stored procedure on this table, I read the last value of col1 and then I increment it by 1, this work fine on development environement, but when many users are accessing this website, I will have wrong values for col1. Any idea??Thanks.
View 6 Replies View RelatedHi Everybody,
Right now i have a DTS package thats populating a fact table,I'm trying to do an incrmental update of the fact table (which has a sequential#id).
any ideas ,please
Thanks,
Paris
hi,
i need to create two instances of db and transfer incremental uploads from one db to another without having to transfer the entire table of data again and again. how should i go about it? what commands should i use?
thanks in advance
ramya.
i have a catalog and add directory which has 10,000 documents and all are index but if i add 1000 documents to that directory and i don't want those 1000 documents to be indexed. i want only previous 10,000 index document and don't want to new document to be indexed. is there any way can stop the new document to be indexed, please let me it's bit urgent.
Thanking you in anticipation
Hi All
I've created an SSIS package that loads data from source to destination, using Lookup and conditional Split to check New rows and changed rows for one table.
Now I want to take this father by loading data for multible table more that 100. I did it in T-SQL using dynamic sql and cursor.
How can I achive this using SSIS.
I was wondering if there is a way to schedule a tast that will dump afixed width text file of all the new entries in a table. So if I hada table with likeusername - varchar(20)created - smalldatetimeI could get a weekly feed each week of all the new users in a textfile. I know I could write a script that would go through and do thisby looking at the time stamp, and the last time that the filepreviously ran and get the new dates but I was hoping there was abuilt way to do this. Or perhaps a more elegant solution.Thanks,Charlie
View 2 Replies View RelatedHi allNeed your help to do this; I got a table with these records:Supplier RegNo Status PoNumberABC sbh1309m 1DCD sbt99x 1FGJ sbg3939m 1FGJ sbg3939m 1OEE ey3939d 1Need to have a sql command to transform to :Supplier RegNo Status PoNumberABC sbh1309m 1 50001DCD sbt99x 1 50002FGJ sbg3939m 1 50003FGJ sbg3939m 1 50003OEE ey3939d 1 50004Any ideas?Thanks in advance.Rashid.
View 1 Replies View Related
I am attempting to perform an incremental load, inserting new rows and updating existing rows. I am using a lookup and everything works fine, except when it is the first load of the destination table. As there are no records in the table at all, the lookup fails. I thought of using a rowcount - if the count variable is zero load everything from temporary table to load table, otherwise perform lookup and incremental load.
Can anyone help?
We are in the process of converting our existing incremental loads from DTS to SSIS.
Currently we get all the data for the past month into temp tables in the warehouse, compare with key fields add the new rows and update changed rows. All this is done using Execute SQL task.
Is there a better way to implement the incremental logic using SSIS any new objects that be used to avoid too much SQL codes? Performance is very important and we do a lot of aggregation after the load for the reports to run faster so that we can meet customer SLA's.
We have around 20 tables that needs to be loaded 4 have large amount of data between 20 and 40 million rows out of which we will be brining over around 100 thousand during each incremental run. The other tables have less than 100,000 rows so does not hurt truncating and reloading the entire table.
Any assistance is appreciated.
Thanks!
Hi everyone. I'm trying to figure out how to run an incremental load into a Staging table.
At this point I'm not trying to Conditional Split it between "New" and "Changed" records... just the load.
The logic in my head says that after each load, you can take the most recent "modified" date/time and store that in an incremental load table. That way, next time you run an incremental load, you just have to look up that "modified" date/time, and only load the source records with a "modified" date/time later than the record in your incremental load table. Does that plan sound feasible?
I think so far my problem is that my source is on an ADO.NET connection, and my incremental load table is on my SQL Server. So when I do my load from the ADO.NET database, I cannot read the data from the incremental load table.
Is my logic flawed?
Any help would be appreciated.
Thank you very much!
Does an Differential Backup contain all the changes since the last full backup, including Inserts, Updates, and Deletions. Our DB has "Truncate on Checkpoint" = True, so Log backups are non-sensical. I want to apply the Differentials to an archive, offline DB.
"one pebble does not fill the void, but it is a start."
"Dedicated to only creating original mistakes."
I'm trying to update every record with a incremental number. I wrote the following query but it updates the records with the same number. Could someone please tell me what I'm doing wrong? Thanks.
declare @NextKey int
SELECT @NextKey = NEXT_KEY FROM TABLE_KEYS
WHERE TABLE_NAME = "tblEmp"
set @NextKey = @NextKey + 1
DECLARE tblNewEmp_cursor CURSOR FOR
select emp_pk from tblNewEmp
open tblNewEmp_cursor
FETCH NEXT FROM tblNewEmp_cursor
WHILE @@FETCH_STATUS = 0
begin
update tblNewEmp
set emp_pk = @NextKey
set @NextKey = @NextKey + 1
FETCH NEXT FROM tblNewEmp_cursor
end
close tblNewEmp_cursor
Hi,
I am working on one application, which retrieves data from multiple tables in the database and all the fields retrieved, are exported as an excel sheet. All the export functionality is done through DTS. And the data is retrieved using an SP.
I am supposed to use an "incremental backup" approach here. Means, for the 1st scheduling of the package, all the database dump will be there in excel file. But, after that only fields that are updated/inserted are to be filled in the excel file. Each time, when the dump of the database is taken, the excel file is stored in an archive folder with Date and tiemstamp(e.g. TEST_11_04_2007_15_00_00.xls)
e.g. For the first dump, I get 100 records from the database. Before next execution of the package, 10 rows get inserted and 1 row gets updated. So, on the second time execution of the package, I should populate the excel sheet with those 11 rows only, and not 110 records.
I am not authorised to change the database schema.
So, is there any approach to try out this?
Hi, all.
I want to Select query with incremental column on the fly.
For example
Use pubs
GO
select * from jobs where job_desc like '%e%' Order by max_lvl
returns
job_idjob_descmin_lvlmax_lvl
1New Hire - Job not specified1010
12Editor25100
13Sales Representative25100
...
I want to add here Rank Column numbering in order
select RankOnTheFly, * from jobs where job_desc like '%e%' Order by max_lvl
Then result will be..
Rankjob_idjob_descmin_lvlmax_lvl
11New Hire - Job not specified1010
212Editor25100
313Sales Representative25100
..
I can get the result using cursor and looping throught and inserting or Using Identity function.
But, I saw before there is just one simple Select query doing that.
Does anyone know this?
Thank you..
Hi All
I have this table which has about 30 thousand records
I want to send all the records everyday at night to the other interface(TIBCO) in form of XML.
I can send all the records at once or maybe 10 records at a time
1. If I make one XML with all the records and send it, its risky as XML would be quite big, also it would be time consuming and would generate a big load on the program(.NET program will create XML). Also, any abnormal error would kill the whole process and we would have to start again
2. Processing 10 or 20 records at a time and then sending the XML(with these 10 or 20 records) and doing this till all the records are sent. If there is any error in between in the program(.NET), the program can start from that step again. Now the problem is how I can keep track of the record number already sent to the other interface
Can someone help?
Thanks
I required query for Incremental Load of CDC using Query
View 1 Replies View RelatedHi;Before I start let me say I know this is a silly way to go aboutthings, but it is one of those "my company made me do it" things.My company is in the process of ( finishing ) migrating from foxpro tosqlserver.They have a foxpro program that does a lot of various updates. Iwould like to write a tsql script to replace it.The trick is that the tsql script would have to have some proceduralprogramming artifacts that I don't think it has.I/my company would like the script to generate output messages thatthe user can see while the script is running. Everytime I have ran ascript in query analyzer I usually don't see the output ("print"statemetns, etc ) until all of the work isdone. Is there a way to run thing so I can see the output statementsas the work is being done?Can print statements work from within a tsql loop?The other thing is that I/my company wants the tsql script to writeoutput messages to a log file while it is operating ( for errorchecking and other logging).Is this possible with tsql?Thanks in advance for the infoSteve
View 1 Replies View Related