I am performing data migration from a legacy sql server (2000) to a new one (SQL 2005). I am using SSIS packages to perform the migration.
The source legacy sql server is a live machine (used by the customer). Therefore, during the data transfer there would certainly be new records inserted/updated/deleted on the source legacy sql server.
Now, after the migration i wish to migrate the delta to the new sql server. how can i do this ?
Does SSIS have any provision to take care of it ?
i'm new to this platform and am wondering if there is an easy way to identify ADDs changes and deletes between two tables - or whether i will have to do a ror for row comparison using the business key?
Suppose a database server and client are separated by a low bandwidthlink such as DSL, and the client repeatedly issues a query for, say, acurrent product list.Suppose the product list is large, but only a handful of entries havetypically changed between queries. It would be nice if only the changesfrom last query to current one could be sent, saving bandwidth.Is there any way to do this?Thanks,--"Always look on the bright side of life."To reply by email, replace no.spam with my last name.
We are trying to configure Delta View in our senario and are confused at a couple of points:
1. What is the difference between modify and modify_attribute? How does modify_attribute works at attribute level? Is it only for multivalued attributes? 2. Rather than having an extra table for Delta. If we would use the prime table itself with an extra field acting as a flag (0/1) (0=Prime Record, 1=Delta Record) and generate the delta view according to the flag. Would it be fine or we should have a different delta table only?
I'm working on a program that reads in zip codes and then does some checking on them. Occasionally, the end user typos or transposes digits. It's easy enough to identify bad zip codes, but I would like to offer up a single recommendation. The problem is that there are often multiples to choose from. Is there a function or has anyone tried to figure out the delta between two strings?
I need to get the replacement records between the 2 tables. I have table A and table B with same structure. I have 5 fields. Table A has 50,000 records and table B has 20,000 records. I have fields id , name, address,meter_flag,end_Date.
Some of the records in Table B are just replacement records of table A. I mean for example I have records like this in Table A
id name address meter_flag end_date
23 john 1201 salt lake dr no 2011-12-28
24 tom 1222 gibson ln yes 2011-12-16
25 alex 1334 qayak dr no 2011-12-17
In Table B
23 john 1344 mc kinney st yes 2011-12-18
24 tom 1222 gibson ln yes 2011-12-16
56 gary 1335 pruitt rd no 2011-12-18
25 alex 1334 qayak dr no 2011-12-17
So here in Table B i have an update for john with id 23 in table A in address field and meter_flag has changed to yes. There is new record with id 25 in table b but that is not in table A. so I need to find all these difference records by querying these 2 table
Currently through my stored procedure, I am pulling the data from my source systems in the way Daily Truncate and loading the data in to my database table. Currently my query looks as follows,
SELECT Â Â Â a.col1,a.col2,b.col3,b.col4,c.col5 FROM a left outer join b ON a.col1 = b.col1 left outer join c ON b.col1 = c.col1.
Now, I want to change the above one to delta process(I have updateddate field in all the tables). The requirement is any of these 3 tables has updateddate = getdate() then i need to take "a.col1,a.col2,b.col3,b.col4,c.col5" columns and push to destination table.
I think usually delta process means we will use to consider the key table, in that table if there is an updated record will take that record and join with the other required tables to pull the updated record. But these logic seems different.how to build the query for the above delta process? Since it is an urgent requirement need to implement asap.
DECLARE @MaxCountHistogram TABLE  (  MaxId  INT IDENTITY PRIMARY KEY NOT NULL,  PublicationId  INT NOT NULL,  ProviderId    INT NOT NULL,  DateLog  DATETIME NOT NULL,  Amount     FLOAT NOT NULL  ) INSERT INTO @MaxCountHistogram VALUES(432,3,'20150530',10.2564),(432,3,'20150630',13.2564),(432,5,'20150530',8),(432,5,'20150630',13),(433,3,'20150530',9),(433,3,'20150630',11),(433,5,'20150530',13),(433,5,'20150630',21)
I need to take for each Publication and Provider  and getting the diferential between two different months, for example:
Period             Delta Amount           Provider    PublicationId 20150530          10.2564                   3 432 20150630           3  Result of (13.2564- 10.2564 )     3            432
In an approach of building an ETL tool, we are into a situation wherein, a table has to be loaded on an incremental basis. The first run all the records apporx 100 lacs has to be loaded. From the next run, only the records that got updated since the last run of the package or newly added are to be pulled from the source Database. One idea we had was to have two OLE DB Source components, in one get those records that got updated or was added newly, since we have upddate cols in the DB getting them is fairly simple, in the next OLEDB source load all the records form the Destination, pass it onto a Merge Join then have a Conditional Split down the piple line, and handle the updates cum insert.
Now the question is, how slow the show is gonna be ? Will there be a case that the Source DB returns records pretty fast and Merge Join fails in anticipation of all the records from the destination ?
What might be the ideal way to go about my scenario.. Please advice...
/*Title: Created By: Create Date: Notes:This function is used to concatenate the fields of a table except any identity and hashbyte column passed into the function. It works for temp tables are phyisical tables.
** NOTE: The temp table has be on the same SQL connection to work. If you use this in SSIS you will need to make your connection persistant.
The original concept came from [URL] .... and was modified.
Revisions:
*/ ALTER FUNCTION [dbo].[get_hash_fields] ( @p_table_name VARCHAR(128), @p_schema_name VARCHAR(20), @chksum_col_name varchar(255) ) RETURNS VARCHAR(MAX) AS BEGIN DECLARE @sqlString as varchar(max)
I am using transfer manager in SQL 6.5 to copy a database and all objects with data to another server. Transfer manager is not recreating all stored procedures. This even happens when I used it to another database on the same server. Any ideas??
Is it possible/advisable when transfering very large amounts of data from server to server to: trasnfer the data to a new table first second alter new table adding indexes, defaults, ets based on original table
if it is what flow item would be used to transfer/alter the indexes and defaults?
I'm very new to ssis so the more detail you can give the better.
I have been trying to run transfer manager to transfer all of the data from the production database on one server to a test database on another server(to refresh it). In order to make sure it runs on the server, I have been scheduling it under EM to do so and I am pointing to the log on the destination server on the EM Transfer panel.
For some reason, I am getting the following message in the destination server error log:
99/02/16 10:24:41.42 ods Error : 17824, Severity: 10, State: 0 99/02/16 10:24:41.42 ods Unable to write to ListenOn connection '.pipesqlquery', loginname 'sa', hostname 'TEMP09'. 99/02/16 10:24:41.42 ods OS Error : 232, The pipe is being closed. 99/02/16 10:24:41.42 spid17 Error : 1608, Severity: 21, State: 2 99/02/16 10:24:41.42 spid17 A network error was encountered while sending results to the front end. Check the SQL Server errorlog for more information.
I checked the event viewer error log and see no messages for today.
Can any one advise me what I need to do for this to run successfully?
Thanks. Any information furnished will be greatly appreciated.
In the full recovery model, if i run a transaction that inserts 10MB of data into a table, then 10 MB of data is moved in the data file. Does this mean then that the log file will grow by exactly 10MB as well?
I understand that all transactions are logged to the log file to enable rollback and point in time recovery, but what is actually physically stored in the log file for this transactions record? Is it the text of the command from the transaction or the actual physical data from that transaction?
I ask because say if I have two drives, one with 5MB/s write speed for the log file and one with 10MB/s write speed for the data file, if I start trying to insert 10 MB of data per second into the table, am I going to be limited to 5MB/s by the log file drive, or is SQL server not going to try and log all 10 MB each second to the log file?
Hi, I have the following method which transfers data SQL to SQL on the same server but when I try to change the destination server it won't transfer the data. The method runs through as expected with no exceptions or errors but with no data transfered.
Private Sub TransferSQLData(ByVal SourceDetails As Admin_upload.EnvironmentDetails, ByVal DestinationDetails As Admin_upload.EnvironmentDetails)
'Transfer database from source to destination. Dim oPackage As New DTS.Package2 Dim oConnection As DTS.Connection2 Dim oStep As DTS.Step2 Dim oTask As DTS.Task Dim oCustomTask As DTS.TransferObjectsTask2 Try oStep = oPackage.Steps.New oTask = oPackage.Tasks.New("DTSTransferObjectsTask") oCustomTask = oTask.CustomTask oPackage.FailOnError = False With oStep .Name = "Copy Database design and data" .ExecuteInMainThread = True End With With oTask .Name = "GenericPkgTask" End With With oCustomTask .Name = "DTSTransferObjectsTask" .SourceServer = "MYSERVER" .SourceUseTrustedConnection = True .SourceDatabase = SourceDetails.MetaDB .SourceLogin = SourceDetails.MetaUser .SourcePassword = SourceDetails.MetaPWD .DestinationServer = "MYSERVER" .DestinationUseTrustedConnection = True .DestinationDatabase = DestinationDetails.MetaDB .DestinationLogin = DestinationDetails.MetaUser .DestinationPassword = DestinationDetails.MetaPWD .CopyAllObjects = True .IncludeDependencies = False .IncludeLogins = False .IncludeUsers = False .DropDestinationObjectsFirst = True .CopySchema = True .CopyData = DTS.DTSTransfer_CopyDataOption.DTSTransfer_ReplaceData
End With oStep.TaskName = oCustomTask.Name oPackage.Steps.Add(oStep) oPackage.Tasks.Add(oTask) oPackage.Execute() Catch ex As Exception LabelUploadMeta.Text = "Failed to Upload MetaData: " & ex.Message Throw ex Finally oConnection = Nothing oCustomTask = Nothing oTask = Nothing oStep = Nothing
If Not (oPackage Is Nothing) Then oPackage.UnInitialize() End If End Try End Sub This works fine but when I set the following within the method: .DestinationServer = "ANOTHERSERVER" It won't transfer the data. I can access the remote server and read and write data to it. Any ideas?
Hi , Can anyone suggest me to transfer a database from 65 to 70. I trasfer a database using DTS, but i am only getting data. I need my users , views, stored procedures, permissions. I need to upgrade three databases from 65 to another 70 server. i also generated a script from 65, but which is showing errors. Could u pls give any suggestions. thank u.
I recently transferred a db from our production machine to a test machine. After the transfer was complete I realized that all copied logins were copied with blank passwords. Is this default behavior? Can I set the transfer to include passwords?
I have recently tried to transfer my database over from one machine to another, but failed to do so because it gets jammed on 30%. I am just wondering if there is any other ways I can take to migrate the database over.
I am currently running sql server 6.5 in concurrent with iis 4 and nt4. I am trying to seperate the iis to a new server machine and sql server to another server.
Please advise me which route to take if you have any suggestions and your solution is important to me. Thanks!
I want to transfer rows from one database to another. Source My Source tables in database 'Staging'
CustomersEng Name Age StCode CityCode InternalID
CustomersSpanish Name Age StCode CityCode InternalID
CustomersEng and CustomersSpanish contains a replica..only difference being that 'Name' in the CustomerSpanish table is written in spanish language. The Stcode, citycode and Internalid together makes the row unique.
Code BlockSample script Create table #CustomersEng (name nvarchar(20),age int, st_code int, citycode int, internalid int) Insert into #CustomersEng values('Aron',23,1,1,12); Insert into #CustomersEng values('Cathy',28,1,1,13); Insert into #CustomersEng values('Zarog',33,1,1,14); Insert into #CustomersEng values('Michale',25,1,2,12); Insert into #CustomersEng values('Linda',43,1,2,13); Insert into #CustomersEng values('Burt',53,1,2,14);
Create table #CustomersSpanish (name nvarchar(20),age int, st_code int, citycode int, internalid int) Insert into #CustomersSpanish values('Arona',23,1,1,12); Insert into #CustomersSpanish values('Cathylo',28,1,1,13); Insert into #CustomersSpanish values('Zarogo',33,1,1,14); Insert into #CustomersSpanish values('Michalo',25,1,2,12); Insert into #CustomersSpanish values('Lindalo',43,1,2,13); Insert into #CustomersSpanish values('Burto',53,1,2,14);
--------------------------------------------------------------------------------------------------------------- Destination tables in database 'CMO'
Hi All; I know that in the SQL Server, the logins are not contained in the database (thanks to Kevin Yu) but I don't know where are they stored. For example if I want to transfer to another server I'll have to backup users too; but HOW? In order to be more clear: When I simply take a backup from my server and restore it to my computer I can not login again with the user login information which is created on the server; However even if I make some changes; after backing the DB up from my computer and restoring to the server the logins are working (with re-mapping) Thanks...
I had Sql Server 2005 installed on my machine, with that I am working on Visual Studio 2005. I just need to add an existing database into the app_data folder. The database is existing in the Sql server, it contains 180 tables so I cannot manually copy that tables in the app_data folder. for that is there any another way to copy all the tables into the database into the folder app_data with the data in the table. Thanks & Regards Girish Nehte
I recently transferred my SQL DB from one server to another, I backed the DB up, copied across and restored it into the new location, the problem is that I get the following error.
System.Data.SqlClient.SqlException: Invalid object name 'Table Name'
I have created a new user for this database and changed my code accordingly.
Dim conPubs As SqlConnection Dim cmdSelectContacts As SqlCommand Dim dtrContacts As SqlDataReader
conPubs = New SqlConnection("Server=Treebeard;UID=Intranet;Pwd=glglgl;Database=Contacts") ConPubs.Open() cmdSelectContacts = New SqlCommand ("Select * From Clientdata", ConPubs)
dtrContacts = cmdSelectContacts.ExecuteReader()
While dtrContacts.Read() Response.Write( "<li>" ) Response.Write( dtrContacts( "c_forename" )) End While
Hi everyone,I need some help transfering my database(SQL2005) to the host's server.In SQL 2000 I had the Management tools > DTS or ImportExport tools.SQL2005 lacks this, all I've got is Express Manager > can't connect to multiple databases.Can connect to one database and has a screen to excute TSQL commands.I don't want to have to recreate the whole database on the server.....This would be crazy... 150 stored procedures and 30 tables. Not to mention all the individual settings for each column.I know there is SQL Studio Management tools for the full version of SQL 2005. Anyone got any idea's ?Any input/suggestions is greatly appreciated.Thanks,JShep
I am trying to transfer my database from my computer where built it to another computer. I have microsoft SQL server 2005 CTP on my computer and the other computer has Microsoft SQL server. Does anyone have any suggestions on how to do this??? I cannot find much info on SQL server 2005 CTP. I have found some info about the copy database wizzard, but I cannot find that in CTP. Please any ideas or info would be wonderful Thanks Sarah
If I’m using the DTS Transfer SQL Object to move all objects and data from one server to another server and I make a schema change to source server, why is the dts package failing the next time I run it? Do I have to do some sort of refresh?
I have a corrupt master db due to a rogue script that inserted lots of information into it. I have two SQL servers, and I could use some advice on the best way to copy the files over to the second server, then back after a reload of the first server. My main concern is the DTS packages I have built- I can't afford to lose those.
Hi, I am in the process of upgrading database in sql server 6.5 to 7.0. For that i am following one computer upgrade. For creating dev environment in my computer i am using Transfer/Objects in 6.5 for transferring database from remote(production) computer to my computer.
Do i need only to transfer Userdatabase alone or do i need to Transfer Master and other ? Or Is it better to restore dump ?
Does anyone have a sample SQL DMO script to transfer SQL Server logins from one server to another including the passwords ? any pointers will be greatly appreciated
Hello. Firstly apologies for what might appear as a basic question, but being a newbie and all :-). .
I have a SQL 7 DB, loads of sp, vw, tables etc. I want to copy this DB to another server so that all the DB and sp permissions stay current, all the sql logins come across etc etc.
I have tried restoring a backup to the new server (which worked fine), but then I appear to have lost the permissions for the tables and sp's.
I have then tried taking a copy of the original SQL master table and restoring this to the new server - but no luck.
Can anyone advise a sure method of copying an entire table and related entries (meaning sql logins etc) to a new server.
