I'm trying to develop an application and I have been told right here in sql server getting started forum that SSIS or DTS are the best bits.
My old post has these questions:
Code Snippet
I'm making an application. I have 2 different sites running Sql Server 2005 workgroup and sql server express.
I have a 3rd main server and what I would need to do is to get some of the data from both of these sites, merge it and save it on Main server and then generate another file which should be placed in a directory on a network.
I don't have any idea of how I would do that. Its entirely a sql server app.
One person told me its an SSIS and the other one said to investigate replication.
I have checked a bit about both. I want to give it a try.
Where can i find tutorials on both of these. Complete tutorials to develop and design it.
I am trying to import part of the following file structure into a SQL2005 table The 1st 10 records are not needed (HEADER DATA), neither are the records that are from ‘TRAILER RECORD’ Can anyone assist me in designing a Simple SSIS to import this file to a table The LedgerKey will be from a Global variable Text within <> are for information and NOT contained in the file In this file there are three records to be imported, but in other files thjere can be upwards of 20,000 records
Table CREATE TABLE [dbo].[TempTransactions]( [LedgerKey] [varchar](20) NOT NULL, [Licence] [varchar](6) NOT NULL, [DestSortCode] [varchar](6) NOT NULL, [DestAccountNumber] [varchar](8) NOT NULL, [DestAccountName] [varchar](18) NULL, [TransValue] [varchar](11) NOT NULL, [DestBankRef] [varchar](18) NULL, [TransCode] [varchar](2) NOT NULL, [tranref] [int] IDENTITY(1,1) NOT NULL ) ON [PRIMARY]
File < HEADER DATA NOT REQUIRED > "Header Data" "Header Data" "Header Data" "Header Data" "Header Data" "Header Data" "Header Data" "Header Data" "Header Data" "Header Data" < HEADER DATA NOT REQUIRED > "11111122222222TEST ACCOUNT 1 BANK REG 0000000020017" "33333344444444TEST ACCOUNT 2 BANK REG 0000000150017" "55555566666666TEST ACCOUNT 3 BANK REG 0000000030017" <FIXED FIELDS LENGTHS 6,8,18,18,11,2 > <TRAILER RECORD NOT REQUIRED > "TRAILER RECORD" "TRAILER RECORD" "TRAILER RECORD" "TRAILER RECORD" "TRAILER RECORD" "TRAILER RECORD" "TRAILER RECORD" "TRAILER RECORD" "TRAILER RECORD" "TRAILER RECORD" "TRAILER RECORD" "TRAILER RECORD" "TRAILER RECORD" "TRAILER RECORD" <TRAILER RECORD NOT REQUIRED >
I'm all mixed up on this one. Please comment on either of the following two remarks.
1. When I create a new SSIS package under BIDS I gather it is intended for development purposes. .I can test the package and run it but I need to"deploy" it (possibly to another server) to make it available for others to use.
2. Is "deploying" essentially copying my package (.ldtsx) to a new server or does it involve a lot more?
3. I read that I can create an SSIS package without actually needing to have sql 2005 installed on my box. Suppose I create a package that makes no reference to any databases. To *run* this package will I need sql2005 installed or can BIDS do this?
If one of our SSIS packages fails because of a communication problem with the backend, and the DBA is not available, my boss wants another individual (probably a senior programmer but not an "sa" type) to be able to re-run the job.
What is the "right" way to do this under SSIS sql 2005?
Consider the Conditional Split task. I can use a C-like language to create statements such as
Ltrim ([Col1] =€?A€?) || Ltrim([Col1] =€?B€?)
This sure ain€™t VB-like and the list of functions in the upper right-hand pane of the task (€œexpression editor€?) does not help me construct such statements.
Where is the reference manual describing this language?
TIA,
Barkingdog
P.S. I have found fragments in BOL but the info does not seem complete and the BOL topic fragmentation impedes learning.
P.P.S. Why MS uses this language instead of VB.NET, which they use in their Script tasks, is beyond me.
Hi i have a issue i am loading data from a flat file into a relational Database and i am loading the data without dropping the Primary and Foreign Key constraints and i am sending the error rows into a error table for each table. This is becoming large over head is there a way to Load the entire error Row as a single Column in one Error Table.
This is occuring coz the input is being parsed and its dividing into columns,
One approach would be to used Derived Column Transformation and Substring all columns but it makes to writing long substring statement as some tables have 80 columns ,
is there a better way to handle these errors so that they can be looked at and Changes can be made to the Data ,
Also one more question is is there a way to Load in case of Duplicates load only the Recent Row By Date Column. and send the old row according to Date into Error output.
In DTS 2000 there was situation where I had to connect source and destination through a dialup connection (56k), where the transferring of data took me ages to finish, so it was not successful.
I have got a set of queries to run on a table stored in my SQL Database and this query creates a new table which I export to excel for final results. NOw I need to design an SSIS package for this query to run and to export the result in excel sheet. I started with the control flow by adding the TSQL statements to it and then in the data flow I connected it with my database. But when I execute it, it always gives me an error. I have just started learning SSIS but have not been able to figure it our properly.
I created a Backup plan using the SSMS Maintenance Wizard. It created an SSIS package for me but the package didn't delete files older than, say, 3 days. So I exported that package from MSDB, added a Script task to it (to do what I needed), and imported it back into MSDB. I can open the new package under SSMS and everything looked good ... until I double-clicked the Script task. That task is simply not supported under the SSMS Maintenance Wizard. Goin in and out of that task under SSMS gave me the following errors:
"Microsoft Visual Studio Macros: The operation could not be compelted. Call was rejected by callee.".
Or the ever popular "cannot show the editor for this task'.
I was asked to "Wait for Completion" or "Switch tasks or "Cancel".
It seems the only way to delete these files is to set up a seperate SSIS job that calls my Script task sometime after the nightly backup has completed. But now the completion of the Backup job and the file deletion processes are decoupled, which is not very good. Can you offer any ideas how to handle the gracefully? (The real problem is that the SSMS Maintenance wizard does not recognize the full set of SSIS tasks. It would be nice if double-clicking on the Maintenance plan pane opened up the SSIS editor but that's for a later day.)
I've been given the task of migrating a DTS package to SSIS (neither of which I am particularly familar with). The first job in the DTS package is to read a .ini file and set a bunch of variables. These variables are then used throughout the DTS package. After running the DTS package through the SSIS migration wizard this job turns into an execute script task and I can't see if it is still reading the .ini file. However, the only real purpose of this step is to allow different parameters to be passed in development, test, production etc. So I am thinking this whole step can be removed and effectively replaced with a package configuration (I'll probably use an XML file). My understanding is that by selecting the name/value pairs as appropriate in the XML package configuration file means this values will be passed in at runtime and achieve the same functionality. Is this the correct way to do this in SSIS, or do I still need the .ini file and variables?
We have an SSIS package that will be used for both our Test and Prod imports on the same server. The SSIS imports are identical expect that Test needs all connections pointing to the Test database while Prod need its connections pointing to the Prod database.
How can I change the connections, based on Test or Prod, used inside a single SSIS package? (I don't want to create two tweaked packages on the same server. If I find a bug in one of them, I have to correct it twice.)
Im new to stored procedure, but here's what I want, if someone can get me started and provide this as an example it would be VERY welcome!I have the following, a SP with parameter IsMale. This parameter may be empty. If it's not empty I want to add some text to my selection query: AND IsMale=paramvalueHere's the SP so far:ALTER PROCEDURE [dbo].[spFindUsersAdvanced] --declare parameters here@IsMale bitASBEGIN -- SET NOCOUNT ON added to prevent extra result sets from -- interfering with SELECT statements. SET NOCOUNT ON; SELECT DISTINCT BirthDate,IsMale, FROM aspnet_Users INNER JOIN tblUserData ON aspnet_Users.UserId = tblUserData.UserID WHERE tblUserData.Username<>''IF @IsMale THENBEGIN AND IsMale='True'ENDELSEBEGIN AND IsMale='False'ENDEND IFEND1. How do I check if the parameter is empty?2. How do I add the text to my selection query?
Hi all,I am trying to write a stored procedure, which has two insert statements.the first insert statement, is simple which inserts data into a table and returns the primary key for the new row added. using this primary key, i am writing another insert statement, which passes a list of elements which would be then entered into another table, with the primary key of the first table.Is this possible to do in a single stored procedure? I have implemented this using two different sp, but am wondering if it can be done other way?thanks for your help!
Hi plz read below is my scenarioI have two diff tables which stores passwords (don't ask why b'coz it was there).Now to make data in both table consistance what I need to do?plz provide optimized solution.Thanks in advance.bye bye.happy coding!!!
I've few questions regarding SQL Server. Appreciated, If any one out there help me with it.
I would like to do T-SQL Programing and would like to know some good book. I'll mostly be doing Stored procedures,Triggers and views programming. Any advice?
On my sql server I've 10 logins. But, I cannot view these logins from the Enterprise GUI. But, When i run a query against the syslogin table, those all 10 logins shows up. Why is that?
I'm trying to construct the following SQL statement but am missing something somewhere.
I get a syntax error with the 1st FROM highlighted.
If i construct a query out of the bit that works and then construct a query on that query then i can get it to work no problem but how to mimic that in SQL ? I can't simply view the SQL of the queries because the query names replace the SQL statements.
I'm trying to SUM the 5 volumes that get returned by the TOP 5 SQL statement. The VotingUniverse_TotalVolumeByType query returns the TotalVolume which will enable me to divide the SUMmed 5 volumes by the TotalVolume to obtain a %age.
SELECT Type, Sum(Volume) AS Volume, TotalVolume FROM >>>>>>>>>>this following bit works............ (SELECT TOP 5 VotingUniverse_TopFirmsVolumeByType.Type, VotingUniverse_TopFirmsVolumeByType.Volume FROM VotingUniverse_TopFirmsVolumeByType WHERE (((VotingUniverse_TopFirmsVolumeByType.Volume) In (SELECT TOP 5 VotingUniverse_TopFirmsVolumeByType.Volume FROM VotingUniverse_TopFirmsVolumeByType ORDER BY Volume DESC))) ORDER BY VotingUniverse_TopFirmsVolumeByType.Volume;) >>>>>>>>>>>end of bit that works INNER JOIN VotingUniverse_TotalVolumeByType ON Type = VotingUniverse_TotalVolumeByType.Type GROUP BY Type, TotalVolume;
I have a xxx.exe running on SQL 2000 server as a sql job every hour. For some reason this job sometime just hung in the middle and never finish. But if I stop it and rerun it, it will be down within 10 seconds. I am wondering if we can set up something so that the job aborts automatically if it ran more than 10 minutes.
We are running Sql 6.5 sp5 on a Pentium II 350 Mhz with 512 Mb. RAM, 1 ScSi 4 Gb hd, one 8Gb ScSi hd and 2 IDE hd (13,6 Gb and 8Gb) computer running windows NT 4.0 sp 6 without any problems. Now have bought a new computer. The new one is a Pentium III 733 Mhz 512 Mb Ram with 2 ScSi ultra wide 2 18Gb each and one 40 Gb Ide Hd. We have installed windows NT 4.0 sp 6 and Sql 6.5 sp 5. We have restored our database to make some tests to it. The database is 7 Gb big. The results of our test are that the old computer is between 20% and 30% faster than the new one when doing sql instructions and using the database even knowing that the hd´s of the new computers are 4 times faster than the old ones.
Would you please give us any help or clue that would make the new computer go faster? We know that upgrading to sql 7 will work but we need first to make this computer work better with the 6.5. Thank you.
I often use DTS to move databases between the servers. When you choose "Transform" data and “Column Mappings and Transformation” window opens, the default option is “Append rows to destination table”. Question: Is there a way to change the default option to “Delete rows in destination table”, so I wouldn’t have to go thru the transformation of dozens of tables every time I move databases around.
P.S. I know that I can save DTS package with my settings and use it later. The problem is that every time it might be different database or objects might change. So, it is not a solution for me.
I have a client requirement. We are integrating two applications. So the problem is that, whenever some updates (meaning to say "create" , "delete" , "update") happens in the SQL DB Tables, i needed to track that data and send it to a different system.
What I have thought abt is that if SQL could generate a file with all the values deleted/updated/created in the DB, then I could take those values and do the needful. Please help me..
Hello guys, I'm new to the forum and to MS SQL 2K. I'm trying to a merge similar rows in a table into a single row and put them in a new table.
Example:- This is my input table TableA ID A B C ------------------------ 1 jk kl bj 2 sd we op 3 io po kl 1 ui gh ew 2 kl re op 1 qw kj nn
My output table should look like this TableB ID A1 B1 C1 A2 B2 C2 A3 B3 C3 ----------------------------------------------------- 1 jk kl bj ui gh ew qw kj nn 2 sd we op kl re op 3 io po kl
Please help me on how to create my output. Thanks in advance,
Columns PartnerId: int NativeId: int FirstName: nvarchar(50) LastName:nvarchar(50)
1)I should get the records with a minimum native id for a particular PartnerId, 2) if duplicates exists in the above condition i should select top 1 (first record)
Hi, I have been SQL developer for past 2 yrs, want to get into Administration(DBA) can any one suggest me some good articles on net and good books for this.
Hey all, I need some help to build some data bases and maybe some morestuff. Please reply only if you can do it volunteerly or very cheap please;but there is incentives in time..TIA
I have a tableCreate Table Payments {paymentid int,customerid int,amount int,date datetime}What I want is the sum of the amounts of the last payments of all customers.Now the last payment of a customer is not necessarily the one with thehighest paymentid for that customer BUT it is the one with the highestpaymentid on the MOST RECENT date. We dont keep the time part just the dateso if there are more than 1 payments of a customer on a date ( and there aremany such cases ) only then the paymentid decides which is the last payment.Further the last payment may be the last as of today but I may want to findthe sum of all the last payments upto say March 1, 2003or any date. My own solution is too slow even it is correct.SELECT SUM( AMOUNT )FROM PAYMENTS AS P1WHERE PAYMENTID =( SELECT MAX( PAYMENTID ) FROM PAYMENTS AS P2 WHERE P1.CUSTOMERID =P2.CUSTOMERID AND DATE =( SELECT MAX(DATE) FROM PAYMENS AS P3 WHERE P3.CUSTOMERID = P2.CUSTOMERIDAND DATE < #9/8/03# ))What would be the most efficient solution to this.Both in SQL Server and in Access 2000thx in advance
I am using the following T-SQL to pull records from a given table that have a start time between 6:59:59PM and 7:00:00AM. However, it's not working. The SQL I'm using is listed here. What am I doing wrong?
select RecNum, convert(varchar(8), StartTime, 8) as Start_Time from TableA
where convert(varchar(8), StartTime, 8) between '18:59:59' and '07:00:00'
By the way, there are a multitude of records that I'm looking for in the table.
I need some Help with VSA in SSIS . Whenever i open VSA for scripting , it shows the following validation errors.
Warning 1 The dependency 'EnvDTE' could not be found. Warning 3 The dependency 'Microsoft.SqlServer.DTSRuntimeWrap' could not be found. Warning 5 The dependency 'Microsoft.SqlServer.msxml6_interop' could not be found. Warning 4 The dependency 'Microsoft.SqlServer.PipelineHost' could not be found. Warning 6 The dependency 'Microsoft.SqlServer.SqlTDiagM' could not be found. Warning 2 The dependency 'Microsoft.SqlServer.VSAHosting' could not be found. Warning 7 The dependency 'Microsoft.SqlServer.VSAHostingDT' could not be found.
I am able to locate these dependencies in the c: programfilesmicrosoftsqlserver90SDKAssemblies.
APPLICATION SUMMARY * Online only application. * Trust url parameter is set. ERROR SUMMARY Below is a summary of the errors, details of these errors are listed later in the log. * Activation of C:UserspmohanDesktopReportBuilder(2).application resulted in exception. Following failure messages were detected: + Downloading file:///C:/Users/pmohan/Desktop/ReportBuilder.exe.manifest did not succeed. + Could not find file 'C:UserspmohanDesktopReportBuilder.exe.manifest'. + Could not find file 'C:UserspmohanDesktopReportBuilder.exe.manifest'. + Could not find file 'C:UserspmohanDesktopReportBuilder.exe.manifest'.
COMPONENT STORE TRANSACTION FAILURE SUMMARY No transaction error was detected.
WARNINGS There were no warnings during this operation.
OPERATION PROGRESS STATUS * [4/3/2008 3:25:51 PM] : Activation of C:UserspmohanDesktopReportBuilder(2).application has started. * [4/3/2008 3:25:51 PM] : Processing of deployment manifest has successfully completed. * [4/3/2008 3:25:51 PM] : Installation of the application has started.
ERROR DETAILS Following errors were detected during this operation. * [4/3/2008 3:25:51 PM] System.Deployment.Application.DeploymentDownloadException (Unknown subtype) - Downloading file:///C:/Users/pmohan/Desktop/ReportBuilder.exe.manifest did not succeed. - Source: System.Deployment - Stack trace: at System.Deployment.Application.SystemNetDownloader.DownloadSingleFile(DownloadQueueItem next) at System.Deployment.Application.SystemNetDownloader.DownloadAllFiles() at System.Deployment.Application.FileDownloader.Download(SubscriptionState subState) at System.Deployment.Application.DownloadManager.DownloadManifestAsRawFile(Uri& sourceUri, String targetPath, IDownloadNotification notification, DownloadOptions options, ServerInformation& serverInformation) at System.Deployment.Application.DownloadManager.DownloadApplicationManifest(AssemblyManifest deploymentManifest, String targetDir, Uri deploymentUri, IDownloadNotification notification, DownloadOptions options, Uri& appSourceUri, String& appManifestPath) at System.Deployment.Application.ApplicationActivator.DownloadApplication(SubscriptionState subState, ActivationDescription actDesc, Int64 transactionId, TempDirectory& downloadTemp) at System.Deployment.Application.ApplicationActivator.InstallApplication(SubscriptionState& subState, ActivationDescription actDesc) at System.Deployment.Application.ApplicationActivator.PerformDeploymentActivation(Uri activationUri, Boolean isShortcut, String textualSubId, String deploymentProviderUrlFromExtension) at System.Deployment.Application.ApplicationActivator.ActivateDeploymentWorker(Object state) --- Inner Exception --- System.Net.WebException - Could not find file 'C:UserspmohanDesktopReportBuilder.exe.manifest'. - Source: System - Stack trace: at System.Net.FileWebRequest.EndGetResponse(IAsyncResult asyncResult) at System.Net.FileWebRequest.GetResponse() at System.Deployment.Application.SystemNetDownloader.DownloadSingleFile(DownloadQueueItem next) --- Inner Exception --- System.Net.WebException - Could not find file 'C:UserspmohanDesktopReportBuilder.exe.manifest'. - Source: System - Stack trace: at System.Net.FileWebResponse..ctor(FileWebRequest request, Uri uri, FileAccess access, Boolean asyncHint) at System.Net.FileWebRequest.GetResponseCallback(Object state) --- Inner Exception --- System.IO.FileNotFoundException - Could not find file 'C:UserspmohanDesktopReportBuilder.exe.manifest'. - Source: mscorlib - Stack trace: at System.IO.__Error.WinIOError(Int32 errorCode, String maybeFullPath) at System.IO.FileStream.Init(String path, FileMode mode, FileAccess access, Int32 rights, Boolean useRights, FileShare share, Int32 bufferSize, FileOptions options, SECURITY_ATTRIBUTES secAttrs, String msgPath, Boolean bFromProxy) at System.IO.FileStream..ctor(String path, FileMode mode, FileAccess access, FileShare share, Int32 bufferSize, Boolean useAsync) at System.Net.FileWebStream..ctor(FileWebRequest request, String path, FileMode mode, FileAccess access, FileShare sharing, Int32 length, Boolean async) at System.Net.FileWebResponse..ctor(FileWebRequest request, Uri uri, FileAccess access, Boolean asyncHint)
COMPONENT STORE TRANSACTION DETAILS No transaction information is available.
into a 1-column recordset containing 2 rows with the values 'Martin' and 'Cyril'
Here's the nearest I have got:
Code Block declare @pElementListXML XML set @pElementListXML = ' <ElementList> <Element>Martin</Element> <Element>Cyril</Element> </ElementList>
' ;WITH nodes_cte as ( SELECT [c].[value]('/', 'NVARCHAR(40)') Element FROM @pElementListXML.nodes('/ElementList/Element') T(c) ) SELECT * FROM nodes_cte
Hi everyone, I€™m pursuing a fast deployment for SSIS. I mean, we€™ve got ten ASP/VB6 developers which often doing some DTS stuff by mean, of course, Enterprise Manager.
Now, they€™d have to handle the same but with SSIS. I don€™t have very clear whether only is necessary install them SSIS component or it€™d requires something else. Framework 2.0 too??
