How To Automate The Import Process?
Jun 12, 2007
Hopefully, there is a way to do this. I work with two SQL servers. One is our production server the other is our test server. In order to test various things, I often need to copy the source data from one server to the other. Most of our programming is in VBA. It's easy enough to open a recordset and fill it with the data I need from the production server, then upload each record, one at a time, to the test server. The problem is that I am dealing with a massive amount of data and this takes a long time.
I have found that I can use the import task in SQL Server Enterprise and it transfers the data extremely quickly. Is there a way, preferably using VBA, that I could automate this import task process?
Thanks
View 1 Replies
ADVERTISEMENT
Jun 20, 2001
Can anyone direct me to either some code or reading, that talk about gather all my server information. What I want to do is gather all the server Used and Free space size. I'm know a little sql-dmo, and I think I can do it that way, but not sure.
View 1 Replies
View Related
Oct 6, 2015
We are using SQL Server 2014 and SSDT-BI 2013. We have a reporting environment where business users create objects which need to be persisted for fiscal year reporting. Let's say for instance SQLSERVER1SRVR1 they create table objects like below in the reporting environment.
Accounting2014, Accounting2015 in AccountingDB;
Sales2014, Sales2015 in SalesDB;
Products2014, Products2015 in ProductsDB;
Inventory2014, Inventory2015 in InventoryDB etc....
These tables are persisted for auditing in a different environment SQLSERVER2SRVR2 for finance & audit folks.We would want to automate this process using SSIS to create tables in corresponding database and load data. I tried using For Each Loop container but the catch is I could loop the source or destination but how do we loop on Source & Destination at the same time (i.e when source is in AccountingDB destination to be AccountingDB, source SalesDB then destination SalesDB so on etc....
View 6 Replies
View Related
Apr 20, 2015
I've got an SSIS solution file with project deployment model in VS 2013 and would like to deploy that to SSISDB on different environments.All these days I followed the regular way to create a project in SSISDB and deploy it to that. Now want to find out if i can automate this process and so got some questions
1. Can we automate the process of creating a project on SSISDB based on our SSIS project name? This will be like when we do a deployment it should check if the project exists or not on SSISDB based on our SSIS project name, if the project exists we just deploy the packages in the project and if the project does not exists in SSISDB it will create that project and deploy the packages.
2. Can we also automate the process of creating environments? In traditional way we manually create the environment variables under environment tab of SSISDB, but can we make that also as part of deployment? Like when we are releasing to Dev server we look if that particular Dev variable exists on that server, if it exists we just update the existing stuff and if it does not exists we just create it.
View 2 Replies
View Related
Jan 29, 2002
Which method in MSSQL 7.0 would best suit being able to de-dupe, basically leave the dupes in the import file.
I have two process, in the first process, I'm importing from two different tables, so that any potential dupes would have their unique RecID's given from either table. I can then de-dupe on the unique ucase(entry)+RecID combo.
This works fine, however in the second process, the import file that has only one source, and therefore I could have real dupes. Currently I've only used a TSQL cursor process to copy all the data into a temp table, delete the data in the live table, then use another cursor in the same process to only copy one instance of an (account_number + RecID) back into the live table.
This too works, but I'd like to make a DTS package that can do this on import in as few steps as possible. I'm thinking to use one connection and a proc(?)
TIA
JeffP....
View 1 Replies
View Related
Oct 17, 2007
Hello, I try to load the following entry via SSIS in SQL 2005.
Account Name,Account ID,CCY,TD Cash Balance,SD Cash Balance,FX Rate,TD Cash Balance (Base),SD Cash Balance ,,EUR,8251439.34,8251439.34,1.4166,11689195.26,-11689195.26
My value in SQL appears in the 4th column as 8251439.5. I define that the source has a float field. Any idea why my value get changed and how to work around it?
Thanks,
Jam
View 2 Replies
View Related
Nov 25, 1998
hi, I am importing data daily to many tables, I want to keep track of the #of rows for each import process. I already have created a trigger as follow:
CREATE TRIGGER tr_bcp_log ON dbo.A
FOR INSERT
AS
declare @name varchar(30),
@row_count int
select @name=name , @row_count= @@rowcount
from inserted
insert into bcp_tracks (name,row_count)
values(@name,@row_count)
GO
The problem is that I am getting a row for each inserted row in table A.for instance if I have 500 rows in table A, I will get 500 rows in the log table like this
table_name,#of rows
A 1
A 1
A 1
etc up to 500 rows for table A
This is not what I want, I want to capture the num of rows for every bcp process , so in the log table I want to see the following :
table_name, #of rows
A 500
B 600
C 450
A 250
etc
Any help ?
thanks
Ali
View 1 Replies
View Related
Nov 14, 2007
Hi,
I was trying to extract data from the source server using OLEDB Source and SQL Server Destination when i encountered this error:
"Transaction (Process ID 135) was deadlocked on lock resources with another process and has been chosen as the deadlock victim. Rerun the transaction.".
What must be done so that even if the table being queried is locked, i wouldn't experience any deadlock?
cherriesh
View 4 Replies
View Related
Dec 3, 2007
Hello all,
I am running into an interesting scenario on my desktop. I'm running developer edition on Windows XP Professional (9.00.3042.00 SP2 Developer Edition). OS is autopatched via corporate policy and I saw some patches go in last week. This machine is also a hand-me-down so I don't have a clean install of the databases on the machine but I am local admin.
So, starting last week after a forced remote reboot (also a policy) I noticed a few of the databases didn't start back up. I chalked it up to the hard shutdown and went along my merry way. Friday however I know I shut my machine down nicely and this morning when I booted up, I was in the same state I was last Wenesday. 7 of the 18 databases on my machine came up with
FCB:pen: Operating system error 32(The process cannot access the file because it is being used by another process.) occurred while creating or opening file 'C:Program FilesMicrosoft SQL ServerMSSQL.1MSSQLDataTest.mdf'. Diagnose and correct the operating system error, and retry the operation.
and it also logs
FCB:pen failed: Could not open file C:Program FilesMicrosoft SQL ServerMSSQL.1MSSQLDataTest.mdf for file number 1. OS error: 32(The process cannot access the file because it is being used by another process.).
I've caught references to the auto close feature being a possible culprit, no dice as the databases in question are set to False. Recovery mode varies on the databases from Simple to Full. If I cycle the SQL Server service, whatever transient issue it was having with those files is gone.
As much as I'd love to disable the virus scanner, network security would not be amused. The data and log files appear to have the same permissions as unaffected database files. Nothing's set to read only or archive as I've caught on other forums as possible gremlins. I have sufficient disk space and the databases are set for unrestricted growth.
Any thoughts on what I could look at? If it was everything coming up in RECOVERY_PENDING it's make more sense to me than a hit or miss type of thing I'm experiencing now.
View 13 Replies
View Related
Mar 11, 2008
Dear list
Im designing a package that uses Microsofts preplog.exe to prepare web log files to be imported into SQL Server
What Im trying to do is convert this cmd that works into an execute process task
D:SSIS ProcessPrepweblogProcessLoad>preplog ex.log > out.log
the above dos cmd works 100%
However when I use the Execute Process Task I get this error
[Execute Process Task] Error: In Executing "D:SSIS ProcessPrepweblogProcessLoadpreplog.exe" "" at "D:SSIS ProcessPrepweblogProcessLoad", The process exit code was "-1" while the expected was "0".
There are two package varaibles
User::gsPreplogInput = ex.log
User::gsPreplogOutput = out.log
Here are the task properties
RequireFullFileName = True
Executable = D:SSIS ProcessPrepweblogProcessLoadpreplog.exe
Arguments =
WorkingDirectory = D:SSIS ProcessPrepweblogProcessLoad
StandardInputVariable = User::gsPreplogInput
StandardOutputVariable = User::gsPreplogOutput
StandardErrorVariable =
FailTaskIfReturnCodeIsNotSuccessValue = True
SuccessValue = 0
TimeOut = 0
thanks in advance
Dave
View 1 Replies
View Related
Jan 30, 2007
How do I use the execute process task? I am trying to unzip the file using the freeware PZUnzip.exe and I tried to place the entire command in a batch file and specified the working directory as the location of the batch file, but the task fails with the error:
SSIS package "IngramWeeklyPOS.dtsx" starting.
Error: 0xC0029151 at Unzip download file, Execute Process Task: In Executing "C:ETLPOSDataIngramWeeklyUnzip.bat" "" at "C:ETLPOSDataIngramWeekly", The process exit code was "1" while the expected was "0".
Task failed: Unzip download file
SSIS package "IngramWeeklyPOS.dtsx" finished: Success.
Then I tried to specify the exe directly in the Executable property and the agruments as the location of the zip file and the directory to unzip the files in, but this time it fails with the following message:
SSIS package "IngramWeeklyPOS.dtsx" starting.
Error: 0xC002F304 at Unzip download file, Execute Process Task: An error occurred with the following error message: "%1 is not a valid Win32 application".
Task failed: Unzip download file
SSIS package "IngramWeeklyPOS.dtsx" finished: Success.
The command in the batch file when run from the command line works perfectly and unzips the file, so there is absolutely no problem with the command, I believe it is just the set up of the variables on the execute process task editor under Process. Any input on resolving this will be much appreciated.
Thanks,
Monisha
View 1 Replies
View Related
Mar 20, 2008
I am designing a utility which will keep two similar databases in sync. In other words, copying the new data from db1 to db2 and updating the old data from db1 to db2.
For this I am making use of the 'Tablediff' utility which when provided with server name, database, table info will generate .sql file which can be used to keep the target table in sync with the source table.
I am using the Execute Process Task and the process parameters I am providing are:
WorkingDirectory : C:Program Files (x86)Microsoft SQL Server90COM
Executable : C:SQL_bat_FilesSQL5TC_CTIcustomer.bat
The customer.bat file will have the following code:
tablediff -sourceserver "LV-SQL5" -sourcedatabase "TC_CTI" -sourcetable "CUSTOMER_1" -destinationserver "LV-SQL2" -destinationdatabase "TC_CTI" -destinationtable "CUSTOMER" -f "c:SQL_bat_Filessql5TC_CTIsql_filescustomer1"
the .sql file will be generated at: C:SQL_bat_Filessql5TC_CTIsql_filescustomer1.
The Problem:
The Execute Process Task is working fine, ie., the tables are being compared correctly and the .SQL file is being generated as desired. But the task as such is reporting faliure with the following error :
[Execute Process Task] Error: In Executing "C:SQL_bat_FilesSQL5TC_CTIpackage_occurrence.bat" "" at "C:Program Files (x86)Microsoft SQL Server90COM", The process exit code was "2" while the expected was "0". ]
Some of you may suggest to just set the ForceExecutionResult = Success (infact this is what I am doing now just to get the program working), but, this is not what I desire.
Can anyone help ?
View 9 Replies
View Related
Aug 20, 2014
I'm pulling data from Oracle db and load into MS-SQL 2008.For my data type checks during the data load process, what are options to ensure that the data being processed wouldn't fail. such that I can verify first in-hand with the target type of data and then if its valid format load it into destination table else mark it with error flag and push into errors table... All this at the row level.One way I can think of is to load into a staging table then get the source & destination table -column data types, compare them and proceed.
should I just try loading the data directly and if it fails try trouble shooting(which could be a difficult task as I wouldn't know what caused error...)
View 3 Replies
View Related
Feb 14, 2007
Hi Folks,
I am having this table locking issue that I need to start paying attention to as its getting more frequent.
The problem is that the data in the tables is live finance data that needs to be changed and viewed almost real time so what I have picked up so far is that using 'table Hints' may not be a good idea.
I have a guy at work telling me that introducing a data access layer is the only way to solve this, I am not convinced but havnt enough knowledge to back my own feeling up. (asp system not .net).
Thanks in advance
View 1 Replies
View Related
Jan 6, 2012
We are facing deadlock issue in our web application. The below message is coming:
> Session ID: pwdagc55bdps0q45q0j4ux55
> Location: xxx.xxx.xxx.xxx
> Error in: http://xxx.xxx.xxx.xxx:xxxx/Manhatta...Bar=&Mode=Edit
> Notes:
> Parameters:
> __EVENTTARGET:
> __EVENTARGUMENT:
[code].....
View 2 Replies
View Related
Feb 17, 2007
Hi,
I'm trying to upload the ASPNETDB.MDF file to a hosting server via FTP, and everytime when it was uploaded half way(40% or 50%)
I would get an error message saying:
"550 ASPNETDB.MDF: The process cannot access the file because it is being used by another process"
and then the upload failed.
I'm using SQL Express. Does anybody know what's the cause?
Thanks a lot
View 1 Replies
View Related
Nov 15, 2007
Hi. When I try to start a package manually clicking the Start Debugging button I get this after a little while:
Cannot process request because the process (3880) has exited. (Microsoft.DataTransformationServices.VsIntegration)
How can I prevent this from happening? This happens every time I want to start the package and
every time the process id is different. Here it is 3880.
Darek
View 13 Replies
View Related
May 7, 2002
Is it possible to execute a DTS package via a stored procedure?
We have created a web-based application with a SQL back-end, I would like the
user to select an option on the .asp page that will then trigger the DTS.
Is this possible? Any suggestions would be greatly appreciated.
Thanks
View 2 Replies
View Related
Mar 2, 2006
Can someone let me know any 10 things that are usally automated using the SQL Server Agent,
I mean besides backup what all can i automate
Thanks
View 1 Replies
View Related
Jan 28, 2014
I want to create a form (Infopath) or web page where the user can request for a new database I want the user to fill in the required info (like the size, name columns.....) and ones they're done filling the required info, they will click on a button and that should trigger the database automation script to run and if the database with the requested name is not created, the script should create a database for the user.
View 1 Replies
View Related
Mar 2, 2006
Can someone let me know any 10 things that are usally automated using the SQL Server Agent,
I mean besides backup what all can i automate
Thanks
View 10 Replies
View Related
Mar 15, 2007
I want to automate a query that I am running everyday. So, what I want to do is run it at a particular time of the day say 11:00pm. Is this possible? Should I create a stored procedure? And also, I want to run an update statement on a particular table on the first of every month. If anyone can give me some advice on this, I would appreciate it.
Jared
View 8 Replies
View Related
May 7, 2007
Folks:
I have a text file which has all "drop" statements (drop proc, drop view)... I want to run this statements on a daily basis. How can I automate this?
Thanks !
View 8 Replies
View Related
Jul 23, 2005
hello people!I am very new to the SQL World. Currently I work on DB2 and ORACLE andthis is a new "fruit" for me.I have the task to automate the following scenario:- backup database A- restore database A into database B (overwrite database B)- perform the above every night at 23:00.Can anyone indicate me where do i find info how to perform the above?I know that you can do it through the Enterprise Manager but I need toscript the above (probably).Thanks
View 6 Replies
View Related
Aug 14, 2006
Hi All,
I have created the package using Import data ( from comma delimeted text file)
Actually I schedule the JOB to run the package from SQL Server Agent
I am getting error when start the job. ( job started successfully and executing time am getting error.)
here is the Log error
[298] SQLServer Error: 3621, The statement has been terminated. [SQLSTATE 01000] (ConnIsLoginSysAdmin)
[298] SQLServer Error: 15404, Could not obtain information about Windows NT group/user 'VGOSVRSaleem', error code 0x2095. [SQLSTATE 42000] (ConnIsLoginSysAdmin)
If I run the .dtsx file from outside. it works fine
do I need to do any kind of setup before running/starting the job ??
Please advise
Cheers
Saleem
View 3 Replies
View Related
Feb 8, 2008
I am running this procedure to copy the call data info from oma11wicacct01 to oma11csql5tst01 and to vsql4shared
INSERT INTO [oma11csql5tst01].[rms sql].dbo.WICCallData ( ClientID, ProgramID, Calls, ActSeconds, UsageDate )
SELECT WICCallData.ClientID, WICCallData.ProgramID, Sum(WICCallData.Calls) AS SumOfCalls, Sum(WICCallData.ActSeconds) AS SumOfActSeconds, WICCallData.UsageDate
FROM WICCallData
GROUP BY WICCallData.ClientID, WICCallData.ProgramID, WICCallData.UsageDate
HAVING (Sum(WICCallData.Calls)>0 or Sum(WICCallData.ActSeconds)>0) AND WICCallData.UsageDate>'2/3/2008'
The usage date has to be greater than the max date on the target database!
How can I automate the execution?
View 3 Replies
View Related
Dec 19, 2006
hi!
i already have a program that when you click a button it will get the new transactions in the database then pass it to another database now i need to automate the passing... once there is a new entry in the database it should automatically pass it to the other database. how could i do this?!
thanks!
View 5 Replies
View Related
Dec 20, 2006
select * from sysprocesses
How can I determine whether a process a system or user?
View 3 Replies
View Related
Oct 11, 2007
Hello,
I have had a full lock on my sql server and I have a few logs to found the origin of the lock.
I know the process at the head of the lock is the 55 process.
Here are the information I have on this process:
Spid 55 55
ecid 5 5
Ecid 0 0
ObjId 0 1784601646
IndId 0 0
Type DB PAG
Resource 1:1976242
Mode S IS
Status TransID GRANT GRANT
TransID 0 16980
TransUOW 00000000-0000-0000-0000-000000000000
00000000-0000-0000-0000-000000000000
lastwaittype PAGEIOLATCH_SH
CMD AWAITING COMMAND
Physycal id 1059
Login time 2007-07-05 04:29:53.873
nat address DFF06EBF974D
Wait type 0x0046
HostName .
BlkBy .
DBName grpprddb
CPUTime 54331
DiskIO 1059
ProgramName
Would someone know a way to identify the origin of the process 55?
I have already tried to execute the following request:
select * from SYSOBJECTS
where id=1784601646
But I have had no returns.
Regards,
Renaud
View 3 Replies
View Related
Aug 24, 2006
I have a File System Task Copy file operation to copy a file in an SSIS package. The package when scheduled as a job fails with the following error:
The process cannot access the file 'C:ETLConsignmentAppleAppleRawFile.txt' because it is being used by another process.".
However when I right click on the package and execute it manually from the Integration Services it runs successfully without any problem. I am not certain on how to resolve this issue any inputs will be much appreciated.
Thanks,
Monisha
View 19 Replies
View Related
Feb 6, 2008
Error: 0xC002F304 at Rename file 1, File System Task: An error occurred with the following error message: "The process cannot access the file because it is being used by another process.".
When running two File System Tasks after each other, with the same file, the file is still locked when running the second task. Resulting in an error: 0xC002F304 at Rename file 1, File System Task: An error occurred with the following error message: "The process cannot access the file because it is being used by another process.".
I found a workaround by addind a Execute Process Task before the second File System Task that pings to the localhost. This results in a 5 second delay, but there must be a better solution. Anyone?
View 9 Replies
View Related
Nov 23, 2006
While configuring log shipping using SQL 2000 Ent, the copy process failed with this message :
The process cannot access the file because it is being used by another process.
does anyone know what cause this ?
SAP R/3 is configured with the system, could this be the problem ?
View 3 Replies
View Related
Dec 11, 2003
Is there a way to automate a sql server restore? I have a db that I need to include in an installation. I could use generate script but there is info in the db that needs to be copied too. What is the best way to do this?
Thanx
weisenbr
View 1 Replies
View Related
