Dear all,
I'm starting an estimation exercise for an ETL/Datawarehouse project.
Does any of you knows a proven methodology and benchmark information to be able to estimate the overall effort of an ETL project. We have an idea on the number of tables/fields in the source systems, and a (very high-level) idea on the complexity of the different transformations.
Is there any website/document/whitepaper somewhere where an estimation technique is presented ?
I was trying to extract data from the source server using OLEDB Source and SQL Server Destination when i encountered this error:
"Transaction (Process ID 135) was deadlocked on lock resources with another process and has been chosen as the deadlock victim. Rerun the transaction.".
What must be done so that even if the table being queried is locked, i wouldn't experience any deadlock?
Hello all, I am running into an interesting scenario on my desktop. I'm running developer edition on Windows XP Professional (9.00.3042.00 SP2 Developer Edition). OS is autopatched via corporate policy and I saw some patches go in last week. This machine is also a hand-me-down so I don't have a clean install of the databases on the machine but I am local admin.
So, starting last week after a forced remote reboot (also a policy) I noticed a few of the databases didn't start back up. I chalked it up to the hard shutdown and went along my merry way. Friday however I know I shut my machine down nicely and this morning when I booted up, I was in the same state I was last Wenesday. 7 of the 18 databases on my machine came up with
FCB:pen: Operating system error 32(The process cannot access the file because it is being used by another process.) occurred while creating or opening file 'C:Program FilesMicrosoft SQL ServerMSSQL.1MSSQLDataTest.mdf'. Diagnose and correct the operating system error, and retry the operation. and it also logs FCB:pen failed: Could not open file C:Program FilesMicrosoft SQL ServerMSSQL.1MSSQLDataTest.mdf for file number 1. OS error: 32(The process cannot access the file because it is being used by another process.).
I've caught references to the auto close feature being a possible culprit, no dice as the databases in question are set to False. Recovery mode varies on the databases from Simple to Full. If I cycle the SQL Server service, whatever transient issue it was having with those files is gone. As much as I'd love to disable the virus scanner, network security would not be amused. The data and log files appear to have the same permissions as unaffected database files. Nothing's set to read only or archive as I've caught on other forums as possible gremlins. I have sufficient disk space and the databases are set for unrestricted growth.
Any thoughts on what I could look at? If it was everything coming up in RECOVERY_PENDING it's make more sense to me than a hit or miss type of thing I'm experiencing now.
Dear list Im designing a package that uses Microsofts preplog.exe to prepare web log files to be imported into SQL Server
What Im trying to do is convert this cmd that works into an execute process task D:SSIS ProcessPrepweblogProcessLoad>preplog ex.log > out.log the above dos cmd works 100%
However when I use the Execute Process Task I get this error [Execute Process Task] Error: In Executing "D:SSIS ProcessPrepweblogProcessLoadpreplog.exe" "" at "D:SSIS ProcessPrepweblogProcessLoad", The process exit code was "-1" while the expected was "0".
There are two package varaibles User::gsPreplogInput = ex.log User::gsPreplogOutput = out.log
How do I use the execute process task? I am trying to unzip the file using the freeware PZUnzip.exe and I tried to place the entire command in a batch file and specified the working directory as the location of the batch file, but the task fails with the error:
SSIS package "IngramWeeklyPOS.dtsx" starting.
Error: 0xC0029151 at Unzip download file, Execute Process Task: In Executing "C:ETLPOSDataIngramWeeklyUnzip.bat" "" at "C:ETLPOSDataIngramWeekly", The process exit code was "1" while the expected was "0".
Then I tried to specify the exe directly in the Executable property and the agruments as the location of the zip file and the directory to unzip the files in, but this time it fails with the following message:
SSIS package "IngramWeeklyPOS.dtsx" starting.
Error: 0xC002F304 at Unzip download file, Execute Process Task: An error occurred with the following error message: "%1 is not a valid Win32 application".
The command in the batch file when run from the command line works perfectly and unzips the file, so there is absolutely no problem with the command, I believe it is just the set up of the variables on the execute process task editor under Process. Any input on resolving this will be much appreciated.
I am designing a utility which will keep two similar databases in sync. In other words, copying the new data from db1 to db2 and updating the old data from db1 to db2.
For this I am making use of the 'Tablediff' utility which when provided with server name, database, table info will generate .sql file which can be used to keep the target table in sync with the source table.
I am using the Execute Process Task and the process parameters I am providing are:
The customer.bat file will have the following code: tablediff -sourceserver "LV-SQL5" -sourcedatabase "TC_CTI" -sourcetable "CUSTOMER_1" -destinationserver "LV-SQL2" -destinationdatabase "TC_CTI" -destinationtable "CUSTOMER" -f "c:SQL_bat_Filessql5TC_CTIsql_filescustomer1"
the .sql file will be generated at: C:SQL_bat_Filessql5TC_CTIsql_filescustomer1.
The Problem: The Execute Process Task is working fine, ie., the tables are being compared correctly and the .SQL file is being generated as desired. But the task as such is reporting faliure with the following error :
[Execute Process Task] Error: In Executing "C:SQL_bat_FilesSQL5TC_CTIpackage_occurrence.bat" "" at "C:Program Files (x86)Microsoft SQL Server90COM", The process exit code was "2" while the expected was "0". ]
Some of you may suggest to just set the ForceExecutionResult = Success (infact this is what I am doing now just to get the program working), but, this is not what I desire.
I'm pulling data from Oracle db and load into MS-SQL 2008.For my data type checks during the data load process, what are options to ensure that the data being processed wouldn't fail. such that I can verify first in-hand with the target type of data and then if its valid format load it into destination table else mark it with error flag and push into errors table... All this at the row level.One way I can think of is to load into a staging table then get the source & destination table -column data types, compare them and proceed.
should I just try loading the data directly and if it fails try trouble shooting(which could be a difficult task as I wouldn't know what caused error...)
I am having this table locking issue that I need to start paying attention to as its getting more frequent.
The problem is that the data in the tables is live finance data that needs to be changed and viewed almost real time so what I have picked up so far is that using 'table Hints' may not be a good idea.
I have a guy at work telling me that introducing a data access layer is the only way to solve this, I am not convinced but havnt enough knowledge to back my own feeling up. (asp system not .net).
Hi, I'm trying to upload the ASPNETDB.MDF file to a hosting server via FTP, and everytime when it was uploaded half way(40% or 50%) I would get an error message saying: "550 ASPNETDB.MDF: The process cannot access the file because it is being used by another process" and then the upload failed. I'm using SQL Express. Does anybody know what's the cause? Thanks a lot
Hi. When I try to start a package manually clicking the Start Debugging button I get this after a little while:
Cannot process request because the process (3880) has exited. (Microsoft.DataTransformationServices.VsIntegration)
How can I prevent this from happening? This happens every time I want to start the package and every time the process id is different. Here it is 3880.
I have had a full lock on my sql server and I have a few logs to found the origin of the lock.
I know the process at the head of the lock is the 55 process.
Here are the information I have on this process: Spid 55 55 ecid 5 5 Ecid 0 0 ObjId 0 1784601646 IndId 0 0 Type DB PAG Resource 1:1976242 Mode S IS Status TransID GRANT GRANT TransID 0 16980 TransUOW 00000000-0000-0000-0000-000000000000 00000000-0000-0000-0000-000000000000
lastwaittype PAGEIOLATCH_SH CMD AWAITING COMMAND Physycal id 1059 Login time 2007-07-05 04:29:53.873 nat address DFF06EBF974D Wait type 0x0046 HostName . BlkBy . DBName grpprddb CPUTime 54331 DiskIO 1059 ProgramName
Would someone know a way to identify the origin of the process 55?
I have already tried to execute the following request: select * from SYSOBJECTS where id=1784601646
I have a File System Task Copy file operation to copy a file in an SSIS package. The package when scheduled as a job fails with the following error:
The process cannot access the file 'C:ETLConsignmentAppleAppleRawFile.txt' because it is being used by another process.".
However when I right click on the package and execute it manually from the Integration Services it runs successfully without any problem. I am not certain on how to resolve this issue any inputs will be much appreciated.
Error: 0xC002F304 at Rename file 1, File System Task: An error occurred with the following error message: "The process cannot access the file because it is being used by another process.".
When running two File System Tasks after each other, with the same file, the file is still locked when running the second task. Resulting in an error: 0xC002F304 at Rename file 1, File System Task: An error occurred with the following error message: "The process cannot access the file because it is being used by another process.".
I found a workaround by addind a Execute Process Task before the second File System Task that pings to the localhost. This results in a 5 second delay, but there must be a better solution. Anyone?
i'm going nuts with SQL server notification thing. I have gone throigh this artical which tells how to set user http://www.codeproject.com/KB/database/SqlDependencyPermissions.aspx. This article show how to create new user and setup for sql server notification.But In my case user was alredy existing in database. which is very common senario in most cases. So i did following( check the SQL script below) but then i get this error "A connection was successfully established with the server, but then an error occurred during the login process. (provider: Shared Memory Provider, error: 0 - No process is on the other end of the pipe.)" this my sql script use [master]Go -- Ensuring that Service Broker is enabled ALTER DATABASE [DatabaseName] SET ENABLE_BROKERGO -- Switching to our databaseuse [DatabaseName]GO CREATE SCHEMA schemaname AUTHORIZATION usernameGO ALTER USER username WITH DEFAULT_SCHEMA = schemaname GO /* * Creating two new roles. We're not going to set the necessary permissions * on the user-accounts, but we're going to set them on these two new roles. * At the end of this script, we're simply going to make our two users * members of these roles. */EXEC sp_addrole 'sql_dependency_subscriber' EXEC sp_addrole 'sql_dependency_starter' -- Permissions needed for [sql_dependency_starter]GRANT CREATE PROCEDURE to [sql_dependency_starter] GRANT CREATE QUEUE to [sql_dependency_starter]GRANT CREATE SERVICE to [sql_dependency_starter]GRANT REFERENCES on CONTRACT::[http://schemas.microsoft.com/SQL/Notifications/PostQueryNotification] to [sql_dependency_starter] GRANT VIEW DEFINITION TO [sql_dependency_starter] -- Permissions needed for [sql_dependency_subscriber] GRANT SELECT to [sql_dependency_subscriber] GRANT SUBSCRIBE QUERY NOTIFICATIONS TO [sql_dependency_subscriber] GRANT RECEIVE ON QueryNotificationErrorsQueue TO [sql_dependency_subscriber] GRANT REFERENCES on CONTRACT::[http://schemas.microsoft.com/SQL/Notifications/PostQueryNotification] to [sql_dependency_subscriber] -- Making sure that my users are member of the correct role.EXEC sp_addrolemember 'sql_dependency_starter', 'username'EXEC sp_addrolemember 'sql_dependency_subscriber', 'username'
Hi I am using VWD 2008, SQL Express 2005, Reporting Services, Win-XP, IIS5Basically let's say I have 2 pages:Page1: has a SQLDataSource control that populates a GridView from a table from a database file myDB.mdf (no code behind)Page2: has a reportviewer control that show a report with data from the same table from myDB.mdf from the reportserver, (no code behind)I have attached myDB.mdf to the SQL Server Express using the SQL Server Management Studio Express.If I first open Page2 to display the ReportViewer it works ok. or using the Report ManagerNow this is the problem:If after that I try to open Page1 then a get an error message:Cannot open user default database. Login failed.Login failed for user 'myServerASPNET'. Exception Details: System.Data.SqlClient.SqlException: Cannot open user default database. Login failed.Login failed for user 'myServerASPNET'.Then I have to restart the SQL Server to fix it,Now I can open Page1 ok, but if after this I try to open Page2 (ReportViewer) againThen I get this error:" An error has occurred during report processing. o Cannot create a connection to data source 'my_Datasource'. &And this error if open the report using the report manager:" An error has occurred during report processing. o Cannot create a connection to data source 'my_Datasource'. ยง Unable to open the physical file "C:InetpubwwwrootWebsiteApp_DatamyDB.mdf". Operating system error 32: "32(The process cannot access the file because it is being used by another process.)". &Now if i check the Management Studio Express again, you can see that myDB.mdf was detached. It seems to be there by it has no Tables or definitions, so I have to attach it again..Do you know how to fix this?Thanks in advance,Ed
I came across a weird problem in the morning in SQL Server 7.0 on NT Cluster Server. * SP_WHO - returned a list of System Process IDs where ID = 6 shown as a sql server User comming from a web server doing rollback on a stored procedure.. a) I tried to KILL the user but I couldn't. b) Next I stopped the IIS service on Webserver and tried to KILL the user but still couldn't. Surprisingly the user was showing up as comming from the same Webserver eventhough the IIS Service was not running.
* When I checked in the Current Status window of Enterprise Manager - the same Process ID was shown as SYSTEM User doing rollback on a stored procedure.
Because of above user problem I was not able to do any Delete or update in the database. Also Restore of database failed because of the above user.
I could get rid of this problem only after STOPPING and STARTING the SQL Server Service.
Does any one one have any IDEA on this type of problem. If so Please let me why it happend and what needs to be done to resolve this from happening again
Dear friends, I'm working on 5 different servers,my work is to write the script and make it to run successfully.
I've two databases for me one to test the script and the other for the final script. after that i'l implement in other servers. after successful execution, the testing team will take care. now my problem is, some times i'm not able to find the exact script which was successfully runned.(on some servers, modifications were done.because some dependencies will be increased).
my question is how to maintain the exact script? may be this is a silly question, but after one week i'm not able to find the exact script.
please help me in this regard with your valuble suggesions.
Hi,Please help.I have 2 tables as followings:CREATE TABLE [dbo].[Master] ([masitemno] [char] (10) NOT NULL ,[masqty] [decimal](10, 3) NOT NULL ,[masunitcost] [decimal](10, 2) NOT NULL) ON [PRIMARY]GOCREATE TABLE [dbo].[Transaction] ([transeqno] [int] NOT NULL ,[tranitemno] [char] (10) NOT NULL ,[tranqty] [decimal](10, 3) NOT NULL ,[tranamount] [decimal](10, 2) NOT NULL ,[tranunitcost] [decimal](10, 2) NOT NULL) ON [PRIMARY]GOALTER TABLE [dbo].[Master] WITH NOCHECK ADDCONSTRAINT [PK_Master] PRIMARY KEY NONCLUSTERED([masitemno]) ON [PRIMARY]GOALTER TABLE [dbo].[Transaction] WITH NOCHECK ADDCONSTRAINT [PK_Transaction] PRIMARY KEY NONCLUSTERED([transeqno]) ON [PRIMARY]GOTable "Transaction" has about 1,000,000 (one million rows) and Table"Master" has about 500,000 rows.I have to update "MASTER" table with "TRANSACTION" table withrow-by-row processing basis sorting byprimary key TRNSEQNO column.Sometimes TRANSACTION can explicitly SET "MASQTY" and "MASUNITCOST"columns (TRANUNITCOST<>0) of MASTERwhich linked byitemno and after that AMOUNT column of next row ofTRANSACTION will used thisnew UNITCOST of MASTER as followed statements.-------------------------------------------------declare @count int, @max intset @count=1set @max = (select max(seqno) from transaction(nolock)while @count<=@maxbeginupdate TRANSACTIONset TRANAMOUNT = TRANQTY * (select MASUNITCOST from MASTERwhere MASITEMNO=TRANITEMNO)where TRANSEQNO = @countand TRANUNITCOST = 0update MASTERset MASQTY = MASQTY + TRANQTYfrom TRANSACTIONwhere TRANSEQNO = @countand TRANUNITCOST = 0and MASITEMNO=TRANITEMNOupdate TRANSACTIONset TRANAMOUNT = TRANQTY * TRANUNITCOSTwhere TRANSEQNO = @countand TRANUNITCOST <> 0update MASTERset MASQTY = MASQTY + TRANQTY,MASUNITCOST = TRANUNITCOSTfrom TRANSACTIONwhere TRANSEQNO = @countand TRANUNITCOST <> 0and MASITEMNO=TRANITEMNOset @count = @count +1end-------------------------------------------------The above sample statements take me more than 10 hrs. (I quit beforeactually done) with MS SQL SERVER 7.5 SP4.on WIN2K SERVER (2 XEON PROCESSORS, 1GB MEM.). I tried to use triggerbut result is not correct.Please advise on shorten running time (in minutes , maybe) and betterperformance.Thank you and appreciate any suggestionsNipon Wongtrakul
In installing sql express/management, during installion I get an error message when it comes to sql db services - It says setup failed because there is no process on the other end of the pipe.
Also, When I go to Start/All Programs/Microsoft Server 2005/Configuration Tool it says empty
This is running on Vista RC1
There is no way for me to enable remote connection
When I start Microsoft Sql Server Management Studio there is no server name to select from or choose from browseing
There is no sqlbrowser in SERVICES for me to start.
Also not install during installation is the work station component.
I'm looking for Ideas on how to handle a Pldege Reminder process. For example; a pledge is made to pay $2400 over the next two years. They will pay $100 per month and each month a reminder will be sent. No real mistery if there is a balance you send a reminder. My problem is how to handle things like what if they want to pay quarterly or annually and how to determine if a payment is really due based on when they paid last, etc... You most likely see what I mean. If anyone has done this in the past and/or has any ideas and is willing to share I would greatly appreciate any help. Some stuff that may help you help me better: tblClient (ClientID) tblPledge (PledgeID, ClientID, PledegedAmt, PledgeDate,Frequency,NumberYears) tblPledgePayments (PmtID, PledgeID,PmtAmt,PmtDate)
I'm going to design a web page for students parking permit application.I'm new to asp.net, I have a process in mind doing it like this: The form asks student to choose a permit type, then fill in applicant info, like name, grade, vehicle info, student may have 2 vehicles, and payment method, check or credit card, I plan to create 3 tables like tblPermit, tblapplicant, tblVehicle.After the students fill in the form, click submit button, I pass the values and call a stored procedure, some thing like this.Dim Cmd As New Data.SqlClient.SqlCommand(MySQL, MyConn) Cmd.CommandType = Data.CommandType.StoredProcedure Then in the stored proceudre, I will do : Begin insert into Permit table, then get permitID, by using something like this:Select @PermitID=@@Identity Then insert into Student table, then use Select @StudentID=@@Identity Then insert into vehicle table. All the above 3 processes I put it in begin, end--So that they will not messed up with other applicants data. I think begin...end is one transaction. SO am I doing correctly this way? Thanks
I have SQL Server 7.0 with SP2 on it and I am not able to kill one of the SPID's who is running a SELECT Statement. When I saw in the current activity in EM I see the STATUS as ROLLBACK, COMMAND as SELECT, APPLICATION as MS SQLEM-DATATOOLS, WAITTIME as 5375 and WAITTYPE as CXPACKET..
I also tried to reboot the machine of that particular SPID , but it still shows up in the Enterprise Manager.
Can anyone tell me how to kill this process and how to get around this Problem
Hi One DTs package job running and we stop the job.Job was stoped but process not killing.We tried using kill spid but no use.please any body give suggestion. this production server.
I have this weird situation on one of the production database. Somebody ran a sql script couple days ago and must have hit cancelled. Somehow this script is still rolling back. The script is actually very simple statement..should not take more than couple miliseconds to run. Is there any way to see whether rollback is really in process or this process has simply hung? is there any other way to get rid of this rollback without stopping and restarting sql server services? Can it cause problems in getting sql server up again if I restart sql server services?
I am seeing a strange scenario in performance monitor that i am hoping someone could help shed some light on. I have a 8 processor box running only sql configured to utilize all processors. When watching the sqlservr process it seems to max out at times of high traffic. Does this not use all processors? Any know problems with this service maxing out?
what does the sleeping process will do I am running the following stored procedure the cpu value is increasing by time. I am killing old process manually. How to stop the sleeping processing.
CREATE PROCEDURE SP_MONITOR2 AS select spid, hostname,cpu,status,datediff(mi,last_batch,getdate ())Diff_in_Min from sysprocesses where hostname <>'' and last_batch <>'1900-01-01 00:00:00.000' order by Diff_in_Min desc
I hope you can help me to find the way to resolve this issue.
When I've accidentally triggered a process in SQL 6.5 without knowing that it's going to hold a lot of resources and making the network very slow and end users will be started complaining about the slowness.
I've no other way other than the killing the process kill <spid>. Funniest part is, even it is not get killed even when I tried multiple times and the process is still active and still running.
When I tried to find the way out, One another guy stopped the SQL server and restarted again. That's it, it took a long time to stop and restart and ends up with the recovery mode of the database and it was running more than 3-4 hours to get into the usual mode.
Based on the scenario, what would be your suggestion when I've encoutered the same situation. I've triggered some application like DBCC checkdb and it's keep running for a long time. But I need to kill the process immediately without affecting any other process. Pls. advise me.
I have a problem running a batch file, now the problem is that when i run the batch file the command prompts the user for an input, but I have all the output of the bat file going into a log file. So when i run the bat file the process just sits there until i hit the 'y' key or unless i nput sumthin manually. This is a problem becuase this batch file is running on the UAT server as a job and there is no one there to input once the job is running. The commnd in the batch only requires an input once a month.
for eg if run: launch_scrt.bat and i want to put sum parameters such as 'y' or 'n' to avoid the manual input once the job is running. Any ideas?
