Dts.Variables Collection, Threading, And Contention
Apr 29, 2008
All,
We are experiencing some problems when running the same package on multiple threads of a C# application concurrently. The package instances seem to share a Dts.Variables collection.
I have created a test application and package to reproduce the issue. The package is very simple. It has two variables - Instance (Int32) and Content (string). The test application runs two concurrent threads that run through 25 iterations running the package. Each thread loads the package, sets the instance, and then runs the package. The package then takes the instance, sticks it in the Content variable and then writes it to a file called "FileX" where X is the Instance. Thread 1 sets the instance from 1 to 25, thread 2 sets it from 26 to 50. The test execution results in about 15% of the files containing the incorrect "instance content". That is, I'll see "42" in the file named "File14", or something along those lines. This tells me that the variables are being overwritten.
I ran another test where I created a copy of the package, so the code is identical, but the VersionGUID is different. Running the thread test application always succeeds. There is no file name/content mismatch. I then ran another test where I manually edit the XML of the package to change the VersionGUID before the package is loaded (by calling LoadFromXml). Again this never fails.
Changing the VersionGUID may work for us, but it is a total hack. Has anyone else experienced this or have any other solutions? Any thoughts on manually updating the VersionGUID to get around this problem? I hate doing this, but we are getting desperate.
Thanks in advance,
Rob
View 2 Replies
ADVERTISEMENT
Mar 27, 2008
Precedence Constraint Editor Values:
Expression Operation: Expression and Constraint
Value: Failure
Expression: @GotRecs == 1
Radio button: Logical AND. All contraints must evaluate to True
Stored Procedure has an IF (that contains a SELECT) statement that checks for records if no records are returned processing stops. I SET the parameter after the BEGIN that follows the IF statement.
The step that contains the above code is step #3. I placed the constraint between step #3 and step #4 (Data Flow task) just Exports to a Flat File.
When I clicked the TEST button in the Precedence Constraint Editor I got the following msg:
Error at Constraint 3: The variable €œGotRecs€? was not found in the variables collection. The variable might not exist in the correct scope.
View 12 Replies
View Related
Oct 1, 2007
As far as I can make out it is not possible to access the variables collection from a custom log provider.
I'm interested to know what the rationale for this is. Why aren't the variables available for logging purposes?
Thanks
Jamie
[Microsoft follow-up]
View 6 Replies
View Related
Sep 7, 2006
I've created a user variable but when I'm going through its variable collections for that TaskHost appears all the system users but not mine.
Why??
?th.variables.count
26
When I have 27 including the mine.
View 1 Replies
View Related
Jan 17, 2008
Hello, I am trying to configure a Script Component as a data source. Although this should be a simple exercise, I am running into a problem.
My control flow contains a Foreach Loop with a file iterator. The Directory Expression of the Foreach Loop Editor is supplied by an expression mapped to a package level variable called inputdirectory. The FileNameRetrieval Expression is mapped to a package scoped variable called filename.
My data flow is encapsulated by the Foreach Loop, and contains a Script Component as Source for the Data Flow Source, and a Flat File for the Data Flow Destination. The contents of the Script Designer are as follows:
Code Block
Imports System
Imports System.Data
Imports System.Math
Imports Microsoft.SqlServer.Dts.Pipeline.Wrapper
Imports Microsoft.SqlServer.Dts.Runtime.Wrapper
Imports System.IO
Public Class ScriptMain
Inherits UserComponent
Dim thisFileDate As Date
Dim thisFileName As String
Dim thisFilePath As String
Public Overrides Sub CreateNewOutputRows()
thisFileName = ReadOnlyVariables("filename").Value.ToString()
thisFilePath = ReadOnlyVariables("inputdirectory").Value.ToString()
thisFileDate = File.GetCreationTime(thisFilePath & "" & thisFileName)
FileSpecBuffer.AddRow()
FileSpecBuffer.FileName = thisFileName
FileSpecBuffer.FullPath = thisFilePath & "" & thisFileName
FileSpecBuffer.CreateDate = thisFileDate
End Sub
End Class
When I debug the package, I get the following error:
The collection of variables locked for read access is not available at this point.
at Microsoft.SqlServer.Dts.Pipeline.ScriptComponent.get_ReadOnlyVariables()
at ScriptComponent_67311120e6eb4162a3ea1f70847f04de.ScriptMain.CreateNewOutputRows()
at ScriptComponent_67311120e6eb4162a3ea1f70847f04de.UserComponent.PrimeOutput(Int32 Outputs, Int32[] OutputIDs, PipelineBuffer[] Buffers)
at Microsoft.SqlServer.Dts.Pipeline.ScriptComponentHost.PrimeOutput(Int32 outputs, Int32[] outputIDs, PipelineBuffer[] buffers)
My googlefu fails me at reconciling this. I have read various posts about changing the line ReadOnlyVariables("filename").Value to ReadWriteVariables("filename").Value, and handling the buffer assignment in PostExecute. The problem is that all examples shown are either for Script Component as Destination or Script Component as Transformation. I have tried playing with the Custom Properties in the Script Commponent set the ReadOnlyVariables and ReadWriteVariables, using a PreExecute method, a PostExecute method, all with different errors returning. I'm at a loss here. Could anybody provide me with a simple working example so that I can correctly populate my output buffer in the context of Script Component as Source?
I fully understand that I could just run a Script Task Using System.IO.Directory, and System.IO.File, but I really want to make this package work in the manner I've described. Any help would be appreciated.
Thanks,
John T
View 5 Replies
View Related
Mar 19, 1999
Has anyone come across the problem of transactions not being able to write to the transaction log while a 'dump transaction' operation is in progress and therefore causing the transaction to timeout??
Is there any way of eliminating the timeouts/contention??
View 1 Replies
View Related
May 19, 2008
hi,
this is sanjeev,
i have SSIS package, using my c# program i want to add one execute package task to this package's sequence container.
it is creating the new package with out any probelm. but when i opened the package and try to move the newly created exeute package task it is giving the following error.
the element cannot be found in a collection. this error happens when you try to retrieve an element from a collection on a container during the execution of the package
this is my code
{
Package pkg = new Package();
string str = (string)entry.Key;
pkg.Name = str;
alEntity = (ArrayList)entry.Value;
ConnectionManager conMgr;
Executable chPackage;
TaskHost executePackageTask;
Microsoft.SqlServer.Dts.Runtime.Application app = new Microsoft.SqlServer.Dts.Runtime.Application();
//string PackagePath = @"c:Genesis.dtsx";
//p = app.LoadPackage(PackagePath, null);
p = new Package();
p.LoadFromXML(parentPackageBody, null);
p.Name = str;
//Sequence seqContainer;
IDTSSequence seqContainer;
//seqContainer = (Sequence)p.Executables["Extract Genesis Data"];
seqContainer = ((Sequence)p.Executables[0]);
string packageLocation = @"Geneva Packages";
conMgr = p.Connections["SQLChildPackagesConnectionString"];
foreach (string val in alEntity)
{
if (seqContainer.Executables.Contains("Load_" + val) == false)
{
chPackage = seqContainer.Executables.Add("STOCK:ExecutePackageTask");
executePackageTask = (TaskHost)chPackage;
executePackageTask.Name = "Load_" + val;
executePackageTask.Description = "Execute Package Task";
executePackageTask.Properties["Connection"].SetValue(executePackageTask, conMgr.Name);
executePackageTask.Properties["PackageName"].SetValue(executePackageTask, packageLocation + ddlApplication.SelectedItem.Text + @"" + executePackageTask.Name);
}
}
app.SaveToXml(Server.MapPath("../SynchronizeScript/Packages/" + ddlApplication.SelectedItem.Text + @"") + str + ".dtsx", p, null);
}
please let me know what is the wrong in my code.
thanks in advance.
regards
sanjeev bolllina
sanjay.bollina@gmail.com
View 14 Replies
View Related
Feb 28, 2008
Hi - have 150,000,000 row table receiving BULK INSERTS all day - I have also found out my client likes to performan READ opertions simultaniously against this table with of course chronic contention issues.
Need to provide a quick-fix, will index views enable me to abstact the reporting data away from the BULK INSERT table and is there any connection issues between the base table and the Index View?
View 2 Replies
View Related
Jun 23, 2015
 I have used Extented event to monitor the occurances of TempDB contention on Production server . I found there are several entried logged in in 30 mints .Now I am trying to determin if Tempdb contention on PFS, GAM or SGAM page then I will decide if I need to increase the number of TempDB data files on Production server . Currently , There are 8 TempDB Data files configured on its separate Disks .There are Page_IDs I found in the extented events for Tempdb files -
Page_IDÂ =1 for PFS page
Page_ID = 2 for GAM page
Page_ID =3 for SGAM page
but I found the Below Page_IDs and I know there is a formula that you can use to identify if page is PFS,GAM or SGAM ? How should I use this formula and what should I look for to determine if page is PFS,GAM or SGAM ? Is there any threshold value for the duration of TempDB contention occured ?
View 10 Replies
View Related
Jun 3, 2002
I have some threading questions, please explain how these work. Thanks!!
How do the threads get created (programmatically, network, or through sequel)?
If I have 10 people in an application, is a thread created for each user?
If I run a large process can I subdivide the process and run it through mulitple threads?
View 1 Replies
View Related
Sep 29, 2005
Ello,
We have quite a few SQL Servers running on servers with hyper threaded enabled processors. We have always left hyper threading on, even on our very busy servers but i've recently begun to question this due to comments i have received.
All our servers run SQL 2000 SP3a as "support" for hyper threading was added in SP3a
So... What do you think of Hyper Threading any why?
View 5 Replies
View Related
Aug 9, 2006
Over some time now, I've been developing a fairly hefty stored procedure, that does a lot of computations, and fairly few table lookups.
When I look at the performance on my server (a dual Xeon HT) I can see that it only uses 1 out of 4 possible "cpus" to work on the calculations, while the three others idle out, and was wondering if I can somehow force it to use max available CPU power?
View 2 Replies
View Related
Aug 23, 2006
I have a vb 2005 app that executes a SSIS package in threads.
The SSIS package imports a large (20mb) file, does many alternation to it, exports it to the database.
The SSIS execution is quite frequent and there for multi-threading is needed.
Here is the issue:
If i take out the threading everything works great. The second that i add threading, and try to execute a few instances of the SSIS it starts crashing. The thing is that it errors out in different task and is completely unpredictable.
My question:
1) Can SSIS be used in a multi-threading environment?
2) If yes, How do i do it?
3) Does anyone have any suggestion as to what i should try?
p.s. Timing is definitely an issue for me. I got to get this working, tested, approved by 8/27/2006
---------
here are the 3 errors that i'm getting when i turn multi-threading on
1)Thread "WorkThread0" has exited with error code 0xC0202009.
2)An OLE DB record is available. Source: "Microsoft SQL Native Client" Hresult: 0x80004005 Description: "Transaction (Process ID 112) was deadlocked on lock resources with another process and has been chosen as the deadlock victim. Rerun the transaction.".
3)The ProcessInput method on component "(i cannot display the name of the task for security reasons" (448) failed with error code 0xC0202009. The identified component returned an error from the ProcessInput method. The error is specific to the component, but the error is fatal and will cause the Data Flow task to stop running.
View 5 Replies
View Related
Jun 22, 2006
Apologies if this is not the right forum.
We have some c~ code which calls a stored proc to trnsfer data from one db to another.
We are using multi threading to allow a whole days data to be processed in parrallel.
However, we are experiencing problems. After a while we get the following message:
"An exception during changed data captured caused all processing threads to abort".
Can anyone tell me what posssible reasons would cause this to happen?
It is not an area in which I have a lot of expertise, the code was written by a third party.
Many thanks
View 1 Replies
View Related
Nov 8, 2007
We're moving in a 4-socket (Xeon Dual Core CPU's) server. What version and product of SQL Server should I be looking for to fully take advantage of the new server's 4 processors. I'm not sure I understand the licensing per CPU. Is there any advantage in waiting for SQL Server 2008 as it concerns multi-threading and what should I be looking to spend for a SQL server solution that utilizes all four processors. Someone has told me that the SQL Server standard edition comes with 4 cpu licenses. Is this correct?
Thanks, Rob Huylo
Rob21379@aol.com
View 11 Replies
View Related
Jan 28, 2004
hi,
I want to execute two user created stored procedures in a multithreaded manner in ms-sql server . Can some tell me how i can do that.
Thanks in advance.
Regards,
Manpreet
View 14 Replies
View Related
Apr 11, 2008
I have an SSIS package that is running the same set of stored procedures on about 50 remote database servers and importing the data through data flow tasks into a local database.
To do this I have a db table set up with the remote server and db names and using variables, i am looping through each one and getting the data. Everything works wonderfully...however, I need to find a way to be able to run multiple calls at the same time. Is it possible to multithread SSIS so that I can run 1 server retrieval on 1 thread and another retrieval on another thread? Any other suggestions on how to optimally do this?
View 4 Replies
View Related
Nov 23, 2005
Hi,I have SQL SERVER 2000 SP4 Enterprise , Windows 2003Enterprise onXeon 4 Processors (now with multi-threading CPU) and I havequestions whichseem weirds to me (used to have the same config withoutMulti-Threading) as following:1. SQL Server see 8 CPUs. Is this because ofmulti-threading CPUs ?2. when use SP_WHO2 ACTIVE ,2.1 Result has user SPID , which blockitself(it is UPDATE and SELECTstatements, most often)2.2 Result has user SPID , show morethan one result rows withthe same processe and command3. when use Properties of SQL Server , it has anerror 15407 in log.many thanks in advanceNiponW
View 2 Replies
View Related
Jul 10, 2015
If I have several stored procedures, all of them making inserts/updates to the same tables, what is the safest way to run them in sequence?
The context is an asp.net application; I have an aspx page where a click on a button (or more) will launch stored procedures in execution. I have encountered the unfortunate situation before, when stored proc #2 started long before stored proc #1 finished and this caused problems.Â
If I group all stroed procs in one that simply calls all of them in the needed sequence:
exec stproc1
exec stproc2
.....
exec stprocn
Will they execute in a single thread? Is there any guarantee that stproc2 will be executed when stproc1 finishes, and so on?
View 2 Replies
View Related
Aug 29, 2007
I have a question:
Is SQL Compact Edition safe to use in ASP.NET WebApps?
And second if so, should i use transation isolation level or row locking hints to assure data quality during iniserts?
E.g. i want to assure that there is only one record named user2@domain2 and thus i want to make sure that my code
will not fail in a mutli-theaded scenario and insert second one or get a key exception back: see sample
transaction or locking hit{
select from users ....
if no user found
insert into users
}
Thanks
Tim
View 5 Replies
View Related
Dec 22, 2006
I've seen a few posts in the MSDN documentation (see links below) stating that the MergeSynchronizationAgent no longer requires the STA threading model in SQL Server 2005 SP1. However, I'm still receiving the following exception message in my synchronization code (where it attempts to access the SynchronizationAgent property):
The MergeSynchronizationAgent class must be instantiated on a Single-Threaded Apartment (STA) thread.
I have Service Pack 1 for SQL Server 2005 installed on both my server as well as my local client (the client is running SQL Server 2005 Express). How can I verify the correct files are there (e.g. are the some specific date/time values for the RMO/COM objects?
Here's the list of MSDN links mentioned above:
http://forums.microsoft.com/MSDN/ShowPost.aspx?PostID=326837&SiteID=1
http://msdn2.microsoft.com/en-us/library/ms146869.aspx
http://msdn2.microsoft.com/en-us/library/microsoft.sqlserver.replication.mergesynchronizationagent.aspx
http://msdn2.microsoft.com/en-us/library/microsoft.sqlserver.replication.mergepullsubscription.synchronizationagent.aspx
View 3 Replies
View Related
Sep 4, 2006
Hi,
I am not comfortable with DTS 2000 but I need to execute a encapsulated DTS 2000 package from a SSIS package. The real problem is when I need to pass SSIS variables to DTS 2000 package. The DTS 2000 package have 3 global variables that I can identify on " Execute DTS 2000 Package Task Editor - Inner Variables ". I believe the SSIS variables must be mapped on " Execute DTS 2000 Package Task Editor - OuterVariables ". How can I associate the SSIS variables(OuterVariables ) to "Inner Variables"? How can I do it? Much Thanks.
João
View 8 Replies
View Related
Jan 24, 2006
Hi,
I would like to design a SSIS package, which have couple of variables. It loads a xls file specified in a variable [varExcelFileFullPath] .
I will run it by commands: exec xp_cmdshell 'dtexec /SQL ....' (pls see an example below).
It seems it does not get the values passed in for those variables. I deployed the package to a sql server.
are there any grammar errors here? I copied it from dtexecui. It worked inside Dtexecui not in dos command.
exec xp_cmdshell 'dtexec /SQL "LoadExcelDB" /SERVER test /USER *** /PASSWORD ****
/MAXCONCURRENT " -1 " /CHECKPOINTING OFF /REPORTING EW
/LOGGER "{6AA833A1-E4B2-4431-831B-DE695049DC61}";"Test.SuperBowl"
/Set Package.Variables[User::varExcelFileName].Properties[Value];"TestAdHocLayer"
/Set Package.Variables[User::varExcelWorkbookName].Value;"Sheet1$"
/Set Package.Variables[User::varExcelFileFullPath].Value;"D: estshareTestAdHocLayer.xls"
/Set Package.Variables[User::varDestinationTableName].Value;"FeaturesTmp"
/Set Package.Variables[User::varPreSQLAction].Value;"delete from FeaturesTmp"
'
thanks,
Guangming
View 2 Replies
View Related
Jan 28, 2008
i want to use datebase wich suport all latin languages but i cant find the collection
esspeacily
eng,
german,
france and Turksh
View 4 Replies
View Related
Feb 27, 2004
I recently migrated a database from server A to server B. The backup jobs I am trying to build on server B are failing because of the following error:
--------------
[Microsoft SQL-DMO] Error 21776: [SQL-DMO]The name 'WinDat' was not found in the Databases collection. If the name is a qualified name, use [] to separate various parts of the name, and try again.
-------------
How do I add this database to the databases collection so it will be recognized?
Thanks in advance for your time and help.
View 9 Replies
View Related
Aug 16, 2007
I can add two reportitem controls, ie reportitems!begbal.value + reportitems!deposits.value, without a problem. However, when I add the 3rd reportitem control to the expression, ie + reportitems!withdrawals.value, some really funky arithmetic occurs. All of these controls I am referring to are in the same group footer.
Any help out there would be greatly appreciated.
View 1 Replies
View Related
Feb 9, 2007
actuallu the earlier obviously worked but now i am fetching values from 2 colums and in collection we can pass 2 strings i did that.but then how to increment the collection c.will just post the code .
query fetching 2 records
can add only 1
NameValueCollection c = new NameValueCollection();
cmd2.CommandText="select Pname ,HoursWorked from TimeSheet1 ts , ProjResource pr,ProjectDetails pd where ts.rid = pr.rid and ts.pcode=pr.pcode and pd.Pcode = ts.Pcode and pr.Rid = '" + Ridtxt.Text + " '";
cmd2 .Connection = con1;
con1.Open();
SqlDataReader dr = cmd2.ExecuteReader();
while (dr.Read())
{
c.Add(dr["pname"].ToString(), dr["hoursworked"].ToString());
}
as earliar
supose c# 12
asp 11
only c# getting added
even though loop is while dr.read()
{
}
actually how to increment key as in c both strings are occupied
View 3 Replies
View Related
Jul 3, 2004
hi evryone
I have a question!
in my ASP.NET page I use a SqlDataAdapter object for working on my database
first I declare it as a global vaiable(SqlDataAdapter objDataAdapter;)
then I construct it in my Page_Load procedure like this
(objDataAdapter=new SqlDataAdapter(strSql,objConnention))
then when I want to use it in my other procedures an error tells me that your object does not exist
for solving this error I should construct my objDataAdapter in evry procedure using it.
but I think such variables should not be erased from Heap automatically(with Garbage Collection)
because it has a reference to Stack. I mean the global variable...
any opinion ?
Thanks
View 3 Replies
View Related
Feb 12, 2005
I am collecting companyID's form a data grid, I want to pass the selected values to a sproc via a variable. Any idea on the syntax?
this works using a query string within my code
WHERE (dbo.Promotions.ExpirationDate > GETDATE()) AND (dbo.Promotions.CompanyID IN (" + selectedCompanies + "))
this doesn't within my sproc
WHERE (dbo.Promotions_ByLink.ExpirationDate > GETDATE()) AND (dbo.Promotions_ByLink.CompanyID IN (@SelectedCompanies))
I also tried
WHERE (dbo.Promotions_ByLink.ExpirationDate > GETDATE()) AND (dbo.Promotions_ByLink.CompanyID IN (SELECT @SelectedCompanies))
and
WHERE (dbo.Promotions_ByLink.ExpirationDate > GETDATE()) AND (dbo.Promotions_ByLink.CompanyID IN (' + @SelectedCompanies + '))
Thanks
View 4 Replies
View Related
Oct 26, 2000
hi , what is the script to change the unicode collection in sql server 7.0
Ahemd
View 1 Replies
View Related
Feb 6, 2006
I'm having trouble obtaining errors raised in a stored
procedure via the ADO Errors collection after the second
FETCH NEXT statement from within that stored procedure.
Consider the following table created in a SQL Server
database:
CREATE TABLE TestTable
(
TestInt int
)
go
INSERT TestTable(TestInt) values(1)
INSERT TestTable(TestInt) values(2)
INSERT TestTable(TestInt) values(3)
This is a very simple table with one column, and three
rows containing the values 1, 2 and 3.
Consider this stored procedure:
CREATE PROCEDURE TestStoredProc
as
BEGIN
set rowcount 0
Set NoCount ON
declare @TestInt int
declare @ErrMsg char(7)
declare TestCursor cursor forward_only for
select * from TestTable
open TestCursor
Fetch next from TestCursor into @TestInt
While @@fetch_status<>-1
Begin
select @ErrMsg = 'Error ' + convert(char, @testint)
raiserror(@ErrMsg, 16, 1)
raiserror(@ErrMsg, 16, 1)
Fetch next from TestCursor into @TestInt
end
Close TestCursor
DeAllocate TestCursor
return
END
This stored procedure simply defines a cursor on all rows
in TestTable. For each row fetched from the cursor, the
error message 'Error n' is raised twice, where n is the
integer that had just been fetched from the cursor.
Finally, consider this VB code using ADO to execute the
above stored procedure. After the stored procedure is
executed, the code loops through the errors collection,
and creates a message box for each error in the collection:
Private Sub Form_Load()
Dim cn As Connection
Dim cm As Command
Dim oErr As Error
On Error Resume Next
Set cn = CreateObject("ADODB.Connection")
cn.Open "Data Source=<Some SQL Server>; Initial
Catalog=<Some Database Name>; Provider=SQLOLEDB; Persist
Security Info=False; Integrated Security=SSPI"
Set cm = CreateObject("ADODB.Command")
Set cm.ActiveConnection = cn
cm.CommandType = adCmdStoredProc
cm.CommandText = "TestStoredProc"
cm.Execute
For Each oErr In cn.Errors
MsgBox oErr.Description
Next
End
End Sub
When this code is executed, only two message boxes appear
with the message "Error 1".
Any help on this matter would be greatly appreciated :)
View 3 Replies
View Related
Nov 13, 2007
If I have N different xml document formats and I want to store those xml data files as typed xml in one table (one column), can I do it using xml schema collection (by adding schema file to schema collection for each document type and assigning xml schema collection to this column)?
Is this possible using xml schema collection? Or did I miss something about xml schema collection usage scenario?
Thank you!
View 4 Replies
View Related
Mar 7, 2008
I need to feed head office sql server with the data from regional servers. Servers are spread through all continents
Data input done locally on Head office server as well and plus need to ship data from other servers.
So clarify this - Head office server is not standby one. Mirroring is out of the picture, I think..
Initially, I thought ship a log every 15 min and restore on Head office server but is this going to create an issue for the local data processing?
Any bright ideas welcome!
View 2 Replies
View Related