Package Hanging On OLE DB Command With Large Result Sets
Jul 13, 2007
I have an SSIS package (SQL 2005 SP2 and Visual Studio SP1) that does the following:
OLE DB Source --> Conditional Split --> OLE DB Command #1 --> OLE DB Command #2
The source reads from database A. Each row is variable-width and up to several KB wide, including two ntext columns.
Command #1 executes a stored proc in db A, using a bunch of inputs and two output parameters.
Cmd #2 executes an update in db B, using the two output params from cmd #1 as inputs.
When the rowset size is small, around 500, everything works fine.
However, when the rowset size is larger, around 5000, SSIS hangs when trying to execute cmd #2. The profiler shows that none of the cmd #2 updates are ever executed. No error messages are produced, and the connection never times out -- it just hangs forever.
If I replace the cmd #2 updates with a simple select, everything works fine. If I replace it with a stored proc that does an update, it hangs.
The work-around I came up with was to create a new table in db B, and do inserts into the table, but unless I'm missing something, this still seems like a bug...
View 1 Replies
ADVERTISEMENT
Jul 2, 2015
so async cursor population is supposed to create the cursor and return the cursor id quickly, while the server works on async populating the results. For a keyset-driven cursor, SQL Server stores the key sets in tempdb, which it then uses to fetch data for cursor results. Anyway, this works fine for smaller tables, but I'm finding for large result sets, the async cursor population is very slow and indeed seems to approximate synchronous time. The wait stat I get while it is running (supposedly asynchronously) is TRANSACTION_MUTEX.
Example:
--enable async cursor
exec dbo.sp_configure 'cursor threshold', 0; reconfigure;
declare @cursor int, @stmt nvarchar(max), @scrollopt int, @ccopt int, @rowcount int;
--example of giant result set
set @stmt = 'select * from sys.all_objects o1, sys.all_objects o1';
[code]...
Note that using the SQL "select * from sys.all_objects o1" is much faster than "select * from sys.all_objects o1, sys.all_objects o2". However, if cursor population is async, I'd expect the time to return a cursor id to be similar between the two.
View 7 Replies
View Related
Mar 2, 2004
For MS SQL 7, an error occurs as shown at bottom.
Basically a piece of embedded sql is being sent to the db which has a syntax like:
select distinct WC_REFERENCE_MANAGER.logicalname,
WC_REFERENCE_MANAGER.referenceid,
wc_site.doclistid,
WC_REFERENCE_MANAGER.shared,
WC_REFERENCE_MANAGER.languageid
from wc_content inner join wc_content_ref on wc_content.contentid = wc_content_ref.documentid
inner join wc_reference_manager on wc_content_ref.referenceid = wc_reference_manager.referenceid
inner join wc_site on wc_reference_manager.referenceid = wc_site.referenceid
where xmlid = @p1 and contains(xmldata, @p2)
and wc_reference_manager.languageid = @p3
and wc_site.siteid = @p4
and wc_site.sectionid IN (
Followed by over 15000 numbers
(dont ask - its a generic product with a generic db design behind it!).
Any comments apart from rewrite the query (which isnt an option).
The error is :
02/27/04 14:30:05 Stack Overflow Dump not possible - Exception c00000fd E at 00499f17
2004-02-27 14:30:05.93 spid8 Address=499f17 Exception Code = c00000fd
2004-02-27 14:30:05.93 spid8 eax=394d29f4 ebx=00000000 ecx=394d29f4 edx=0000000d
2004-02-27 14:30:05.93 spid8 esi=394d29f4 edi=433a499c eip=00499f17 esp=3c263000
2004-02-27 14:30:05.93 spid8 ebp=3c26300c efl=00010a97
2004-02-27 14:30:05.93 spid8 cs=1b ss=23 ds=23 es=23 fs=3b gs=0
2004-02-27 14:30:05.93 spid8 Input Buffer 250651 bytes -
2004-02-27 14:30:05.93 spid8
2004-02-27 14:30:05.93 spid8 s p _ p r e p a r e & @ T @ p 1 i n t , @ p 2 n v a r c h a
2004-02-27 14:30:05.93 spid8 r ( 4 0 0 0 ) , @ p 3 i n t , @ p 4 i n t c s e l e c t
2004-02-27 14:30:05.93 spid8 d
View 9 Replies
View Related
Aug 29, 2015
Using merge replication + web synchronization, I have a situation when there are large amount of data changes to upload to the publisher, Merge agent would create a large request and send it over. The publisher gets it and is able to work on it. After few minutes it has finished but (I assume) the connection has been dropped. At the subscriber's side, it appears that the merge agent is hung. The output would look like something like this:
Upload request size is XXX bytes.
Uploaded a total of 100 chunks.
Uploaded a total of 200 chunks.
Uploaded a total of 211 chunks.
The request message was sent to [URL] ....
Normally, when the publisher finishes working, the merge agent then continues processing. But when it takes more than few minutes (it seems to break about at 2 or 3 minutes), merge agent will hang as long as the InternetTimeout setting is (currently 20 minutes) before finally failing and retrying.
But that's not right. The publisher was done and can't communicate back to the merge agent (presumably because the connection was dropped). As a result, merge agent will try to re-enumerate changes on top of giving appearance that it's hung.
I've already fiddled with settings such as MaxUploadChanges, UploadGenerationsPerBatch, UploadReadChangesPerBatch, and UploadWriteChangesPerBatch. However, none of those setting actually ensure that the request message is too large. It has worked in breaking up the changes into separate batches (e.g. processing a single table rather than all tables) which results in more frequent updates and thus avoid the problem.
However, when a single table has several changes, it is still lumped into one large request which then takes more than 2-3 minutes to process on publisher's side and thus I still end up with the same symptom of merge agent hanging.
Is there anything else I could try to get merge agent to keep its connection alive even during processing a large request?
View 0 Replies
View Related
Mar 20, 2008
Hi,
I'm currently trying to retrieve results from a large dataset, there are over 45000 records and I need to use them all to peform counts etc. I have set up views, but my page is still being returned slowly, is there anything I can do to speed this up?
Thanks
Gemma
View 2 Replies
View Related
May 22, 2000
We are trying to limit are query that returns items from our database. The
query currently returns 32,000 records. We are trying to figure out an effecient way so we can request the 1st 50, or the 3rd 50, or the 5th 50 to display to the screen. We dont want to return the entire 32,000 then limit whats displayed to the screen in ADO. We want the select statment to only return 50 at a time. Any suggestions?
View 1 Replies
View Related
Mar 9, 2007
I need to copy data from TableA to TableB (>5 millions rows). The two are in the same database.
What is the best way of doing this?
I was thinking about using a simple INSERT INTO ... SELECT statement. Is there a faster way to do it with SSIS?
Thanks
View 13 Replies
View Related
Jul 20, 2005
Hi!Another silly question:If a stored procedure returns multiple result sets, how do I choose the oneI want to insert into a table?For example sp_spaceused returns two result sets if object name is ommited.I only need the first result set. How do I do that?Tnx!Darko
View 2 Replies
View Related
Feb 29, 2008
Hi, I'm trying to return 2 different result sets using the below query mapped to 2 different variables in my execute sql task. I've tried this with one task and 2 seperate tasks but can't get it to work. Is this possible using only one task? It keeps giving me a result set error.
SELECT COUNT(*) AS DeceasedCount
FROM AMGR_User_Fields_Tbl
WHERE (Client_Id = '' or Client_Id is NULL) and Type_Id = 53
SELECT COUNT(*) AS LostCount
FROM AMGR_User_Fields_Tbl
WHERE (Client_Id = '' or Client_Id is NULL) and Type_Id = 469
View 5 Replies
View Related
Jul 31, 2007
Hi,
I am aware that SSRS does not support stored procedures with multiple data sets, but what if the same stored procedure may only return one set based on a parameter?
For example, lets say if parameter @TableID = 0, the stored procedure returns 2 result sets. But if @TableID = 1, or @TableID = 2, the SP only brings back 1 result set. Is this also not supported?
When I try to do this it still only brings back the fields from the first table. Am I doing something wrong?
Thanks for any insight.
View 2 Replies
View Related
Aug 28, 2006
What is the recommended mechanism for selecting paged results from SQL.
Presently I pass various params including the request Max Items Per Page and the requested page.
The I execute the query as a count with the search params.
Then comes the paging logic, which validates the page number against the request page and number of hits etc.
Then a temp table and record variables are created for the results.
Then I run the query again with a cursor and select the appropriate Items into the temp table based on the paging values (First Item and Last Item).
Then I return the temp table & some additional return params with the Total Hits etc.
The Stored procedure is accessed via an ADO.Net client and the system.data.IDBReader populates a .Net strongly typed collection and is for read only display.
Thanks for any input,
Martin.
View 11 Replies
View Related
Jun 12, 2008
My stored procedure displays two result sets. How can i use that result sets in my 3-tier application. I want to bind first resultset to repeater control and second to label control. I am using SqlDataReader...
View 6 Replies
View Related
Mar 29, 2007
Hi,
do u know
has sql server 2000 support MARS?
View 6 Replies
View Related
Mar 1, 2007
I am new to SQL Server Integration Services. I need to help here.
If there is only one result set, I could get rank info by putting DENSE_RANK expression inside of my select. My case is that I have two data source. After union of them, I need rank the result set and save result set plus rank info into the destination. I tried to use Derived Column component by using expression DENSE_RANK() over ( order by columnName ). But it does not like it.
Does anyone know how to achieve this?
Thanks!
View 3 Replies
View Related
Feb 21, 2008
Scenario: Launch SQL Server 2005 Management Studio & Connect to Server, ie: ServerNameInstanceName. Click "New Query" Button
Running the following SQL Statement returns 842 rows but should return 1343:
SELECT A.id, C.PName, B.School
FROM users AS A
LEFT OUTER JOIN Bio AS B ON A.id = B.id
LEFT OUTER JOIN Groups AS C ON A.PName = C.PName ORDER BY A.Id
I get 1343 rows when I change Line 2 of SQL Statement to: "FROM dbName.dbo.users AS A".
Clicking "New Query" w/out first opening the database must run the query in the context of the Master Database??
I noticed in Sql Server 2000 if I connect Query Analyzer to the Server and run the query I get 842 & I can see that the default selected database is master. When I make the change mentioned above I then get 1343.
Why doesn't this produce an error though? I'd rather get an error than the wrong result set.
View 4 Replies
View Related
Sep 7, 2006
Hi all,
I am using SSAS 2005. The mining model works fine. But it crashes when I run the 'Mining Model Predictions' against large data sets.
I ran it against 5,000,000 records and it went fine.
But exactly same model failed for 5,100,000 records and beyound.
The message is 'Query Execution Failed' and then Visual Studio crashes.
Pl. let me know if anybody has the same experience or knows the solution.
Thanks,
Vikas
View 3 Replies
View Related
May 20, 2008
In MySQL we use "SELECT (....) LIMIT 0, 10" to only return the first 0 to 10 records. Alternatively we could do "LIMIT 10, 20" to return the 10th to 20th records.
What's the equivilent of this in SQL Server?
Thanks
View 4 Replies
View Related
Jun 11, 2014
I am looking for a tool to compare the result sets.
Is there any free tool or Microsoft built in tool that we can use to compare the result sets. Suppose if I change one thing in code and needs to test the am I getting the same result set or not? Instead of doing manually is there any way to compare both result set.
View 5 Replies
View Related
Feb 10, 2015
I 've got the below RESULT SET after several JOINs & deriving columns from different sources
STUD_ID, ANNUAL_YEAR, COURSES_ID, SUBJECTS_OPTED, CHAPTERS, YEAR_JOINED, Course_Tenure, Sub_ID, SUB_MARKS, EXAM_ID, ATTEMPT_YEAR, ATTEMPT_NO and many more.
Now, I've only one result set available in my DATA FLOW. And I want the below few,
1) STUD_RESULTS:TABLE : Evaluate COURSE-Annual year completion:
Have columns Students, Annual_yr, Results (IF ANY SUB _MARKS are below 35? 'PASS': 'Fail') and few more.
<--I've got to group the result set on STUD_ID and ANNUAL_YEAR.
2) STUD_COURSE_PERF: TABLE: for course performance.
I've got to group the result set on STUD_ID,ANNUAL_YEAR, Course_ID and the other essential
3) STUD_SUB_PREF: TABLE: Evaluating Subjects difficulty
I've got to group the result on STUD_ID,ANNUAL_YEAR, Course_ID, SUBJECTS_OPTED and other essentail.
How to use the only result set to carry on different evaluations?I Applied Multicast on the only result set.
View 1 Replies
View Related
Aug 9, 2007
Hello,
Is it possible to save the results of several SP calls in a script, to one file?
Here's what I mean:
I want to run these 4 sp calls--
exec EPC_SP1 'aph','live'
exec EAUI_SP2 'noble','newswire'
exec EAUI_SP3 'noble',1
exec EAUI_SP4 5507,'live'
And save the results of each one of those calls to the same file, in other words, an APPEND
Is this or something like it possible in TSQL?
Help appreciated!
Thank you,
--PhB
View 3 Replies
View Related
Sep 28, 2006
I have a stored procedure like the following. This returns 2 result sets, however i only want it to return 2nd (SELECT SomeField FROM SomeTable). How is this done? Note - it is not possible to change 'SomeSPThatReturnsAnIntAndAResultSet '
CREATE PROCEDURE [dbo].[SomeSP]
@SomeParam int
AS
BEGIN
SET NOCOUNT ON;
declare @SomeScalar int
exec @SomeScalar = SomeSPThatReturnsAnIntAndAResultSet @SomeParam
if @SomeScalar = 0
BEGIN
SELECT SomeField FROM SomeTable
END
END
View 6 Replies
View Related
Aug 4, 2015
Periodically throughout the day a report is manually pulled from a SQL Server database. Â Is their a way w/o me adding a field to the database to have the result set return the "new" results? Â For example, lets say this is our DDL
Create Table OneTwoThree
(
id int
,date11 datetime
,firefly varchar(10)
)
Insert Into OneTwoThree Values (1, '08/03/2015 18:43:32.012', 'Hi'), (2, '08/03/2015 18:44:11.011', 'No'),
(3, '08/03/2015 19:36:33.011', 'Second'), (4, '08/03/2015 19:37:33.011', 'Alpha')
Prior I could use this syntax, but that was only with needing to generate 1 result set. Â
Select id, convert(varchar(10), date11, 101) As [Date], firefly from onetwothree
where CONVERT(varchar(10), date11, 101) < CONVERT(varchar(10), GetDate(), 101)
Looking at my datetime values, let's say the 1st was generated at 18:45, obviously the 1st two records in the table would be returned. And let's say a 2nd time I need to generate I want to exclude the 1st two entries as they have already been verified. How can I do such w/o adding a field to the table?
View 11 Replies
View Related
Nov 8, 2007
hi all.....
i select some records from my database using mysqldatareader.......but i want to load it's result sets into my datatable or dataset....
is it possible and how can i do it this way ?
thanks
View 1 Replies
View Related
Jan 8, 2001
I need to export out the data in a result set from temp tables. It is in a rather large Dts Stream. Thanks for your help. Brett
View 1 Replies
View Related
Jul 6, 2004
Greetings
I have a SQL Server stored procedure that performs several queries and therefore returns several "result sets" at any one time.
Wiring it up via ADO.NET I populate a DataSet with a number of items in the Tables collection - which is great - and I can give each item a name for identification purposes once the DataSet is populated.
But I'd like to know if there is some way I can set the names of each result set *within the text of the stored procedure*, i.e. before the DataSet gets populated.
Any help greatly appreciated.
Stuart
View 14 Replies
View Related
Jan 28, 2007
hi
i read that stored procedures can return multiple result sets?how is that?
thanks in advance.
View 4 Replies
View Related
Aug 28, 2006
I have a novice question. How does one suppress result sets returned from a stored procedure?
I have created a procedure which makes use of multiple stored procedures . The purpose of this procedure (lets call it procA), is to count the rows returned from other procedures. The €œOther€? procedures will return rows having an unknown number of columns. I would like to limit any changes which may be needed to be made to the €œOther€? procs.
Once procA has collected all of the information (@@rowcount) from the inner procedures, then it will return a result set having several columns €“ mainly the subProcedure name and number of rows returned.
The purpose of procA is to query several subsystems and identify which ones need attention.
Cursor While Loop
exec @ProcName @ObjectName,@userID,@syncDate
set @recs = @@rowcount;
My c# program calls the sp as follows:
cmd = DataUtility.GetSQLCommand();
cmd.CommandType = CommandType.StoredProcedure;
cmd.CommandText = "FetchAdminData";
cmd.Parameters.AddWithValue("@userID", userAlias);
cmd.Parameters.AddWithValue("@adminDate",userDate);
reader = cmd.ExecuteReader();
I do not wish to iterate over each resultSet when I am only interested in the last one. Any suggestions?
View 2 Replies
View Related
Dec 29, 2006
Seasons greetings to everyone,A simple question. Could someone show me the syntax to produce multiple (2 or 3) result sets in a stored proc and how you access those sets from a c# program (ASP.NET)..Couldn't find a reference on Google, maybe I was asking the wrong question! Thanks for any help regardsDavej
View 3 Replies
View Related
Mar 6, 2007
Hi there everyone. I have a stored procedure called “PagingTable� that I use for performing searches and specifying how many results to show per ‘page’ and which page I want to see. This allows me to do my paging on the server-side (the database tier) and only the results that actually get shown on the webpage fly across from my database server to my web server. The code might look something like this:
strSQL = "EXECUTE PagingTable " & _
"@ItemsPerPage = 10, " & _
"@CurrentPage = " & CStr(intCurrentPage) & ", " & _
"@TableName = 'Products', " & _
"@UniqueColumn = 'ItemNumber', " & _
"@Columns = 'ItemNumber, Description, ListPrice, QtyOnHand', " & _
"@WhereClause = '" & strSQLWhere & "'"
The problem is the stored procedure actually returns two result sets. The first result set contains information regarding the total number of results founds, the number of pages and the current page. The second result set contains the data to be shown (the columns specified). In ‘classic’ ASP I did this like this.
'Open the recordset
rsItems.Open strSQL, conn, 0, 1
'Get the values required for drawing the paging table
intCurrentPage = rsItems.Fields("CurrentPage").Value
intTotalPages = rsItems.Fields("TotalPages").Value
intTotalRows = rsItems.Fields("TotalRows").Value
'Advance to the next recordset
Set rsItems = rsItems.NextRecordset
I am trying to do this now in ASP.NET 2.0 using the datasource control and the repeater control. Any idea how I can accomplish two things:
A) Bind the repeater control to the second resultset
B) Build a “pager� of some sort using the values from the first resultset
View 3 Replies
View Related
Aug 22, 2007
Hi all,
I want to know how to handle multiple result sets return from Stored Procedure? I know one way is to insert the result sets into the table, but the limitation is the result sets must have the same data structure. If the result sets have different data structure, how can I handle it.
Thanks,
View 5 Replies
View Related
Jun 10, 2015
I have a general question I need a few pointers or explanations. I have a top level Data Flow query in a SSIS package that has been running for years for which I have made some minor changes to pull additional fields and rectify a filter condition.Â
I have noticed that when I run this same query, exactly as entered in the Data Flow task, in the SQL Server 2012 Management Studio, the query returns some 105,000+ records but when run in development by executing the package the top level task only indicates some 21,000 records returned. Since there was not other transformations or filtering performed, I had expected the same record count and am not getting it.Â
View 5 Replies
View Related
Jun 22, 2007
Hi folks,
I have a simple SSIS package I built to migrate data from a SQL db on one server to another SQL db on another server. All it does is: Execute SQL task to disable some triggers on the target table, Data Flow Task to pull the data from the source (a simple sql query) into the destination table (using OLEDB), then another Execute SQL task to re-enable the triggers.
This package ran fine several times during my testing. However, this morning it's not working anymore. When I run it and check its progress, the "Validation has started" is the only thing runs, then nothing else happens... no error messages, no timeouts, nothing in the output window besides "DTS.Pipeline: Validation phase is beginning".
The only thing that has changed is one change to a date in the where clause of my source query, so I'm baffled on why it no longer runs.
I'm pretty new to SSIS, so I could easily be missing something obvious here.
Anyone have any ideas on what's up, or who can point me in the right direction? I'm currently reading up on how to debug and add more comprehensive error handling, in hopes of finding a better description of the problem.
Thanks in advance,
Geoff
View 4 Replies
View Related
Mar 14, 2006
My SSIS package will just hang (do nothing) after validation of the package tasks. I realised that it does 2 validation. It then hits "starting exectuing" and then nothing. I mean nothing. It just stays the same. When I look into the logg file, the same message as in the output window. My package has parallel extract of data from the same datasource, but different tables. I dont know if that the problem but i really doubt it because i have done parallel table downloads countless times in version 2000. When i go into the data task window, the source task does no even indicate that its downloading (color yellow). Is there any reason why this will happen? Ooo, but the tasks executes just fine when i execute them individually (right click > execute).
View 17 Replies
View Related