Package Hanging On OLE DB Command With Large Result Sets

Jul 13, 2007

I have an SSIS package (SQL 2005 SP2 and Visual Studio SP1) that does the following:



OLE DB Source --> Conditional Split --> OLE DB Command #1 --> OLE DB Command #2



The source reads from database A. Each row is variable-width and up to several KB wide, including two ntext columns.



Command #1 executes a stored proc in db A, using a bunch of inputs and two output parameters.



Cmd #2 executes an update in db B, using the two output params from cmd #1 as inputs.



When the rowset size is small, around 500, everything works fine.



However, when the rowset size is larger, around 5000, SSIS hangs when trying to execute cmd #2. The profiler shows that none of the cmd #2 updates are ever executed. No error messages are produced, and the connection never times out -- it just hangs forever.



If I replace the cmd #2 updates with a simple select, everything works fine. If I replace it with a stored proc that does an update, it hangs.



The work-around I came up with was to create a new table in db B, and do inserts into the table, but unless I'm missing something, this still seems like a bug...

View 1 Replies


ADVERTISEMENT

SQL Server 2012 :: Asynchronous Cursor Population Slow For Large Result Sets

Jul 2, 2015

so async cursor population is supposed to create the cursor and return the cursor id quickly, while the server works on async populating the results. For a keyset-driven cursor, SQL Server stores the key sets in tempdb, which it then uses to fetch data for cursor results. Anyway, this works fine for smaller tables, but I'm finding for large result sets, the async cursor population is very slow and indeed seems to approximate synchronous time. The wait stat I get while it is running (supposedly asynchronously) is TRANSACTION_MUTEX.

Example:
--enable async cursor
exec dbo.sp_configure 'cursor threshold', 0; reconfigure;
declare @cursor int, @stmt nvarchar(max), @scrollopt int, @ccopt int, @rowcount int;
--example of giant result set
set @stmt = 'select * from sys.all_objects o1, sys.all_objects o1';

[code]...

Note that using the SQL "select * from sys.all_objects o1" is much faster than "select * from sys.all_objects o1, sys.all_objects o2". However, if cursor population is async, I'd expect the time to return a cursor id to be similar between the two.

View 7 Replies View Related

SQL 7 Hanging With Large In() Statement

Mar 2, 2004

For MS SQL 7, an error occurs as shown at bottom.
Basically a piece of embedded sql is being sent to the db which has a syntax like:
select distinct WC_REFERENCE_MANAGER.logicalname,
WC_REFERENCE_MANAGER.referenceid,
wc_site.doclistid,
WC_REFERENCE_MANAGER.shared,
WC_REFERENCE_MANAGER.languageid
from wc_content inner join wc_content_ref on wc_content.contentid = wc_content_ref.documentid
inner join wc_reference_manager on wc_content_ref.referenceid = wc_reference_manager.referenceid
inner join wc_site on wc_reference_manager.referenceid = wc_site.referenceid
where xmlid = @p1 and contains(xmldata, @p2)
and wc_reference_manager.languageid = @p3
and wc_site.siteid = @p4
and wc_site.sectionid IN (

Followed by over 15000 numbers

(dont ask - its a generic product with a generic db design behind it!).

Any comments apart from rewrite the query (which isnt an option).

The error is :



02/27/04 14:30:05 Stack Overflow Dump not possible - Exception c00000fd E at 00499f17
2004-02-27 14:30:05.93 spid8 Address=499f17 Exception Code = c00000fd
2004-02-27 14:30:05.93 spid8 eax=394d29f4 ebx=00000000 ecx=394d29f4 edx=0000000d
2004-02-27 14:30:05.93 spid8 esi=394d29f4 edi=433a499c eip=00499f17 esp=3c263000
2004-02-27 14:30:05.93 spid8 ebp=3c26300c efl=00010a97
2004-02-27 14:30:05.93 spid8 cs=1b ss=23 ds=23 es=23 fs=3b gs=0
2004-02-27 14:30:05.93 spid8 Input Buffer 250651 bytes -
2004-02-27 14:30:05.93 spid8
2004-02-27 14:30:05.93 spid8 s p _ p r e p a r e & @ T @ p 1 i n t , @ p 2 n v a r c h a
2004-02-27 14:30:05.93 spid8 r ( 4 0 0 0 ) , @ p 3 i n t , @ p 4 i n t c s e l e c t
2004-02-27 14:30:05.93 spid8 d

View 9 Replies View Related

SQL 2012 :: Merge Agent Hanging After A Large Request

Aug 29, 2015

Using merge replication + web synchronization, I have a situation when there are large amount of data changes to upload to the publisher, Merge agent would create a large request and send it over. The publisher gets it and is able to work on it. After few minutes it has finished but (I assume) the connection has been dropped. At the subscriber's side, it appears that the merge agent is hung. The output would look like something like this:

Upload request size is XXX bytes.
Uploaded a total of 100 chunks.
Uploaded a total of 200 chunks.
Uploaded a total of 211 chunks.

The request message was sent to [URL] ....

Normally, when the publisher finishes working, the merge agent then continues processing. But when it takes more than few minutes (it seems to break about at 2 or 3 minutes), merge agent will hang as long as the InternetTimeout setting is (currently 20 minutes) before finally failing and retrying.

But that's not right. The publisher was done and can't communicate back to the merge agent (presumably because the connection was dropped). As a result, merge agent will try to re-enumerate changes on top of giving appearance that it's hung.

I've already fiddled with settings such as MaxUploadChanges, UploadGenerationsPerBatch, UploadReadChangesPerBatch, and UploadWriteChangesPerBatch. However, none of those setting actually ensure that the request message is too large. It has worked in breaking up the changes into separate batches (e.g. processing a single table rather than all tables) which results in more frequent updates and thus avoid the problem.

However, when a single table has several changes, it is still lumped into one large request which then takes more than 2-3 minutes to process on publisher's side and thus I still end up with the same symptom of merge agent hanging.

Is there anything else I could try to get merge agent to keep its connection alive even during processing a large request?

View 0 Replies View Related

Large Data Sets

Mar 20, 2008

Hi,
 I'm currently trying to retrieve results from a large dataset, there are over 45000 records and I need to use them all to peform counts etc.  I have set up views, but my page is still being returned slowly, is there anything I can do to speed this up?
 Thanks
 Gemma

View 2 Replies View Related

Limiting Large Query Results Sets

May 22, 2000

We are trying to limit are query that returns items from our database. The
query currently returns 32,000 records. We are trying to figure out an effecient way so we can request the 1st 50, or the 3rd 50, or the 5th 50 to display to the screen. We dont want to return the entire 32,000 then limit whats displayed to the screen in ADO. We want the select statment to only return 50 at a time. Any suggestions?

View 1 Replies View Related

Copying Large Sets Of Data In Same Database

Mar 9, 2007

I need to copy data from TableA to TableB (>5 millions rows). The two are in the same database.

What is the best way of doing this?

I was thinking about using a simple INSERT INTO ... SELECT statement. Is there a faster way to do it with SSIS?

Thanks

View 13 Replies View Related

Multiple Result Sets

Jul 20, 2005

Hi!Another silly question:If a stored procedure returns multiple result sets, how do I choose the oneI want to insert into a table?For example sp_spaceused returns two result sets if object name is ommited.I only need the first result set. How do I do that?Tnx!Darko

View 2 Replies View Related

Return 2 Result Sets

Feb 29, 2008

Hi, I'm trying to return 2 different result sets using the below query mapped to 2 different variables in my execute sql task. I've tried this with one task and 2 seperate tasks but can't get it to work. Is this possible using only one task? It keeps giving me a result set error.


SELECT COUNT(*) AS DeceasedCount
FROM AMGR_User_Fields_Tbl
WHERE (Client_Id = '' or Client_Id is NULL) and Type_Id = 53

SELECT COUNT(*) AS LostCount
FROM AMGR_User_Fields_Tbl
WHERE (Client_Id = '' or Client_Id is NULL) and Type_Id = 469

View 5 Replies View Related

Multiple Result Sets

Jul 31, 2007

Hi,

I am aware that SSRS does not support stored procedures with multiple data sets, but what if the same stored procedure may only return one set based on a parameter?

For example, lets say if parameter @TableID = 0, the stored procedure returns 2 result sets. But if @TableID = 1, or @TableID = 2, the SP only brings back 1 result set. Is this also not supported?

When I try to do this it still only brings back the fields from the first table. Am I doing something wrong?

Thanks for any insight.

View 2 Replies View Related

Paged Result Sets

Aug 28, 2006

What is the recommended mechanism for selecting paged results from SQL.

Presently I pass various params including the request Max Items Per Page and the requested page.

The I execute the query as a count with the search params.

Then comes the paging logic, which validates the page number against the request page and number of hits etc.

Then a temp table and record variables are created for the results.

Then I run the query again with a cursor and select the appropriate Items into the temp table based on the paging values (First Item and Last Item).

Then I return the temp table & some additional return params with the Total Hits etc.

The Stored procedure is accessed via an ADO.Net client and the system.data.IDBReader populates a .Net strongly typed collection and is for read only display.

Thanks for any input,

Martin.

View 11 Replies View Related

Two Result Sets From Stored Procedure..

Jun 12, 2008

My stored procedure displays two result sets. How can i use that result sets in my 3-tier application. I want to bind first resultset to repeater control and second to label control. I am using SqlDataReader...

View 6 Replies View Related

Multiple Active Result Sets

Mar 29, 2007

Hi,
do u know
has sql server 2000 support MARS?

View 6 Replies View Related

How To Rank An Union Of Two Result Sets?

Mar 1, 2007

I am new to SQL Server Integration Services. I need to help here.

If there is only one result set, I could get rank info by putting DENSE_RANK expression inside of my select. My case is that I have two data source. After union of them, I need rank the result set and save result set plus rank info into the destination. I tried to use Derived Column component by using expression DENSE_RANK() over ( order by columnName ). But it does not like it.

Does anyone know how to achieve this?

Thanks!

View 3 Replies View Related

How Come The Result Sets Do Not Match? Why No Error?

Feb 21, 2008

Scenario: Launch SQL Server 2005 Management Studio & Connect to Server, ie: ServerNameInstanceName. Click "New Query" Button

Running the following SQL Statement returns 842 rows but should return 1343:

SELECT A.id, C.PName, B.School
FROM users AS A
LEFT OUTER JOIN Bio AS B ON A.id = B.id
LEFT OUTER JOIN Groups AS C ON A.PName = C.PName ORDER BY A.Id

I get 1343 rows when I change Line 2 of SQL Statement to: "FROM dbName.dbo.users AS A".

Clicking "New Query" w/out first opening the database must run the query in the context of the Master Database??
I noticed in Sql Server 2000 if I connect Query Analyzer to the Server and run the query I get 842 & I can see that the default selected database is master. When I make the change mentioned above I then get 1343.

Why doesn't this produce an error though? I'd rather get an error than the wrong result set.

View 4 Replies View Related

SSAS Crashes - Mining Predictions For Large Data Sets

Sep 7, 2006

Hi all,

I am using SSAS 2005. The mining model works fine. But it crashes when I run the 'Mining Model Predictions' against large data sets.

I ran it against 5,000,000 records and it went fine.

But exactly same model failed for 5,100,000 records and beyound.

The message is 'Query Execution Failed' and then Visual Studio crashes.

Pl. let me know if anybody has the same experience or knows the solution.

Thanks,

Vikas

View 3 Replies View Related

Limiting Rows Returned By Result Sets

May 20, 2008

In MySQL we use "SELECT (....) LIMIT 0, 10" to only return the first 0 to 10 records. Alternatively we could do "LIMIT 10, 20" to return the 10th to 20th records.

What's the equivilent of this in SQL Server?

Thanks

View 4 Replies View Related

SQL 2012 :: Tool To Compare Result Sets

Jun 11, 2014

I am looking for a tool to compare the result sets.

Is there any free tool or Microsoft built in tool that we can use to compare the result sets. Suppose if I change one thing in code and needs to test the am I getting the same result set or not? Instead of doing manually is there any way to compare both result set.

View 5 Replies View Related

SQL 2012 :: Requery Result Sets In SSIS

Feb 10, 2015

I 've got the below RESULT SET after several JOINs & deriving columns from different sources

STUD_ID, ANNUAL_YEAR, COURSES_ID, SUBJECTS_OPTED, CHAPTERS, YEAR_JOINED, Course_Tenure, Sub_ID, SUB_MARKS, EXAM_ID, ATTEMPT_YEAR, ATTEMPT_NO and many more.

Now, I've only one result set available in my DATA FLOW. And I want the below few,

1) STUD_RESULTS:TABLE : Evaluate COURSE-Annual year completion:
Have columns Students, Annual_yr, Results (IF ANY SUB _MARKS are below 35? 'PASS': 'Fail') and few more.
<--I've got to group the result set on STUD_ID and ANNUAL_YEAR.

2) STUD_COURSE_PERF: TABLE: for course performance.
I've got to group the result set on STUD_ID,ANNUAL_YEAR, Course_ID and the other essential

3) STUD_SUB_PREF: TABLE: Evaluating Subjects difficulty
I've got to group the result on STUD_ID,ANNUAL_YEAR, Course_ID, SUBJECTS_OPTED and other essentail.

How to use the only result set to carry on different evaluations?I Applied Multicast on the only result set.

View 1 Replies View Related

Save Multiple Result Sets To One File??

Aug 9, 2007

Hello,

Is it possible to save the results of several SP calls in a script, to one file?

Here's what I mean:
I want to run these 4 sp calls--
exec EPC_SP1 'aph','live'
exec EAUI_SP2 'noble','newswire'
exec EAUI_SP3 'noble',1
exec EAUI_SP4 5507,'live'

And save the results of each one of those calls to the same file, in other words, an APPEND

Is this or something like it possible in TSQL?

Help appreciated!

Thank you,
--PhB

View 3 Replies View Related

Multiple Result Sets From A Stored Procedure

Sep 28, 2006

I have a stored procedure like the following. This returns 2 result sets, however i only want it to return 2nd (SELECT SomeField FROM SomeTable). How is this done? Note - it is not possible to change 'SomeSPThatReturnsAnIntAndAResultSet '



CREATE PROCEDURE [dbo].[SomeSP]

@SomeParam int

AS

BEGIN

SET NOCOUNT ON;

declare @SomeScalar int

exec @SomeScalar = SomeSPThatReturnsAnIntAndAResultSet @SomeParam

if @SomeScalar = 0

BEGIN

SELECT SomeField FROM SomeTable

END

END

View 6 Replies View Related

Transact SQL :: Get 2 Different Result Sets With GetDate Function

Aug 4, 2015

Periodically throughout the day a report is manually pulled from a SQL Server database.  Is their a way w/o me adding a field to the database to have the result set return the "new" results?  For example, lets say this is our DDL

Create Table OneTwoThree
(
id int
,date11 datetime
,firefly varchar(10)
)

Insert Into OneTwoThree Values (1, '08/03/2015 18:43:32.012', 'Hi'), (2, '08/03/2015 18:44:11.011', 'No'),
(3, '08/03/2015 19:36:33.011', 'Second'), (4, '08/03/2015 19:37:33.011', 'Alpha')

Prior I could use this syntax, but that was only with needing to generate 1 result set.  

Select id, convert(varchar(10), date11, 101) As [Date], firefly from onetwothree
where CONVERT(varchar(10), date11, 101) < CONVERT(varchar(10), GetDate(), 101)

Looking at my datetime values, let's say the 1st was generated at 18:45, obviously the 1st two records in the table would be returned. And let's say a 2nd time I need to generate I want to exclude the 1st two entries as they have already been verified. How can I do such w/o adding a field to the table?

View 11 Replies View Related

Load Result Sets From Mysqldatareader Into Datatable Or Dataset

Nov 8, 2007

hi all.....
i select some records from my database using mysqldatareader.......but i want to load it's result sets into my datatable or dataset....
is it possible and how can i do it this way ?
thanks

View 1 Replies View Related

Can You Caputure And Export Out Result Sets In Temp Tables???

Jan 8, 2001

I need to export out the data in a result set from temp tables. It is in a rather large Dts Stream. Thanks for your help. Brett

View 1 Replies View Related

Naming The Result Sets Returned By Stored Procedure

Jul 6, 2004

Greetings

I have a SQL Server stored procedure that performs several queries and therefore returns several "result sets" at any one time.

Wiring it up via ADO.NET I populate a DataSet with a number of items in the Tables collection - which is great - and I can give each item a name for identification purposes once the DataSet is populated.

But I'd like to know if there is some way I can set the names of each result set *within the text of the stored procedure*, i.e. before the DataSet gets populated.

Any help greatly appreciated.
Stuart

View 14 Replies View Related

How Stored Procedures Return Multiple Result Sets?

Jan 28, 2007

hi

i read that stored procedures can return multiple result sets?how is that?

thanks in advance.

View 4 Replies View Related

How To Suppress Result Sets Returned From A Stored Procedure?

Aug 28, 2006

I have a novice question. How does one suppress result sets returned from a stored procedure?

I have created a procedure which makes use of multiple stored procedures . The purpose of this procedure (lets call it procA), is to count the rows returned from other procedures. The €œOther€? procedures will return rows having an unknown number of columns. I would like to limit any changes which may be needed to be made to the €œOther€? procs.

Once procA has collected all of the information (@@rowcount) from the inner procedures, then it will return a result set having several columns €“ mainly the subProcedure name and number of rows returned.

The purpose of procA is to query several subsystems and identify which ones need attention.

Cursor While Loop
exec @ProcName @ObjectName,@userID,@syncDate
set @recs = @@rowcount;


My c# program calls the sp as follows:

cmd = DataUtility.GetSQLCommand();
cmd.CommandType = CommandType.StoredProcedure;
cmd.CommandText = "FetchAdminData";
cmd.Parameters.AddWithValue("@userID", userAlias);
cmd.Parameters.AddWithValue("@adminDate",userDate);
reader = cmd.ExecuteReader();

I do not wish to iterate over each resultSet when I am only interested in the last one. Any suggestions?



View 2 Replies View Related

Syntax For Returning And Accessing Multiple Result Sets In MSSQL 2k

Dec 29, 2006

Seasons greetings to everyone,A simple question. Could someone show me the syntax to produce multiple (2 or 3) result sets in a stored proc and how you access those sets from a c# program (ASP.NET)..Couldn't find a reference on Google, maybe I was asking the wrong question! Thanks for any help regardsDavej 

View 3 Replies View Related

Data Binding To A Stored Procedure That Returns Two Result Sets

Mar 6, 2007

Hi there everyone.  I have a stored procedure called “PagingTableâ€? that I use for performing searches and specifying how many results to show per ‘page’ and which page I want to see.  This allows me to do my paging on the server-side (the database tier) and only the results that actually get shown on the webpage fly across from my database server to my web server.  The code might look something like this:
 
strSQL = "EXECUTE PagingTable " & _
"@ItemsPerPage = 10, " & _
"@CurrentPage = " & CStr(intCurrentPage) & ", " & _
"@TableName = 'Products', " & _
"@UniqueColumn = 'ItemNumber', " & _
"@Columns = 'ItemNumber, Description, ListPrice, QtyOnHand', " & _
"@WhereClause = '" & strSQLWhere & "'"
 
The problem is the stored procedure actually returns two result sets.  The first result set contains information regarding the total number of results founds, the number of pages and the current page.  The second result set contains the data to be shown (the columns specified).  In ‘classic’ ASP I did this like this.
 
'Open the recordset
rsItems.Open strSQL, conn, 0, 1
 
'Get the values required for drawing the paging table
intCurrentPage = rsItems.Fields("CurrentPage").Value
intTotalPages = rsItems.Fields("TotalPages").Value
intTotalRows = rsItems.Fields("TotalRows").Value
 
'Advance to the next recordset
Set rsItems = rsItems.NextRecordset
 
I am trying to do this now in ASP.NET 2.0 using the datasource control and the repeater control.  Any idea how I can accomplish two things:
 
A) Bind the repeater control to the second resultset
B) Build a “pager� of some sort using the values from the first resultset

View 3 Replies View Related

How To Handle Multiple Result Sets Return From Stored Procedure

Aug 22, 2007

Hi all,

I want to know how to handle multiple result sets return from Stored Procedure? I know one way is to insert the result sets into the table, but the limitation is the result sets must have the same data structure. If the result sets have different data structure, how can I handle it.

Thanks,

View 5 Replies View Related

Integration Services :: SSIS - Database Query Result Sets Not Right

Jun 10, 2015

I have a general question I need a few pointers or explanations.  I have a top level Data Flow query in a SSIS package that has been running for years for which I have made some minor changes to pull additional fields and rectify a filter condition. 

I have noticed that when I run this same query, exactly as entered in the Data Flow task, in the SQL Server 2012 Management Studio, the query returns some 105,000+ records but when run in development by executing the package the top level task only indicates some 21,000 records returned.  Since there was not other transformations or filtering performed, I had expected the same record count and am not getting it. 

View 5 Replies View Related

SSIS Package Hanging Without Errors?

Jun 22, 2007

Hi folks,



I have a simple SSIS package I built to migrate data from a SQL db on one server to another SQL db on another server. All it does is: Execute SQL task to disable some triggers on the target table, Data Flow Task to pull the data from the source (a simple sql query) into the destination table (using OLEDB), then another Execute SQL task to re-enable the triggers.



This package ran fine several times during my testing. However, this morning it's not working anymore. When I run it and check its progress, the "Validation has started" is the only thing runs, then nothing else happens... no error messages, no timeouts, nothing in the output window besides "DTS.Pipeline: Validation phase is beginning".



The only thing that has changed is one change to a date in the where clause of my source query, so I'm baffled on why it no longer runs.



I'm pretty new to SSIS, so I could easily be missing something obvious here.



Anyone have any ideas on what's up, or who can point me in the right direction? I'm currently reading up on how to debug and add more comprehensive error handling, in hopes of finding a better description of the problem.



Thanks in advance,

Geoff

View 4 Replies View Related

SSIS Package Is Hanging After Validation

Mar 14, 2006

My SSIS package will just hang (do nothing) after validation of the package tasks. I realised that it does 2 validation. It then hits "starting exectuing" and then nothing. I mean nothing. It just stays the same. When I look into the logg file, the same message as in the output window. My package has parallel extract of data from the same datasource, but different tables. I dont know if that the problem but i really doubt it because i have done parallel table downloads countless times in version 2000. When i go into the data task window, the source task does no even indicate that its downloading (color yellow). Is there any reason why this will happen? Ooo, but the tasks executes just fine when i execute them individually (right click > execute).

View 17 Replies View Related







Copyrights 2005-15 www.BigResource.com, All rights reserved