Hello,
It seems strange to me that once the UNION ALL component waits for at least one row from each input buffer before it puts anything into the output buffer (that's the behaviour that I have observed anyway).
Why does it do that?
Thanks
-Jamie
[SSIS Team Followup]
Hi
A script component receives some input. But I just can't get at the first row??
Basically, if i use the NextRow method in the in the Do statement, then it advances the row collection to the second row before it gets into the code inside the loop?? BUT, if I use the EndOfRowset property to define my loop then I get an error:
[PipelineBuffer has encountered an invalid row index value]
I'm guessing this means...I have to call NextRow before i access the data in the collection? But thats retarted because then I miss the first row?? what? What am I missing??
This is the code which works but I miss the first row:
Public Overrides Sub Input0_ProcessInputRow(ByVal Row As Input0Buffer)
Dim strConcept As String
Do While Row.NextRow()
strConcept = Row.concept
updateDb(strConcept)
Loop
End Sub
This is the code which throws the invalid row index error:
Public Overrides Sub Input0_ProcessInputRow(ByVal Row As Input0Buffer)
Dim strConcept As String
Do While Not Row.EndOfRowSet()
strConcept = Row.concept
updateDb(strConcept)
Row.NextRow()
Loop
End Sub
I've put some try catches in there an the error happens on the line which calls Row.concept....?
Can anyone help, it must be something I'm messing up
thanks!!
andy
Hello,
I have a Union All component with 7 inputs (and it will grow to 13 inputs shortly). Is there any way to correlate each of the €œUnion All Input X€? with actual inputs?
When I get an incompatible datatype message for one of the input, how do I know which input stream to pick from?
Is this a silly question or it€™s a feature request?
Kapil Kelaiya
I have an SSIS package I've been working on that has been working just fine until now.
There are four inputs to this package, all OLEDB Inputs based on SQL Statement expressions.
These four inputs are then merged with a Union All step.
One of the merged columns is called "unit" and now suddenly this column cannot be added to the union all step for one of the inputs.
I get the following error:
Error at Import mat [Union All [330]]: The metadata for "input column "unit" (8979)" does not match the metadata for the associated output column.
Error at Import Mat [Union All [330]]: Failed to set property "OutputColumnLineageID" on "input column "unit" (8979)".
My question is
Can anyone suggest how I can diagnose this issue? I can't see the metadata in question, but when I physically inspect the results of the source queries, they all seem consistant with each other. The unit column is derived in similar fashion in each query. I'm a bit lost with this. I can't seem to shake off the error. Nothing has changed in the database to cause this.
Cheers.
I encountered the following error while attempting to preview an RDL report I was developing in VS2010 using SSDT:"The size necessary to buffer the XML content exceeded the buffer quota"
View 3 Replies View RelatedWe have a set of reports with same header section in all the reports. So while developing a new report i used to copy that header section to the new report with same dataset names (without any change) , but while rendering the report it is throwing error " The size necessary to buffer the XML content exceeded the buffer quota".
View 2 Replies View RelatedHi
I have a master package that executes a series of sub packages run from a SQL Agent job. One of those sub packages has been stable for a week, running at least once per day, but it just failed despite having been run once already today with the same set of input data.
There were a series of errors showing in the event log for the Execute Package Task starting with "Buffer Type 15 had a size of 0 bytes.", then "The buffer manager failed to create a new buffer type.", then "The Data Flow task cannot register a buffer type. The type had 32 columns and was for execution tree 3.", then "The layout failed validation." and finally "Error 0xC0012050 while loading package file "C:[Package].dtsx". Package failed validation from the ExecutePackage task. The package cannot run.".
SQLIS.com reports the constant for the error code as DTS_E_REMOTEPACKAGEVALIDATION ( http://wiki.sqlis.com/default.aspx/SQLISWiki/0xC0012050.html ).
I then ran the package on my dev machine in BIDS and it worked fine, so I re-ran the job on the server and this time that package executed ok, but another one fell over but did not put anything in the event log.
Does any one have any idea what happened?
TIA . . . Ed
Good day everyone,
I'm experiencing a completely random warning from any given row count component within any given data flow task. It occurs sporadically. Whilst distracting, I don't see any adverse effects to the data after the packages complete. Can someone weigh in on this warning and let me know if it is indeed benign or what I maybe able to do to fix it?
Here's the warning:
"A call to the ProcessInput method for input 75997 on component "CNT Rows sent for STG table" (75995) unexpectedly kept a reference to the buffer it was passed. The refcount on that buffer was 4 before the call, and 5 after the call returned."
Thanks,
Langston
SQL Server 2008 r2...
I have a query which does 3 selects and Union ALLs each to get a final result set. The performance is unacceptable - takes around a minute to run. If I remove the Union All so that the result sets are returned individually it returns all 3 from the query in around 6 seconds (acceptable performance).
Any way to join the result sets together without using Union All.
Each result set has exactly the same structure returned...
Query below [for reference]...
WITH cte AS (
SELECT A.[PoleID], ISNULL(B.[IsSpanClear], 0) AS [IsSpanClear], B.[SurveyDate], ROW_NUMBER() OVER (PARTITION BY A.[PoleID] ORDER BY B.[SurveyDate] DESC) rownum
FROM[UT_Pole] A
LEFT OUTER JOIN [UT_Surveyed_Pole] B ON A.[PoleID] = B.[PoleID]
[Code] .....
The scenario is as follows: I have a source with many rows. Each row has a column called max_qty_value. I need to perform a calculation using another column called qty. This calculation is something similar to dividing qty/(ceiling) max_qty_value. Once I have that number I need to write an additional duplicate row for each value from the prior calculation performed. For example, 15/4 = 4. I need to write 4 rows to the same target table as in line information for a purchase order.
The multicast transform appears to only support fixed and/or predetermined outputs. How do I design this logic in SSIS to write out dynamic number of rows to a target table.
Any ideas would be greatly appreciated.
thanks
John
Why the sequence different?
select * from (
select id=3,[name]='Z'
union all select 1,'G'
union all select 2,'R'
union all select 4,'Z'
) as t
order by [name]
--result:
---------
--1 G
--2 R
--4 Z
--3 Z
select * from (
select id=3,[name]='Z'
union select 1,'G'
union all select 2,'R'
union all select 4,'Z'
) as t
order by [name]
--result:
----------
--1 G
--2 R
--3 Z--changed
--4 Z
I am running into problems while running a large procedure, and i think it may have something to do with a PAGEIOLATCH_SH wait problem.
My server, whose sole purpose is to run this one procedure, is doing plenty of disk i/o, and the CPU’s bouncing around, so I assume it’s working. But when I look at its process info, it seems to be sleeping a lot of the time on PAGEIOLATCH_SH. No other users are in the DB, so I'm quite confused. I don't find much info on this anywhere, so any insight would be very appreciated.
I'm new to SQL Sever 2005 and I'm trying to do what Informatica (Power Center - ETL) is trying does.
I have created a work flow and it is scheduled to run at every night 1:00 AM .The process is to load a flat file (CRV.data) into the database from a shared location.The flat file is transfered from a 3rd party and once the file transfer is complete it will create a indicator file (0 byte eg: CRV.DONE file) which indicates the CRV.data transfer is complete.
In my workflow I will be waiting for the CRV.DONE indicator file and once it is avaiable I will start loading the CRV.data and once the load is completed I will delete CRV.DONE file and be ready for the next day load.
Please let me know if there is any way in SQL Server 2005 to achieve it.Thanks
Hi. We are migrating a mainframe datacom database to SQL Server. One of our client-server applications already uses SQL Server. This application uses a middleware product to query and update the datacom database being migrated. We are considering using Service Broker to replace the middleware.
In many cases the client does not need a response provided the message is queued and will eventually get delivered. However, in some cases the client would like to wait for the message to be processed before proceeding. Is there an easy way to both submit and optionally wait for a response - with data - in a single stored procedure? If client does not want to continue to wait, is there a way to use a procedure to check for the returned message later?
We have not used Service Broker before and are doing for a "sanity" check before proceeding. We do not want to tightly couple the two databases at this time.
I have installed performance dashboard on 2 different servers. The first server have User Session CPU Time 71% and Wait Time =28%, The other server have Cpu Time of 20% and Wait Time of 79%. Have I understand that stands in SQL Server Waits And Queues that I have some typ of wait problem in my second server?
Then I tries to run this
Select
'%signal waits' = cast(100.0 * sum(signal_wait_time_ms) / sum (wait_time_ms) as numeric(20,2)),
'%resource waits'= cast(100.0 * sum(wait_time_ms - signal_wait_time_ms) / sum (wait_time_ms) as numeric(20,2))
From sys.dm_os_wait_stats
First Server
%signal waits %resource waits
--------------------------------------- ----------------------------
0.07 99.93
Second Server
%signal waits %resource waits
--------------------------------------- ----------------------------
0.12 99.88
Messy in my head€¦ Help please
My second server have
CLR with 50% in the historial wait. Rest in Sleep
My first server have
99% in sleep wait category.
I'm doing an update on a table with about 113m rows, the update-statement is fairly simple: update tab set col = null where col is not null.
The col column is mostly null.
Sysprocesses shows three rows for this statement: 1 CXPACKET (its a dual processor, 2000 box with sp3 installed), 2 PAGEIOLATCH_SH (waitresource is filled). My guess would be that the where-clause is executed in a seperate process blocking the update.
I changed the statement into update [...] set col = null; sysprocesses shows one row with PAGEIOLATCH_SH. Executing forever.
I checked other processes including those outside sqlserver but none are using the db, let alone accessing the table involved. Even restarted sqlserver to be sure there's no dead process blocking the update. Didn't help.
So I added a search condition to the where-clause, involving a clustered index in order to reduce the rowcount. The execution plan shows a 97% hit on the clustered index, but sysprocesses shows the three rows again...
So far the profiler didn't help me out either: there's a SP: CacheInsert on the update-statement... then nothing.
What should I do?
I have an ASP.NET web application that hangs on a single database UPDATE command for 5+ minutes. I can see this occur in SQL Profiler. This is a one row UPDATE statement on a small table (~600 rows). There are no JOINs or sub queries. There are no other users using the system. During this 5+ minutes, I can see the job in Enterprise Manager with a wait type of NETWORKIO. Since both IIS And SQL Server are running on the same system, the network shouldn't be an issue. Any ideas?
View 5 Replies View RelatedHello, my first post??
View 4 Replies View RelatedI'm writing a small vbscript to backup a db and some related files, so I used a WSShell calling OSQL to run a Sql BACKUP command, then after it's finished I XCOPY the resulting file plus some other related files. But the problem is that OSQL ends it's execution as soon as the BACKUP command is sent to SqlServer, not when the backup itself ends.
Anyone knows how to synchronize the two? How to wait, inside OSLQ, for the end of the BACKUP execution?
TIA
Luigi
I setup a SQL Agent to send me an email when the Average Latch Wait Time is greater than 300ms. Now I receive an email every 15 seconds stating that the current ALWT is 3916ms. That value never changes with the emails. However, the perfmon shows nothing at all (shows zero).
I also have a Buffer cache hit ratio of 2848.00.
These numbers are when there is NOBODY on the DB at all It is just sitting there. When I reboot the server, as soon as SQL starts it starts to send the emails again.
Server: Intel Xeon Quad Core 2.66
RAM: 4GB (with /3GB in the boot.ini)
RAID 1: OS
RAID 1: Data (DB and logs)
CPU Utilization: 0-1%
RAM Utilization: 527MB
OS: Server 2003 R2 With SP2
SQL: 2005 Standard with SP2
How can I determine if the ALWT is really 3916?
I executed 'Select * from sysprocesses where SPID>50 and waittime>0'
Which showed;
BROKER_RECEIVE_WAITFOR
waittime=110640ms
waittype=0x0075
status=suspended
program_name=DatabaseMail90 - Id<3780>
Any assistance is greatly appreciated.
Thank You
Magnum
Oracle has a 'wait interface' for monitoring bottlenecks, etc... does SQLServer have utilities like this?
View 3 Replies View RelatedHi,
I have a data-flow-task that imports data to sqlserver.
Now I want to check, if a special column of an imported record is null.
If yes, I have to wait 10 minutes and jump to the data-flow-task again. (Cjeck and wait).
How can I do this with the integration services?
Thanks
Gerd
We have a few SSIS jobs that we are currently manually kicking off after we are sure that certain AS400 jobs have run. We want to completely automate this process, so that we don't have to babysit. What is the most efficient way to do this? In the past (on SQL Server 7 no less) I've seen the 400 job setting a flag to 'Y' in a 400 file, FTPing it down to a flat file, and then the SQL job running every five minutes checking the flag. When it was 'Y', the SQL job would run. We do not have the option of using FTP here. Any suggestions would be appreciated! After the job runs, we'd like it to kick off a report as well
So, I'm fairly new to SQL, and I'm working with a SQL2k5 Database with pre-made packages and what-not. This database was setup before I started this job, and now I'm trying to improve part of the processing in SQL, and so far so good, but I can't figure a couple things out.
The main problem is when I start a SQL command to launch a DTS package from a .sql file, how can I make it wait for the package to complete or fail before moving onto the next part of the .sql script? Hope it's a simple question, I've just taught myself enough SQL to get by in a couple of weeks.
Thanks!
I am tring to fiqure out how i can run a TSQL and the have it start again 60 sec after it compleats, without me have to push the button. I just need it to loop over and over until my data is deleted. I have to do it this way so my site will still allow customers to login and I need the break so they can. Any help would be great.
View 3 Replies View Relatedare there any tasks in SSIS that watches a folder and waits for incoming files?
View 9 Replies View RelatedI have re-initialize some subscription. After generated snapshot successfully, the synchronization view showing 'The process is running and is waiting for a response from the server.' and I found the replication process is under 'async_network_io wait'. The job has run for more than 8 hrs. Any idea to improve it? Thanks in advaise.
View 3 Replies View RelatedGood morning,
I will start off with the default warning message: I am a beginner. That said, I have an SSIS process that calls an external executable to transform a data file through a homegrown C program. (This will eventually be converted, but for the moment needs to remain.) The end of the run creates a *.done file. How do I use the SSIS tasks to pause/wait, checking periodically, for the existence of this file before continuing with processing tasks? I apologize if this is easy, but I am stumped.
Thanks in advance,
Roger
Info:
SQL 2005
Microsoft SQL Server Integration Services Designer
Version 9.00.3042.00
Hi all,
I have a Union All transformation with 4 inputs and one output when I debug the package the sum of the different inputs rows does not match the row count in output.
I don't understand, I've used the Union All transform many times and I've never seen this.
Any idea why this could happen ?
When I run some of the quiries, in current activity, wait type would say:
PAGEIOLATCH_SH what does it means ? Is there a source I can see all
types of wait types ?
Help is much appreciated.
Thanks,
I keep getting an alert from a Third-Party Tool stating my server is reporting failed components with the following message
Components Reporting Problems:
Average Lock Wait Time(ms)(Critical)
When I look at sp_who2 or sp_lock I don't see anything blocking processes or anything else out of the ordinary.
CXPACKET wait type current makes up 63% of all the wait types which is causing latency. I need to identify the specific workloads responsible for waits so I can optimise or MAXDOP them. I already know how to retreived the top IO, CPU, Memory consuming queries but how do I identify the statements and order them by wait time?
Can someone point me in the direction of a command, DMV or will the top CPU list be adequate?
Thanks
I would like to know what a particular session is waiting on (cpu, io, memory etc).
For that I try to figure out how to use extended events. I would prefer to avoid the use files written to db-server, why I tried "histogram target".
Well my attempt does not produce usefull output,
-- check current event sessions
SELECT a.name,
CASE WHEN b.name IS NOT NULL
THEN 'Started'
ELSE 'Stopped'
END AS current_state
[Code] ....
The intention was to get a pareto chart, to see the top wait events in the session. I would also like to see amount of cpu used. Based on this tuning effort could be prioritized.