I have a data-flow containing 150000 rows that is taking a long-g-g-g- time to run.
The bottleneck seems to be a SORT component. The SORT is immediately preceded by a MULTICAST.
Now, all 150000 rows have gone down the other output from my MULTICAST all the way to the destinaton, admittedly very slowly. My SORT is still caching the 150000 rows.
I'm wondering if the MULTICAST is causing a problem. Does each output fro the MULTICAST have its own buffer?
Can an asynch component that caches all the data (e.g. SORT) cause backpressure on the MULTICAST that might cause problems.
As I write, the package has been running 20 mins and the SORT component still has not cached all the rows.
I have a dataflow where each row passes through about 20 validations. This data flow could contain 200,000 to 20 million rows. Initially I set it up with a multicast component sending data each of the validations.
This got to be pretty messy.
Then I thought about using several multicast to subdivide the validations and make the data flow easier to follow. I tested using two multicasts to see if it would affect the performance of the dataflow. Results were inconclusive.
Before I continue adding and testing different multicast configurations, I thought I would check here to see if anyone else has tried and measured this?
I split a source into two. I get an Id from one after doing couple of lookups and maxId from other after doing a lookup and and an Aggregate. I want to compare Id and max Id in the same data flow. How can I do that please?
I have to do various controls on a dataset - I created a multicast. After performing controls (one control per copy), I merge my (7) multicasted datasets using a Union All transformation. The problem I'm having are the duplicate rows created by merging the multicast copies.
How do I get rid of the duplicates? Is the Sort Transformation the solution by setting the option Remove rows with duplicate sort values to True? I have a unique key by which I'm able to discard the duplicates correctly. Are there any other ways (at a Union All level)? Is there sth like Union and Union All like in SQL?
I'm working on my 1st integration serv. project and it seems that more I work more questions I have. Shoudn't be the opposite? Thank you for the help.
Hello, I am using a multicast with 5 outputs, I attempting to pass about four thousand rows to different destinations. However, upon execution only two of the outputs send the rows to the destination. After deleting the multicast and reinserting it, different destinations received the data. The number of destinations is also not consistant, sometimes 3 work and sometimes only 1 works. Thank you.
Wondering how this is handled by Multicast component. If multiple copies are created in the memory and the size of the dataset is large, this could cause some performance problems. Any thoughts? TIA
I am trying to set sorting up on a DataGrid in ASP.NET 2.0. I have it working so that when you click on the column header, it sorts by that column, what I would like to do is set it up so that when you click the column header again it sorts on that field again, but in the opposite direction. I have it working using the following code in the stored procedure: CASE WHEN @SortColumn = 'Field1' AND @SortOrder = 'DESC' THEN Convert(sql_variant, FileName) end DESC, case when @SortColumn = 'Field1' AND @SortOrder = 'ASC' then Convert(sql_variant, FileName) end ASC, case WHEN @SortColumn = 'Field2' and @SortOrder = 'DESC' THEN CONVERT(sql_variant, Convert(varchar(8000), FileDesc)) end DESC, case when @SortColumn = 'Field2' and @SortOrder = 'ASC' then convert(sql_variant, convert(varchar(8000), FileDesc)) end ASC, case when @SortColumn = 'VersionNotes' and @SortOrder = 'DESC' then convert(sql_variant, convert(varchar(8000), VersionNotes)) end DESC, case when @SortColumn = 'VersionNotes' and @SortOrder = 'ASC' then convert(sql_variant, convert(varchar(8000), VersionNotes)) end ASC, case WHEN @SortColumn = 'FileDataID' and @SortOrder = 'DESC' THEN CONVERT(sql_variant, FileDataID) end DESC, case WHEN @SortColumn = 'FileDataID' and @SortOrder = 'ASC' THEN CONVERT(sql_variant, FileDataID) end ASC And I gotta tell you, that is ugly code, in my opinion. What I am trying to do is something like this: case when @SortColumn = 'Field1' then FileName end, case when @SortColumn = 'FileDataID' then FileDataID end, case when @SortColumn = 'Field2' then FileDesc when @SortColumn = 'VersionNotes' then VersionNotes end
case when @SortOrder = 'DESC' then DESC when @SortOrder = 'ASC' then ASC end and it's not working at all, i get an error saying: Incorrect syntax near the keyword 'case' when i put a comma after the end on line 5 i get: Incorrect syntax near the keyword 'DESC' What am I missing here? Thanks in advance for any help -Madrak
We are using a modeling technique called Anchor Modeling in our data warehouses. You can read more about the technique itself at our homepage http://www.intellibis.se, where we have published a fact sheet and a recently held presentation (TDWI European conference). One of the features with this technique is its simple way to historize data. This is done by having a fromDate column which together with the surrogate key will yield a unique combination. On the tables that has this kind of historization we add a primary key, which in turn will create a clustered index, with the following specification (surrogateKey asc, fromDate desc). This will physically order data on the storage media according to the specificed columns and ordering. Now I move on to create a "latest view" of this table which does a subselect to find the latest version for every surrogateKey using max(fromDate). Should not the optimizer now figure out that data is ordered so that the latest version always comes first for every surrogateKey, hence any sorting would be unneccessary? If I look at the actual execution plan after running a query that uses the view there is a sort in the plan, but the cost is always 0%. Does this mean that it did not sort the data, or that it did call a sorting routine, but it actually took very little time to do the sorting? If so, is there a reason that is has to do the sorting or could it have been left out by an even smarter optimizer?
I would also like to applaud the people behind the optimizer, since it will figure out which tables are in fact necessary to query and eliminate others, even if I have left joined them into the view I am using. This speeds up performance and makes anchor modeling feasible. Unfortunately optimizers from other vendors seem to have trouble doing this...
I've been racking my brain all day and I finally decided to ask for help. I've got two tables with rows from the first that need to be sorted by the second. The problem is that the rows don't always exist in the second table. I've tried various forms of INNER, LEFT, RIGHT, OUTER, LEFT OUTER, CROSS, etc., etc., etc. and nothing (oh yeah UNION too). Every time I get close, I lose the records that don't have matches.
Something close-
SELECT A.IDDoc, B.First FROM A LEFT JOIN B ON A.IDDoc = B.IDDoc WHERE B.Dept = 'A' ORDER BY B.First
SELECT LEFT(CONVERT(CHAR(11),convert(datetime,task_date),109),3) + ' ' + RIGHT(CONVERT(CHAR(11),convert(datetime,task_date),109),4) as Date,SUM(CASE a.status_id WHEN 1000 THEN b.act_point ELSE 0 END) as Programming,SUM(CASE a.status_id WHEN 1016 THEN b.act_point ELSE 0 END) as Design,SUM(CASE a.status_id WHEN 1752 THEN b.act_point ELSE 0 END) as Upload,SUM(CASE a.status_id WHEN 1032 THEN b.act_point ELSE 0 END) as Testing,SUM(CASE a.status_id WHEN 1128 THEN b.act_point ELSE 0 END) as Meeting,SUM(CASE a.status_id WHEN 1172 THEN b.act_point ELSE 0 END) as OthersFrom task_table a,act_table b where a.status_id=b.act_id and a.user_id=(select user_id from user_table where user_name='Raghu') and a.task_date like '%/%/2006' GROUP BYLEFT(CONVERT(CHAR(11),convert(datetime,task_date),109),3) + ' ' + RIGHT(CONVERT(CHAR(11),convert(datetime,task_date),109),4)Output :Aug 2006 294 0 0 80 0 0 Jan 2006 14 0 0 0 0 0 Oct 2006 336 0 0 0 0 0 Sep 2006 3262 20 24 8 16 0 How to sort the date in ascending Order ?Jan 2006Aug 2006Sep 2006Oct 2006
I have: 4 tables and 1 table variable. CCenters (ID, Name) Campaigns (ID, Name) Rel (ID, CCenterID, CampaignID) - [many to many] and @SCampaigns (ID, CampaignID) - represents the selected campaigns by the user
performing the commands below I would get the centers associated with the campaigns selected.SELECT CCenterID FROM Rel INNER JOIN @Campaigns ON @SCampaigns.CampaignID = Rel.CampaignID But what I really want are the common centers to the selected campaigns. Thanks
I am trying to select a record from a table where it has the smallest priority how would you go about doing this is there a cool sort command or is there a select command syntax that can do this thanks
I've made this example and it loads a picture into a database. (MsSql )Take a look at the code, it works just fine however it leaves a process in sleeping mode "avaiting command" in Enterprise manager under "Management/current Activity/Process Info"Is it supposed to be like this or is it supposed to be reemoved after .net is finished??Code snip_______________________________________________________ Dim conn As New SqlConnection("Data Source = (local);Initial Catalog = " & "test;User ID = NAME; Password=PASSWORD;") Dim cmd As New SqlCommand("Select * from tab_bild", cnn) Try conn.Open() Dim myDatareader As SqlDataReader myDatareader = cmd.ExecuteReader(CommandBehavior.CloseConnection) Do While (myDatareader.Read()) Response.ContentType = myDatareader.Item("PersonImageType") Response.BinaryWrite(myDatareader.Item("PersonImage")) Loop conn.Close() Response.Write("Picture info succesfully retrieved") Catch SQLexc As SqlException Response.Write("Read failed, Reason: " & SQLexc.ToString()) End Try End Sub________________________________________________________________Please can someone explain this for me or sort this out for me.All help is welcome even if its only points me too a direction.RegardsTombola
I have a table that most of the data has the same value, but there are only a few that do not match that value. I want to populate a listbox with all values from the table, but I'd like to have the majority listed first, followed by the others (the few that don't matach). What's the best way to approach this with SQL?
I'm trying to setup a duplicate of an old SQL Server 4.2 server to put in place while we upgrade the server, but I can't get the sort-order right. I know the existing server uses sort order id 40, but I can't find which sort-order that corresponds to during the install process. If anyone can give me a system table that lists all the sort orders names and id's, or can tell me what the text name for sort order 40 is, I would be very grateful.
I have a table with 3 columns. Product, Location and Value. The data looks like this:
NULL NULL 100 Atlanta NULL 50 Atlanta Cookie1 30 Atlanta Cookie2 20 Dallas NULL 120 Dallas Cookie1 80 Dallas Cookie2 40
This table gets filled with a Groupby with Rollup option. The NULLS show subtotals/total. Is there a way to build a query that returns the results with NULLs at the bottom of each section like:
Atlanta Cookie1 30 Atlanta Cookie2 20 Atlanta NULL 50 Dallas Cookie1 80 Dallas Cookie2 40 Dallas NULL 120 NULL NULL 100
I need to copy the structure and data of an existing SQL 6.5 server to one with a different sort order. Normally, I would use the transfer tool to accomplish this, but the servers are on different networks. My question is, is BCP the answer? In other words, will the data copied via BCP from the sending server be able to be copied on the recieiving server. Also, is there a way to automatically generate the BCP statements for all tables? What I would really like is to be able to get at the scripts and data files created by the transfer tool.
Hi, Can any one pls tell me what this sort order id 42 corresponds to and how its different from 52 ? What options i need to check during installation for sort oreder id. ?
I have a table with 3 columns. Product, Location and Value. The data looks like this:
NULL NULL 100 Atlanta NULL 50 Atlanta Cookie1 30 Atlanta Cookie2 20 Dallas NULL 120 Dallas Cookie1 80 Dallas Cookie2 40
This table gets filled with a Groupby with Rollup option. The NULLS show subtotals/total. Is there a way to build a query that returns the results with NULLs at the bottom of each section like:
Atlanta Cookie1 30 Atlanta Cookie2 20 Atlanta NULL 50 Dallas Cookie1 80 Dallas Cookie2 40 Dallas NULL 120 NULL NULL 100
Hi There i have two windows 2000 servers which are both running SQL and i would like to restore a backup from one server to the other. Which in my opinion should be an easy task but when i go into the restore option and point it at the file i would like to restore i get the follwoing error
"The database you are attempting to restore was backed up under a different sort order ID (52) than the one you are currently using on this server (50) and at least one of them is a non binary sort order. Backup or restore operation operation terminating abnormally."
The server that i am trying to restore to already has databases on this so i cannot just reinstall SQL and change the sort order not that id know how to do that but this is what i have read.
Is htere anyway that i can put insome script for the database to fix this ???
MIN(CAST(PARSENAME (ipaddress,1) as decimal)) + '&' + MAX(CAST(PARSENAME(ipaddress,1) as decimal)) as LowestAndHigestIP
I get error message "Error converting data type varchar to numeric." i guess that '&' is the issue. I want to be able to sort high and low IP like number not string.
How do I create a sort order column in a view? I could use Row_Number() in 2005 but unfortunately I need to create a sort order column in a 2000 view and the View is not letting me to use an ORDER BY.
Any inputs?
Prakash.P The secret to creativity is knowing how to hide your sources!
