Validation Of For Loop Very Slow On Load
Aug 31, 2007
Hi there,
I have recently added a nested for loop into my SSIS package (The 'parent' for loop loops through years with a variable @DAT_YEAR, the 'child' for loop loops through months of those years using @DAT_MONTH). The task contained within the 'child' loop is a data flow transformation which connects to a source ODBC database and writes out data to a text file a month at a time, supplying the @DAT_YEAR and @DAT_MONTH variables as expressions in the SqlCommand statement.
The logic of the data flow seems to work fine, however, I notice that when I load the package now, it seems to take forever (> 5 minutes, certainly) to regain control in Visual Studio, and the status message at the bottom left of the VS window reads "Validating For Loop Container".
I know that you can work in 'offline' mode to temporarily turn off the validation, but I cannot manage any of the connections in this mode - why is it taking so long? Is it actually doing the loop and validating the ODBC connection each iteration?
yours puzzled,
t.
View 2 Replies
ADVERTISEMENT
Nov 15, 2006
I have one aspx page that is very slow loading it is connected to SQL 2005 database. I am looking for any preformance tips out there
View 3 Replies
View Related
Apr 14, 2008
Each day, the first user who launches our RS reports always gets a long wait time. Subsequent report launches are normal.
Does anyone know what is going on? If yes, what is the remedy?
Thanks
View 1 Replies
View Related
May 9, 2008
Hi I have created a web service task that sits inside of a foreach loop. The loop is a ado enumerator. This loop is going to loop through about 35,000 records and make 35,000 web service calls.
So far through my testing, I have gotten it to read and write everything correctly, but each call to the web service takes about 1-2 seconds and maxes out the CPU. This would take almost all day for this web service to run at 100% CPU power all day. We want to be able to run this a few times a day or at least once a day.
I have disabled all of the other tasks in the foreach loop so it just calls the web service task and found that it's the web service task that takes up 100% of the CPU and takes 1-2 seconds to run. I have also tried creating a mock web service on the server that just sends back the same mock data, so it's extremely fast, but the Web Service Task still maxes out the CPU and takes 1-2 seconds per call.
I have tried this out on 2 different servers, all with a few gigs of ram they're not using. It seems more like it's a CPU thing, or that the Web Service Task has a high overhead that each time in the loop it takes 1-2 seconds to create the task.
Anyone have any advice?
Thanks,
-Will
View 6 Replies
View Related
Jul 25, 2006
Hi all,
I have a webpage with a Datagrid that populates using a table adapter from a Stored procedure that exists in my SQL Database...If I run the Stored procedure in SQL Directly then it takes 20 Secs to return all records...If I run the webpage then it takes just over 20 Secs..
Great you say..But If I have the sorting option set in ASP.net and I click on a column to sort then off the page goes for another 20 secs to sort the data..
Is there a better way to do what I am doing here that will speed up the page load..
Ie..the data is returned once and then sorted...
Is it Better / Quicker for me to create a table using the stored procedure and link to this from the website..Updating the table every couple of minutes ?
Any advice please ?
Ray..
View 4 Replies
View Related
Feb 13, 2006
Is there any information around what the SSIS packages are doing in the first 5-10 seconds of execution, and ways to speed this process up?
View 3 Replies
View Related
Sep 21, 2015
I have a dynamic sql query where in I am comparing two tables and loading data for last 15 days. e.g today 2050921 then I am going to load till 20150906.
I pass on 2 variables @currentdate and @currentdate-1 to the query which are in date format 'yyyymmdd'
I need to do this for last 15 days how do I do this using while loop.
Note my date format is YYYYMMDD.
DECLARE @SQL VARCHAR(MAX)
@sql = ' insert into targetÂ
select from table_1_currentdate a
LEFT JOIN Table_2_currentdate-1 b
on a.col1=b.col1 Â where b.col1 is null '
exec(@sql)
I have to use while loop and decrement it every time and load data for last 15 days comparing two tables. I tried so many times I am not getting it right .
View 3 Replies
View Related
Oct 28, 2015
I have a requirement to load multiple flat files in target table .
I have created the package which used to load files into target table using For each loop container.
But now requirement has been changed now I have to take only those files from table where status="Success" and max JobId. By the query I am to get those records which need to load into table.
Below query I am using to get the files which need to load.
select [JobLogKey],[SrcNm],[DestNm]
FROM [ConfigRep].[dbo].[JobLog]
Where [JobId]=
(Select Max(cast([JobId] as Int)) Jobid
FROM [ConfigRep].[dbo].[JobLog]
Where [JobStat]='Success')
Output:-
JobLogKey SrcNm  DestNm
268 H:Data PlatformSource FileClient2LocHGSSpecLocation.txt Location.txt
269 H:Data PlatformSource FileClient1LocHGSSpecLocation.txt Location.txt
I have to load using above 2 files which are under SrcNm. I have created one variable called FileToLoad as Object and mapping to result set of above query. I have create JobId,SrcNm and DestNm variable to catch the record at every loop. I have  created 2 For each Loop containerÂ
Below screen shot of outer Foreach loop. Till here Its working fine. Inner for each loop container not executing any task under that. How to get it done.
View 3 Replies
View Related
Oct 24, 2007
I am getting the following error when trying to load multiple excel files using for each loop container in SSIS, I tried to put the quotes in several different ways but still can't get rid of this error. I was able to successfully load single excel file, but when I use the for each loop container that's when I am having problems. Any help is greatly appreciated. Thx.
TITLE: Package Validation Error
------------------------------
Package Validation Error
------------------------------
ADDITIONAL INFORMATION:
Error at Package1 [Connection manager "SourceConnectionExcel"]: The connection string components cannot contain unquoted semicolons. If the value must contain a semicolon, enclose the entire value in quotes. This error occurs when values in the connection string contain unquoted semicolons, such as the InitialCatalog property.
Error at Package1: The result of the expression ""Provider=Microsoft.Jet.OLEDB.4.0;Data Source=" + @[User::Folder] + @[User::file] + ";Extended Properties="Excel 8.0;HDR=NO";"
" on property "ExcelFilePath" cannot be written to the property. The expression was evaluated, but cannot be set on the property.
(Microsoft.DataTransformationServices.VsIntegration)
------------------------------
BUTTONS:
OK
------------------------------
View 8 Replies
View Related
Aug 26, 2015
In my SSIS package I am using Foreach loopcontainer to load multiple flat files.
Now my requirement is that I want to load only those file which contains %vendor%.In source folder I have many files but I am interested in to load only those file which contains the string %vendor% in file name.
View 2 Replies
View Related
Apr 9, 2008
Hello,
I have developed some packages to load data into "Fact" tables in the data warehouse.
Some packages are OK, other ones not. What is the problem?: some packages load fact tables with lots of "Lookup - Data Flow Transformation" into the "data flow task" (lookup against dimension tables) but they are very very slow, too much slow to be choosen as a solution.
Do you have any other solutions to avoid using "Lookup - Data Flow Transformation"? Any other solution (SSIS, TSQL and so on....) is welcome to speed up the Fact table loading process.
Thank in advance
View 7 Replies
View Related
Dec 14, 2007
Hi,
Here I will describe my problem.
1. We are loading large amount of data from database on background thread which is starting on Application_start event in global.aspx.cs file.The data is later cached for subsquent request to improve the performance.
2. Now when we put the application on web farm garden, it is not able to load the application.
3. We are sending the request the servers through Router kind of application.
4 This application is working fine on single server enviornment.
Please help us.
Ajay Kumar Dwivedi
View 1 Replies
View Related
May 28, 2010
difference between FOR LOOP and FOREACH LOOP with example(if possible) in SSIS.
View 4 Replies
View Related
Apr 27, 2007
I just have done the SSIS example in the tutorial document included when install SQL 2005 ENT. I have a problem that whenever I test to run, the service load all data from source with out noticing about the data (I mean it load all the data to the destination), I do it several time and it continue to load all without checking. That mean the data is dublicated when the schedule run???
I think there should be a paramete or something like that to help the engine just load the new data to the destination. Could you help please?
Thank
View 3 Replies
View Related
Feb 22, 2006
I have a table with RowID(identity). I need to loop though the table using RowID(not using a cursor). Please help me.
Thanks
View 6 Replies
View Related
Mar 3, 2006
I have a foreach loop that is supposed to loop through a recordset, however it doesn't loop. It just repeats the same row, row after row.
I would like to look into the recordset variable but I can't because it is a COM object and the ADODB namespace is not available in the script task.
Any solution to this? anyone experienced anything similar
View 1 Replies
View Related
Jul 8, 2006
I have a table called Tbltimes in an access database that consists of the following fields:
empnum, empname, Tin, Tout, Thrs
what I would like to do is populate a grid view the a select statement that does the following.
display each empname and empnum in a gridview returning only unique values. this part is easy enough. in addition to these values i would also like to count up all the Thrs for each empname and display that sum in the gridview as well. Below is a little better picture of what I€™m trying to accomplish.
Tbltimes
|empnum | empname | Tin | Tout | Thrs |
| 1 | john | 2:00PM | 3:00PM |1hr |
| 1 | john | 2:00PM | 3:00PM | 1hr |
| 2 | joe | 1:00PM | 6:00PM | 5hr |
GridView1
| 1 | John | 2hrs |
| 2 | Joe | 5hrs |
im using VWD 2005 for this project and im at a loss as to how to accomplish these results. if someone could just point me in the right direction i could find some material and do the reading.
View 18 Replies
View Related
Feb 23, 2006
I have source and destination table names in the database(one table) and I need to read the source and destination tables one by one...
My Lookp table is like the following...
Srn srctable desttable
1 SRC1 DEST1
2 SRC2 DEST2
3 SRC3 DEST3
Now I want one package to load from source to destination.. how do I do it.. I dont know how to use....
How do I run the pacakge for each of the rows... ..............................
View 1 Replies
View Related
May 19, 2006
Hello,
I have a big problem and i'm not able to find any hint on the Network.
I have a window2000 pc, VS2005,II5 and SQLServer 2005(dev edition)
I created an SSIS Package (query to DB and the result is loaded into an Excel file) that works fine.
I imported the dtsx file inside my "Stored Packages".
I would like to load and run the package programmatically on a Remote Scenario using the web services.
I created a solution with web service and web page that invoke the web service.
When my code execute:
Microsoft.SqlServer.Dts.Runtime.Application.LoadFromDtsServer(packagePath, ".", Nothing)
I got the Error:
Microsoft.SqlServer.Dts.Runtime.DtsRuntimeException: The package failed to load due to error 0xC0011008 "Error loading from XML. No further detailed error information can be specified for this problem because no Events object was passed where detailed error information can be stored.". This occurs when CPackage::LoadFromXML fails.
The error message doesn't help so much and there is nothing on the www to give me and advice....
Is it a SSIS problem???
Thank you for any help!!
Marina B.
View 10 Replies
View Related
Mar 10, 2004
I am working on a query application, and I want to do syntax validation before I submit the dynamically sql to the database. The expression will include ANDs, ORs, IN, (,),>,<,etc. Anyone done this already? any code snippets?
Thanks!
View 1 Replies
View Related
Aug 10, 2006
Hi
I am running the following:
DECLARE @h uniqueidentifier --conversation handle
DECLARE @msg xml; --will hold the message
BEGIN DIALOG CONVERSATION @h
FROM SERVICE [ewx.co.za/Service/store001_ewx_sb_service]
TO SERVICE 'ewx.co.za/Service/ewx_sb_hub_service'
ON CONTRACT [ewx.co.za/Contract/ewx_Contract];
SET @msg = '<InventoryUpdate>
<TitleId>STORE001TEST1</TitleId>
<Quantity>7777</Quantity>
</InventoryUpdate>';
SEND ON CONVERSATION @h
MESSAGE TYPE [ewx.co.za/Message/ewx_sendmsg](@msg);
Now to test errors comming back on the aueue i sed to make the xml tags wrong, then the target would send a error back on the queue with xml validation failed (both queues have validation well_formed_xml). However now in testing i cannot even send the message i get an invalid xml error straight away, i am not sure why this is , i know the xml is not valid but the send used to work and i would get an erro rback, as the xml is validated by the ttarget, but this no longer works it ails strainght away, with no thing in any queue. What could be causing this ?
Thanx
View 1 Replies
View Related
Mar 27, 2007
I think I have read online a recommendation about not using XML VALIDATION in a production environment, due to performance reasons. Is it recommended using other that NONE validation in production, and is there available documentation for a scale that grades performance hits for various types of validations?
Thanks
View 1 Replies
View Related
Apr 10, 2007
Afternoon all,I want my SQL SP to do some validation on a form submit to do the following before committing to the table.If email address (txt.Email.Text) doesn't exist in the table, commit values.if email address (txt.Email.Text) does exist and option (radOptions.SelectedValue) equals 1, print message to say 'you're already subscribed'if email address (txt.Email.Text) does exist and option (radOptions.SelectedValue) equals 0, print message to say 'you're not subscribed'if email address (txt.Email.Text) does exist and option (radOptions.SelectedValue) equals 1 or 0, update row to 0 or 1 (depending on subscribe or unsubscribe - 1 = subscribed, 0 = unsubscribed) The simple SP is currently:ALTER procedure [dbo].[sp_customerSignups]@name varchar(50),@email varchar (50),@subscribed intasBEGININSERT INTO tblCustomerSignups(Name, EmailAddress, Subscribed)VALUES(@name, @email, @subscribed)END Does anyone have the correct syntax for this?Thanks,Brett
View 4 Replies
View Related
Sep 21, 2007
Hi,
I have following XML-file and XML-schema:
<template xmlns="http://www.mycompany.com/xsd/PageTemplate">
...
</template>
<xs:schema attributeFormDefault="unqualified" elementFormDefault="qualified" xmlns:xs="http://www.w3.org/2001/XMLSchema" xmlns="http://www.mycompany.com/xsd/PageTemplate" targetNamespace="http://www.mycompany.com/xsd/PageTemplate"><xs:element name="template"><xs:complexType>...</xs:complexType>
</xs:element></xs:schema>
When I create and query the XML file using LINQ, everything works just fine. I also get no compilation errors.
But when I try to add the XML file to a database-field of type xml(CONTENT dbo.Common7), I get following error:
XML Validation: Declaration not found for element 'http://www.mycompany.com/xsd/PageTemplate:template'. Location: /*:template[1] Any ideas? Thanks,Thomas
View 1 Replies
View Related
Jul 10, 2006
Hey all,
Quick question regarding validation!
I have a table of contact details containing the usual name, adresss etc fields.
I want to validate the fields Country and telephone together so that fopr example if country = 'UNITED KINGDOM' the telephone has to begin with +44 if it doesnt i want it to add the +44!
I can do this through writing a little program but just wanted to explore the possibility of doing this with SQL or functions already available with MSSQL Server
cheers
ed
View 3 Replies
View Related
Feb 8, 2005
i have two params..
@startdate datetime,@enddate datatime
1.what's the best way to validate them
2.Best way to check whether @startdate is less than @enddate
View 1 Replies
View Related
Feb 27, 2004
I need to make sure that a string contains at least one number, at least one letter and has no repeating chars. Anyone done something similar? What's the best way to do it? Thanks.
View 14 Replies
View Related
Mar 17, 2004
I am new to SQL SERVER,
Can anybody help me to write a procedure to validate a given string like
09461020(Y2K compliant date), I need to check wether this is a valida date or not??
0946 = should convert to 1946
10 = month
20 is the date.
If it is 20461020.....then year = 2046, month = 10 and day = 20..and it is valid.
procedure should return validate date or not ...return TRUE OR FALSE.
View 1 Replies
View Related
May 23, 2008
Is there a way in TSQL to validate a string to ensure it has only numbers or letters (no special characters like @,%, etc)?
I want to write a function that validates a number that should only contain letters and numbers. The function should return 1 if valid 0 if not. If it contains anything other than 0-9 or A-Z or a-z then it should return 0.
Any ideas on how to do this in TSQL? I could do this in C# easily with regular expressions, but I don't want to use SQLCLR or any other external resource like that...just straight TSQL.
Thanks!
View 2 Replies
View Related
Jun 13, 2008
How can we validate email id in sql server 2005..
i use the following code...
if((len(@default_to_mail)-len(replace(@default_to_mail,' ','')))=0 and @default_to_mail like ('%@%.%'))
its not validating correctly...
can u give me any solution.
Thanks in advance.
cool...,
View 4 Replies
View Related
Feb 23, 2006
I'm sorry to repost this, but I didn't get an answer yet for this seemingly simple issue. I need to make sure that an email address has the @ symbol before I insert it from one table to another. Does anyone know the syntax? What I have that isn't working is -
select * from table where email charindex ('@',email) > 0
View 7 Replies
View Related
Mar 7, 2007
Guys,
I hope I will be able to get this.
I'm looking a cripts/code that will validate date in my form when the user key in date. The date has to be in the following format 00/00/0000 (month/ day/ year)
HELP
Thk
View 3 Replies
View Related
Mar 13, 2007
hi,
this is my table values:
select * from empmaster
emp.no fromdate todate
----------------------------------
E3/13/20074/7/2007
E4/13/20076/13/2007
i am going to pass some date for example 5/20/2007.
so i have to compare this 5/20/2007 which lies in between 3/13/20074/7/2007 or 4/13/20076/13/2007.if means i have to print not allowed other wise allowed. and the date should not be less(if 2/10/2007). like thi i have many from date and todate in my table. if i give 7/20/2007 means it should print allowed becos it doesnot falls in between any date o the emp.no=e. this should be depent on the emp.no also.
so please help me to do this.please show me query for this.or spc
View 10 Replies
View Related