Validation Of For Loop Very Slow On Load

Aug 31, 2007

Hi there,

I have recently added a nested for loop into my SSIS package (The 'parent' for loop loops through years with a variable @DAT_YEAR, the 'child' for loop loops through months of those years using @DAT_MONTH). The task contained within the 'child' loop is a data flow transformation which connects to a source ODBC database and writes out data to a text file a month at a time, supplying the @DAT_YEAR and @DAT_MONTH variables as expressions in the SqlCommand statement.

The logic of the data flow seems to work fine, however, I notice that when I load the package now, it seems to take forever (> 5 minutes, certainly) to regain control in Visual Studio, and the status message at the bottom left of the VS window reads "Validating For Loop Container".

I know that you can work in 'offline' mode to temporarily turn off the validation, but I cannot manage any of the connections in this mode - why is it taking so long? Is it actually doing the loop and validating the ODBC connection each iteration?

yours puzzled,

t.

View 2 Replies


ADVERTISEMENT

Slow Access On Page Load

Nov 15, 2006

I have one aspx page that is very slow loading it is connected to SQL 2005 database. I am looking for any preformance tips out there

View 3 Replies View Related

Initial Report Load Is Slow

Apr 14, 2008

Each day, the first user who launches our RS reports always gets a long wait time. Subsequent report launches are normal.
Does anyone know what is going on? If yes, what is the remedy?

Thanks

View 1 Replies View Related

Slow Web Service Task In Foreach Loop

May 9, 2008



Hi I have created a web service task that sits inside of a foreach loop. The loop is a ado enumerator. This loop is going to loop through about 35,000 records and make 35,000 web service calls.

So far through my testing, I have gotten it to read and write everything correctly, but each call to the web service takes about 1-2 seconds and maxes out the CPU. This would take almost all day for this web service to run at 100% CPU power all day. We want to be able to run this a few times a day or at least once a day.

I have disabled all of the other tasks in the foreach loop so it just calls the web service task and found that it's the web service task that takes up 100% of the CPU and takes 1-2 seconds to run. I have also tried creating a mock web service on the server that just sends back the same mock data, so it's extremely fast, but the Web Service Task still maxes out the CPU and takes 1-2 seconds per call.

I have tried this out on 2 different servers, all with a few gigs of ram they're not using. It seems more like it's a CPU thing, or that the Web Service Task has a high overhead that each time in the loop it takes 1-2 seconds to create the task.

Anyone have any advice?

Thanks,
-Will

View 6 Replies View Related

Stored Procedure - Page Load Slow

Jul 25, 2006

Hi all,
I have a webpage with a Datagrid that populates using a table adapter from a Stored procedure that exists in my SQL Database...If I run the Stored procedure in SQL Directly then it takes 20 Secs to return all records...If I run the webpage then it takes just over 20 Secs..
Great you say..But If I have the sorting option set in ASP.net and I click on a column to sort then off the page goes for another 20 secs to sort the data..
Is there a better way to do what I am doing here that will speed up the page load..
Ie..the data is returned once and then sorted...
Is it Better / Quicker for me to create a table using the stored procedure and link to this from the website..Updating the table every couple of minutes ?
Any advice please ?
Ray..

View 4 Replies View Related

Performance - Slow Load Times For SSIS Packages

Feb 13, 2006

Is there any information around what the SSIS packages are doing in the first 5-10 seconds of execution, and ways to speed this process up?

View 3 Replies View Related

Transact SQL :: Load Data For Last 15 Days By Decrementing Current Date And While Loop

Sep 21, 2015

I have a dynamic sql query where in I am comparing two tables and loading data for last 15 days. e.g today 2050921 then I am going to load till 20150906.

I pass on 2 variables @currentdate and @currentdate-1 to the query which are in date format 'yyyymmdd'

I need to do this for last 15 days how do I do this using while loop.

Note my date format is YYYYMMDD.

DECLARE @SQL VARCHAR(MAX)
@sql = ' insert into target 
select from table_1_currentdate a
LEFT JOIN Table_2_currentdate-1 b
on a.col1=b.col1  where b.col1 is null '

exec(@sql)

I have to use while loop and decrement it every time and load data for last 15 days comparing two tables. I tried so many times I am not getting it right .

View 3 Replies View Related

Integration Services :: Load Multiple Files From Table Using For Each Loop Container

Oct 28, 2015

I have a requirement to load multiple flat files in target table .

I have created the package which used to load files into target table using For each loop container.

But now requirement has been changed now I have to take only those files from table where status="Success" and max JobId. By the query I am to get those records which need to load into table.

Below query I am using to get the files which need to load.

select [JobLogKey],[SrcNm],[DestNm]
FROM [ConfigRep].[dbo].[JobLog]
Where [JobId]=
(Select Max(cast([JobId] as Int)) Jobid
FROM [ConfigRep].[dbo].[JobLog]
Where [JobStat]='Success')

Output:-

JobLogKey SrcNm   DestNm
268 H:Data PlatformSource FileClient2LocHGSSpecLocation.txt Location.txt
269 H:Data PlatformSource FileClient1LocHGSSpecLocation.txt Location.txt

I have to load using above 2 files which are under SrcNm. I have created one variable called FileToLoad as Object and mapping to result set of above query. I have create JobId,SrcNm and DestNm variable to catch the record at every loop. I have  created 2 For each Loop container 

Below screen shot of outer Foreach loop. Till here Its working fine. Inner for each loop container not executing any task under that. How to get it done.

View 3 Replies View Related

Trying To Load Multiple Excel Files Into A Db Table Using For Each Loop Container In SSIS

Oct 24, 2007

I am getting the following error when trying to load multiple excel files using for each loop container in SSIS, I tried to put the quotes in several different ways but still can't get rid of this error. I was able to successfully load single excel file, but when I use the for each loop container that's when I am having problems. Any help is greatly appreciated. Thx.

TITLE: Package Validation Error
------------------------------

Package Validation Error

------------------------------
ADDITIONAL INFORMATION:

Error at Package1 [Connection manager "SourceConnectionExcel"]: The connection string components cannot contain unquoted semicolons. If the value must contain a semicolon, enclose the entire value in quotes. This error occurs when values in the connection string contain unquoted semicolons, such as the InitialCatalog property.

Error at Package1: The result of the expression ""Provider=Microsoft.Jet.OLEDB.4.0;Data Source=" + @[User::Folder] + @[User::file] + ";Extended Properties="Excel 8.0;HDR=NO";"

" on property "ExcelFilePath" cannot be written to the property. The expression was evaluated, but cannot be set on the property.

(Microsoft.DataTransformationServices.VsIntegration)

------------------------------
BUTTONS:

OK
------------------------------

View 8 Replies View Related

Integration Services :: Foreach Loop Container To Load File Which Contains The String

Aug 26, 2015

In my SSIS package I am using Foreach loopcontainer to load multiple flat files.

Now my requirement is that I want to load only those file which contains %vendor%.In source folder I have many files but I am interested in to load only those file which contains the string %vendor% in file name.

View 2 Replies View Related

Load Fact Table, Very Slow With Lookup - Data Flow Transformation

Apr 9, 2008



Hello,


I have developed some packages to load data into "Fact" tables in the data warehouse.
Some packages are OK, other ones not. What is the problem?: some packages load fact tables with lots of "Lookup - Data Flow Transformation" into the "data flow task" (lookup against dimension tables) but they are very very slow, too much slow to be choosen as a solution.

Do you have any other solutions to avoid using "Lookup - Data Flow Transformation"? Any other solution (SSIS, TSQL and so on....) is welcome to speed up the Fact table loading process.

Thank in advance

View 7 Replies View Related

Not Able To Load The Application In Case Web Farm Garden When We Load Data Through Background Thread.

Dec 14, 2007

Hi,

Here I will describe my problem.
1. We are loading large amount of data from database on background thread which is starting on Application_start event in global.aspx.cs file.The data is later cached for subsquent request to improve the performance.
2. Now when we put the application on web farm garden, it is not able to load the application.
3. We are sending the request the servers through Router kind of application.
4 This application is working fine on single server enviornment.

Please help us.

Ajay Kumar Dwivedi

View 1 Replies View Related

SQL Server 2008 :: Difference Between FOR LOOP And FOREACH LOOP?

May 28, 2010

difference between FOR LOOP and FOREACH LOOP with example(if possible) in SSIS.

View 4 Replies View Related

Load All Data Without Knowing Old One Was Load In The Previous Time???

Apr 27, 2007

I just have done the SSIS example in the tutorial document included when install SQL 2005 ENT. I have a problem that whenever I test to run, the service load all data from source with out noticing about the data (I mean it load all the data to the destination), I do it several time and it continue to load all without checking. That mean the data is dublicated when the schedule run???



I think there should be a paramete or something like that to help the engine just load the new data to the destination. Could you help please?



Thank

View 3 Replies View Related

Loop Though Table Using RowID, Not Cursor (was Loop)

Feb 22, 2006

I have a table with RowID(identity). I need to loop though the table using RowID(not using a cursor). Please help me.
Thanks

View 6 Replies View Related

Foreach Loop Doesn't Loop

Mar 3, 2006

I have a foreach loop that is supposed to loop through a recordset, however it doesn't loop. It just repeats the same row, row after row.

I would like to look into the recordset variable but I can't because it is a COM object and the ADODB namespace is not available in the script task.

Any solution to this? anyone experienced anything similar

View 1 Replies View Related

Fishing For A Clue. To Loop Or Not To Loop

Jul 8, 2006

I have a table called Tbltimes in an access database that consists of the following fields:

empnum, empname, Tin, Tout, Thrs

what I would like to do is populate a grid view the a select statement that does the following.

display each empname and empnum in a gridview returning only unique values. this part is easy enough. in addition to these values i would also like to count up all the Thrs for each empname and display that sum in the gridview as well. Below is a little better picture of what I€™m trying to accomplish.

Tbltimes

|empnum | empname | Tin | Tout | Thrs |

| 1 | john | 2:00PM | 3:00PM |1hr |

| 1 | john | 2:00PM | 3:00PM | 1hr |

| 2 | joe | 1:00PM | 6:00PM | 5hr |

GridView1

| 1 | John | 2hrs |

| 2 | Joe | 5hrs |

im using VWD 2005 for this project and im at a loss as to how to accomplish these results. if someone could just point me in the right direction i could find some material and do the reading.

View 18 Replies View Related

ForEach Loop Or For Loop??

Feb 23, 2006

I have source and destination table names in the database(one table) and I need to read the source and destination tables one by one...

My Lookp table is like the following...

Srn srctable desttable

1 SRC1 DEST1

2 SRC2 DEST2

3 SRC3 DEST3

Now I want one package to load from source to destination.. how do I do it.. I dont know how to use....

How do I run the pacakge for each of the rows... ..............................

View 1 Replies View Related

Load A SSIS Package Via Web Service: The Package Failed To Load Due To Error 0xC0011008 Error Loading From XML.WHAT IS THAT?

May 19, 2006

Hello,

I have a big problem and i'm not able to find any hint on the Network.

I have a window2000 pc, VS2005,II5 and SQLServer 2005(dev edition)

I created an SSIS Package (query to DB and the result is loaded into an Excel file) that works fine.

I imported the dtsx file inside my "Stored Packages".

I would like to load and run the package programmatically on a Remote Scenario using the web services.

I created a solution with web service and web page that invoke the web service.

When my code execute:
Microsoft.SqlServer.Dts.Runtime.Application.LoadFromDtsServer(packagePath, ".", Nothing)

I got the Error:
Microsoft.SqlServer.Dts.Runtime.DtsRuntimeException: The package failed to load due to error 0xC0011008 "Error loading from XML. No further detailed error information can be specified for this problem because no Events object was passed where detailed error information can be stored.". This occurs when CPackage::LoadFromXML fails.

The error message doesn't help so much and there is nothing on the www to give me and advice....

Is it a SSIS problem???

Thank you for any help!!

Marina B.



View 10 Replies View Related

SQL Validation

Mar 10, 2004

I am working on a query application, and I want to do syntax validation before I submit the dynamically sql to the database. The expression will include ANDs, ORs, IN, (,),>,<,etc. Anyone done this already? any code snippets?

Thanks!

View 1 Replies View Related

Xml Validation

Aug 10, 2006

Hi

I am running the following:

DECLARE @h uniqueidentifier --conversation handle

DECLARE @msg xml; --will hold the message

BEGIN DIALOG CONVERSATION @h

FROM SERVICE [ewx.co.za/Service/store001_ewx_sb_service]

TO SERVICE 'ewx.co.za/Service/ewx_sb_hub_service'

ON CONTRACT [ewx.co.za/Contract/ewx_Contract];

SET @msg = '<InventoryUpdate>

<TitleId>STORE001TEST1</TitleId>

<Quantity>7777</Quantity>

</InventoryUpdate>';

SEND ON CONVERSATION @h

MESSAGE TYPE [ewx.co.za/Message/ewx_sendmsg](@msg);

Now to test errors comming back on the aueue i sed to make the xml tags wrong, then the target would send a error back on the queue with xml validation failed (both queues have validation well_formed_xml). However now in testing i cannot even send the message i get an invalid xml error straight away, i am not sure why this is , i know the xml is not valid but the send used to work and i would get an erro rback, as the xml is validated by the ttarget, but this no longer works it ails strainght away, with no thing in any queue. What could be causing this ?

Thanx

View 1 Replies View Related

VALIDATION

Mar 27, 2007

I think I have read online a recommendation about not using XML VALIDATION in a production environment, due to performance reasons. Is it recommended using other that NONE validation in production, and is there available documentation for a scale that grades performance hits for various types of validations?


Thanks

View 1 Replies View Related

SQL Validation Before Committing Row

Apr 10, 2007

Afternoon all,I want my SQL SP to do some validation on a form submit to do the following before committing to the table.If email address (txt.Email.Text) doesn't exist in the table, commit values.if email address (txt.Email.Text) does exist and option (radOptions.SelectedValue) equals 1, print message to say 'you're already subscribed'if email address (txt.Email.Text) does exist and option (radOptions.SelectedValue) equals 0, print message to say 'you're not subscribed'if email address (txt.Email.Text) does exist and option (radOptions.SelectedValue) equals 1 or 0, update row to 0 or 1 (depending on subscribe or unsubscribe - 1 = subscribed, 0 = unsubscribed) The simple SP is currently:ALTER procedure [dbo].[sp_customerSignups]@name varchar(50),@email varchar (50),@subscribed intasBEGININSERT INTO tblCustomerSignups(Name, EmailAddress, Subscribed)VALUES(@name, @email, @subscribed)END Does anyone have the correct syntax for this?Thanks,Brett

View 4 Replies View Related

XML Validation Error

Sep 21, 2007

Hi,

I have following XML-file and XML-schema:

 

<template xmlns="http://www.mycompany.com/xsd/PageTemplate">
...
</template> 
 <xs:schema attributeFormDefault="unqualified" elementFormDefault="qualified" xmlns:xs="http://www.w3.org/2001/XMLSchema" xmlns="http://www.mycompany.com/xsd/PageTemplate" targetNamespace="http://www.mycompany.com/xsd/PageTemplate"><xs:element name="template"><xs:complexType>...</xs:complexType>
</xs:element></xs:schema>

When I create and query the XML file using LINQ, everything works just fine. I also get no compilation errors.
But when I try to add the XML file to a database-field of type xml(CONTENT dbo.Common7), I get following error: 
XML Validation: Declaration not found for element 'http://www.mycompany.com/xsd/PageTemplate:template'. Location: /*:template[1] Any ideas? Thanks,Thomas 

View 1 Replies View Related

SQL Server Validation

Jul 10, 2006

Hey all,

Quick question regarding validation!

I have a table of contact details containing the usual name, adresss etc fields.

I want to validate the fields Country and telephone together so that fopr example if country = 'UNITED KINGDOM' the telephone has to begin with +44 if it doesnt i want it to add the +44!

I can do this through writing a little program but just wanted to explore the possibility of doing this with SQL or functions already available with MSSQL Server

cheers

ed

View 3 Replies View Related

Date Validation

Feb 8, 2005

i have two params..
@startdate datetime,@enddate datatime

1.what's the best way to validate them
2.Best way to check whether @startdate is less than @enddate

View 1 Replies View Related

String Validation

Feb 27, 2004

I need to make sure that a string contains at least one number, at least one letter and has no repeating chars. Anyone done something similar? What's the best way to do it? Thanks.

View 14 Replies View Related

Date Validation

Mar 17, 2004

I am new to SQL SERVER,

Can anybody help me to write a procedure to validate a given string like
09461020(Y2K compliant date), I need to check wether this is a valida date or not??
0946 = should convert to 1946
10 = month
20 is the date.

If it is 20461020.....then year = 2046, month = 10 and day = 20..and it is valid.

procedure should return validate date or not ...return TRUE OR FALSE.

View 1 Replies View Related

String Validation

May 23, 2008

Is there a way in TSQL to validate a string to ensure it has only numbers or letters (no special characters like @,%, etc)?

I want to write a function that validates a number that should only contain letters and numbers. The function should return 1 if valid 0 if not. If it contains anything other than 0-9 or A-Z or a-z then it should return 0.

Any ideas on how to do this in TSQL? I could do this in C# easily with regular expressions, but I don't want to use SQLCLR or any other external resource like that...just straight TSQL.

Thanks!

View 2 Replies View Related

Email Id Validation

Jun 13, 2008

How can we validate email id in sql server 2005..

i use the following code...

if((len(@default_to_mail)-len(replace(@default_to_mail,' ','')))=0 and @default_to_mail like ('%@%.%'))

its not validating correctly...
can u give me any solution.


Thanks in advance.


cool...,

View 4 Replies View Related

Email Validation

Feb 23, 2006

I'm sorry to repost this, but I didn't get an answer yet for this seemingly simple issue. I need to make sure that an email address has the @ symbol before I insert it from one table to another. Does anyone know the syntax? What I have that isn't working is -

select * from table where email charindex ('@',email) > 0

View 7 Replies View Related

DATE VALIDATION

Mar 7, 2007

Guys,
I hope I will be able to get this.

I'm looking a cripts/code that will validate date in my form when the user key in date. The date has to be in the following format 00/00/0000 (month/ day/ year)


HELP

Thk

View 3 Replies View Related

Date Validation

Mar 13, 2007

hi,
this is my table values:

select * from empmaster
emp.no fromdate todate
----------------------------------
E3/13/20074/7/2007
E4/13/20076/13/2007

i am going to pass some date for example 5/20/2007.

so i have to compare this 5/20/2007 which lies in between 3/13/20074/7/2007 or 4/13/20076/13/2007.if means i have to print not allowed other wise allowed. and the date should not be less(if 2/10/2007). like thi i have many from date and todate in my table. if i give 7/20/2007 means it should print allowed becos it doesnot falls in between any date o the emp.no=e. this should be depent on the emp.no also.


so please help me to do this.please show me query for this.or spc

View 10 Replies View Related







Copyrights 2005-15 www.BigResource.com, All rights reserved