File Watcher Task (Konessans) Vs For Each Loop Task
Dec 6, 2007
HI, I need to trigger some packages upon existance of specific files in a particular directory. Sound lkike the file watcher task (from SQLIS) would do the job but I am wondering what is the difference of using this tool instead of a for each loop container. I mean, If a file exists in a directory, the for each loop container will detect it. Since the file watcher is not a service, the package containing it needs to ne scheduled on a regular basis for the filewatcher to detect the file, right? So, a for each loop container would do the job? So, waht wouldbe the advantage of using the file watcher task?
Thank you,
Ccote
View 11 Replies
ADVERTISEMENT
Apr 30, 2008
Installed File watcher task, then created a new SSIS package by adding the file watcher task in the control flow, below are my setting's inside the file watcher task editor:
2. Options
Filter - *.*
Find Existing Files - True
Timeout warning - False
3. Path
Path input type: Direct imput
Path - \server1Sourcefolder
Include subdirectories - False
Actually my understanding about this task is:
This file watcher task will constantly check for the file and then will execute the SSIS package.
If my understanding is correct, then where do we specify the name of the SSIS package?? And also do we need this SSIS package which has the File watcher task to run every 5 minutes or so, so that it can keep checking for the file existence.
Please help, thanks!!
View 9 Replies
View Related
Sep 24, 2007
In the file watcher task in the filter section does it look for exsiting file name which we have mentioned or is it possible to give first few letters of the file name like
pat*.txt
Please let me know
View 1 Replies
View Related
May 25, 2006
Where the earth is this task??? I am not be able to see it anywhere!!
Thanks for any input,
View 1 Replies
View Related
Jul 7, 2006
Hey there
Ive built an SSIS package which generates a file from a legacy system and then downloads the file into a designated folder on the server. I need the file watcher task to wait for a the file to completely finish loading before it says it is complete. Currently, as soon as the file is created, the WMI step finishes.
Any help would be greatly appreciated!
Kind Regards
David
View 4 Replies
View Related
Jan 28, 2007
Hi All,
I have a problem in which I want the run an SSIS based on a file being dropped in a directory. I've tried the WMI event watcher as well as the File Task Watcher component. The problem that I'm seeing is that if the process is currently processing in the pipeline while another file dropped, the newly dropped file doesn't get picked up. Is there a way to create a FileWatcher Task in SSIS which will spawn a SSIS job and return immediately for watching files?
View 3 Replies
View Related
Jun 21, 2007
OK. I give up and need help. Hopefully it's something minor ...
I have a dataflow which returns email addresses to a recordset.
I pass this recordset into a ForEachLoop configuring the enumerator as (Foreach ADO Enumerator). I also map the email address as a variable with index 0.
I then have a Execute SQL task which receives this email address as a varchar variable (parameter 0) which I then use in my SQL command to limit the rows returned. I have commented out the where clause and returned all rows regardless of email address to try to troubleshoot this problem. In either event, I then use a resultset to store the query result of type object and result name 0.
I then pass this resultset into a script variable to start parsing the sql rows returned as type object. ( I assume this is the correct way to do this from other prior posts ...).
The script appears to throw an exception at the following line. I assume it's because I'm either not passing in the values properly or the query doesn't return anything. However, I am certain the query works as it executes just fine at the command prompt.
Try
ds = CType(Dts.Variables("VP_EMAIL_RESULTS_RS").Value, DataSet)
My intent is to email the query results to each email address with the following type of data by passing the parsed data from the script to a send mail task. Email works fine and sends out messages but the content is empty. I pass the parsed data as string values to the messagesource and define the messagesourcetype as a variable in the mail task.
part number leadtime
x 5
y 9
....
Does anyone have any idea what I might be doing wrong?
thanks
John
View 5 Replies
View Related
May 30, 2006
hello
i would like to know how i can retrieve the file path+name of a file detected by the WMI Event Watcher task, so that I can assign this to a variable for an Import task
thanks
View 8 Replies
View Related
Oct 4, 2007
I am trying to write a SSIS to distribute files to our client directories. In my source directory ( C:ClientPending ) i have a variable amount of file with variable file names and extensions. I need to sort these and place then in to different cient direcories. Each file name will contain the clients Referance Number ( ie Q123456.007 , A123456.exe, S123456.523 etc ). this clients directory would be C:ClientOutbound123456
Im presuming that i need a for loop container and a system file task, and a variable that i can parm in the ClientRef, i just dont know how to configure it.
Can anyone shed some light on this?
ta
View 1 Replies
View Related
Apr 6, 2007
I would like this task to watch for a particular text file in the particular folder.
here
Connection managers:
servename=\localhost
namespace=
ootcimv2
WMIQuerySourceTypeirectinput
WMIQuerySource:
SELECT * FROM __InstanceCreationEvent WITHIN 10 WHERE Targetinstance ISA "CIM_DirectoryContainsFile" and TargetInstance.GroupComponent= "Win32_Directory.Name="c:\\ImportData\\ est.txt""
Rest of the fields i leave as they are..
I get this error
[WMI Event Watcher Task] Error: Watching for the Wql query caused the following system exception: "Unparsable query.". Check the query for errors or WMI connection for access rights/permissions.
I am looking for test.txt on my local machine.
What do i need to do??
View 11 Replies
View Related
Jul 10, 2007
I want to watch a directory 24 hours a day to launch SSIS packages as files arrive. There are several options I have found to accomplish this:
Use the WMI Event Watcher task in SSIS to launch the packages as files arrive. Leave the package with the WMI Event Watcher task running all day long everyday
Create a Windows service that uses WMI to detect file arrival and launch packages
Schedule SQL Server Agent to run the package when a WMI event is raised
What is the best practice? Are there better approaches that I have not listed?
View 15 Replies
View Related
Apr 4, 2007
I am using this task as the first task in my control flow.I want to check if a particular file exists in the particular location.IF exisists run the package else no.
How do i configure this...please help..
View 1 Replies
View Related
Jan 18, 2007
Hi ALL,
Getting Access Denied To FileName Error When Using the Execute Sql Task (With File Connection) into a Foreach Loop Container.
Please Note :
I have a folder containing .sql files. I have to dynamically loop through the files and send them as a File connection Folder to the Execute Sql Task.
When I run this Package I am getting the follwoing error :
[Execute SQL Task] Error: An error occurred with the following error message: "Access to the path 'C:ProjectsFuzzy Lookup DataFlow ExampleScripts' is denied.".
Also I have logged in to the machine as Administrator and to Sql Server with sa.
Please help.
Thanks.
Regards,
Salil
View 1 Replies
View Related
Mar 28, 2007
In the Control flow tab, I have an Execute SQL Task that outputs full Result set into a variable of an object type. Now how can I write the contents of the Full Result Set into a text file using Script Task. I also want to format the following way while I output into a file:
Column Name 1 : Column Value
Column Name 2: Column Value and so on
I tried writing the contents of the Object Variable into a file, but the file had an output of single word: System.__ComObject.
Code for Writing the Full Result Set into a Text File
Dim RSsqloutput as String = Dts.Variables("objVariable").Value.ToString
Dim strVal as String = "File completed on " & Now() & vbCrLf & "------------------------------------------------------" & vbCrLf
oLogFile.WriteAllText("C:MyFile.txt", strValue)
oLogFile.WriteAllText("C:MyFile.txt", rsSQLOutput)
I went through this link that explains how to write XML Result Set into a File, But this doesn't help as it writes in XML format.
Would you please give me a hint of code how I can go upon.
View 7 Replies
View Related
Nov 7, 2007
A common issue that I run across with clients is they want only want to process a file if it's finished transmitting to the server. This SQL Server 2005 task reads the properties of a file and writes the values to a series of variables. For example, you can use this task to determine if the file is in use (still be uploaded or written to) and then conditionally run the Data Flow task to load the file if it's not being used. You can also use it to determine when the file was created in order to determine if it must be archived.
http://www.pragmaticworks.com/filepropertiestask.htm
View 5 Replies
View Related
Apr 8, 2008
I have been testing with the WMI Event Watcher Task, so that I can identify a change to a file.
The WQL is thus:
SELECT * FROM __InstanceModificationEvent within 30
WHERE targetinstance isa 'CIM_DataFile'
AND targetinstance.name = 'C:\Program Files\Microsoft SQL Server\MSSQL.1\MSSQL\Backup\AdventureWorks.bak'
This polls every 30 secs and in the SSIS Event (ActionAtEvent in the WMI Task is set to fire the SSIS Event) I have a simple script task that runs a message box).
My understanding is that the event polls every 30 s and if there is a change on the AdventureWorks.bak file then the event is triggered and the script task will run producing the message.
However, when I run the package the message is occurring every 30s, meaning the event is continually firing even though there has been NO change to the AdventureWorks.bak file.
Am I correct in my understanding of how this should work and if so why is the event firing when it should not ?
View 2 Replies
View Related
Jul 10, 2007
Hi all,
I got this problem and before post I've read many post similar in the forum (sorry for my english I try to do my best....i'm italian)
This is my scenario.
1) SQL
SELECT
UPPER(SUBSTRING(FileImportazione, 7, 50)) AS str,
Id
FROM Sources
WHERE (TipoImportazione = 0)
This is an SQL task and from this I create a Object variable "ObjDs" on scope package1
2) I introduce a loop (Foreach ADO Enumerator) on my object variable "ObjDs".
Here I mapping two variables "IDsources" and "str" (as you caqn see from my first sql)
Now I set a breakpoint into the beginning of my cicle and really see the dinamic change of the values from "watch" windows.
Now start my problem.
Into this loop I must do "X" operation....but first of all I must Update a table...so I simple SQL TASK...on the dinamic ID of my variable.
Ok, I put a SQL tast into the loop, set SQLsourceType = Variabile, resulset = none, and create an expression like this (SQLstatementsource):
"UPDATE New_pangea.dbo.Sources SET InAggiornamento = "+ (DT_WSTR, 4) @[Utente::IDsources]
If I don't cast (DT_WSTR, 4) I got always an error...
The data types "DT_WSTR" and "DT_I4" are incompatible for binary operator "+". The operand types could not be implicitly cast into compatible types for the operation. To perform this operation, one or both operands need to be explicitly cast with a cast operator.
Attempt to set the result type of binary operation ""UPDATE New_pangea.dbo.Sources SET InAggiornamento = " + @[Utente::IDsources]" failed with error code 0xC0047080.
so I put and validate myexpression.
before my global variable @[Utente::IDsources] change the value to true of EvalutateAsExpression.
Something goes always wrong...When I try debug...this is the error
Errore in Execute SQL Task: Failed to lock variable "UPDATE New_pangea.dbo.Sources SET InAggiornamento = 0" for read access with error 0xC0010001 "The variable cannot be found. This occurs when an attempt is made to retrieve a variable from the Variables collection on a container during execution of the package, and the variable is not there. The variable name may have changed or the variable is not being created.".
Errore in Execute SQL Task: There were errors during task validation.
(Microsoft.DataTransformationServices.VsIntegration)
Hope someone can help me, thanks Alen
View 2 Replies
View Related
Apr 24, 2006
hi guys,
i'm trying to do a bulk insert through a t-sql which accepts 3 params.
i frist run a SQL statement to get a list of DB's to update through a SQL task, then i save the output to a variable object. which is been used by the For each loop.
i've done the mapping with the object fine. but i want to call a SQL task within for each loop, and parse the above results as parameters. i tried parsing it as
exec mysp3 @[User::fundid], @[User::dbCode], @[User::subfund]
where fundiid, dbcode and subfund are user defined variables that gets updated by the ado recordset.
this fails with the error msg:
SSIS package "SP test.dtsx" starting.
Error: 0xC002F210 at Execute SQL Task, Execute SQL Task: Executing the query "exec ConnectODBC @[User::fundid], @[User::dbcode], @[User::subfund] failed with the following error: "Incorrect syntax near 'User::fundid'.". Possible failure reasons: Problems with the query, "ResultSet" property not set correctly, parameters not set correctly, or connection not established correctly.
Task failed: Execute SQL Task
Warning: 0x80019002 at Shred the contents of the variable: The Execution method succeeded, but the number of errors raised (1) reached the maximum allowed (1); resulting in failure. This occurs when the number of errors reaches the number specified in MaximumErrorCount. Change the MaximumErrorCount or fix the errors.
SSIS package "SP test.dtsx" finished: Failure.
i also tried
exec mysp3 ?, ?, ?
then mapping the inputs still doesn't work
same thing with
Error: 0xC002F210 at Execute SQL Task, Execute SQL Task: Executing the query "exec ConnectODBC ?,?,? failed with the following error: "Syntax error, permission violation, or other nonspecific error". Possible failure reasons: Problems with the query, "ResultSet" property not set correctly, parameters not set correctly, or connection not established correctly.
Task failed: Execute SQL Task
View 3 Replies
View Related
Nov 8, 2006
Hi,
My Foreach Loop container has 10 different task inside. I want to execute the first task only one time. I have a variable with increases for each repition. How can I put precedence contstraint on the first task so that it should execute only first time and other task has to execute all the time.
Thanks
View 4 Replies
View Related
Jan 29, 2007
In Dts you can loop a package by creating an Active-X script with something like this ...
"oPackage.Steps("DTSStep_1").ExecutionStatus = DTSStepExecStat_Waiting"
"oPackage.Steps("DTSStep_2").ExecutionStatus = DTSStepScriptResult_DontExecuteTask"
What is the way to do in SSIS? Depending on a condition i want to rerun my package but have not found a smooth way to do it. I guess you can do it in a better way than using the ActiveX Script Task
have a nice day
/Erik
View 4 Replies
View Related
Oct 18, 2007
I want to iterate through a for loop and run some SQL update statements:
I have a variable wk_day that has a scope of the container
I want it to run this code:
update daily_sls_curr set mtd_ty_sls = (
select sum(ty_sls) from daily_sls_curr ci
where ci.fmonth = (select max(fmonth) from daily_sls_curr)
and wk_day <=@wk_day
and ci.store = daily_sls_curr.store
group by store)
where fmonth = (select max(fmonth) from daily_sls_curr)
and wk_day = @wk_day
repeatedly from the loop for 35 days of the fiscal month
When I parse the query, it gripes at the variable
How do I set up the query to work with the variable?
Thanks
JPS
View 7 Replies
View Related
Jun 14, 2006
Okay, I have a For each loop container that has an enumeration of folder locations. I run a script to filter the files in those folders and prepare them to the next task to be copy. I want the script to "continue" the loop if none of the files meet the criteria. Can someone point me into the right direction? I need some way to advance through the loop when this happens.
Thanks!
View 1 Replies
View Related
May 3, 2007
Okay, can someone explain why when I execute children packages within a loop that only a couple of them work?
When I first started this I had Child Package 1 and Child Packge 2 working from the Parent Package 1. It would loop through twice (I had 2 organizations in my outer loop) and pass in the Parent Package variables correctly.
I am using a simple "Foreach Item Enumerator" with a collection of string enumerators that are the Names of the children packages - these are assigned to a variable that is scoped to the outer loop.
Now, when I add a new Child Package 3 and set it up the same as the other 2, when it goes to execute Child Package 3 it fails with the error:
"Error: The connection manager "[My Package Name].dtsx" is not found. A component failed to find the connection manager in the Connections collection. "
What in the world have I done? I was under the assumption that since the first 2 packages worked, that any other packages I added would work the same.
View 3 Replies
View Related
Aug 29, 2006
I have a .bat file that calls a script file to log in to a server and upload a file to my pub account. When I run the .bat file it just executes the command over and over and never executes the script file. Does anyone know why? Am I missing something in my script or .bat file commands below?
my .bat file contains this--->ftp -s:script.txt
my script file contains this:
open myserver.com
username
password
put W:UPLOADdoc.txt
bye
exit
Thanks,
Mike
View 6 Replies
View Related
May 9, 2008
Hi I have created a web service task that sits inside of a foreach loop. The loop is a ado enumerator. This loop is going to loop through about 35,000 records and make 35,000 web service calls.
So far through my testing, I have gotten it to read and write everything correctly, but each call to the web service takes about 1-2 seconds and maxes out the CPU. This would take almost all day for this web service to run at 100% CPU power all day. We want to be able to run this a few times a day or at least once a day.
I have disabled all of the other tasks in the foreach loop so it just calls the web service task and found that it's the web service task that takes up 100% of the CPU and takes 1-2 seconds to run. I have also tried creating a mock web service on the server that just sends back the same mock data, so it's extremely fast, but the Web Service Task still maxes out the CPU and takes 1-2 seconds per call.
I have tried this out on 2 different servers, all with a few gigs of ram they're not using. It seems more like it's a CPU thing, or that the Web Service Task has a high overhead that each time in the loop it takes 1-2 seconds to create the task.
Anyone have any advice?
Thanks,
-Will
View 6 Replies
View Related
Mar 3, 2008
I have written a recursive loop, the function using dataview.rowfilter ="something".
On some servers I am getting a weired issue. when I am interating through rows in dataview its throwing me an error.
"No row at position 1", I am using rowView("ColumnName")
strange thing is the underlying data row is returning me the value but row view is throwing an error.
I thought its an issue with SP1, but after installing SP2 still failing..
View 1 Replies
View Related
Oct 5, 2007
Hi,
I have a FOR EACH LOOP that loops through a bunch of folders in search of particular filename (including path). That part works fine.
Then, INSIDE the FOR EACH LOOP, I have a script task that evaluates the filename and grabs the date portion of the filename. That part works fine too.
Now, here's my dilemma. If the date portion of the file name IS GREATER THAN the the value of another date in a variable, I want it to load the file into a table using a data flow task.
So, this is what my package looks like:
[ FOR EACH LOOP (Script task --- evaluate the date using an expression (see below) ---> Data flow task) ]
My expression looks like this:
@[User::fileDate] > @[User::maxLoadDate]
You think that this would work, right??
However, after the VERY FIRST iteration of the loop, the script task turns green, then it goes right to the data flow task, which then says that the file name is "", and can't be loaded. (I'm not sure why this is occuring)
My question is, is my logic correct? Can I use an expression inside the for each loop and have it work?
View 1 Replies
View Related
Aug 22, 2007
All:
I am sure I am missing something really silly but I am not able to figure out what. The For Each Loop uses an ADO Enumerator and passes variable values to a data flow. In executing the package the loop runs fine but nothing is happening to the data flow. When I move the data flow out of the loop it runs fine. What is going on?
Thanks!
desibull
View 14 Replies
View Related
Feb 15, 2008
Hi, I'm trying to loop thru a table and insert records into another table in ssis. So far I have been able to get the data using a execute sql task set up to store the full result set into a variable called data. I then drug a foreach loop container out and selected the foreach ADO enumerator and used my variable data as the ADO object source variable. I then set up a new variable under variable mappings with index 0 to get the collection values. How do I take that variable and update another table using another sql task inside the foreach loop container? Is this possible?
thanks,
View 8 Replies
View Related
Oct 19, 2015
I have ForEach Loop using Foreach File Enumerator. Within this loop I have SQL Task containing an Insert statement. When I run the Insert statement in query builder the transaction inserts data into a table as expected.
However, when actually running the process I am getting the error message:
Executing the query "INSERT INTO dbo.TEST_TABLE
..." failed with the following error: "Value does not fall within the expected range.". Possible failure reasons: Problems with the query, "ResultSet" property not set correctly, parameters not set correctly, or connection not established correctly.
I currently have the ResultSet to "None" and have defined the parameter I am using. Where the process seems to joke is on my file_Name variable will I am trying to insert only part of the file name.
My insert statement looks as follows:
INSERT INTO dbo.TEST_TABLE
(IID, AN8, File_Type, IB_OB, File_Name, Processed_Flag, ReceiptDateTime)
VALUES
(
'1111',
'123',
'888',
'IB',
RIGHT(LEFT(?, LEN(?) - 4), LEN(?) - 24),
'I',
GETDATE())
View 0 Replies
View Related
Apr 23, 2008
I everyone,
I'm learning SSIS from a book but the most simple example is not working.
The "For each task" I assigned "xyz" as name.
I added a variable onto the "variables" window and called it "counter" Its scope was set to "xyz".
Inside the "for loop" ("xyz") editor i assigned values as follows:
"initexpression"-"@counter=0"
"eval expression"-"@counter<5"
"assign expression"-"@counter=@counter+1"
I added "Script task" onto the "For each" container and into the "Script task editor" I assigned
the following values:
"ReadOnlyVariables"-"counter"
Its design script is set to:
Imports System
Imports System.Data
Imports System.Math
Imports Microsoft.SqlServer.Dts.Runtime
Public Class ScriptMain
Public Sub Main()
Dim variables As Variables
If Dts.Variables.Contains("Counter") = True Then
Dts.VariableDispenser.LockOneForRead("Counter", variables)
End If
MsgBox("You are in iteration: " & CStr(variables("Counter").Value))
Dts.TaskResult = Dts.Results.Success
End Sub
End Class
Runnig it i get the following error message:quote:
"DTS Script task has encountered an exeption in user code:
Project name:ScriptTask_e7d98dbad0de4041bcdc9079a5c2fa65
Object reference not set to an instance of an object.."
The line where the error occurs is from within the above script:
"MsgBox("You are in iteration:" " & CStr(variables("Counter").value))
Anyone understands what that means? What is "object reference" and how do i set it to an instance of an object?
Thanks
View 4 Replies
View Related
Sep 10, 2007
Hello All
I was trying to insert some row from one table to another of different database.
I was using Execute SQL task along with Foreach loop container.
In my execute SQL task I am using this query
SET IDENTITY_INSERT dbo.Table1 ON
INSERT INTO dbo.Table1
SELECT * FROM DB2.dbo.Table2
WHERE TableKey = ?
When executed I get this error:
failed with the following error: "Syntax error, permission violation, or other nonspecific error". Possible failure reasons: Problems with the query, "ResultSet" property not set correctly, parameters not set correctly, or connection not established correctly.
While the same query when executed in Management Studio Its successful.
The properties I set
For Each Loop Editor Settings:
1) Collection: a) Enumerator Set to ForEach ADO Enumerator
b) ADO Object Source Variable: User:bjectVariablename
c) Checked Rows in the first table
2) Variable Mapping: New Int Variable2 and Index = 0 to set it to first colunm.
3) Expression: Left blank
Execute SQL Task Editor:
1) General: a) Timeout : 0
b) CodePage: 1252
c) Result Set: None
d) SQLSourceType: Directinput
e) SQL Statement: SET IDENTITY_INSERT dbo.Table1 ON INSERT INTO dbo.Table1 SELECT * FROM DB2.dbo.Table2 WHERE TableKey = ?
f) BypassPrepare: False
2)Parameter Mapping: Variable Name : New Integer variable2 selected
Direction: Input
DataType: Long
ParameterName: 0
Can somebody help me in this regards.
Reference:
a) http://www.whiteknighttechnology.com/cs/blogs/brian_knight/archive/2006/03/03/126.aspx
View 10 Replies
View Related
Jun 4, 2007
Hi,
I am using an XML task for validating the XML data against the XML Schema. Let me describe the scenario, I am working on.
I have XML Files with different schema. I have different XSD files for each XML file type. I have used a for loop container which has a script task and an XML task. Script task gets the XML File and XSD file contents into variables which are, in turn, used by XML Task for validation.
XML task is configured in the option "Validate". I have provided the XML Data in variable and XSD file content in other variable. XML Task stored the result in another variable. FailOnValidationFail property set to false.
Every thing works fine until result of XML Task is false. If XML Task fails for one of the files, it fails for each subsequent files. If first file does not compile to schema and other files do, then XML task fails all the files because first file failed.
Is this a problem with XML Task or I have missed any property. Please, guide me through.
View 3 Replies
View Related
