FTP Task And Temporary Internet Files
Apr 27, 2006
I have created a package within SQL Server SSIS which includes an FTP Task, deployed it to our SQL Server (2005 SP1) msdb database and am running this job under SQL Agent on Windows Server 2003. Due to company security requirements this job has to be run under a service account within SQL Agent. The problem with this is that even though a directory is specified within the FTP Task to place any downloaded files into, the files are first written to the TIF (Temporary Internet Files) directory of "Default User" which is on the system drive. Based on corporate standards the system drive (C:) on our servers are only configured with enough space for the OS and other system files. All of the files being transferred are compressed, but some are still well over 1GB in size. The result is that many of our downloads are failing due to the system drive running out of space.
I have attempted to run IE by using "Run As" with the service account credentials, and have changed the location of the TIFs to a different drive, rebooted and verified the settings. When the SQL Agent job was run again, the files were still being written to the "Default User" directory on the system drive. I also created a new template account with the TIFs pointing to a non-system drive and used the User Profiles functionality of System Properties to copy the new template account to "Default User", but still the files are being written to the system drive.
My questions are:
is there a way to stop the FTP Task from using TIF (i.e. just directly write the file to the location specified)
is there a best practice around how to setup a service account and have it create a proper user profile that can be managed separate from "Default User"
short of specifying during the OS install, is there a way to move the "Default User" profile directory to a different drive
View 3 Replies
ADVERTISEMENT
Jul 9, 2004
I created a DTS FTP task to connect to an FTP site. The problem I'm having is the folder I want to access you cannot see under the Files Tab. So if I put the ftp site/subdirectory in the FTP Site box, I get the error "Unable to connect to internet! Please check the source site information, your username and password"
If I go to the FTP site/subdirectory in IE all the files show up fine. I believe this is limitation of DTS, but I would like to hear from you all first.
To see what error I'm talking about:
Create a DTS FTP task
Use: ftp.microsoft.com/bussys/ with no username or password
Click the Files Tab
Normally you can just put the server name in, and see all the folders/files, but the way my FTP site is set up I cannot see the folder I want, I can only access it.
View 1 Replies
View Related
Feb 1, 2007
Why are some SSIS files, generated by the Import/Export Data wizard put into the local users temp folder? Why are these not compiled with the package when the solution is built?
Is there some setting I am missing?
This architecture is kind of silly, as the server always needs access to the temp folder on the local machine to run.
How can I get these temp files packaged with the rest of the package and deployed to the server so the server can run independent of the machine I develop the package on?
Thanks,
Jeff
View 8 Replies
View Related
Sep 17, 2014
I've been using replication for a long while now but have never come across this error. It's a basic transactional replication from ServerA to ServerB, where ServerA is also the distributor. Everything had been running fine on it until yesterday, when this error started popping up and no further transactions could be delivered.
After some quick googling I was able to determine that the distribution agent account needed write access to C:Program FilesMicrosoft SQL Server100Com. According to the MSFT article it's because the distribution agent is running under a non-default profile. I didn't change this. However, what I did change around the time that these errors started occurring was the server's Replication Max Test Size setting. It would be far too coincidental for this to not be the cause, but what I don't understand is *why* that would have changed it.
how do I change this? It is definitely not preferable to create temp files in this directory in our environment.
View 1 Replies
View Related
Feb 2, 2007
I have a virtual machine that previously had SQL Server 2005 installed. Since I changed the computer name, I have to reinstall SQL. I removed all the SQL components and am trying to reinstall the database and reporting components. The installation is failing on the "Removing temporary files" step. According to the setup progress screen, the following components have been installed:
OWC11
SQL Server Bakcward-Compatibility Files
SQL Server Database Services
Reporting Services
Visual Studio Integrated Development
SQL Server Books Online
SQLXML4
Workstation Components, Books Online - this step is still in "configuring components" status.
The virtual machine is running Windows 2003 Server SP1 with all current patches, using just under 2 GB of memory (1920 MB of RAM), the host is Windows XP SP2 with all current patches, with 4 GB of memory.
I had no problem doing this procedure on a different virtual machine, so I'm not sure what the problem is here.
View 2 Replies
View Related
Apr 24, 2008
In the first step of my SSIS package I need to get files from FTP and dump it/them in a local directory, but it's more than that, the logic is like this:
1. If no file(s) found, stop executing and send email saying no file(s) found;
2. If file(s) found, then compare it/them with existing files in our archive folder; if file(s) already exist in archive folder, stop executing and send email saying file(s) already existed, if file(s) not in archive folder yet, then transfer it/them to the local directory for processing.
I know i have to use a script task to do this and i did some research and found examples for each of the above 2 steps and not both combined, so that's why I need some help here to get the logic incorporated right.
Thanks for the help in advance and i apologize for the long lines of code!
example for step 1:
----------------------------------------------------------------------------------------------------------
' Microsoft SQL Server Integration Services Script Task
' Write scripts using Microsoft Visual Basic
' The ScriptMain class is the entry point of the Script Task.
Imports System
Imports System.Data
Imports System.Math
Imports Microsoft.SqlServer.Dts.Runtime
Imports Microsoft.SqlServer.Dts.Pipeline.Wrapper
Imports Microsoft.SqlServer.Dts.Runtime.Wrapper
Imports Microsoft.VisualBasic.FileIO.FileSystem
Imports System.IO.FileSystemInfo
Public Class ScriptMain
' The execution engine calls this method when the task executes.
' To access the object model, use the Dts object. Connections, variables, events,
' and logging features are available as static members of the Dts class.
' Before returning from this method, set the value of Dts.TaskResult to indicate success or failure.
'
' To open Code and Text Editor Help, press F1.
' To open Object Browser, press Ctrl+Alt+J.
Public Sub Main()
Dim cDataFileName As String
Dim cFileType As String
Dim cFileFlgVar As String
WriteVariable("SCFileFlg", False)
WriteVariable("OOFileFlg", False)
WriteVariable("INFileFlg", False)
WriteVariable("IAFileFlg", False)
WriteVariable("RCFileFlg", False)
cDataFileName = ReadVariable("DataFileName").ToString
cFileType = Left(Right(cDataFileName, 4), 2)
cFileFlgVar = cFileType.ToUpper + "FileFlg"
WriteVariable(cFileFlgVar, True)
Dts.TaskResult = Dts.Results.Success
End Sub
Private Sub WriteVariable(ByVal varName As String, ByVal varValue As Object)
Try
Dim vars As Variables
Dts.VariableDispenser.LockForWrite(varName)
Dts.VariableDispenser.GetVariables(vars)
Try
vars(varName).Value = varValue
Catch ex As Exception
Throw ex
Finally
vars.Unlock()
End Try
Catch ex As Exception
Throw ex
End Try
End Sub
Private Function ReadVariable(ByVal varName As String) As Object
Dim result As Object
Try
Dim vars As Variables
Dts.VariableDispenser.LockForRead(varName)
Dts.VariableDispenser.GetVariables(vars)
Try
result = vars(varName).Value
Catch ex As Exception
Throw ex
Finally
vars.Unlock()
End Try
Catch ex As Exception
Throw ex
End Try
Return result
End Function
End Class
example for step 2:
-------------------------------------------------------------------------------------------------------
' Microsoft SQL Server Integration Services Script Task
' Write scripts using Microsoft Visual Basic
' The ScriptMain class is the entry point of the Script Task.
Imports System
Imports System.Data
Imports System.Math
Imports Microsoft.SqlServer.Dts.Runtime
Public Class ScriptMain
' The execution engine calls this method when the task executes.
' To access the object model, use the Dts object. Connections, variables, events,
' and logging features are available as static members of the Dts class.
' Before returning from this method, set the value of Dts.TaskResult to indicate success or failure.
'
' To open Code and Text Editor Help, press F1.
' To open Object Browser, press Ctrl+Alt+J.
Public Sub Main()
Try
'Create the connection to the ftp server
Dim cm As ConnectionManager = Dts.Connections.Add("FTP")
'Set the properties like username & password
cm.Properties("ServerName").SetValue(cm, "ftp.name.com")
cm.Properties("ServerUserName").SetValue(cm, "username")
cm.Properties("ServerPassword").SetValue(cm, "password")
cm.Properties("ServerPort").SetValue(cm, "21")
cm.Properties("Timeout").SetValue(cm, "0") 'The 0 setting will make it not timeout
cm.Properties("ChunkSize").SetValue(cm, "1000") '1000 kb
cm.Properties("Retries").SetValue(cm, "1")
'create the FTP object that sends the files and pass it the connection created above.
Dim ftp As FtpClientConnection = New FtpClientConnection(cm.AcquireConnection(Nothing))
'Connects to the ftp server
ftp.Connect()
'ftp.SetWorkingDirectory("..")
ftp.SetWorkingDirectory("directoryname")
Dim folderNames() As String
Dim fileNames() As String
ftp.GetListing(folderNames, fileNames)
Dim maxname As String = ""
For Each filename As String In fileNames
' whatever operation you need to do to find the correct file...
Next
Dim files(0) As String
files(0) = maxname
ftp.ReceiveFiles(files, "C: emp", True, True)
' Close the ftp connection
ftp.Close()
'Set the filename you retreive for use in data flow
Dts.Variables.Item("FILENAME").Value = maxname
Catch ex As Exception
Dts.TaskResult = Dts.Results.Failure
End Try
Dts.TaskResult = Dts.Results.Success
End Sub
End Class
View 16 Replies
View Related
Jun 2, 2006
Hi,
I'm just starting off in SSIS and have a question that I can't find an answer to...
I'm loading in a number of files (in separate Data Flows) and performing some transformations on them before merging them back together. What I'm not sure about is what I should be doing with the data at the end of each of my "Import Data From XXXX Flat File" Data Flows. Am I better off using OLE DB Destinations (or SQL Server Destinations) and saving this intermediate data to temporary tables, or am I better off using a Raw File Destinations and saving this intermediate data to files? Or is there, perhaps, a better option that I'm currently unaware of?
If the Raw File Destination is the way to go, then isn't there a maintenance issue with cleaning up all the files created? And will there not be a management issue to ensure that there is sufficient disc space available on the drive you are saving to?
I'm a bit confused and overwhelmed by SSIS at the moment, so any help would be much appreciated!
Thanks in advance,
Lawrie.
View 3 Replies
View Related
Jan 21, 2006
Is there a way to do this with some sort of task ?
View 7 Replies
View Related
Mar 15, 2007
I cannot set a breakpoint in a Script task and have it complete successfully. I am running Vista 32-bit and only the workstation tools for SQL Server 2005 SP2. The steps to recreate are:
1) Create a new SSIS project/package.
2) Add a Script Task.
3) Set a breakpoint in the Script Task.
4) Run the package.
When I remove the breakpoint, the script task succeeds. When I put it back the task fails. The execution results say "Error: The script files failed to load." I completely uninstalled all SQL Server 2005-related entries using the Programs and Features control panel, rebooted, reinstalled the Workstation Tools, then applied a freshly-downloaded SQL Server 2005 SP2 and rebooted. If I change "RecompileScriptIntoBinaryCode" to false the script fails whether there's a breakpoint or not. If it's true (the default), the script only fails when a breakpoint is set. I would like to be able to set a breakpoint to assist in debugging the package. For the time being, I can move the package to a different (non-Vista) OS, where it works fine, but I would like to be able to develop and debug on Vista.
View 4 Replies
View Related
Nov 3, 1998
How do you specifiy a log file for scheduled tasks. I have consistency checks done at night and when it fails, I cannot find out why because no log file is created. I checked for a log file in /MSSQL/LOGS/*.log. The maintenance wizard creates a log file, but how does a regular TSQL scheduled task create a log file?
Thanks in advance for the help.
View 1 Replies
View Related
Dec 18, 2007
Hi i have a FTP task on my SSIS package where i want to select 3 files from a directory, this directory already contains other files but i only want 3 of them how do i only pull down the 3 files from the FTP site i can't seem to find a option on the FTP task to select what files i want from the direcroty.
View 3 Replies
View Related
Dec 2, 2007
Hi,
I'm tring to copy files from FTP address, the problem is that sometimes the FTP folder is empty, and then the FTP Task is failed.
Why is it failed if there are no files? Any suggestion how to avoid the error?
Thanks,
Hadar
View 4 Replies
View Related
Oct 7, 2005
I'm using package configurations to store my server/database name in an XML File. I need to be able to dynamically change the database at runtime when I execute the package. Can I use the XML Task in another package to make a change to that xml file? I'm not familiar at all with XML, so I don't really know the syntax for XPath or XSLT or anything. Basically I'm just looking for an example of how to this with XML, but I haven't found anything on the web to explain this to me. BOL isn't very helpful with the XML Task.
View 3 Replies
View Related
May 5, 2006
Hi everyone,
I want to design a FTP task to download all the xml files from a FTP site. And I don't know what the file's name is.
How can I design this task?
Thank you for your helps!
Tony
View 5 Replies
View Related
Sep 18, 2007
Hi Guys,
I've one Dafta flow task where I'm getting data from OleDb source and then doing some scripting using script component and then generating a file.
Now I would need to get the same data and apply some different things and generate another file.
Can I used this same task for doing the secondry work? If yes how woulld I put the thing in place, I would need to get the same data but I would need to use a seperate scripting and generate a seperate file?
TA
Gemma
View 1 Replies
View Related
Feb 22, 2007
I could not find the exact details on how to create a SSIS script that would ftp files on these forums, so I am adding my code to help save time for anyone else that might be wanting to do something similar. Here is the VB code for my script task to FTP files (hope this helps someone):
' Microsoft SQL Server Integration Services Script Task
' Write scripts using Microsoft Visual Basic
' The ScriptMain class is the entry point of the Script Task.
Imports System
Imports System.Data
Imports System.Math
Imports Microsoft.SqlServer.Dts.Runtime
Public Class ScriptMain
Public Sub Main()
Try
'Create the connection to the ftp server
Dim cm As ConnectionManager = Dts.Connections.Add("FTP")
'Set the properties like username & password
cm.Properties("ServerName").SetValue(cm, "Enter your Server Name here")
cm.Properties("ServerUserName").SetValue(cm, "Enter your FTP User Name here")
cm.Properties("ServerPassword").SetValue(cm, "Enter your FTP Password here")
cm.Properties("ServerPort").SetValue(cm, "21")
cm.Properties("Timeout").SetValue(cm, "0") 'The 0 setting will make it not timeout
cm.Properties("ChunkSize").SetValue(cm, "1000") '1000 kb
cm.Properties("Retries").SetValue(cm, "1")
'create the FTP object that sends the files and pass it the connection created above.
Dim ftp As FtpClientConnection = New FtpClientConnection(cm.AcquireConnection(Nothing))
'Connects to the ftp server
ftp.Connect()
'Build a array of all the file names that is going to be FTP'ed (in this case only one file)
Dim files(0) As String
files(0) = "Drive:FullPathYourFileName"
'ftp the file
'Note: I had a hard time finding the remote path directory. I found it by mistake by creating both the FTP connection and task in the SSIS package and it defaulted the remote path setting in the FTP task.
ftp.SendFiles(files, "/Enter Your Remote Path", True, False) ' the True makes it overwrite existing file and False is saying that it is not transferring ASCII
ftp.Close()
Catch ex As Exception
Dts.TaskResult = Dts.Results.Failure
End Try
Dts.TaskResult = Dts.Results.Success
End Sub
End Class
View 66 Replies
View Related
May 23, 2006
Hi,
I have created a FTP task that logs into FTP server and receives files and scheduled it to run every 15min. However, it fails when there are no files on FTP. How would I check the if files exist? How can I catch the FTP task error and compare it to Hresults.NoFilesFound in a script task?
Thanks in advance for any help.
View 7 Replies
View Related
Oct 15, 2007
Hi,
I have an FTP task on a package that runs every hour, downloads files from FTP and processes them.
If there are no files, the package should stop successfully after the FTP task, but the problem is that it fails because there are no files.
How can I avoid this?
And another question is there another way to download files and delete then with out another FTP task?
Thanks
View 4 Replies
View Related
May 11, 2007
Hello all,
I am new to SSIS and have a (maybe) simple question:
I use the FTP-Task to download two excel files from a FTP server. The task succeeds but only one file is downloaded instead of
two. I tried both: RemotePathVariable = false with /FTPTest/*.xls as RemotePath and RemotePathVariable = true with a variable
containing the same path. Both bring up the result described above.
Thank you in advance
Sandra
View 4 Replies
View Related
Jun 12, 2006
Is it possible to ftp files using code in a Script Task? I need to read the contents of an xml file and if it has a a specific file name in there then I ftp the corresponding pdf file which is at the same location as the xml file. However I cannot do this using the provided FTP Task in SSIS, I would need to use code to do this as there are close to 50 xml files which I need to read and upload the corresponding pdf file file it meets a certain criteria.
I do not see a way of looping thru all the files in a folder unless I do this in a Script task. Any inputs or alternative comments on doing this will be appreciated.
Thanks,
MShah
View 1 Replies
View Related
Mar 11, 2006
Hello,
I have two FTP Tasks configured in my SSIS package. One is for "Receive files" and the other is set for "Delete remote files." Both use variables for the source/destination paths. My remote path variable contains a wild card in the name field such as /usr/this/is/my/path/*.ext and it is working to FTP all the .ext files to my working directory. I then rename the files and want to remove the original files from the FTP server. I use the same variable as the remote path variable in the delete as I do in the receive.
Using the same FTP connection manager for both tasks I am always getting a failure on the delete. The FTP connection manger is setup to use the root user. Using a terminal I am able to open an FTP connection to the server and remove the files manually. There doesn't seem to be any detailed documentation on the FTP Task configured for Delete remote files so I'm hoping someone might have some insight to the problem.
I receive the same message for each of the files that was downloaded:
Error: 0xC001602A at MyPackage, Connection manager "FTP Connection Manager": An error occurred in the requested FTP operation. Detailed error description: 550 usr hisismypathdatafile1.ext: No such file or directory.
The attempt to delete file "usr hisismypathdatafile1.ext" failed. This may occur when the file does not exist, the file name was spelled incorrectly, or you do not have permissions to delete the file.
With the root user/working manually I'm not understanding the permission reason, the file does exist and is spelled correctly.
Dan
View 21 Replies
View Related
Nov 25, 2006
Hi all
For selecting different load targets we use package configurations to override the connection parameters in the connection manager.
To organize the different tasks we splittetd our load processes into multiple package files (dtsx) which are then put together into a workflow using "Execute Package Task" component.
With the option /CONFIGFILE its possible to override the configuration file for the package that is called with DTExec. Unfortuanetly it does not override the configuration of subpackages calles by the "Execute Package Task".
Are there some ways to configure subpackages on the command line?
Thomy
View 1 Replies
View Related
Nov 3, 2006
We are Downloading files from FTP Site using "FTP Task" and we need to remove those files after downloading.
We need to delete files of specific names, like "Names_*.TXT" (All the txt files and Names starting from "Names_") so We are using Delete Remote Files and Remote path we are specifying as "/Names_*.TXT"
This doesn't work its not able to delete the files so, We tried using "*.*" and also given the specific filename like "Names_A1.TXT" then also its not deleting the remote files.
Please let me know the solution.
Thanks,
Priaynk Gajera
View 6 Replies
View Related
Sep 28, 2006
I can create remote folder and delete it
I also can create remote files but i can not delete remote files
error warning is like subject
any ides? thanks
View 7 Replies
View Related
Feb 26, 2008
I want to rename files in a folder using filesystem task. My files are like these:
C:FOLDERDatabase20080225.bak
C:FOLDERDatabase120080225.bak etc
I want to remove this datepart and put in a C:FOLDERDatabase.bak file.
I am not quite sure about the string functions usage here...I think I have to map the destination variable to a string function output.
Any other ways to do this job ?
Thnaks for your help,
-Gish
View 2 Replies
View Related
Jan 25, 2007
Hi All,
I don't know if anyone faced this issue. We are having a strange problem. Our process was working well when it was implemented on 32 bit processor.IT ran perfectly for 6 months with out a problem. But when we moved the packages to a 64 bit machine, this issue along with some other issues started to show up.
The issue is we are missing files in the source folder.
Our process is designed such that a source process, brings in a file and updates a status for the file in a audit table. The ETL process picks up the file, then assigns the status as €˜running€™ when SRC process is complete and loads into Target DB, and updates ETL status to complete. But current problem is the ETL is losing files after it assigns the status as running. When we looked into the DB weather the data is loaded, we could not find any data related to these files.
we are have mapping level parameters for source path and target path.
We are using a For Each Loop task, and processing files(which are simple flat files) in the source path. The file name is stored in the mapping level parameter. Once the file is process we are moving them into a target path.
Our src and target file paths are on the same drive, just have src folder, inside src folder we have processed folder and failed folder. So files are picked from the source folder and moved into processed folder after processing. The files are not even moved to a failed folder.
There are lot other processing going on this box, and the trend observed is that when more processors are running at peak hour, the missing files€™ count is more.
Right now we are refetching those files, as a work around, but does any one has any suggestion why this is happening or any better implementation suggestions?
Thanks
View 1 Replies
View Related
Feb 2, 2008
Hi,
I am new to SSIS. I need to read the files in a remote server folder and ftp them to the other remote server folder.
I know , I have to set up FTP task and FTP task must use a variable to provide the path information. But I am not sure how I can do this. Anyone suggest me with the script task to populate the variable and steps involved in this.
Thanks,
Gish
View 1 Replies
View Related
Feb 13, 2007
Hi,
I have a FTP task in my control flow that download files from a FTP server. This ftp task is inside a foreach container that loops over a ADO recordset for the file name. The files that the ftp task pulls are huge. If the FTP task fails then I want the FTP task to restart and only download those files that have not been downloaded. Is this possible?
What possible configurations do I have to make to the foreach container and the filetask?
Thanks a lot in advance for your help and time.
Regards,
$wapnil
View 2 Replies
View Related
Feb 27, 2008
Hi all,
I'm trying to capture the file name and insert to the Database with Loop Container and Execute Sql task...However when I run, I get error with the input parameter.
In my Sql Task, the parameter mapping:
Variable name: user::variable,
direction: input,
Data Type: Varchar,
parameter name: @xVariable,
parameter size: -1
connectiontype: OleDB
sourcetype: direct input
statement : insert into xtable(xcolumn) values(@xVariable)
Anyone have any other suggestion how to capture the filename and input in the database?
thanks
View 4 Replies
View Related
Nov 2, 2015
When my ForEach Loop runs, when a file does not exist on the server, I am getting a File does not exist error.I would prefer to write a mesaage to my log and then move on to the next step successfully.When I got to Event Handlers and select OnTaskFailed, what do I want to do from here?
View 0 Replies
View Related
Jul 6, 2015
while i am trying to unzip files using execute process task ,getting below error
[Execute Process Task] Error: In Executing "C:Program Files7-Zip7z.exe" "a -tzip D:excel.zip D:unzipfileexcel.xls" at "", The process exit code was "1" while the expected was "0".
Warning: SSIS Warning Code DTS_W_MAXIMUMERRORCOUNTREACHED. Â The Execution method succeeded, but the number of errors raised (1) reached the maximum allowed (1); resulting in failure. This occurs when the number of errors reaches the number specified in MaximumErrorCount. Change the MaximumErrorCount or fix the errors.
i want to know more about unzip and zip files and folders using execute process task.
zip folder: Â C:Program Files7-Zip7z.exe
SQL version:Â SQL server 2008 R2
do not having win rar so please instruct using 7z.its quite interest to work but i don't know to get desired result.
View 6 Replies
View Related
Mar 24, 2008
I used the toolbox to select maintenance cleanup task to create the job to do this. In reading similar notes regarding this problem, some people mentioned that there was a choice to include subfolders. I do not have this choice. When I execute select @@version I get Microsoft SQL Server 2005 - 9.00.3042.00 (X64) Feb 10 2007 00:59:02 Copyright (c) 1988-2005 Microsoft Corporation Enterprise Edition (64-bit) on Windows NT 5.2 (Build 3790: Service Pack 2) . This is running on a cluster. Any idea what is going on here? Thanks.
View 6 Replies
View Related
Aug 4, 2015
I'm using the FTP Task in SSIS to send files. They task succeeds but the files get uploaded to the wrong folder. Instead of being sent to the cg268301 folder they are being sent to the cg268300 despite selecting /cg268301 from the remote path field in the FTP task editor.
I've tried uploading this file to the /cg268301 file using an execute process task and it works fine. I just don't know why it won't work with the FTP task.
View 3 Replies
View Related
