Using the below script task I am checking for the excel file existence and upon file existence using the data flow task will load the excel data into sql table. After the data is loaded from one file or however number of excel files present, I want to move those into a archieve folder with datetimestamp to the filenames,please let me know how I can move those files with datetimestamp to the filenames, any help is greatly appreciated. Thanks!!
Imports System
Imports System.Data
Imports System.Math
Imports Microsoft.SqlServer.Dts.Runtime
Imports System.IO
Public Class ScriptMain
Public Sub Main()
If File.Exists(ReadVariable("FileNameVariable").ToString()) Then
Dts.TaskResult = Dts.Results.Success
Else
Dts.TaskResult = Dts.Results.Failure
End If
End Sub
'From Daniel Read's Blog - http://www.developerdotstar.com/community/node/512/
Private Function ReadVariable(ByVal varName As String) As Object
Dim result As Object
Try
Dim vars As Variables
Dts.VariableDispenser.LockForRead(varName)
Dts.VariableDispenser.GetVariables(vars)
Try
result = vars(varName).Value
Catch ex As Exception
Throw ex
Finally
vars.Unlock()
End Try
Catch ex As Exception
Throw ex
End Try
Return result
End Function
End Class
Using the below script task I am checking for the excel file existence and upon file existence using the data flow task will load the excel data into sql table. After the data is loaded from one file or however number of excel files present, I want to move those excel files into a archieve folder with date×tamp to the filenames, please let me know how I can move those files with datetimestamp to the filename, any help is greatly appreciated. Thanks!! Imports System Imports System.Data Imports System.Math Imports Microsoft.SqlServer.Dts.Runtime Imports System.IO Public Class ScriptMain Public Sub Main() If File.Exists(ReadVariable("FileNameVariable").ToString()) Then Dts.TaskResult = Dts.Results.Success Else Dts.TaskResult = Dts.Results.Failure End If End Sub 'From Daniel Read's Blog - http://www.developerdotstar.com/community/node/512/ Private Function ReadVariable(ByVal varName As String) As Object Dim result As Object Try Dim vars As Variables Dts.VariableDispenser.LockForRead(varName) Dts.VariableDispenser.GetVariables(vars) Try result = vars(varName).Value Catch ex As Exception Throw ex Finally vars.Unlock() End Try Catch ex As Exception Throw ex End Try Return result End Function End Class
In the first step of my SSIS package I need to get files from FTP and dump it/them in a local directory, but it's more than that, the logic is like this: 1. If no file(s) found, stop executing and send email saying no file(s) found; 2. If file(s) found, then compare it/them with existing files in our archive folder; if file(s) already exist in archive folder, stop executing and send email saying file(s) already existed, if file(s) not in archive folder yet, then transfer it/them to the local directory for processing.
I know i have to use a script task to do this and i did some research and found examples for each of the above 2 steps and not both combined, so that's why I need some help here to get the logic incorporated right.
Thanks for the help in advance and i apologize for the long lines of code!
example for step 1: ----------------------------------------------------------------------------------------------------------
' Microsoft SQL Server Integration Services Script Task ' Write scripts using Microsoft Visual Basic ' The ScriptMain class is the entry point of the Script Task.
' The execution engine calls this method when the task executes. ' To access the object model, use the Dts object. Connections, variables, events, ' and logging features are available as static members of the Dts class. ' Before returning from this method, set the value of Dts.TaskResult to indicate success or failure. ' ' To open Code and Text Editor Help, press F1. ' To open Object Browser, press Ctrl+Alt+J.
Public Sub Main()
Dim cDataFileName As String Dim cFileType As String Dim cFileFlgVar As String WriteVariable("SCFileFlg", False) WriteVariable("OOFileFlg", False) WriteVariable("INFileFlg", False) WriteVariable("IAFileFlg", False) WriteVariable("RCFileFlg", False) cDataFileName = ReadVariable("DataFileName").ToString cFileType = Left(Right(cDataFileName, 4), 2) cFileFlgVar = cFileType.ToUpper + "FileFlg" WriteVariable(cFileFlgVar, True) Dts.TaskResult = Dts.Results.Success End Sub Private Sub WriteVariable(ByVal varName As String, ByVal varValue As Object) Try Dim vars As Variables Dts.VariableDispenser.LockForWrite(varName) Dts.VariableDispenser.GetVariables(vars) Try vars(varName).Value = varValue Catch ex As Exception Throw ex Finally vars.Unlock() End Try Catch ex As Exception Throw ex End Try End Sub Private Function ReadVariable(ByVal varName As String) As Object Dim result As Object Try Dim vars As Variables Dts.VariableDispenser.LockForRead(varName) Dts.VariableDispenser.GetVariables(vars) Try result = vars(varName).Value Catch ex As Exception Throw ex Finally vars.Unlock() End Try Catch ex As Exception Throw ex End Try Return result End Function End Class
example for step 2: -------------------------------------------------------------------------------------------------------
' Microsoft SQL Server Integration Services Script Task
' Write scripts using Microsoft Visual Basic
' The ScriptMain class is the entry point of the Script Task.
Imports System
Imports System.Data
Imports System.Math
Imports Microsoft.SqlServer.Dts.Runtime
Public Class ScriptMain
' The execution engine calls this method when the task executes.
' To access the object model, use the Dts object. Connections, variables, events,
' and logging features are available as static members of the Dts class.
' Before returning from this method, set the value of Dts.TaskResult to indicate success or failure.
'
' To open Code and Text Editor Help, press F1.
' To open Object Browser, press Ctrl+Alt+J.
Public Sub Main()
Try
'Create the connection to the ftp server
Dim cm As ConnectionManager = Dts.Connections.Add("FTP")
At present our snaopshot folder for merge replication(mainly) and our snapshot replications are on a seperate server. We now need to move the folder to a new server. Anybody explain how we will do this and what affects it will have
I have set up an FTP connection that tests successfully. I can log on to the FTP site with the same credentials and see my root folder, and within that, two more folders. In my FTP task, I want to receive files to my local machine. The problem is that the only remote path available is not at the root level, and the only thing I can see are files from one of the child folders, but not the child folder I want.
Is it possible in the remote path to change folders? The remote path just shows a "/", which I thought would be the root level, but it is somehow linked to a child folder.I've tried various combinations to get to the folder I want, /root folder/child folder, but that gives the error that the folder does not exist.
If I open the FTP Task Editor, click on File Transfer, and click the ellipses for the Remote Path, the Browse For File box opens, with a "/" in the Location section, and a list of files in the child folder that I do not want to be in. If I click the "Up Directory" button, I get the message "Already at top level directory".
how I can get the files from one particular folder on the FTP site?
I attempted to use Move Directory to move the contents of one directory to another. I encountered the 'different volume' issue that others have experienced. While this error is frustrating I can work past this particular issue. My more pressing question is why is the move directory command overwriting a destination directory? When I setup the Move directory file task I provided two vars to hold src and dest location:
dest var: estserveroutput src var: devserverdev estfiles Set overwrite destination = TRUE
Why would Move Directory overwrite output folder at destination? Shouldn't it only overwrite if the testfiles directory exists at destination? This is very frustrating since I cannot find enough information in the official documentation to understand what is happening here.
Is it just me or does the documentation for Move directory seem.....incomplete?
I need to use .txt files as data source, those files are in same folder. Now I connect to them with ODBC, and allow client select which file they want to view by input the name of file as a parameter. The problem is all those file are tabdelimited and the default of ODBC driver is CSV delimited. So once new file is added in the folder, we must refresh the ODBC otherwise the drive will deal all the data as one column.
I am having a huge xml file with nested section. i also have a xsd file for that xml. i have a destination table where the data from the xml should be loaded into. i am using the xml source transformation. But o get all the data i need to use multiple merje joins to get the data in a single row which i can insert into the destination.i was not quiet convinced with using so many joins.
so i tried using the script source transformation where i am using xml objects to get the node and dynamically construction the data row. and the output is then inserted into the destination. on comparing the two approach the one using the script source is working much faster than the xml source transformation.
i wanted to know is there any limitaion using the script source to parse through xml files. also i would like to know any other better way of getting the data from xml source without using the joins.
Hi All,I have a multiple files, but they are store in different directory onthe server. I want open those files and insert it into the databaseusing bcp.Example files structure dir:\xyz123abc ext1.txt\xyz123abc ext2.txt\zyz123999 ext2.txtbcp "dabase" in \xyz123abc ext1.txt -c -S"servername' -Usa-Ppassword -T".is there away to loop througth each dir, get the files, excecute thebcp, then go to next folder.Please help. Thanks in advance.Ted Lee
I am new to SSIS and I am trying to do following thing using WMI Reader Task.
I have developed SSIS package which import data from flat files. Now I have to add following functionality to SSIS package
Before SSIS package load data I would like to check If there are all files in source folder and check the files are with current date. If both condition true then only load data.
After some research i found that WMI uses specialized query language known as WQL, which is similar to SQL, to obtain information on files and directories.
Also i found that Under WMI Reader Task editor properties you can write WQL query to receive file information from source folder.
Can Anyone suggest me What WQL query i have to write to retrive file information from Source folder?
Or
Any Other suggesion i should approach?
Please advise me!...Any help would really appriciated...
I'm tring to copy files from FTP address, the problem is that sometimes the FTP folder is empty, and then the FTP Task is failed. Why is it failed if there are no files? Any suggestion how to avoid the error?
I am using a Excel Source to get the data from an excel file to sql server 2005 table. A couple columns are coming in a double precision float, but some values have characters in them, but those values are coming out as null, even though I changed the datatype from float to unicode string. Any inputs on resolving this will be much appreciated.
I am trying to get the contents of the Excel Files dynamically and dumping into the SQL Database using SSIS. Through WMI Event Watcher, I could find when one or more Excel files dumped in a particular folder and using ForEach Loop Container I was able to take all the filenames and pass it through Variables. But at the same time in the Data Flow, I have to pass each Sheet of an Excel File to the Excel Source control and export the data to my SQL Database using OLEDB Destination.
For that I need to get the names of each sheets in an Excel File and pass it to the Excel Source Control through variables. But when I give Data Access Mode as "Table name or view name variable" and provide the variable name in that, then it is giving an error message as "A destination table name has not been provided".
And at the same time, Since I was not able to provide an static Filename (as I am passing through Variables), when I tried to map the columns in the OleDB Destination, it is not allowing me to map the columns.
So all these things I should do at Run-time using Variables in SSIS. I don't want to hard-code any filenames or Sheet names. If any one of you have a solution, please share with me.
I have inherited a SSRS setup and am trying to do some clean up. Problem is I cannot see the Data Sources Folder. I am setup as SA and SU on both the RS Server and SQL Server. When I go check the catalog in SQL db it does truly exist as well as a number of data sources in the folder that reports are currently using. How can I see this folder?
I am developing application with SQLCE2.0, NETCF1.0,Sp1,VS2003.
I found there is some files are created under "Temp" folder by the system with size "0B".
Why/when these file are created? Do I need to clean it periodicly? If not, will this cause exception like "Not enough storage is available to complete this operation"?
1) I read that the path can be changed without any problem, in the same server we have a D drive with 40GB of space, so I am planning to change the path to: D:RSTempFiles. Any experience on changing that path?
2) If the change of path is successfull, how I can delete the files on the RSTempFiles on the C: drive. Can I directly delete the files? Ther just previous snapshots or I will lose something? Any experience with this will be great.
I have installed SSAS on a 64 bit machine and am trying to connect to Oracle but having little success. Some of the solutions I've read indicate to uninstall SSAS and then reinstall, making sure that the reinstall does not use the Program Files (x86) folder since Oracle has a problem with the parentheses. Here's my question... How do you NOT install in the Program Files (x86) folder. It seems SQL Server always wants to put a small part in there. Thanks in advance and sorry if this is a dumb question.
I'm wondering if its possible to return the number of files in a folder suing something like DIR command.I'm wanting to do something like, If count(DIR) >0 then do something
I have a challenge i am trying to overcome, hopefully soneone would have come across this issue before..
I am creating a DTS package that will be scheduled to run at a certain time everyday. A source folder exists that get a set of new files everyday.The DTS Package will then read each file and copy the data into a load table in my database the challenge is this:
I am trying to load files from a source folder into my load table, Within each file, the entires are in a specific format using pipes to seperate the data that goes into which column e.g
example of a file entry:
column1 | column2 | column3
data1 | data2 | data3
data1 | data2 | data3
data1 | data2 | data3
And now i am using DTS to specify the file format and map the cloumns as apprporiate to my table...all this is well and good, but my problem is each file has a different name as well as being timestamped, now how do i use DTS to specify the source folder, open each file sequentially and read (or more appropriate, copy the entries into my table, inserting new data from each file into my load table as well as overwriting old data in the load table from the files in the folder ?) is there a way in specifying your source folder in DTS rather than specifying the file in the Menu options (in the transformation data task properties )given, and or do i need to write a script for this(reading the file?)
can someone please give me a solution and how to approach this?
I have a problem with retreving a excel data through excel source component.
I have source component as Excel Source which will connect to my .xls sheet. To retrieve the values from the sheet i am using a query as, "SELECT F14,F3 FROM [Charac Defn & Assgnment$]"
The column F14 is not formatted so that the format of the cell is "General" I have a different type of values in the F14 column such as "PE","PES",15,20,20.00,8888.9999 etc.. While i click on preview button of Excel source it shows only the text values and not the int or decimal values, its returning NULL for those cells. I tried to use convert function, its throwing an error as
TITLE: Microsoft Visual Studio ------------------------------ There was an error displaying the preview. ------------------------------ ADDITIONAL INFORMATION: Undefined function 'Convert' in expression. (Microsoft JET Database Engine)
Is there any other function to change the format of the cell or i need to some thing else Please help me how to solve this issue.
I am creating an SSIS package witha a Dataflow task, which reads from an Excel source and then uses script component to dumpt the data to multiple tables in Sql Server database
I need to some how make my Excel source dynamic, that is my excel template which i would be using to map the excel columns to script component's input columns would be dynamic..
In other words, I should be able to define the Excel Source, Column Mapping Information, Precedence constraint to the Script component dynamically
Copy a full directory from source to destination (Done)
then for each file on the destination directory,it must process that file and insert rows on the table.
So I created a foreach loop, and created a varriable aclled CURRENTFILENAME, and assigned it into the foreachloop to index 0.
Inside the foreachloop I created a dataflow task, in the dataflow task I dragged a flat file source and an oledb destination, but I noticed that the flat file source requires flat file manager, and the flat file manager requires a unique FILE NAME. I cant put this c:copia*.txt.
I took a loot at flat file source properties and it has associated the flat file manager, but I can not assign the filename to the variable from the foreach.
I have a question about ssrs security. In report manager I have set a list of users as browsers, and the builtin administrators are content manager in the parent folder where reports are. A user that is not an administrator and neither a user I added, has access to reports.
I'm thinking this person is having access because the data source of the reports use a administrator account to connect to the analysis services using the option Credentials stored securely in the report server with the 2 checkboxes marked (use as windows credentials when connecting to the data source and impersonate the authenticated user).
I simply marked this option because SSRS is in 1 server and SSAS in other and I think kerberos need to be configured and I haven't looked into it, but I think the report access is separated from the data source.
Can someone tell me how to move the log file for a database from one drive to anoyher. I want to keep the data on one drive and the logs on another so I need to move the log files to another drive. Thanx, Kelly
Without restoring the database, has anyone written a slick little piece of code that moves a file existing on on filegroup to another newly created filegroup without doing it from a restore.
The purpose of doing this is to get rid of one data file and placing the data into the data file we are retaining. We have some more physical file space and are moving data into one data file and one transaction log file.
We have already ran the DBCC SHRINKFILE with EMPTYFILE to move the data, We have already ran the ALTER DATABASE statement but here, because the filegroup is identified as PRIMARY in the sysfilegroup table, we can not REMOVE the data file because of indexes relating to the PRIMARY filegroup.
So if I could ALTER DATABASE and create a bogus filegroup and move the data file which I want to get rid of into the bogus filegroup, I could ALTER DATABASE with REMOVE filegroup and solve the situation.
Does anyone know of an easier way to do it other than BACKUP database and RESTORE database.....please help!!!
Hi, I am wondering how can we return each file in a folder ? I am trying to get each file in a folder to perform other job. Below is the description ... while folder is not empty foreach file run the job end for end while