MS SQL Server :: How To Load Zip Files Using FTP Or Other SSIS Tasks

Delete Files Older Than 3 Days Using SSIS Tasks

Aug 11, 2006

I want to delete all files in a given folder that are older than 3 days from today's date.

I tried using a "Foreach Loop container" with a "File System" task inside it but found I couldn't access any file properties such as the file creation date. Am I using the wrong task for this job?

TIA,

Barkingdog

P.S. History of the problem:

I used the sql 2005 Database maintenance program to setup our database backup jobs. One day I noticed that the free space on the drive where we keep the dumps had grown small. I found that we had at least 4 weeks of dumps in there, not the 3 days I wanted to keep! I looked but could not find the "delete file" option in the SSIS package generated by the Database Maintenance Wizard. No wonder the files were piling up.

View 14 Replies View Related

Problem With WMI Tasks : Waiting For Files In Parallel In A SSIS Package

May 28, 2007

Hi,

I have a problem with the task "event watcher".

I've made a query like the one in msdn (SELECT * FROM __InstanceCreationEvent WITHIN 10 WHERE Targetinstance ISA "CIM_DirectoryContainsFile" and TargetInstance.GroupComponent= "Win32_Directory.Name="e:\\temp""). I have 20 similar tasks for watching in different folders, but when there are too much tasks in parallel, it doesn't work anymore. I change the numbers of executables to 128 (in the general properties of the package (to test)) but it doesn't seems to work.

I don't understand why it works when there are only 1 or 2 (6 seems to be the maximum) tasks and not if there are more than 6.

Could you help me with this issue?

Configuration : Windows Server 2003, SQL Server 2005, SSIS, Sql Server Agent

Thanks a lot.

Julien.

View 3 Replies View Related

Load XML Files Into Tables Via SSIS

Jul 19, 2007

Hi

I am new to SSIS and am looking to load XML files (with a DTD definition) into tables via a SSIS package. I have created a XML task and am able to load the XML and output it to file. I have also stripped out the DTD definition and am able through a dataflow using XML source an OLE db Destination load and map the XML to table sin my DB. But have no idea how to get the data in when it has a DTD definition included. I either want to put each file into a row in a table then query it. Or from the SSIS package input the relevant info into a set of staging tables or the real tables.

Any help/pointers would be appreciated.

View 10 Replies View Related

SSIS Packages That Load Different Text Files Into Tables

Jan 25, 2013

So i have about 80 different SSIS PAckages that load different text files into tables. each package has a config file that contains password, server name and user id. I want to forward these packages to a use who will execute them. So instead of giving the user 80 packages to execute manually, how do i put these packages into one parent package so that the user has to only execute this parent package instead of executing 80 different packages NB. package exec is manually. (we will automate later) so this is a temp measure.

View 2 Replies View Related

SQL 2012 :: Load Files In Date Order In SSIS

Apr 10, 2014

I have files which has date in file name and I want to load all files in sequential order.

Here I am giving you an example of my file name.

"NetworkActivity_869_403722_01-01-2014.log"
"NetworkActivity_869_403722_01-02-2014.log"
"NetworkActivity_869_403722_01-03-2014.log"
"NetworkActivity_869_403722_01-04-2014.log"
"NetworkActivity_869_403722_01-05-2014.log"

These are my files in that I want to load files sequentially means jan 1st, 2nd like this way.

View 5 Replies View Related

SQL 2012 :: SSIS To Load Files From Two SFTP Location

May 18, 2015

How to load files with similar format , from two different locations into same database with same ssis.

Lets say
Location 1: C:LoadFilesCust1APP_123445.txt
Location 2: D:LoadFilescust2VDD_543121.txt

Currently we have one ssis which loads and process files from C:LoadFilesCust1 only. we have to modify the existing package it to load files from Location 2 (D:LoadFilescust2) as well. Also while loading, the ssis should assign a value to existing column CustID depending upon the file name. File names always start with APP_ in first location. VDD_ in second location

Assign CUSTID as 100 if file name starts with APP_
Assign CUSTID as 200if file name starts with VDD_

View 1 Replies View Related

Integration Services :: SSIS - How To Load 2 Different Excel Files To 2 Different Tables

Jul 29, 2015

I have 2 different excel files file1 and file2. file1 should be loaded to table1 and file2 should be loaded to table2. Both of the files will have 1 sheet inside. Do I need to create separate excel source for file1 and file2? I mean file1 in one excel source and that will be connecting to 1 execute sql task. file2 in other excel source and that will be connecting to another execute sql task. Is this the way I should proceed or is there any looping should be done? I need to schedule this activity to run every week. So, I'll get new files every week with the same file names and sheet names. Do I need to consider anything for this requirement also?

I'm planning to do truncate and reload not an incremental load.

View 5 Replies View Related

Trying To Load Multiple Excel Files Into A Db Table Using For Each Loop Container In SSIS

Oct 24, 2007

I am getting the following error when trying to load multiple excel files using for each loop container in SSIS, I tried to put the quotes in several different ways but still can't get rid of this error. I was able to successfully load single excel file, but when I use the for each loop container that's when I am having problems. Any help is greatly appreciated. Thx.

TITLE: Package Validation Error
------------------------------

Package Validation Error

------------------------------
ADDITIONAL INFORMATION:

Error at Package1 [Connection manager "SourceConnectionExcel"]: The connection string components cannot contain unquoted semicolons. If the value must contain a semicolon, enclose the entire value in quotes. This error occurs when values in the connection string contain unquoted semicolons, such as the InitialCatalog property.

Error at Package1: The result of the expression ""Provider=Microsoft.Jet.OLEDB.4.0;Data Source=" + @[User::Folder] + @[User::file] + ";Extended Properties="Excel 8.0;HDR=NO";"

" on property "ExcelFilePath" cannot be written to the property. The expression was evaluated, but cannot be set on the property.

(Microsoft.DataTransformationServices.VsIntegration)

------------------------------
BUTTONS:

OK
------------------------------

View 8 Replies View Related

Integration Services :: SSIS Fails To Load Files From Network When Executed Via Agent (2008 R2 SP2)

Aug 5, 2014

I have a simple SSIS package that reads a flat file and copies it into a SQL Server table.

When the flat fiel is on the C drive I have no problem runnign this package from SQL Server Agent, but as soon as I update the path to a network location the package only works when I run it manually, but fails when is executed via the SQL Server agent job.

The error says "cannot open the datafile", while the datafile location is valid.

Is this a kind of limitation of a SQL Server Agent that only local files are allowed to be processed?

View 4 Replies View Related

Cannot Load Data Flow Tasks

Aug 3, 2007

When I copy over an SSIS package I have been developing from my laptop to my desktop with Windows File Sharing (shared folders) across a home network, the moved package fails to load properly. I can see the Control Flow tasks but not the Data Flow tasks - they simply disappear! I have update the Connection Managers to point to the new machine, and tested them (OK).

Its the same as this issue posted here

And Here

Creating an empty package and clicking to create a new data flow cause this error:

TITLE: Microsoft Visual Studio
------------------------------

Failed to create the task.

------------------------------
ADDITIONAL INFORMATION:

The designer could not be initialized. Microsoft.DataTransformationServices.Design)

Seems the only way is to uninstall and reinstall Client Components.

How do I uninstall and reinstall Client Components? when I try to do that, SQL Server Setup tells me "None of the selected features can be installed or upgraded. Setup cannot procees since there is no effective change being made to the machine...."

I had selected "Workstation components, Books online and Development tools". This is already installed - so how I can reinstall it if I get this message above which does not let me proceed.

View 2 Replies View Related

SSIS Job Fails For Package Containing Script Task With Error The Script Files Failed To Load.

Apr 10, 2006

I have developed a SQL Server 2005 Integration Services (SSIS) package that includes a Script Task on a 32-bit machine. The PrecompileScriptIntoBinaryCode property is set to True. After I build the package, the .dtsx file includes a <BinaryItem> element for that Task. Package runs fine on the dev machine, both in BIDS and as SQL Server Agent job.
When I deploy the package to a 64-bit server, it runs fine when I execute the package ad hoc from SQL Server Management Studio. However, when I schedule the package for execution as a SQL Server Agent job, the package fails with the message: "the script files failed to load."
I have reviewed posts on this error from late 2005, but the solutions don't work in this case. Specifically:
1. The Precompile property is already set to True.
2. I have already verified that the script was compiled.
Any further suggestions?

View 1 Replies View Related

Could Not Serialize The SSIS Runtime Objects When Attempting To Copy Tasks Created By SSIS Wizard

Jul 11, 2007

Run the SSIS import/export wizard.

(xls -> sql table)

Select the tasks created.

Copy.

Here is the error that occurs.

p.s. Does anyone have any needles I can borrow? I think sticking them in my eyes would be nicer than working with SSIS.

===================================

An error occurred while objects were being copied. SSIS Designer could not serialize the SSIS runtime objects. (Microsoft Visual Studio)

===================================

Could not copy object 'Preparation SQL Task' to the clipboard.
(Microsoft.DataTransformationServices.Design)

------------------------------
For help, click: http://go.microsoft.com/fwlink?ProdName=Microsoft%u00ae+Visual+Studio%u00ae+2005&ProdVer=8.0.50727.762&EvtSrc=Microsoft.DataTransformationServices.Design.SR&EvtID=SerializeComponentsFailed&LinkId=20476

------------------------------
Program Location:

at Microsoft.DataTransformationServices.Design.DtsClipboardCommandHelper.SerializeRuntimeObjects(ICollection logicalObjects)
at Microsoft.DataTransformationServices.Design.ControlFlowClipboardCommandHelper.InternalMenuCopy(MenuCommand sender, CommandHandlingArgs args)

===================================

Invalid access to memory location. (Exception from HRESULT: 0x800703E6) (Microsoft.SqlServer.ManagedDTS)

------------------------------
Program Location:

at Microsoft.SqlServer.Dts.Runtime.PersistImpl.SaveToXML(XmlDocument& doc, XmlNode node, IDTSEvents events)
at Microsoft.SqlServer.Dts.Runtime.DtsContainer.SaveToXML(XmlDocument& doc, XmlNode node, IDTSEvents events)
at Microsoft.DataTransformationServices.Design.DtsClipboardCommandHelper.SerializeRuntimeObjects(ICollection logicalObjects)

View 14 Replies View Related

Load Files

Jan 10, 2007

the connection string in my application daynamic ..changed by changing the development environmet ..how can i load a data from file to sql server destination without hard coded the connection

thx

View 5 Replies View Related

SSIS Tasks

Oct 26, 2007

Hello everybody,

I need some help :

I want to transfer a few access mdb kept in a folder to sql server. I am using foreach file enumerator. I want to perform the following actions :

1. At time of loading the file, check if the file name starts with "ED". If not, move the file to error folder.
2. Count no. of records that were there in the access db and that were transferred to the sql table.
3. I also want to run the package from command line. Even though i figured out that, i want to display only limited messages on the console. For eg: File processing begins, File currently being processed, TOtal rows transferred, File processing completed etc.

Please guide me with some examples.

thanks mint.

View 3 Replies View Related

SSIS Tasks

Jun 21, 2006

Hi,

I need some suggesstion regarding which SSIS tasks to use regarding my job in hand. Basically what I need is to extract data from joining multiple tables and then load to a particular output database.

How I can go about it?

Thanks

View 1 Replies View Related

Help Regarding Ssis Tasks

Nov 16, 2007

Hi,

Can anyone help me out on the following tasks with examples

XML Task.

Message queue Task.

Send Mail Task
Thanks in Advance.

Regards
su..........

View 1 Replies View Related

Load 4 Different Csv Files Into 4 Different Tables?

Feb 10, 2015

I have written bulk insert and doing this manully but need to automate it and file location changes evertime .

BULK INSERT dbo.alpha
FROM 'C:posted 2015-01-16HOSPITALHOSPFY2010hosp_2010_ALPHA.CSV'
WITH
(
FIELDTERMINATOR = ',',
ROWTERMINATOR = '',
ROWS_PER_BATCH = 10000,
TABLOCK
)
BULK INSERT dbo.RPT
FROM 'C:posted 2015-01-16HOSPITALHOSPFY2010hosp_2010_RPT.CSV'
WITH
(
FIELDTERMINATOR = ',',
ROWTERMINATOR = '',
ROWS_PER_BATCH = 10000,
TABLOCK
)

how to proceed?

View 6 Replies View Related

Using SSIS Instead Of Scheduled Tasks?

Jan 22, 2008

What are peoples opinions on using SSIS as a "central repository" and replacement for all scheduled tasks. Example, we have a bunch of servers which we have installed services which we have written. Currently we have scheduled tasks on each machine to stop and start the services. One of my collegues is using SSIS to run a system which runs tasks on multiple machines by remotely running programs on other machines via scheduled tasks and then collects the data and puts it into a database.

He's now pitching the idea that we remove the scheduled tasks on each machine and start and stop our services via SSIS so that it's centralized. In addition, we can also check for holidays in our database before starting services. Since it doesn't seem like SSIS was meant for this type of use, I'm weary of using the tool to do something it wasn't intended for.

Any opinions? I'm also worried that the learnnig curve for everyone is going to be too high.

View 11 Replies View Related

Best Way To Load Data From Text Files

Jan 22, 2006

Hi,
I have problem I'm hoping someone can give me some pointers with.

I need to load data from several text files into one table. The format of the files are simple - each line is comma separated, with double quotes around each element e.g.

"parameter 1","value 1","parameter 2","value 2"....
"parameter 12","value 12","parameter 13","value 13"...

However, the files themselves will have different numbers of columns e.g file 1 may have 8 columns, file 2 may have 16 columns.

I'm going to load the data into a table that has at least as many columns as the longest file. The table columns are all varchar, and are named simply as [Col001] [Col002] [Col003] etc...

The first two columns of this table must be left empty during the load (I use these later on), so the data entry will start at [Col003].

My question is what is the best way to do this? I thought perhaps using a BULK INSERT in a stored procedure might do the trick, but I haven't used it before and haven't got very far. I gather another approach might be to use bcp utility. Someone has also suggested a DTS package, but the filenames will be suffixed with current date/time stamp, so i don't think that will work.

My preferred appraoch would be the BULK INSERT..but i'm open to any pointers.

Many Thanks
Greg

View 2 Replies View Related

Load Text Files In To Tables

Oct 15, 2007

Hi all,

I have around 140 text files in a folder where all the file entries are delimited by tabs. Currently i am copying the data and then pasting it in to the tables and it is perfectly working fine. I have already created tables for those 140 files. But now i want to load those files in to the table at a single shot. How can i do it? I am using SQLServer 2005.

Sarvan

View 2 Replies View Related

How To Load .dbf Files In Sql Server 2005

May 1, 2007

hi

i have a dbf file which i need to import in sql server 2005.

anyone having any idea to import these files either through integration services or any other tool ???

View 3 Replies View Related

Parse XML To Load Files Inside Them

Apr 3, 2007

Hi all,

is there a way we can parse a XML(file name lister) file that has a structure say,

<Node>
<Root>
<FullName>/My Documents/Documents/Feature/17470_652</FullName>
<ShortName>17470</ShortName>
<XMLFileName>17470_652.xml</XMLFileName>
<LastModifiedOn>12 December 2006</LastModifiedOn>
</Root>
...
...
...
</Node>

and get the values of all the <XMLFileName> attribute and load them into some variables,say here the filename is "17470_652.xml", I have to look for this file in a specified location, open the XML file and load the contents to a table ? It can be presumed that all the XML files will be stored in a same location

View 11 Replies View Related

How To Load And UnZip Files From FTP Directory

Mar 5, 2008

HI All,
I have Uzip Files to be loaded From ftp.companyname.com, the zip files get updated everyday thus i have to load the newly added files, the Zip file has got 13 text Files within it, the issue is:
1. how do i load it for the new zip file only, the zip files are shown below, i am trying using FTP Task, but need more info.
2. How do i UnZip it and and take the text files and then load them to sql server 2005, each text file has to be loaded to sql server tables.
3. How do i automate it, i mean every time i run the package (on Job based) it has to look the new file only, see the zip file below to understand what i am saying about.

02/25/2008 09:02PM 2,780,729 CompanyName_2008-02-22.zip
02/27/2008 09:04PM 1,274,557 CompanyName_2008-02-23.zip
02/25/2008 08:57PM 1,383,112 CompanyName_2008-02-24.zip
02/26/2008 04:21PM 3,327,882 CompanyName_2008-02-25.zip
02/27/2008 08:44PM 3,623,334 CompanyName_2008-02-26.zip
02/28/2008 05:27PM 3,570,243 CompanyName_2008-02-27.zip
02/29/2008 04:20PM 3,444,375 CompanyName_2008-02-28.zip
03/01/2008 04:17PM 2,655,782 CompanyName_2008-02-29.zip
03/03/2008 04:09AM 1,179,338 CompanyName_2008-03-01.zip
03/03/2008 05:21PM 1,267,777 CompanyName_2008-03-02.zip
03/04/2008 04:49PM 3,238,121 CompanyName_2008-03-03.zip

As you can see the Zip file names are the blues color, they are added at different time, thus what i need is only the current Zip file (i mean i have to Load only the newly added Zip file (recent one)). Thus the SSIS Task has to go to this FTP server and look the newly added zip file. After that i have Unzip it b/se i have 13 text files there, and then Load them to Sql Server 2005.

Please get help on this, the deadline is near by time, if possible try as soon as possible, I would like to say Thank you for every help you do and try.
Thanks,

View 1 Replies View Related

SQL 2012 :: Using Variables In SSIS Tasks

Dec 2, 2014

I have this procedure to remove certain characters from file names.

The SQL Task has this: exec dbo.spCleanseFileName @strFileName = ?, @strFileNameCleansed = ?

The stored procedure:
CREATE PROCEDURE [dbo].[spCleanseFileName](@strFileName varchar(40),@strFileNameCleansed varchar(40) output)

I have it in an SSIS package and my problem is that, after that SQL Task completes, the value for the
),@strFileNameCleansed variable is blank. I HAVE confirmed that the procedure DOES set the correct value inside the SP.

View 2 Replies View Related

BIG SSIS PACKAGE (200 DATAFLOW TASKS)

Mar 29, 2008

Thanks in advance in reading this thread.

I have developed a big SSIS package to extract data from flat-files ( + 200 Dataflows ).

The situation is the following, inside de SSIS package, there are a lot of validations before extracting & loading the flat-files, i'm running this validations in paralell, so that when a file arrives, it enters the "validation process" and start extracting the file.

When i run the SSIS package from BIDS it works the way i have concepted it... but when i run the ssis in the server, the tables that are loaded through the process are only "available" when the SSIS PACKAGE ends, it is imperative that trough the process, when a table receives new data, it becomes ready, and don't just be available when the SSIS package finishes...

I have attached the an lousing .jpeg.

It is importart for the tables to be available, so the stored procedures(OUTSIDE SSIS PACKAGE) that are dependent of some tables, start working before the SSIS package Ends.

Thanks in Advance.

View 4 Replies View Related

SSIS Sceduled Tasks Failing.

Feb 15, 2008

Hi
I have a strange problem with SSIS packages.
(brief description - packages select some data from DBs , write them to CSV file and then CSV file is
copied and renamed to a folder made up of the date)
I have 5 packages scheduled to run, these jobs run perfectly when test scheduled during the day,
(so its not a user permissions problem).
However it seems the 1st package to run at night will fail.
The reason I say 1st is the following,
I had Package A scheduled at 11:20PM and package B at 11:30PM , Package B always succeeded
but package A always failed.
I would test A during the day and it would run fine (the jobs would run successfully aswell as just executing the package manually) .

The I changed the time with B to 11:50PM and it succeeds and A fails ! without changing the packages themselves.

This counts out a possibilty of a DB backup causing the problem (pack always succeeded at 1130, now fails at the same
time)

I was thinking maybe as the folder wasn't created when the 1st pack ran this was causing the fail, but when i test run the job
this morning it succeeds .. and todays folder doesn't exist either !

any ideas ?

View 5 Replies View Related

Communication Between Tasks In An SSIS Package

Feb 6, 2007

Hi,
I have a Flat File Source and I want to retrieve few properties of it in an Script Component. How do I?

Also, How could I make the file path of Flat File Source or Connection manager dynamic or configurable through some file ?

any input is appreciated.

Fahad

View 1 Replies View Related

SSIS Execution Of Transformation Tasks.

Feb 17, 2006

I'm not exactly sure how to ask this question ... but here goes!!

I want to get an idea of how SSIS actually executes transformation tasks.

Do transformation tasks (eg a lookup) complie down to managed code or are the executed as SQL commands in a SQL server database?

Thanks.

View 1 Replies View Related

Active X Transformation Tasks In SSIS

Feb 1, 2008

Hi
I have migrated a DTS that had some activeX transformation tasks within data pump flow tasks.

Those parts were migrated as "DTS 2000 tasks" .. so activeX transformation tasks aren't possible in SSIS ?
I know ActiveX script tasks are but for transformations ?

1. IF i leave these Encapsulated DTS 2000 tasks in the migrated SSIS package, will it run independently of the original
DTS or does it need the old DTS running to "call" that part from ? (I hope im making sense here)
is it possible to load this functionality internally into the new SSIS ?

2.
How could I (if i can't do ActiveX transformation tasks) achieve this is SSIS ?
can I achive this using the script tasks in SSIS ?

View 6 Replies View Related

Load Some Old CVS Files Created In The Year 2005?

Jun 2, 2012

I'm having an issue trying to load some old CVS files created in the year 2005. I'm using SQL server 2008 express edition for now. I've never seen files in this format. For reason, i can't go into, i can't user SSIS and need to load them using the bulkinsert comand. the files contain about 20 fields with a mix of numerics and text. the text fields are delimited by 2 sets of quotes, and a , and the numeric fields by just a , with each line contained in double quotes.

ex """abc"",12,198504,""tom"",""Smith"",49,3322,""mai n st."""

how to define the format file so that i can remove the "" quotes before and after each text field and how to remove the set of quotes that each record is enclosed with.

View 4 Replies View Related

How To Load Dynamic Multiple Flat Files

Apr 23, 2008

Hi Evry one,

I Have Multiple Flat Files in Source Folder(They have Naming Conventions With Todays Date ex: Flatfile_20082204_1,Flatfile_20082204_2,Flatfile_20082204_3 ),
I need to Extract Each and Evry file by Dynamically, and Transform the Flat File then load that Flat file into the Destination Folder with Standard Prefix and Todays Date with a Sequence No ex:Flatfile_20082304_A,Flatfile_20082304_B, Flatfile_20082304_C

Please HELP Me
Thanks In Advance.

View 20 Replies View Related

How Do You Load .JPG Files Into A Image Datatype Column

Sep 9, 2006

I have been fighting with this all day. If you can point me in the right direction I'd appreiciate it. I need to load about 500 jpg files into a table. The table has 3 columns an "ID", "Filename" which has the filename of the jpg in it already, but not the unc path, and a ("Photo" Image Datatype)column which is not populated yet. I need to store the Image file in the photo field so that I can run reports in reporting services and so on and so forth. I am not sure how to complete this task.

View 3 Replies View Related