Hello,
Can we keep a database connection information in .ds (datasource) files. Is it possible to use these instead of the standard approach of putting the connection string in the configuration file.
Furthermore, when they run the SSIS package from the command line (using DTEXEC), is it possible to pass the .ds file as a parameter?
Hello Experts,
I am createing one task (user control) in SSIS. I have property grid in my GUI and 2 buttons (OK & Cancle).
PropertyGrid has Properties like SourceConnection, OutputConnection etc....right now I am able to populate Connections in list box next to Source and Output Property.
Now my question to you guys is depending on Source Connection it should read that text file associated with connection manager. After validation it should pick header (first line of text file bases on record type) and write it into new file when task is executed. I have following code for your reference. Please let me know I am going in right direction or not..
What should go here ?
->Under Class A
public override DTSExecResult Execute(Connections connections, VariableDispenser variableDispenser, IDTSComponentEvents componentEvents, IDTSLogging log, object transaction)
{
//Some code to read file and write it into new file
return DTSExecResult.Success;
}
public const string Property_Task = "CustomErrorControl";
public const string Property_SourceConnection = "SourceConnection";
public void LoadFromXML(XmlElement node, IDTSInfoEvents infoEvents)
{
if (node.Name != Property_Task)
{
throw new Exception(String.Format("Invalid task element '{0}' in LoadFromXML.", node.Name));
}
else
{
try
{
_sourceConnectionId = node.Attributes.GetNamedItem(Property_SourceConnection).Value;
}
catch (Exception ex)
{
infoEvents.FireError(0, "LoadFromXML", ex.Message, "", 0);
}
}
}
public void SaveToXML(XmlDocument doc, IDTSInfoEvents infoEvents)
{
try
{
// // Create Task Element
XmlElement taskElement = doc.CreateElement("", Property_Task, "");
doc.AppendChild(taskElement);
// // Save source FileConnection
XmlAttribute sourcefileAttribute = doc.CreateAttribute(Property_SourceConnection);
sourcefileAttribute.Value = _sourceConnectionId;
taskElement.Attributes.Append(sourcefileAttribute);
}
catch (Exception ex)
{
infoEvents.FireError(0, "SaveXML", ex.Message, "", 0);
}
}
In UI Class there is OK Click event.
private void btnOK_Click(object sender, EventArgs e)
{
try
{
_taskHost.Properties[CustomErrorControl.Property_SourceConnection].SetValue(_taskHost, propertyGrid1.Text);
btnOK.DialogResult = DialogResult.OK;
}
catch (Exception ex)
{
Console.WriteLine(ex);
}
#endregion
}
I want to enumerate all *.xls and *.csv file. How to fill the Files box? I tried
*.xls, *.csv
*.xls *.csv
*.(xls|cvs)
all doesn't work
Hi all,
In a foreachloop, I am inserting records into a flat file which is working fine. But the thing is that as the file grows, it takes longer for it to locate the EOF(End of File) of the flat file so as to insert the records.
I have around 70-100 lines written to the file at each loop and there are more than 20k records to be looped. wihich means that at the end I should be having 1400k - 20000k line in the text file.
One solution would be to insert the records at the start of the file itself so that it does not has to lookup the EOF each time before writting.
Another would be to generate separate files and then merge it.
Any idea how can this can be done?
Beside this I have to zip the file and then SFTP to a given address.
Any suggestion or help would be welcome.
Rdgs
David
Historically I've always written a VB script to copy a file from a sharepoint library. I don't like this method because I have to input a username & password in the script and maintain a config file.
Yesterday I was playing around with using a file system task. The sharepoint file has a UNC path so why not? I created a simple test package with a single file system task that copies the sharepoint file (addressed via UNC) to another network location. Package runs fine locally.
I try running on our utility server but am getting a "The file name [SHAREPOINT UNC PATH] specified in the connection was not valid" error. Package is running with a proxy on the server and the proxy account has the same permissions to the sharepoint site (so far as I can tell) as me.
I am trying to create and later read a data file from a package deployed in SSISDB, but it is not reading it while it is successfully creating the file. The same package when run from the file system package, runs successfully. Generating ispac and deploying in SSISDB is running for infinite time. Is it a permission issue?
View 7 Replies View RelatedI need to build an asp.net/C# application to read values from an Excel spreadsheet. Once the values are read from the spreadsheet, the C# code will do some elementary statistics on the values read. Then the values read and their computations will be written to a sql server database.
My manager suggested that SSIS might be a good candidate technology for doing this type of work. Does that sound correct? My only hesitation with using SSIS is that I want to keep the application as simple as possible, so that the code can be more portable. Maybe might argument is not a good one, but maybe someone can help me out here.
Ralph
Dear Friends,
I store several configurations in the main database of my SSIS packages. I need to get the servername from a xml or txt file in order to get those configurations stored in my database.
How you think is the better way to do that?
Using a FlatFileSource to read the file and a script to save the value into a SSIS variable?
Using the package configuration I cant do that... maybe I dont know, but I can save the SSIS variale in the configuration file, but what I need is to do the inverse, read the configuration file and save the value in the SSIS variable.
How the best way you suggest?!
Regards!!
Thanks.
Hi,
I am pulling text files in gzip format from UNIX system. I want to unzip these files and then import data from these files into database using SSIS.
I am trying to create an ssis package with dynamic csv file as output. and out format contains query output.
sample file name:
Unique identifier + query output + systemdate();
The expression is looking like this.
@[User::FilePath] + @[User::FileName] + ".CSV"
-- user filepath is a variable from ssis package. File name is the output from SQL query. using script task i have assigned the values to @[User::FileName] .
When I debugged the script task the value getting properly but same variable am using for Flafile destination. but its not working.
Is there anyway to send excel file from ssis using send mail task without saving the excel file locally. I need to automate the process which involves loading the excel file from the database and send it to some people.
View 6 Replies View RelatedOriginally in my DTS, the logfilename property is assign by a variable in a dynamic properties task.
After migrate, it appear commented in the script task. So may i know how can i assign the logfilename with the variable value?
Please advice, thank you.
Hi,
I am using an XML source component to integrate it into an SQL server data base.My problem is that when I try to open my data flow containing the XML source component, it take much time to open it, so I don't know why beacause I have a very good machine with 2GO of RAM. is it a problem of SSIS with XML files?
thank you in advance
I want to rename a file that is a variable to a fixed file name in a specific folder in SQL Server System Integration Services using the File System Task Editor. Example: File1.txt to Users.txt or File2 to Users.txt. The source file is a variable the destination file is not. Here is an example of the File System Task EditorHOW DO I SPECIFY A FIXED OR VARIABLE FOLDER NAME WITH A SPECIFIC FILE NAME?????????File System Task EditorProperties ValueDestination ConnectionIsDestinationPathVariable TrueDestinationVariable User::ArchivePathOverWriteDestination TrueGeneralName Rename a fileDescription File System TaskOperationOperation Rename fileSource ConnectionIsSourcePathVariable TrueSourceVariable User::ImportFilePath Thanks
View 1 Replies View RelatedWe are deploying our SSIS packages into different folders. For example:
Test1
est2
production
Test1 points to the Test1 database, etc.
So, I configured my SSIS package to use a database connection called dbMAIN.
I then setup the SSIS File Configuration (XML) so that "dbMain" points to Test1. This xml file, called Global.dtsConfig, sits in C: est1, the same place as my ssis.dtsx file.
I want to copy my dtsx file and my dtsconfig file into est2 and production. The problem is that the the location of dtsconfig is HARDCODED in the ssis package!
How do I dynamically change it?
I tried to hardcode the location to ".Global.dtsconfig", but that did not work! Please advise!
have a URL that I get from a partner by calling another URL and feeding it some query parameters. That call returns to me a URL which is what I need to then go get the file I'm downloading. I have no problem getting that URL into a variable in my SSIS package. URL....there are several key-value pairs after that question mark which denote the login information, the report ID that it's picking up, etc.
If I manually take that URL and dump it into IE and hit enter, I get the standard "Open/Save/Cancel" prompt. If I open it, I see the XML that should be in my file in the browser. If I save it, I get the standard save dialoge box so that I can save the xml file.How do I do this in SSIS? I can create a file connection for where I want this XML file to be saved, but I don't see an option in the XML task to simply save the file from a URL.
Hi,
I need to zipped files using SSIS, but not sure how to do that.
Zip application: 7-zip
Any help?
thanks
I am encountering a weird issue with SSIS.
I have a very simple SSIS package that executes a .bat file. Here's
the
actual file it executes:
@Echo Off
c:
F:
cd ReportingServicesScripts
rs -i NoteBlankSnapshots.rss -S http://10.90.160.13/ReportServerTest
EXIT
The .bat file executes a Reporting Services .rss script using the
rs.exe
utility. When I run the command from the http://10.90.160.13/ReportServerTest
EXIT
The .bat file executes a Reporting Services .rss script using the
rs.exe
utility. When I run the command from the command prompt, it runs just
fine.
When I execute the .bat file manually, it runs and completes. When I
execute
the actual package, it also completes. But when I schedule the package
as a
job, it just hangs...it never finishes.
The owner of the job is an administrator in SQL Server. I have the
SQL
Server Agent configured to interact with the desktop - although
my .bat file
requires no input from the user.
I've created other jobs that just execute plain old SQL using the
same
owneer and these jobs complete just fine.
What am I missing?
Any help would be appreciated!
Thanks!!
__________________
Anthony Robinson
"If I'm curt with you, it's because time is a factor here. I think fast, I talk fast, and I need you guys to act fast if you want to get out of this. So, pretty please - with sugar on top..."
I can use MS Access to import an XML file and it builds the columns and loads the data. I am trying to use SQL Server 2005 to do the same thing and there doesn't seen to be a way to easily accomplish the same task that works so well in Access.
Has anyone tried this? If so and you were successful, what were the steps to set up the package?
Thanks.
i need to download a file over HTTP and get a xml file then upload it into SSIS.
Only 2 methods are available in SSIS: FTP and web method. How come there is no HTTP transfer task in SSIS? pretty strange.
then i found a script at
http://www.sqljunkies.com/howto/49e823fd-d126-4134-893d-1fd8bd3bd3ba.scuk
What kind of SSIS tasks do i need to perform this operation? I got the script above but I don't know where to key it in.
What is raw file in SSIS, could you tell me samples for usage of raw file
View 4 Replies View RelatedHi ,
I need to create a CSV file by reading a table data , Can any one know how i can do this using SSIS .
My Requirment is that -
I have a temp table in which i inserted all the incorrect data during the load of a flat file using SSIS. Now i need to write all the incorrect data of the temp table in a CSV . Is there is a way i can do this using SSIS , Please help as i am new to SSIS.
Regards,
Ashish Soni
OK, I just want to know if I can use SSIS to open one text file create a table with the info in
SQL Server. Then open another fixed length file and insert rows in this table. I want to do this from an application in .Net. for e.g. I have a file that says "Col1 String 20, Col2 String 20,". This will create a table "Table1" in database DB1. then it will open a text file that has 200 rows for Col1 and Col2 with fixed length as 20. The table will be filled with the rows in the second text file. I want to give the user the ability to select the above files and when he clicks submit, the table will be filled. Is it possible?
Hi,
Using ssis, is it possible to refresh the data inside an excel file which is sitting on a separate machine (i.e. machine B) on the network?
When I try this using the importy wizard, the error message is:
The microdoft jet database engine can not open the file \pathnamedata.xls. It is already opened exclusively by another user, or you need permission to view it's data.
I have checked and the file is closed. No excel is running when checking the processes.
I have given myself full permission on that file from that machine where data.xls is sitting.
Thanks
Hello, I have a SSIS package that is making one database connection (dbMAIN). I am using the XML File Configuration to specify dbMAIN's properties (server, database, password, etc).
The problem is that the File Configuration contains entries for multiple databases (dbMAIN, dbTEST, dbSTAGING, etc)
When the SSIS is run through dtexec.exe, I get this error:
Error: 2007-04-10 16:52:23.39 Code: 0xC001000E Source: TEST Description: The connection "DDMAIN" is not found. This error is thrown by Connections collection when the specific connection element is not found.End Error
I do not want it to give me an error. The XML File must house all our database connections. What can I do?
`Le
is it possible that i can retrieve the list of all records that are duplicate and put them in a excel files
View 3 Replies View RelatedHello All,
In my project, developers enable configuration on the packages they develop. This means that if a project has 100 packages, each one has its own configuration file.
If our primary database server changes, I have to go an update 100 configuration files.
This defeats the purpose of having a configuration file in the first place.
I want to have a configuration file which contains common settings for all the packages in the project.
At the same time, I should have the flexibility of having individual configuration files for the project.
Is this possible and how can I do this?
Hi All,
I am looking for help trying to configure an XML dtsconfig file, I am using different connection managers for my Source and Destination Databases, I am also using an XML dtsconfig file to set these parameters dynamically, however I dont want to use userid/password to connect to my source/destination databases and instead use trusted connection or windows authentication, I remember we had an option in earlier DTS where we could set it up as 0 or 1and use windows authentication, I cannot find this in the new SSIS package properties.
Appreciate your help in advance.
Hi,
I have migrated a DTS pakage into SSIS. The DTS package validates a Textfile Source File using an ActiveX Script task.
Could somebody tell me how to validate a FlatFile in SSIS. Based on whether the file exist or if exist then whether its empty or not, I have to execute a database proc.
It'll be very helpful if somebody can assist me in this.
Thanks in advance.
Neetu
I'm getting an error when running my SSIS package in a SQLAgent job....
Message
Executed as user: DOITLHHQSQLT01SYSTEM. The process could not be created for step 1 of job 0x35D6B42101FB6646B20036599D407110 (reason: The system cannot find the file specified). The step failed.
This is the command line executing the SSIS package....
/SQL "ES PackagesES DEDUCTIONS" /SERVER DOITLHHQSQLT01 /MAXCONCURRENT " -1 " /CHECKPOINTING OFF /LOGGER "{6AA833A1-E4B2-4431-831B-DE695049DC61}";DestinationConnectionOLEDB
I'm assuming the file it cannot find is the SSIS package..."ES PackagesES DEDUCTIONS".
The step is running under the SQL Agent Service Account.
Anyone have any hints for me? The file is there when I look in integration services.
PS. I looked at the Configuration Manager and Integration Services in running under the Network Service account while the SQL Agent service is running under the Local System account. Could this make a difference? Should all the services be running under the same account? (I don't install this stuff so excuse my ignorance.)
I have created a package that creates a CSV file by exporting data from SQL Server 2005 sp2.
I am looking to have the process of naming the files automated so i don;t need to change the package every time i need the files generated.
is there away that SSIS can automatically name the CSV files using the database name that the data is coming from?
eg.
Database Filename
DB1 DB1.File.csv
DB2 DB2.File.csv
where file remains constant.
Hi,
i am dev new project which will be having 10 packages which can be having many source conn mgrs (flat file connection managers).
usually we will give file path when creating filat file conn manager. like that i many do for all packages.
when package is executed it searches for that file path in conn manager and file get execute where data is available normally.
But my req is i have to create file (say xml) where the path of folder(bec all the files which are used by all packages will be available in the same folder) should be available.
so server while executing package it has to search that xml first where it should find the path of the file and finally file has to be executed.
how can i acheive that req.plz guide me
Hi,
i am creating a SSIS package, for copying files from local meachine (where the pacakge) to other destination (\servernamefoldernamefilename) using script task. it's work fine while running package.
but if am schedule a job for the same its giving error (package execution failed)
Can anyone please help me this issue it's urgent
thanks in advace
sunil.