when using a raw file destination it would be nice to be able to use an environment variable for the filename property.
like
%my_extract%data.txt
instead of
c:my_extractdata.txt
I have about 40 DTS packages that I want to run against three different databases on the same server. Can someone suggest an easy way to run these jobs and differentiate each time which SQL or INI file to use.
Thanks,
John Shaening
Hi all,
does anybody know if it is possible to use environment variables when calling dtexec utility?
I'd like to run packages stored on server's file system from directory that I've had specified in an environment variable called SSIS_PackagesPath.
Now, I'am trying to write dtexec command, where path to the actual SSIS package would be concatenation of environment variable (i. e. path to package directory) and name of package itself (written explicitly). Is this syntactically possible?
The reason behind is to be able to easily modify package storage directory for multiple scheduled jobs that run SSIS packages.
Any other ideas are hapilly welcomed.
Thanks,
Marek
I am migrating the BE of an Access app. to SQL server 2012. I need to get the user's login name (Windows Authentication login). This can be done using xp_cmdshell, but, xp_cmdshell is considered dangerous and I wouldn't be able to run it once I deploy the app. to the company servers (currently I have SQL server on my computer and as an admin I can enable xp_cmdshell to run, but IT doesn't allow it in company servers for security reasons).
Another question, is it possible to send data from the logged in user from Access to SQL server? What I need to do is let SQL know the username of the logged in user, then, use it to filter the data on SQL. Idea is that user can only run queries for his data (he can't view other user's data unless he is a manager or an admin (currently the app. in Access logs the user in automatically if his Windows Domain username is found in the user's table, and set's his role found in the Roles table). It is this functionality that is giving me some problems to migrate to SQL.
I created a function that uses the System_User SQL built-in function, this retrieves the SQL login username, but, the app. uses 1 SQL local account to connect to the server, so in essence it doesn't work as I need the Windows Domain account username.
Is it possible to export the environment variables for an SSIS 2012 project? And if so how is it achieved?
View 2 Replies View RelatedI noticed accidentally today on my PATH and discovered that I have several variables for different sql versions. I can understand this as I have upgraded from 2008, to 2012 then 2014. Here are the list of variables
C:Program FilesMicrosoft SQL Server100ToolsBinn;
C:Program FilesMicrosoft SQL Server110ToolsBinn;
C:Program FilesMicrosoft SQL Server120DTSBinn;
C:Program Files (x86)Microsoft SQL Server120DTSBinn;
[Code] ....
Can I delete all variables related to older versions safely without any issue.
Hi all,
We're just in the middle of performing our first release of SSIS packages through various environments.
The way we are set up currently is the developer will check the package(s) and related config files out of source control, develop on their own machine and check everything in again. Then we deploy the packages consecutively to the Dev, Tst and Prd servers.
We are going down the path of using one environment variable for every config file. some packages share config files (e.g. we've only one config file for each database or ftp connection etc.) and some config files are package specific (error log file connections and success/failure e-mail sources etc.).
What we want ideally is a script that we can check into source control that will create the environment variables on a server at deployment. The "set" command at the command line can be used to change the value of an environment variable, or to create a session-specific variable, but not to create environment variables.
So far the only method that we're using is manually typing in the environment variable names via control panel and copying and pasting the paths into the value fields. Given that we're deploying potentially hundreds of config files, it's obvious that this new-fangled GUI point-and-click and copy-and-paste method of deployment is absolutely foolproof and totaly not prone to any error whatsoever.
Please tell me there's a way to create and set environment variables without going through control panel. running a script or something to do it automatically will: Ensure that each environment is set up accurately and identically, eliminating human error. Ensure that when a developer checks out a package to their local drive, although they may have to change the variable values, he can at least create the relevant variables without having to type them in. Enable efficient migration to another new server (for example during Disaster Recovery).
Can anyone point me to some example scripts at all?
Kind Regards,
Andrew.
I have deployed a project with multiple packages to SSIS 2012 db. I am able to configure the project parameters fine. But, I am not able to replace the package variable values with the 'Environment' variables.
View 3 Replies View RelatedAny fix for the seemingly random sort order of the variables in the dropdown list when configuring parameters and connection managers in the SSISDB catalog?
I imported all of our connection strings into an environment (about 200 of them). They were inserted in alpha order and the ID values within the internal.environment_variables table shows them in order as well, by ID and by name.
When I run profiler and capture the command that retrieves them and run it in ssms they are in order but in the dropdown they seem random.
There are no values within any of the tables that accounts for the order they are in.
If a package has 5 connections you need to go through the unsorted list 5 times to find them.
Sometimes you get lucky and they are in the first 20 or so.
I know I can write a script, just wondering if there is a fix for the sorting.
Hello -
I am making good progress with my ssis package. However, there is one new thing which I cannot graps yet. That is, how to use variables when I want to update or insert a new row. I have some columns in my tables that require the datetime that the update/insert occured, the person making the change, and a few other things that are not part of the incoming data source (an excel file).
I created some user variables for these things, but I cannot figure out how to use them with my OLE DB Command and OLE DB Destination. One handles Inserts and the other handles the Updates based on whether a row in the Excel file is new (an Insert) or already exists (an update). Along with the insert or update, I'd like to set the Lastupdate, Who, etc.
Thanks for any help
- will
I have SSIS 2012 Enterprise, using catalog deployment and have more that 50 environment variables for connection to databases across my enterprise.
The problem when i go to configure the packages after deployment and pick the proper env variables, that are not sorted, so i have to browse all entries in order to find the proper entry in environment variables.
Im using the SQL Server destination object to load some data (about 10gb) into a table. However I would like to use a variable to define what table I load into. All the tables have the same structure. I dont see a way to bind a configuration to it, or to use a variable as the loading table name.
Thanks.
Need to know how I can get the dynamic filename created in the FlatFile destination for insert into a package audit table?
Scenario: Have created a package that successfully outputs Dynamiclly named flat files { Format: C:Test’Comms_File_’ + ‘User::FileNumber’+’_’+Date +’.txt’
E.g.: Comms_File_1_20150724.txt, Comms_File_2_20150724.txt etc} using Foreach Loop Container :
* Enumerator Set to: “Foreach ADO Enumerator” with the ADO object source variable selected to identify how many total loop iterations there are i.e. Let’s say 4 thus 4 files to be created
*Variable Mappings : added the User::FileNumber – indicates which file number current loop iteration is i.e. 1,2,3,4
For the DataFlow task have a OLDBSource and a FlatFile Destination where Flat File ConnectionString is set up as:
@[User::Output_Path] + "Comms_File"+ @[User:: FileNumber] +"_" + replace((DT_WSTR, 10) (DT_DBDATE) GETDATE(),"-","")+ ".txt"
All this successfully creates these 4 files:
Comms_File_1_20150724.txt, Comms_File_2_20150724.txt, Comms_File_3_20150724.txt, Comms_File_4_20150724.txt
Now the QUESTION is how do I get these filenames as I need to insert them into a DB Audittable. The audit table looks like this:
CREATE TABLE dbo.MMMAudit
(
AuditID INT IDENTITY(1, 1) NOT NULL,
PackageName VARCHAR(100) NULL,
FileName VARCHAR(100) NULL,
LoadTime DATETIME NULL,
NumberofRecords INT NULL
)
To save the Filename & how many records in each file in our Audit Table, am using an Execute SQL Task and configuring it as this:
Execute SQL Task
Parameter mapping - Mapped the User Variable (RecordsInserted) and System Variable( PackageName) to Insert statement as shown below
SQLStatement: INSERT INTO [dbo].[MMMAudit] (
PackageName,NumerofRecords,LoadTime)
(?,?.GETDATE)
Again this all works terrific & populates the dbo.MMMAudit table as shown below BUT I also need to insert the respsctive file name – How do I do that?
AuditID PackageName FileName NumberOfRecords
1 MMM NULL 12
2 MMM NULL 23
3 MMM NULL 14
4 MMM NULL 1
I have a ssis package where I need to have excel destination. In the Excel file, I need to have few rows with some text and then populate data below the text. One the text is like this:
Data as of: 08/25/2015
if the report ran today, then Data as of will have Yesterday. So, if the user opens that excel file after a week, then user should see same Data as of: 08/25/2015. not today()-day(1).
I was planing to handle on excel side with today()-day(1). but it only works the day it was run. Then the excel file is open after few days later, then it might as Data as of: 08/30/2015 which is not true. It should still stay Data as of:
08/25/2015 on what ever date the excel file is open. The SSIS package runs only once.
How do I handle this so that whenever user open the file, they will see Data as of: 08/25/2015. This is not a column in excel. It is like a description of data in excel.
I am having an issue with the File System Task.
I was wondering if there is a way to 'Move File' with the File System Task inside of a For Each Loop container but to dynamically set the Destination path variable.
Currently, this is what I have:
FileDestinationPath variable - set to C:TestFiles
FileSourcePath variable - set to C:TestFiles
FileNameAndLocation variable - set to blank
For Each Loop Container Iterates through a folder C:TestFiles that has .txt files in it with dates in the file name. Ex: Test_09142006.txt. Sets the file path (fully qualified) to the Variable Mapping FileNameAndLocation.
Script Task (within For Each Loop, first step) Sets the FileDestinationPath to the correct dated folder within C:TestFiles. For example, if the text files I want to move are for the 14th of September, it takes FileDestinationPath and appends the date folder to the end of it. The text files have a date in the file name (test_09142006.txt) and I am picking this apart (from FileNameAndLocation in the For Each Loop) to get the folder date. (dts.Variables(User::FileDestinationPath?).Value = dts.Variables(User::FileDestinationPath?).Value & ? Month & _? & Day & _? & Year & ?) which gives me C:TestFiles 9_14_2006?.
File System Task (within For Each Loop, second step) This is where the action is supposed to occur. I want it to take the FileDestinationPath and move the FileNameAndLocation file (from the For Loop) into this folder for each run.
Now as for my problem. I want this package to run everyday but it has to set the FileDestinationPath variable dynamically according to that days date. Basically, how do I get this to work since I cant hard code the destination path variable from the start? I have the DestinationVariable on the File System Task set to the FileDestinationPath variable, after the script task builds it. However, using FileNameAndLocation as the SourceVariable on my File System Task tells me that the Variable FileNameAndLocation? is used as a source or destination and is empty.?
Let me know if I need to clarify further...I may be missing something very simple. Any help would be greatly appreciated!
I am running my package in sql server 2012, in which i am giving network path for flat file destination. And its working fine. But if i give m local path, its giving me error " cannot open data file" ...
Nothing is wrong with package.
Hello friends,
I have the following (simplified):
1. Flat File Source
2. Conditional Split, Case Good = !ISNULL(KEY) Case Error = ISNULL(KEY)
3. Case Good -> Writes to Good Flat File (with timestamp in the title)
4. Case Error -> Writes to Error Flat File (with timestamp in the title)
Most job runs have no errors but the error file is created as a zero byte file anyway. If there are no error records I don't want the error file created. How might I accomplish this?
Thanks
Hi,
I'm using connection managers for all the connections i have in my packages in one project. However, if i change from one environment to another, i have to go to each connection manager in each package just to set the connection.
is there a faster way to change them like a configuration file lookup or something?
cherriesh
I have finished a change request from our client. I need to update clients' database with the one in developments.Here is the changes i made to database:Added/Changed some tablesAdded/Changed some stored proceduresAdded data to some dictionary tableThe data in clients' current database MUST be kept. So how can I merge the changed information to clients' database?
View 3 Replies View RelatedI'm doing a simple ETL that reads a database table and dumps the content to a text file. The text file will be named Employee.txt. This file name will remain the same across my environments, but I may want to vary the directory location to where I want this file dumped.
So, I defined an environment variable called "DataTargetDir" in all my environments. Now, I want to utilize this variable in the "File name:" box within the Flat File Connection Manager Editor. How do I do this? I'm thinking I can write something like "%DataTargetDir%Employee.txt" in the "File name:" box, but it's not working.
Am I approaching this the right way?
Hi,
I am testing SSIS and have created a Flat File Destination. I defined the Flat File Connection as New for the first time and it worked fine. Now, I would like to go back and modify the Flat File Connection in the Flat File Destination Editor, but it allows only to create a New connection rather allowing me to edit the existing one. For testing, I can go back and create a new connection, but if my connection had 50-100 columns then it would be an issue to re-create it from scratch.
Did someone else faced this issue?
Thanks,
AQ
We have a valid full backup of a database. We know it is valid, we have restored it twice from the network with no problems, but we do not have access to the network location from our sandbox environment.
The .bak file is sizable at about 9GB. The .bak file resides on our internal network, in a SAN drive. No problems there. When we copy the file from there into a sandbox environment to attempt the restore in the sandbox environment it gets corrupted. We've tried three different times and all three different times it gets corrupted. First time we copied the file over to the sandbox the restore went up to 50% and failed. The second time we copied the file again and attempted the restore again it failed at 70%.
The third time it failed at 60%. The error message we get during the restore is "...Read on ... failed: 13(The data is invalid) Msg 3013, Level 16, State 1, Line 1 Restore database is terminating abnormally."
Now some background here. To move the file our network team is doing this: they have this .vmdk file that they mount out in our production environment (which has access to the network location where the .bak file is), copy the file into the drive, then move the .vmdk file into the sandbox environment(which does not have access to the network location), mount the drive in the sandbox environment, and then I have access to the .bak file from within the sandbox environment.
Something in the process of using the .vmdk file to move the .bak file from production into the sandbox is causing the file to get corrupted.
Hi,
Is it possable to have control which name the flatfile has? F.ex. have the same process created files with the name:
filename_2008-01-01
filename_2008-01-02
filename_2008-01-03
Thank you.
Hello,
I'm trying to send a file with FTP.
In the Flat File connection manager I have to write the name of the file.
Is there any way of deciding at runtime what the name of the file should be?
Thank you.
I am transferring data from an OLEDB source to a Flat File Destination and I want the column width for all of the output columns to 30 (max width amongst the columns selected), but that is not refected in the Fixed Width Flat File that got created. The outputcolumnwidth seems to be the same as the inputcolumnwidth. Is there any other setting that I am possibly missing or is this a possible defect?
Any inputs will be appreciated.
M.Shah
Hello All, I want to know.
I have an SSIS Map which I am running. The destination is Raw File Destination.
I want to know what is the Raw file destination.
I mean in what format and where I can view this data.
I tried notepad but its not showing proper results
Please let me know thankss
Hi,
I am exporting a table to an excel format. The package should change the name of the destination file everytime the package is executed(preceded by date). I need to deploy this urgently.
Hi, I am new to SQL Server. Trying to convert a pc sas program to a SQL Server 2000 DTS package. Need fixed format comma delimited destination text file from a DB2 data source. Problem: SQL wants to make financial field 19 bytes long but I want it to be 12. Tried casting as a DECIMAL(7,2) but SQL still wants it to be 19 bytes long. Tried converting to CHAR but then it is still left-justified. I need decimal to be in the third byte from the right. Is this possible?
View 1 Replies View RelatedHi,
How can I check if my Flat File Destinatino is empty or not?
Im sending it with FTP, but I dont wanna send an empty file.
Any ideas?
Thank you.
Hi,
I try to use Data Transaction Service (SQL Server 7) to copy information from an Sql Server Table to an text file (fixed field). When I run the process, no problem, the text file is created and a Ok message appears. But when I look in my text file, every time a field was NULL or empty in the source table, the following fields are not aligned !
Somebody knows what the problem ?
Thanks
Vincent
Ps: I'm french so excuse me for my english
Hi,
I'm trying to output the results of a query straight into a raw text file. The problem I'm facing is that the tool seems to write some unwanted characters at the top of the file and within it. For example, I get the name of the column that was used as input at the top of the text file:
[] [] <<name of column>> [] []
I need to have a clean file containing strictly the results of the query.
Any ideas?
It's funny:
Everyone thinks that CSVs are awesome to transport data. I mean after all, SSIS defaults to Comma Delimited files. Even Excel defaults to it. Microsoft is supposed to be our leader! They should get this right. And get the terminology correct too. How many people describe the file as comma separated? It's delimited! Even SQL server calls it delimited by the "delimited" drop down.
CSVs suck and i will tell you why:
When you transport any text field (especially Address) it has the possibility of containing a comma. This causes data to be parsed into the wrong fields. Why in God's grace would you EVER get in the habit of choosing a delimiter that SOMETIMES doesn't work?
I'll tell you a little short story:
I have been waiting (at work) for like a month for a guy to export data and give me this file. Well today i finally got that file. He was in a hurry and used the Defaults to export this file. I don't blame him for being in a hurry and using the defaults. Well the defaults made the import NOT work because the data parsed into the wrong columns. Comma delimited would work if there was a text qualifier. But the default is comma delimited and NO text qualifier. What idiot thought that would be good. Or it was the separation of duties that eff'd this one up? Who knows?
Fix your products!
That's my .02
I am using SSIS to create a weekly data extraction that will be emailed to an external agency. I can extract the data, create the file and email it without any problems but I want to compress and encrypt it before I send to give some measure of protection to sensitive data it would contain.
If I did this manually, I would simply use Winzip to compress and encrypt the file to be sent. How can I achieve a similar result programtically?