Hi All,
I'm new on SSIS, but have worked for some time with DTS and a long time with other ETL tools like Informatica or OWB.
I would like to know in which way can i, easily, control my project/package versions. At the same time i need to implement a concurrency management system, which will control what developer is using which package, and when finished update the central repository (As it does Informatica or even OWB).
I have heard that i could implement versioning with source safe, but can i implement this in the way that i've referenced before. Can i use CVS?
I am having a hard time with what appears to be something simple. I want to import an excel spreadsheet into a table on a daily basis from a command line. I created a package from the Import Wizzard in the SQL Management Studio and saved it. Since I want a clean table each day, my process needs to be create a temp table, import from the Excel file into the temp table. If that is successful, delete the original table and rename the temp table the original name. The point of this process is to provide for a fail-safe if there is some unforseen problem downloading the data on a particular day.
When I run the package, the first thing it does is delete the original table. I know this because the process shows the time that it finished is before anything else has started or finished. The time shown for the completion of the data flow task is about 2 minutes after that time.
This is maddening!!! The one thing I do not want to happen I can not seem to prevent. I have my control flow set on success. Why does it do this?
Our team has the following problem: I create a SSIS project in VS and add it to Team Source Control. When the project and all it's files exist on a machine and I want to open the project from Source Control, everithing works fine, but when another developer, who doesn't have the files on his machine, wants to open the project from Source Control he is required to check out the project file.
I've investigated the problem and found out that each project has a database file (with .database extension) and project file contains a reference to it. Once the user opens from Team Source Control a project that isn't presented on a target machine, the VS creates a new database file on a local machine and insists to change the reference in the project file.
I have all the config files for my various packages in a folder call configs. It seems that I cannot add folders to the SSIS solution. I can add config files, but they get copied into the root folder, which breaks the directory structure that I have.
Am I missing something? Is there a recommended to manage config files, and other external files used by the SSIS package, in SourceSafe and in the SSIS solution? Thank you.
Dear friends,In the area of GIS (Geographic Information Systems) there is a featureknown as versioning (long transactions).This feature allows databases to maintain different versions of data,in a hierachycal structure, in order to do simulation (what if ...),historical snapshots, concurrent editing, etc.Each version can be reconciled with its parent version in any moment(merge-post changes).I have recently seen that Oracle supports this feature from version9i.I am very interested in knowing if SQL server will support thisfeature in future versions. Looking at SQL Server 2005 doc. I haven'tseen any related info.Thanks,Jerry
I'm having problems to keep proper versioning control in place between the development and production environments. I'm running SQL2000 and we have hundreds of packages that runs daily. Some on shedules and some not.
Every time a package is saved, it creates a version in SQL Server. After development I want to be able to use something like "Visual Source Safe" and book the final version in that was moved into production. Something with a version number etc.
This is especially a problem if I want to roll back to a prior version of a package. I do not know which one of the 1000 versions to choose from that were created while developing the package.
Another problem is that I do not know if someone is working on a package if I want to work on it.
I can not run a search on all the packages to get a list of which tables/fields are used where to determine the impact of a program/database/design change that needs to be implemented.
I have an EDI file with Different Transaction types. I would like to read for a Header, capture some specific info on the header, and read for another specific tran type that comes after and capture additional info. I have a couple of ideas like tagging the records with a key and loading into two temp tables for matching later but that means I would have to do double processing. Maybe a conditional split for the two transaction types and then unioning them downstream, but not sure if the right records would be associated. Possibly tagging sequential key and writng to raw files and matching on the raw file keys downstream.
I have a lot of ideas but I am looking for the best proven practice here so i don't spin my wheels or have to go back and re-engineer later.
I have a big delimma. We are developing and application that requires parallel work to be done one different copies of databases at the same time. Then when one when group is done and ready to ship their bug fixes/features, the changes they make to the database and data have to be merge back into the baseline database.
Here's the specifics. 4 databases (that make up the product), 4 copies of the 4 database (one for each team).
I was thinking about using SQL DMO to attach to each database and comare each table's schema and data against the baseline (the current release) then scripting out the changes that were made.
Can someone give me some tips on how to maintain parallel database development and the merge process that can make this happen?
I replicate (transactional replication) my data entry database to aread-only database. Both are SQL 2000+SP4. The web server reads theread-only database. At times, there will be lots of changes in the dataentry database, thus lots of replications to the read-only database. Iam concerned that the replication may lock the data in the read-onlydatabase, causing slow response to the web server.I would like to use row versioning so that the read-only database cansupply old data when the same row is being written by replication. Iread that row versioning is a feature in SQL 2005. Is there anyversioning capability in SQL 2000?Thanks
I have a question about how I should go about handling different database versions (schema changes) with my application. I am using an sdf database as a local data store (either on the .NET framework or Compact Framework).
I set it up so that the database file has the database schema, but no actual data, which is copied to the AppData folder if it isn't already there. The I load the database into the dataset, and can store data in the database with no problems.
What I want to figure out is what happens when I later decide to change the database schema. For example, say I add a column to a table. When I load the existing database into the dataset, I get an exception because the existing database doesn't have that column.
It seems that there should be some way to update the existing database so that it adds the column into the table, and sets the rows to just have NULL for that new column.
I am not sure if the TableAdapter or some other object should handle updating the existing database so it matches the latest dataset schema, or if I need to manually write SQL statements to modify the existing database.
I was wondering if anyone knows if there is a way to version a report, after changes have been made kind of like there is to an application. when changes are made to an app with a version # of say 4.0.1 you can changge to 4.0.1. Without using a program like SourceSafe. Thanks in advance for the speed responses.
Could someone point me in the right direction? I have an internal development database and a production database. Is there an easy way to replicate the changes that have been made to the development version on the production server without modifying the actual data in the tables? So, if I add a new user in my development version I offcourse don't want to see it pop up in the live version. But adding/deleting/updating a new table or column should.
And if possible I'd also like to know how you could do the following: Let's say we have an OrderDetail table containing information about the purchased product. Let's say I'd like to add a new column 'total' to skip calculation on the database every time I want to know the totals. It should be able to initialise the value by doing 'times ordered * price' for every existing row. Is that possible as well?
Is there documentation somewhere about multiple execution paths in SSIS control flow? I didn't find documentation anywhere. I have a situation where I have two tasks that take considerable time, but could be executed in parallel (to speed up things) and I was wondering whether SSIS supports parallelism.
To illustrate the issues in simultaneous execution, I created a test SSIS package. In the package, I have five tasks, let's call them T1, T2, T3, T4 and T5. The taks are connected with "green arrows" like this:
T1->T2
T1->T3
T2->T4
T3->T4
T5 is not connected. The tasks can be e.g. Send Mail tasks, that's not relevant to this issue. I put a breakpoint in each task and execute the package.
When I execute package, T1 and T5 become active, i.e. the arrow that displays where the package execution currently is, is in two tasks simultaneously. Now F10 (step over) doesn't seem to work "Unable to step. Not implemented". If I press F5 nothing happens. After I press F5 for a second time tasks T1 and T5 and executed. Why don't they execute with the first pressing of F5? I would additionally like to know whether these two tasks are executed in parallel or sequentially, i.e. in the same thread or in two threads? Is there documentation of this?
The execution stops at T2&T3. Again, pressing F5 doesn't do anything, but the second time I press F5 T2 and T3 are executed.
I have created one custom task with PropertyGrid Control and two button on it. I have everything under one class library project. Problem I am facing is when i load task and clik on Edit I can not see those properties into that GUI and even functionlity of those two buttons (OK and Cancel) not working but I am able to see those properties in default property window.
If I create this GUI as a seperate window application then I am able to see those properties in GUI and buttons also working but in SSIS I am not able to load the task.
After reading on internet about SSIS they suggest to create everything under one project which I did.
Basically I am trying to populate connection managers like Source Connection and Destination Connection when I load this task and there are much more backend functionlity but at first step i m stuck and not able to see those properties in GUI.
Please help and give your input on it. I was following "Increment Task" example given by MSDN. If you need more info let me know.
How do you go about setting up configuration files to control connection strings when SSIS packages are migrated from a test environment to a production environment? Specifically, changing the server you are connecting to and possibly different login and password.
This is a question to the SSIS development team. I would like to know what are the requirements to implement custom SSIS Control Flow task in C++ . There is a documentation describing the process when implementing a managed task, but no such documentation exists for implementing a task in C++.
HI i need to write the Condition for Insert and Update Reccord depending upon the Prod_ID. How to write the Follwing condtion in the Condition Split? pls Anyone give me the Solution?
" if Prod_ID Exist then UPDATE Records
if Prod_ID Not EXIST then INSERT Records "
how to write the above conditon in the Condional Split?
I want to get the start time of data load and end time after data load and store it in a table which has mapping_id , mapping_name,start_time,end_time.
i use ActiveXScript task to get the start time before data load and store the mapping_id in a global variable,then data flow transformation occurs.
i want to use a global variable to store the mapping id ,so that i can update the end time after data load with that variable.how to do this?
is there any other way, i can get the start and end time of data load (other than the logging information)?
At ScottGu's Blog about "Database Publishing Wizard", AlexD from codeplex said: "Regarding the multiple requests for versioning, backup/restore of remote database, and selection of individual objects - these are all things we are actively looking at for our next release in the first half of 2007." After so many searches, I still don't know if this tool performs Versioning, i.e, when deploying the database, just update de diferences between the local and server database. Did Visual Studio Express 2008 have somethingh like that? (I know that VS Team Edition 2005 had). If this tool can't make it (versioning), which tool/method did you recomend me? Thanks in advance. Alberto
I did a seach here and found some posts but none that answered my specific problem.I am a programmer tasked with building an application for generating Quote Proposals. The database is for the most part fairly simple except when it comes to versioning and history. Basiclly every quote can be revised and modified several times prior to making a final decision (final approved quote), so I need to keep track of the changes that occur durning the revision process. I am not a DBA but I have had some database experience. From what I can tell I have two choices a.) Duplicate all the data everytime the quote is revised. While this method is does cause a lot of duplicate data it is very straitforward and easy to explain (or turnover to someone else) and reporting becomes very easy as well.Reporting.... this is my biggest area of concern as the users of the app should have the ability to print out the original quote proposals as well as the revised quote proposals. Duplicating all the data makes reporting very easy.b.) Create a history table and record the original data (along with who and when) before recording the new value in the main table. While this method does conserve disk space it make reporting a bit difficult as you would have to pull the specific value for a specific Quote Revision and display the orginal values on the report instead of the current ones.Table info: I looking a 10 to 12 tables to record and store the data. The largest table will have about 40 fields. Current estimates are producing about 5 to 8 quotes per week. Each quote is revised an average of 2 to 3 times.Are the pros and cons I listed the main ones to be concerned with and are there any other options?Thanks
I have a small webapplication, sql server 2000. The users can only update the data in the system. However my client needs a report that display changes. The changes are that needs to be monitored are only change of order status, change of delivery date and when a user splits an order.
What is the best practice to keep track of changes? A mirror table for each table with changes?
I am working on an SSIS project where I create two flat files for submission to a data contractor. This contractor requires a control record be the first line in the file. I create the control record based on the table information being exported.
What I would like to know is, is it possible to utilize the Header Section of the Flat File Destination Editor to insert the control record? And, as it is dynamic, what kind of coding must I do in order to utlise this functionality?
For the past few months I've been developing an DW and ETL with SQL 2005 / SSIS. My packages are being deployed to a SQL Server. Although in the end game we will have a Dev/Staging/Production environments, I would still like to archive production packages when we push staging to production. Essentially I would like to archive the last X packages that were deployed to production where X is a reasonable number (3 - 5). I don't necessarily need to have them accessible to run. One of the purposes is to have another safeguard should we miss anything in user testing and need to roll back a deployment.
I am utilizing VSS and we will have backups running on the production server, but I would prefer to have a archive that is a little more accessible.
I just wondering if anyone has any thoughts on how to extract/archive production packages when the push is made. I could easily develop an app that queries the MSDB and exports the packages to the file system.
I installed the Feature Pack Balanced Data Distributor control on my PC to use with SQL Server 2014 64 bit. I have used the control with SQL Server 2014 and SSDT so I was familiar with the process. Unfortunately, I cannot get the control to appear in the toolbox. No error messages appear, BDD just doesn't appear in the toolbox. I have tried un-installing, reinstalling, installing SQL Server 2014 SP1, installing again, rebooting a number and nothing works. The control just does not appear in the toolbox. It doesn't not appear when I go to choose items either. What does it take to get BBD to appear in the SSIS Toolbox for VS?
Why isn't there some documentation on how to do this. This should be really simple and it has taken me 2 weeks and I still haven't gotten an answer. Please Help Does anyone know the answner or some place where there is some documentation!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
I get the following error when I try to substitute the strings in the databasedetails collection with variables: Error: Object reference not set to an instance of an object. StackTrace: at Microsoft.SqlServer.Dts.Tasks.TransferObjectsTask.TransferObjectsTask.CheckLocalandDestinationStatus(Database srcDatabase, DatabaseInfo dbDetail) at Microsoft.SqlServer.Dts.Tasks.TransferObjectsTask.TransferObjectsTask.TransferDatabasesUsingSpAttachDetach()
I created the following variables: strDestinationDB = AirCL2Exp_new3 strDestinationDBPath = C:Program FilesMicrosoft SQL ServerMSSQL.2MSSQLDATAAirCL2Exp_new3_Data.mdf strDestinationLGPath = C:Program FilesMicrosoft SQL ServerMSSQL.2MSSQLDATAAirCL2Exp_new3_Data.ldf strSourceDB = AirCL2Exp strSourceDBPath = C:Program FilesMicrosoft SQL ServerMSSQL.2MSSQLDataDataNewAirCL2Exp_Data.mdf strSourceLGPath = C:Program FilesMicrosoft SQL ServerMSSQL.2MSSQLDataDataNewAirCL2Exp_Log.ldf
I then assigned those variable to DatabaseDetails Collection:
Inaddtion I also assigned the following to the two DatabaseFiles Collection: for 0: DatabaseFileSize = 0 DestinationFilePath = @strDestinationDBPath FileType = DatabaseFile SourceFilePath = @strSourceDBPath SourceSharePath = @strSourceDBPath
Edited:Maybe I should have posted this to the "managed" newsgroup. If any admins think that would be better, then let me know. I don't want to duplicate unnecessarily.
Hi,
We developed a custom Control Flow task for SSIS (2005, not yet had a lot of time to look at 2008 yet) and found that it does not handle versioning, or an uninstall and the resulting lack of an addressable component very gracefully.
Here is a typical scenario:
Baseline
Install component MyCustomTask 1.0 Create Project Save Project
Action 1
Uninstall MyCustomTask 1.0 and don't install the new version (a typical user scenario!) Open Project SSIS acts like the world has ended, especially if the user forgot to manually remove the item from the toolbox Fix:None, obviously, but it would be nice to be a bit more graceful and informative. Backdoor Toolbox fix: "Cleanse" the toolbox when it goes haywire by deleting the toolbox.tbd, in Documents and Settings<UserName>Local SettingsApplication DataMicrosoftVisualStudio8.0
Action 2
Uninstall MyCustomTask 1.0 cleanly, plus removing the toolbox item by hand. Install MyCustomTask 1.1, with identical interfaces etc, and add the toolbox item by hand. SSIS acts like the world has ended, and fails to ask you a sensible question like "do you want to upgrade the project to use the new version of the component" Fix:Identify major and minor version component changes and throw the user a rope. Backdoor Fix: Go into the DTSX manually - attack the DTSExecutable ExecutableType and DTS Name, for a Task in our case and replace it with the new version info. Even if the interface for the component has changed slightly, it seems to deal with that OK.
Given the fact that it seems to be very likely that there will need to be SSIS version specific builds of components (I am assuming that a task created in 2005 will not work with 2008), what is the best way to deal with the current lack of SSIS smarts.
Would this be the best approach:
Version the interfaces, but never the builds within a version i.e. My.CustomTask90 v1.0, My.CustomTask100 v1.0 etc.
This is a bit of a pain, rather than the simpler My.CustomTask v9.0 / v10.0 etc.
Or, are there some nice improvements in the pipe to alleviate this, plus perhaps even a way to programatically add components to the toolbox, rather than the low-rent method of getting the user to do it by hand.
I recently upgraded to on 2012 SP1 CU5 and have found the SSDT gui for SSIS to be almost unusable. I can't drag or resize items. Any time i try they either automagically shrink to the tiniest possible size, shoot off to some extreme or just shake uncontrollably I didn't have these problems on previous versions (dont remember what It was).
I am Using Derived column between Source and Destination Control. the Source input column PriceTime is String Data type. but in the Destination is should be a DATE TIME column. How to Convert this string to DateTime in the Derivied Column Control.
I already tried to in the Derived column control
PRICEDATETIME <add as new column> ((DT_DBTIMESTAMP)priceDateTime) database timestamp [DT_DBTIMESTAMP]
But still throwing Error showing type case probelm
