SSIS Layout/Design Basics

Nov 29, 2007



I have 3 'separate' but related processes that I am developing in SSIS. Each process shares a few global variables and connection objects. None of the processes depend on inputs or outputs from the other processes. However, there is a sequence order in which they are exectuted.

Should I develop these three process in the same package or 3 separate packages in the same solution?

From the point of view of sharing the same variables and connection objects, maybe just one package? If so, would it be advise to use the toolkit's 'Sequence Container' to ensure the order in which the 3 processes are run? I noticed that 'global variables' are limited to the scope of individual packages so it would appear that a variable can't be shared between packages in the same solution? I was also thinking that in terms of configuration settings, there might be an advantage of keeping everything in the same package. The only disadvantage I can think of in relation to keeping everything in the same package is that the design layout might look a bit cluttered.

I'm tending to think one package here but would welcome comments and suggestions from those that have experience of similar design scenerios.

Thanks,
Clive

View 4 Replies


ADVERTISEMENT

Report Layout Design Tip

Mar 25, 2008

I'm trying to create a report like following format. Any idea how can I design it?
'ID', 'Title', 'Answer' and 'Comment' have to span multiple rows which will list 'Response'.

----------------------------------------------------------------------------------------------------------------------------
ID Title Answer Response Comment
------------------------------------------------------------------------------------------------------------------------------
1 | Q1 | B | A | aaaaaaaaaaaaaaaaa
| | | B | aaaaaaaaaaaaaaaaaaaaaaaaa
| | | C | aaaaaaaaaaaaaaaaaaaaaaaaa
| | | D |
---------------------------------------------------------------------------------------------------------------------------------------
2 | Q2 | C | A | bbbbbbbbbbbbbbbbbbbbbbbbbb
| | | B | bbbbbbbbbbbbbbbbbbbbbbb
| | | C | bbbbbbbbbbbbbbbbbbbbbbb
| | | D |
---------------------------------------------------------------------------------------------------------------------------------------

View 12 Replies View Related

SSIS Basics Working With The IDE

Apr 1, 2008

Is the intent of Microsoft to enforce a vertical presentation of the SSIS Control Flow, Data Flow etc? I like to work down and to the right and I am having problems with the connectors (precedent) or in DTS 2000 world Success or On Success implied. Is there away to connect from the lower center anchor point to the side of the next object in the flow? For some reason, I can't get a similiar look to DTS 2000 environment (going from the bottom middle of the object to the side anchor pint (small box) symbol to the left. It wants to anchor on the bottom right, left, or middle, if going horizontally. I must be missing something, here??

Thanks again!

Skyseek


View 2 Replies View Related

SSIS Package Designer Error Displaying The Layout

Aug 10, 2007

Our Visual Studio installation suddenly starting producing the error below upon trying to open any .dtsx package; although by "suddenly" I'm sure it was due to some Windows update or other program install, update, or uninstall. I've spent two days uninstalling, reinstalling, and repairing everything that is seemingly related to VS, but no luck. I've searched the forums and could not find this specific error mentioned, although I have tried some of the suggestions to fix possibly related 'class not registered' errors, but again no luck.

===================================
There was an error displaying the layout. (Microsoft Visual Studio)
------------------------------
For help, click: http://go.microsoft.com/fwlink?ProdName=Microsoft%u00ae+Visual+Studio%u00ae+2005&ProdVer=8.0.50727.762&EvtSrc=Microsoft.DataTransformationServices.Design.SR&EvtID=DDSErrorDeserializeDiagram&LinkId=20476
===================================
Class not registered (Exception from HRESULT: 0x80040154 (REGDB_E_CLASSNOTREG)) (msddsp)
------------------------------
Program Location:
at MSDDS.IDdsDiagram.CreateShape(String ProgID, Boolean Visible, Int32 X, Int32 Y)
at Microsoft.DataWarehouse.Controls.Interop.AxMSDDS.CreateShape(String progID, Boolean visible, Int32 x, Int32 y)
at Microsoft.DataWarehouse.Design.ComponentDiagram.OnCreateShape(Object logicalObject, Int32 x, Int32 y, Boolean visible)
at Microsoft.DataTransformationServices.Design.DtsComponentDiagram.OnCreateShape(Object component, Int32 x, Int32 y, Boolean visible)
at Microsoft.DataWarehouse.Design.ComponentDiagram.CreateShape(Object logicalObject, Int32 x, Int32 y)
at Microsoft.DataTransformationServices.Design.DtrControlFlowDiagram.CreateExecutableShape(DtsContainer dtsContainer, Int32 x, Int32 y, Int32 width, Int32 height)
at Microsoft.DataTransformationServices.Design.DtrControlFlowDiagram.RefreshSequenceExecutables(IDTSSequence sequence)
at Microsoft.DataTransformationServices.Design.DtrControlFlowDiagram.AfterDeserialize()



Any help would be greatly appreciated.

TIA,

Scott

View 7 Replies View Related

SSIS Loses Layout Of Precedence Constraints When Saving.

Oct 29, 2006

This is more a medium level annoyance than a problem. When I save a package, close it and later re-open it, the precendence constraints end up all over the place making my neatly layed out diagram a shambles.

Call me anally retentive if you like (you wont be the first ), but I find this extremely annoying. Does anybody else have this problem? Does anyone have a solution?

View 12 Replies View Related

Confusing Layout In SSIS With Regard To Execute SQL Task.

Feb 22, 2007

I hope someone can help.

I'm trying to read rows from a SQL Server Table and for each row use a few columns as parameters into a query to be run against oracle which will delete oracle rows.

I add OLDEB connections for Oracle and SQL and then I try to add a "Execute SQL Task". I've also tried a "OLE Command" but I can't get the mapping of the columns to the parameters to work.

There is lots of articles on the web that talk in general around parameterized queries but no clear examples.

I also find the difference between the Control Flow and Data Flow tabs confusing as its not intuitive where to place things. It also appears to force me to re-define things that it should already know (this is no doubt because I'm interpreting what I've done / acheived wrongly).

I have my source and destination on the "Data Flow" tab along with a "Execute SQL Task" object in the middle.

I'm setting its "connection manager" the Oracle (i.e. the destination where I want the deletes to be executed). I don't follow why this also has a "connection property, surely this it set when I drag the output of the SQL Server OLEDB Source to the input of the "Execute SQL Task".

Perhaps I'm expected too much from the wizards / dialogs and I have to create "variables" and "parameters" myself?

Any help or suggestions would be very much appreciated.

Thanks in advance

Craig

Scotland

View 3 Replies View Related

View Layout Is Different Than Print Layout

Nov 21, 2007



I designed a report in reporting services 2005. When i look at the view layout for the report everything is aligned correctly but if i go to print preview one matrix is shifted up or down. I have tried to put the matrix inside a rectangle but the issue is still present. Does anyone know why this is happening or any suggestions on what to do to fix it.

Thanks
Scott

View 3 Replies View Related

SSIS Design Question - New To SSIS, Please Comment! :)

Feb 20, 2008

Hello everyone!!

I am completely new to SSIS and have been given a large project (of course with a tight deadline) that has the absolute requirement of using SSIS. I am/was very, very good with DTS and could easily accomplish what I need to do with an ActiveX script task in DTS in no time, but as this is new development, we are not to use ActiveX script tasks within SSIS since it will not be supported in the next SQL Server release. I'm thinking script task, but please give some comments on how you would accomplish the following in SSIS (please remember I'm new to SSIS, so don't assume I know anything. )

I must accomplish this: in a nutshell, I need to create separate tab delimited text files of customer informaion. One for each region. Each region consists of X amount of states and we have X amount of regions. (Pseudo code followed by standard explanation)

Select a max value from region lookup table in SQL (this is the # of regions)

for N=1 to MyMaxValue

select states from region lookup table where region code = N (the current region we are on)
'this returns a list of states in a region, need these in array or recordset object or something
Open an output file which will be a tab delimited text file we will write results below in loop to (in DTS I would programatically kick off a transformation task in the package)
'loop thru states returned, so if in a rs object...
do while not rs.eof

execute customer stored procedure, passing as a variable the current state we are on
'this will return all customers within a state, this whole result set (approx 1 million) needs to go to the tab delimited file
'I have to execute this stored procedure for each state & then write results to the SAME file, until we are onto a different region

rs.movenext
close file
loop
next

OK, so basically, as you can see, Its sort of simple in a way what I need to do, i just have no idea how to go about doing this in SSIS. I can not hard code any state or region values. I MUST read them in from the lookup tables as region codes are constanatly changing and we are constantly adding in new states and new regions, so with above coding idea, it would always dynamically pick up any new states, new regions or changes.

So in a nutshell, I need to create separate tab delimited text files of customer informaion. One for each region. Each region consists of X amount of states and there are X amount of regions. Pretty strait forward, huh? The requirements are strait forward, but SSIS is throwing me for a loop... it does not seem flexible enough to be as dynamic as I need it to be to do this. I'm sure it is, just my understanding of it is very basic so far.

Please provide your suggestions! I think a lot of newbies would benefit from some SSIS design info... how to do common things in SSIS, but beyond just retrieving a recordset and writing it to a file... what do you do when you need to add just a few layers of decision processing, and retriving recordsets and writing files based on that decision processing?????

THANK YOU!!!!!!
Kathy

View 7 Replies View Related

Basics Of SQL

Oct 24, 2005

Hi, I am used to using Access but I need to learn SQL for larger web sites. I normally build the database in Access visually, but how do you do this with SQL? I just installed SQL developer and I can't find the area that builds the database.

View 3 Replies View Related

Basics

Jun 11, 2007

Hi, I m new to sql server...
I want to know the difference b/w unicode class and character class.. Pls help... That is difference b/w varchar and nvarchar

anup

View 2 Replies View Related

SQL DB Basics...maybe

Jun 28, 2007

I've a number of SQL DBs that are managed by a php browser front end. These were developed in php/html to a SQL 2000 DB.



The DB and tables have been recreated in SQL 2005, permissions are that of the owner (SA) and the php code has been adjusted accordingly. Although the reading of the data is okay, I cannot update nor insert data from the php script through to the SQL 2005 DB whereas it works perfectly fine through the same php script and SQL 2000.



Obviously, there's something I'm not seeing??? There are no noticable messages and the last transaction does show up in the php display.... closing the portal then reopening, the record is "gone" and the SQL 2005 DB remains untouched.

View 5 Replies View Related

Basics

Jan 21, 2007

Hi there,

I've recently bought Visual Studio 2005, and I got SQL server 2005. I have no idea how to work it. I can't even find where you make a table.

Any basic help about SQL server 2005 is needed.

Thanks,
AliQ

View 3 Replies View Related

SQL Server Basics

Jan 9, 2007

Hi,
I am a DB2 DBA and plan to learn SQL Server for professional reasons. But I am not able to find the material to begin with. So I would be greatful if you guys can send me links to start with the architecture and other basics of SQL server. I just want to accquire additional skills so that it can help me in the future.

View 1 Replies View Related

Sybase Basics

May 19, 2008

Hi Gurus,

I am not sure is this the right forum to get solution to my question.

From where i can get sysbase basics.
I am posting this query in this forum coz this forum and gurus in this forum had helped me in lot critical situtions.

Hope this happens now also..

Thanks,
ServerTeam

View 2 Replies View Related

Clustering Basics

Mar 20, 2008

hi all,

can you help me with some informations...

in a case of clustering, is instance of sqlserver installed on each of the nodes, or there is only one instance shat is "shared" between node servers?

or it depends on the type if clustering?

tnx in adv!

View 2 Replies View Related

Restoring To A New DB - The Basics

Sep 13, 2006

Hiya,

I've recently gotten into the administration side of SQL and am muddling my way through it as best I can. We have a DB server with three or four databases on it. Maintenance routines do a full backup of each one nightly and weekly and put all sets into one backup (.bak) file.

I've never had to restore them, but I thought I'd put myself through a test run since this is unexplored territory for me and I would prefer not to have to learn in an emergency.

A scenario I can foresee is where I'd have to restore just one table, perhaps to a new location (i.e. not overwrite the live db, just bring up a copy of part of it elsewhere so I can do a data dump from some old data). But when I go to 'restore database' or 'restore file/filegroups' and select the file, it seems to really want to restore the full backup directly over top of the existing database. If I try to restore it to a new, non-existent database, I get an error about how 'logical file master is not a part of database 'new-backup'), and if I try to restore it to a new db that I've created, I get an error about how the databases don't match.

It seems to me that my problem is a philosophical one - perhaps I don't understand the nature of backups, but I feel like what I'm trying to do is pretty simple. I want to take the full backup of database A and restore it, in part (or in full if I have to) to new database B. What do I need to do to accomplish this?

Thanks for indulging me - this must be brain-dead stuff for most of you!

View 8 Replies View Related

Distribution Agent Basics?

Nov 6, 2006

Hey everyone,
 sorry for the lame questions, but:
1) I have a transactional replication service set up. However, whenever we select a new table or column within a table to replication to the subscribers, the changes never seem to get picked up unless we make an actual change within some of the other data. Any idea why this is? Even after I've reinitialized snapshots, the new changes don't want to proprogate to teh subscribers until data is actually changed.
2) where do I need to go to manually stop and restart a a distribution agent to force these changes to be proprogated without having to change any data within the tables?
Thanks
 

View 1 Replies View Related

Basics With Stored Procedures.....please

Jan 2, 2004

Hi there, I'm still fairly new to asp.net / c#

I've setup my stored procedure, but now what?
I want to display the information (only one record it will return) on the page / NOT using a datagrid or list, just simple.....I guess using an asp:label?????
Could someone tell me what I need to do!!!

Also, are there any proper resources out there that just show you right out of the box howto insert / update / delete with sp's?, show / view, just proper simple source code?

THANKS!

my code:
// Creating the database connection
SqlConnection objConn = new SqlConnection(System.Configuration.ConfigurationSettings.AppSettings["MM_CONNECTION_STRING_conn_main"]);

// Creating the stored procedure 4 the webpage
// Instantiate a Sql Command
SqlCommand webpage_view = new SqlCommand("stpr_webpage_view", objConn);
// Declare that the T-SQL statement that will be executed is a stored procedure
// Use a CommandType.Text to execute a T-SQL Query
webpage_view.CommandType = CommandType.StoredProcedure;
SqlParameter webpage_id_parm = webpage_view.Parameters.Add("@webpage_id", SqlDbType.Int);
webpage_id_parm.Value = "1";
// Create a SqlDataAdapter to manage the connection and retrieve the data
SqlDataAdapter da_1 = new SqlDataAdapter();
// Instantiate a Dataset that will hold the data retrieved from the database using the select command
DataSet ds_webpage = new DataSet();
da_1.SelectCommand = webpage_view;
// Retrieve the data into the dataset using the SqlDataAdapter.Fill command
da_1.Fill( ds_webpage, "webpage_title" );
DataView dv2 = ds_webpage.Tables[0].DefaultView;
if ( !this.IsPostBack ) {
this.DataBind ( );
}

View 1 Replies View Related

2005 Database Basics

Apr 24, 2007

Hi,

Im trying to justify to people in a meeting why we have to break down some large financial instrument tables in a new database design we are implementing.

They have currently broken the tables down into instruments1 & instruments2, containing approx 300 & 150 columns respectively (further columns will be added!). then they have a view that joins virtually all of these columns. I've asked for the tables to be broken down into more logical groups. they are saying that this will cost more overhead as their views will span more tables. i also complained about the width of the rows, but they claimed that sql 2005
will not have a problem as only 1/3 of the row will have any data at one point so will not cause an overhead. can someone give me some definate points on this relating to 2005 as I am fairly new to it.

can someone help me with the basic points why they should be split into more logical groups.

they are also intending on adding millions of records a year to this table and partioning it will different filegroups. Should you place the indexes for tables on different filegroups as well?

View 1 Replies View Related

Failure Audit - The Basics

Aug 19, 2006

Hi,

I'm a bit new to administering a SQL Server and this seems like a pretty basic question, but I'm not sure how to phrase it for the searches. So I apologize for seeking an indulgence...

I have SQL Server 2005 Standard edition running on a server exposed to the Internet. A handful of clients have to connect to it via TCP/IP using SQL Server Studio and ODBC links. But my windows logs are chok-full of failure audits of what I presume to be your garden variety crackers trying default passwords -- several times a minute.

What's the best solution to this, and how would I go about implementing it? Restrict TCP/IP access to certain IP ranges? Is there a 'max login attempts' somewhere? The server uses SQL authentication (not windows) if that makes a difference.

Thanks for the help!

View 1 Replies View Related

Transactions - Back To Basics

Oct 30, 2006

Hi all,

I am having no end of trouble with transactions in the package which i am building. I now just want to go back to basics and see if someone can tell me where i should set specific transaction options.

Firstly, my package runs a for each loop which loops through a directory of directories. In each of the sub directories there are 2 files. The first steps in the loop are to check if a folder has been processed previously, if so then it moves it to a specified directory. The reason that this is done first is that i cannot move the directory whilst it is being read in the foreach loop, so i pass the path to the next iteration of the loop. There is another file system move directory task outwith the foreach loop to deal with the last directory.

Once this has been done, i parse the file name of the xls file within the directory to get a serial number which is assigned to a variable.

The next step is where i envisage that the transactions should be implemented:
I have a sequence container which contains 2 data flow tasks to be run in parallel. each of these reads data from a seperate work sheet in the xls file. and writes it to a database table. Each dataflow task consists of an excel source task, derived column task, look up task (used to derive an ID from the serial number stored in the variable), and an oledb destination.

Upon completion, if the sequence container fails i want to set the destination folder path to the qurantine location. If it succeeds i want to copy the csv file contained in the same directory to a seperate location and then set the out put folder to the archive location.

What i need to know is where do i set the transaction option if i want to roll back the data that has been inserted into the database if either data flow task fails?

Please somebody help, as this is not working at all.

Many thanks,

Grant

View 18 Replies View Related

Connection Pooling Basics.

Feb 15, 2008



We have an large SQL database being accessed by a C# application. Each hit on the database opens a connection then closes it when done. We are currently experiencing errors when more than 20 users access the database at one time. One of the things I am wondering is if we are using connection pooling. From BOL I am lead to believe connection pooling is off by default. Can I turn on connection pooling as a default in SQL Server or do I pass 'pooling=true' as a parameter on each connection string?

Thanks.

View 5 Replies View Related

MS SQL Basics: Insert Into Multiple Tables At Once.

Aug 15, 2006

I'm trying to do something very basic here but I'm totally new to MS SQL Server Manager and MS SQL in general.
I'm using the Database Designer in MS SQL Server Manager. I created two tables with the following properties:
Schedule (ScheduleID as UniqueIdentifier PK, Time as dateTime)
Course (CourseID as UniqueIdentifier PK, Name as VarChar(50))
I create a relationship by dragging the PK from the first table over to the second and I link on ScheduleID to CourseID columns (I'm not certain what type of relationship is created here N:N?). It appears to work, I can do a Select * and join the two tables to get a joined query.
The problem starts when I try to populate the tables: a course will have a schedule. I can't seem to get the rows to populate across both tables. I try to select the pk from the first table and insert it into the second but it complians about it not being a uniqueidentifier. I know this is very basic but I can't seem to find this very basic tutorial anywhere.
I come from the Oracle world of doing DB's so if you have some examples that relate across that would be great or better yet if you can point me to a good reference for doing M$ DB stuff that would be great.
Thanks.

View 9 Replies View Related

SSIS Design Problem.

Jul 12, 2007

Hi,

I would like to merge data from several excel sheets and load into a SQL Server 2005 database. The excel sheet are uploaded by users from several locations and I'm storing them on a folder on the web-server. I've written the SSIS packages for the required transformations, and for loading the data onto the database. Now, my question is:

1. What is the most efficient way to trigger execution of the SSIS package each time a file is uploaded onto the server (and stored in that specific folder)?

2. Would it be possible to not statically set the "excel source" for the SSIS package data flow transformations, but instead use the folder as the source and start the SSIS package each time a file is newly transfered onto the folder? (Should I use the Script task to get a list of excel files in the folder ordered by the system time?)

I tried using the Service Agent and scheduling it to execute the package on the server. But, my current SSIS design has a static excel source filename and thus when the package is executed more than once, it fails due to DB key constraints.

Thanks!
-Ganesh
gv1@cise.ufl.edu

View 1 Replies View Related

SSIS Design Issue

Jul 11, 2007

Hi,



I need some help in designing the SSIS Package. The requirement is as follows:



I need to import data from an excel sheet to a table in the database. I have 2 lookup tables which has the master data. Before importing, I need to check whether the values of 2 columns in Excel sheet exist in those Master data tables or not. If not exist i need to log that error along with the row number and proceed to next row. It has to loop through all the rows and log the errors. The Import happens when there is no error occurred ie i need to roll back (but loop through all rows to log the errors) if there is any error occurs.



It would be great if you could help me out in desining this scenario in SSIS Package.



Thanks in advance.

Gnan

View 1 Replies View Related

VB2008 SQL Database Programming Basics Question

Feb 25, 2008

I am writing a program where I will be calculating some values that need to be sent to an SQL Database row by row. I am new to SQL Database programming and I was wondering what I need to include in my program as far as declarations, and data adapter connections in order to accomplish this. Or is it enough to already be connected to my database via a manual data connection? I also need to know some of the basic commands for inserting a new row, and adding data under existing columns, prior to moving on to the next row and repeating the process. If you can point me to a basic example that demonstrates this, including how to properly close out my database, I would appreciate it. If you know of a good VB2008 SQL Database book that covers all the basic & some advanced SQL Database programming, I would be interested in that as well.
For Example:
Row# Name CalcA CalcB CalcC CalcD
1 ? ? ? ? ?
2 ? ? ? ? ?
Etc€¦
Thanks.
Techno
Visual Studio Express 2008 user

View 5 Replies View Related

SSIS Design Question - Best Pratices

Apr 9, 2007

I have an application with 4 enviornments, test, dev, qa, prod - each with dedicated databases - some on seperate servers - some on same server seperate instance, some in same instance.

Plan to develop one SSIS Project with multiple packages and use DTSConfig files to store the connection strings.

Question one:
Where to store deployed packages - in SQL (msdb) or File System

Question two:
Where to store DTSConfig files - default directory or dedicated SQL Configuration directory.

Question three:
Retain one DTSConfig file per package or merge the connection strings into one global DTSConfig file per enviornment.


I know there is no right or wrong answer to any of these questions - just looking for ideas.

My plan (as of now)
- Deploy the SSIS packages to SQL Server (stored in msdb) for each Server

- Create dedicated folder (F:SQLConfigs) to store DTSConfig files for each server

- Manually combine the each packages DTSConfig into one global DTSConfig for each enviornment. Use this DTSConfig when configuring job steps in SQL Server Agent.

Thanks for your ideas.

View 2 Replies View Related

SSIS Package Very Slow In Design

Feb 21, 2007

Hello,

I have designed a quite small SSIS package and everything worked just fine .

Untill I moved the SSIS project folder to another solution folder and added whole solution to Visual source safe.

Since then opening the package for design has become unbelievably slow, debugging it even worse.

I think the cause is in moving the SSIS project folder. But I have no idea how to fix this.

Any suggestions ?

Janez















View 10 Replies View Related

SSIS Package Design Question

May 15, 2008

Hi

I am busy designing a ETL solution and have a question about how to design the packages.

We have over 30 source systems for different customers. We are building a WH that will combine all this data for analysis. The main issue is that these systems are always at different versions of the software. When a patch is released, it usually goes to one or two for a Beta process before it moves to the rest. These patches can affect the DB design, and we would want to be able to extract any new data as it comes available from the systems that have it.

Solution 1 - Package Versions
The idea was to create the SSIS packages with a version number in their name. For each change, you would create a new version. The Batch control application that is being developed will store which system needs to use which version of the package.

Solution 2 - Multiple paths within a package
This was to create a single package for each table, with a conditional split as the first task. The batch system will still provide which path the package needs to take with different Data Flow tasks containing the different column mappings.

Both have pros and cons, but was wondering if anyone has experience with this type of setup and which way worked best, or if there are any other options.
Many Thanks
Michael

View 3 Replies View Related

SSIS Package Design - Ideas

Nov 28, 2006

Greetings SSIS friends,

I am in the process of designing a package for populating a Dimension table for my new data warehouse. I would like to discuss with you my proposed solution in the hope that some of you will be able to advise me on any mistakes that I have made so far.

I have 3 source tables which are used to populate my Dimension table (I will restrict each source table to a few columns to keep this simple).

The source tables are :

Meetings (MeetID, NameID, Meeting_Start_Date, Meeting_End_Date, Meeting_Time_Stamp)

Events (EventID, MeetID, Event_Start_Date, Event_End_Date, Event_Time_Stamp)

Names (NameID, Long_Name, Short_Name, Names_Time_Stamp)

My Dimension table structure is very basic as follows :

Product_DIM (EventID, MeetID, Meeting_Long_Name, Event_Long_Name..... Meeting_Time_Stamp, Event_Time_Stamp)

Please note that I haven't included the start and end dates to keep this simple, but in my real solution, the Dimension table has many more columns.

The Meetings and Events tables contains hundreds of thousands of records so when my package is run, I should only retrieve information that has changed since the last time my Dimension table was populated. This is why I store the time stamps in my Dimension table.

So when my package runs, I get max(Meeting_Time_Stamp) and max(Event_Time_Stamp) from my Dimension table then retrieve all records from the source table where their timestamps are GREATER than the max values.

The problem is this :

If in the source system, an event is changed, the time stamp associated with that record will change value but the meeting record associated with that meeting will not have its time stamp changed.

How do I ensure that when my package is run, I retrieve all meeting and events information even if one hasn't changed?

 

Should I build a table made of IDs?! And if I do need to build a table made up of IDs, should I store this as a permanent table in my database or is there an SSIS data structure I can use to hold this information?

I hope this makes sense!
 

View 5 Replies View Related

SSIS Package Design Question

Oct 20, 2006

We are downloading 4 large (500mb) zip files in the package. Those come from 4 different FTP sites. Sometimes the FTP download on one of those fails.

Zip files contain images which need to be uploaded for existing listings. Each zip file is processed the exact same way. The question is how can I make the SSIS package so that each downloaded zip file can be processed immediately after the download - and not duplicating the processing logic?

View 3 Replies View Related

Ssis Runtime Vs Design Time How To Tell?

Nov 15, 2006

It appears that some of the methods in a custom source data flow component are called (repeatedly) at both design time and run time?

Is there a way to determine that "I'm in runtime" vs "I'm in design time"?

View 2 Replies View Related

Best Practice For SSIS Package Design.

Apr 2, 2008

Hello,

I am new to SSIS. I need to transfer data from SQL Server 2005 Operational Database to SQL Server 2005 Report Database. The upload needs to work every night. There are few master tables and remaining are transactions tables.

I am planning to create 2 packages one for master tables and other for transactions tables.
Is it the good approach?

Also few of transaction tables are heavy in terms of number of records. Will it better if i further break them in many packages?


I am using book "Microsoft SQL Server 2005 Integration Services Step by Step".
Can you suggest any other good book?

Regards,
Shailendra Shelar

View 7 Replies View Related







Copyrights 2005-15 www.BigResource.com, All rights reserved