SSIS As ETL Tool?
Sep 10, 2007
SSIS as a ETL tool
Guys,
I currently doing microsoft's hands-on training tutorial for SSIS. I realized that its not an comprehensive ETL tool as it was advertised to me. I see it more of a package tool for all imports, exports which include DML scripts.
In my scenario I have to convert 10 source data tables to destination schema tables. i normally use DML scripts to convert the data. Once I have mapping between source and destination tables it does not take more than hour to write DMl scripts. But when I try to use SSIS its taking me more time to create data flow tasks and create a package. And does SSIS support all the sql server 2005 string functions.
Any comments/inputs would help.
Thanks
View 1 Replies
ADVERTISEMENT
May 30, 2007
We have a web application using SQL Server 2005 to store and serve the data to forms made using .Net 2. The forms are used by customers to enter fifteen or so items if data. An important part of what the app does is to validate these 15 items of data against data brought in from the main application which is in Access 2003 (for example, did the customer enter a valid part #, did they actually purchase that part number in the qty stated, etc).
I want to periodically (nightly) schedule a copy of the lookup data from Access (2 different databases) into tables in SQL Server. The copy process would simply overwrite the existing lookup data in SQL server with the new data each night. My guess that about 250 mb of data would be copied. The two different databases are identical in structure, but different in data. They are used in two different countries.
The process as I see it would be to:
1. Delete existing lookup data.
2. Copy in data from database 1
3. Copy in data from database 2
I need to use something with enough programming power to resolve issues with primary keys, eg not copy Access table columns which contain them, and perhaps populate a column in the SQL Server lookup tables which indicate which Access database a given record came from.
If we use SSIS, then I will need to learn it, which is great as long as I am going down the right path in the first place.
Would SSIS be the right tool? Is there something more appropriate? Is there a good tutorial?
Many thanks
Mike Thomas
View 3 Replies
View Related
Jan 14, 2008
Hello,
At our company we are considering building an architecture for file
imports and processing and considering both Biztalk and SSIS at this
time.
My understanding from reading the material out there regarding this
subject that Biztalk is more suited for integrating applications and
real time communication of information and SSIS is more suited for
bulk loads into databases/data warehouses and data manipulation.
Currently we are somewhat along the lines described above, but there
is a desire to use one technology over the other for importing files
and data manipulation, and I am not sure that is practical. Also
there is a debate currently as to whether Biztalk does better at
handling logic than SSIS.
I have read through the article on microsoft site that outlines the
above - http://www.microsoft.com/technet/prodtechnol/biztalk/2004/whitepapers...
However, I would like to get some input from people who have actually
used both these tools in the real world for ETL process and could
provide some insight so as to help us make an informed decision.
Thanks
KR
View 2 Replies
View Related
Sep 28, 2007
I have some questions on SSIS tool I tried to read many docs to understand SSIS.
Now I am looking forward for below questions.
How to use SSIS for :
1. How to Export data from SQL Server 2005 in to Access database using SSIS?
2. How to Import data from Access .mdb file to SQL Server 2005 database using SSIS?
3. How to Import Oracle10g data in to SQL Server 2005 databvase using SSIS?
4. How to Import Oracle10g Data to Access file using SSIS?
5. How to Import SQL Server 2000 data in to SQL Server 2005?
It will be great to have step by step explanation.
Thanks in advance.
View 1 Replies
View Related
May 12, 2008
Hi all,
I am looking for any information about using SSIS as a workflow automation (job engine) tool. My company is looking into buying a 3rd party app to do our job scheduling and I think that SSIS could do all that we need. The only issue they found with SSIS is the lack of a GUI/dashboard to view all jobs at once. We need more flexibility than the job scheduler in SQL Server Agent will allow. I have heard that it is possible to build a C# GUI app that can serve as a job engine front page.
What we need is a way to view all jobs in the system and be able to start, stop, pause all jobs manually in a graphical interface. I know of a few companies that are doing this but I am unable to find anything on line about it. My bosses are ready to give SSIS a shot if I can prove that we can build such an interface. Does anyone have any first hand knowledge of such an application or have any tips on where I should look?
Thanks in advance!
-- Craig ***
View 4 Replies
View Related
Jun 20, 2007
Who knows whether exists embedded tool in win2003 or SSIS for extracting the files from ZIP?
Thanks in advance.
View 4 Replies
View Related
Sep 24, 2007
How can I add a reference to the Microsoft excel 2003 in the script tool in SSIS 2005? I need to automate Excel for formatting.
Thank you.
Dan
View 1 Replies
View Related
Jun 15, 2006
I am trying to migrate our processing from command line based scripts and foxpro to SQL so I need to run the SSIS packages using dtexec. I copied the dtexec file and a few dll's that are missing to our production servers but i cant execute the packages. I dont want to install the full client tools (particularly managment/business inteligence studio) on our production servers due to the overhead and limited system disk space.
Can somebody tell me what the minimum install would be so I would be able to run SSIS packages using the dtexec or dtexecui tools? I would also like to install some of the other command line client tools like osql etc.
View 8 Replies
View Related
Feb 12, 2008
Hi, I am less of a technical but more of a analyst professional and right now investgating on various tools / options for the new conversion project I will be leading in insurance client. One of the tools that client want to use is SSIS but the source and target database is not on SQL server but plans are to build a staging SQL server database for transformation. Does SSID supports this kind of ETL process where both source and target system are non SQL servers.
Thanks,
H Gill
View 4 Replies
View Related
Mar 2, 2006
is it possible to change the settings on this tool. i want to have it so that it does not time out at all? or is that possible??
View 1 Replies
View Related
Jul 23, 2005
Hi all.I'm looking for a tool which should act like some kind of middleware/logical layer bewtween the SQL server and the webbased user interface.- It should be possible to easily create simple web forms (only datainput and output) without programming effort by "clicking" the fieldsand their order on the web mask within an admnistrative interface.- It should also be bossible to add "new fields" to the database,including field type, permissions, value lists etc. (excluding anybusiness logic) without programming effort by administration.I know, that most workflow tools or "Trouble Ticket Tools", based ondatabases have this feature to easily configure new masks and add newfields to the database, but i need it as an tool-unspecific layer forthe MS SQL Server.Thank you very much for your feedback, any ideas are welcome!Best regards, Heiko.
View 1 Replies
View Related
Jul 22, 2005
hiis there any tool to write stored procedurei have long queries to write instead of that i want a tool or editor where i can click or drag and drop to write procedurei make my work to complete fasterif any please tell me the site to downloadthanx byeramesh
View 1 Replies
View Related
Aug 5, 2005
Hello,I'm looking for recommendations for SQL tools used for comparing schemas from different databases. I would like the tool to be able to compare schemas for at least two databases and list the difference in tables, sps, functions, and view. Even if the both schemas contain the same structures I would like to know what differences exist. Also, I would like to be able to type in a table field name and get a list of every place in the db that field is referenced. On another note, how do others go about maintaining a version history of db structures? Any tools available for this functionality? Currently, I use visual source safe for my app. code. I could place copies of my SPs in visual source safe and edit them within visual studio. I'm not sure how I would easily get the edited version into my sql database without having to create a script or copy the edited version back to the db. Any thoughts?Any information is appreciated.Oscar
View 1 Replies
View Related
Apr 23, 2008
Hi guys, Im trying to use Bulk copy to transform a csv file into a table
BULK INSERT [BULK_INSERT_TEST]
FROM 'c:stock.csv'
WITH
(
FIELDTERMINATOR = ',',
ROWTERMINATOR = 'N'
)
--*STOCK LOOKS LIKE THIS
stockcode,barcode,qty
10002,"10002087",0
TABLE Looks like this
id, stock,barcode, qty
can anyone give any guidance on performing this - at present this doesn't seem to work. Worth mentioning id is autoincrementing. I would also need to strip quotes around the second field. Note. Stock file has a header - do I need to strip this to use BCP?
Any help appreciated.
2,10002,"10002087",0
View 4 Replies
View Related
Oct 18, 2007
Hi,
I am using bcp to copy some data from a flat file to a data table using the format file that look like this.
9.0
7
1 SQLCHAR 0 4 "" 1 Col1 SQL_Latin1_General_CP1_CI_AS
2 SQLCHAR 0 9 "" 2 Col2 SQL_Latin1_General_CP1_CI_AS
3 SQLCHAR 0 8 "" 3 Col3 ""
4 SQLCHAR 0 7 "" 4 Col4 SQL_Latin1_General_CP1_CI_AS
There are 4 columns in total, however some rows have missing values for Column 4, and but given there are no field separators (all columns have fixed width and data is truncated if needed), how should the format be written to define a row terminator to apply to Column 3 if Column 4 is null.
Thanks for your suggestions.
View 3 Replies
View Related
May 23, 2008
I'm in the process of delivering a SQL Server BI solution to a customer. I have the cube built, SSIS working and an SSRS package with several hundred reports. They would like to roll their own reports.
I've looked at ProClarity (or whatever MS calls it now) and Report Builder. Both seem to have their pros and cons.
What tool does the community recommend?
Thanks.
View 4 Replies
View Related
Jul 20, 2006
I think Service Broker is the right tool for what I want to do but need confirmation. Here is what I want to do.
Our company sells small business consulting services. When a client signs up with us, we put a server at their site that runs our app (using SQL Server 2005). Once the server is setup at their site and operational we constantly send certain information from that server to our central corporate server. There could be thousands of records every day being sent from each of these servers to our corporate server. We're using webservices now to transfer this data from each server but I'm afraid we're reaching a point where webservices will no longer work because of the volume of data.
1) I assume that Service Broker was created, in part at least, to address these types of situations?
2) If #1 is correct, is there any step by step examples/instruction that shows me how to use Service Broker to communicate between two SQL Server databases on two different servers?
Thanks very much.
Amos.
View 1 Replies
View Related
Jul 3, 2007
Hi, I'm not sure this is the right place to ask this question (please re-direct me if it isn't). I would like to learn about SubSonic tool:1. do you recommend this tool ? if so what are its best uses?2. is its license allows me to work with it for free for commercial uses? any license limitations I should be aware of for using it as a tool? thanks,
View 6 Replies
View Related
Nov 10, 2007
Hi all,
I downloaded the StudentActivities sample app from http://technet.microsoft.com/en-us/library/ms345152.aspx
When I want to go through this tutorial I encounter the following problem:
Database 'C:WebSitesStudentActivitiesRegistryDataStudentActivities.mdf' cannot be upgraded because its non-release version (587) is not supported by this version of SQL Server. You cannot open a database that is incompatible with this version of sqlservr.exe. You must re-create the database.Could not open new database 'C:WebSitesStudentActivitiesRegistryDataStudentActivities.mdf'. CREATE DATABASE is aborted. (Microsoft SQL Server, Error: 950)
So attaching the DB seems to doesn't work. The reason for this is probably that this sample app src was written in an older version of Visual WebDeveloper Express 2005 +SQL Server Express 2005.
Therefore I would like if someone can provide me with the sql src of this StudentActivities database. Then I can create the database myself from this. Or is there some tool which canconvert a mdf to sql?
I hope someone can help me with this. Thanks in advance.
GJ
View 1 Replies
View Related
Nov 12, 2007
Hi all,
I downloaded the StudentActivities sample app from http://technet.microsoft.com/en-us/library/ms345152.aspx
When I want to go through this tutorial I encounter the following problem:
Database 'C:WebSitesStudentActivitiesRegistryDataStudentActivities.mdf' cannot be upgraded because its non-release version (587) is not supported by this version of SQL Server. You cannot open a database that is incompatible with this version of sqlservr.exe. You must re-create the database.Could not open new database 'C:WebSitesStudentActivitiesRegistryDataStudentActivities.mdf'. CREATE DATABASE is aborted. (Microsoft SQL Server, Error: 950)
So attaching the DB seems to doesn't work. The reason for this is probably that this sample app src was written in an older version of Visual WebDeveloper Express 2005 +SQL Server Express 2005.
Therefore I would like if someone can provide me with the sql src of this StudentActivities database. Then I can create the database myself from this. Or is there some tool which canconvert a mdf to sql?
I hope someone can help me with this. Thanks in advance.
GJ
View 4 Replies
View Related
Mar 6, 2008
I have a typed dataset where I used the sql query builder to build the following:
SELECT DealManager.*, Contact AS Expr1FROM DealManagerWHERE (Contact = @Contact)
How can I incorporate the "where % % " clause into this statement and still except the required parameter. In other words, I want something like this:
SELECT DealManager.*, Contact AS Expr1FROM DealManagerWHERE (Contact like %@Contact%)
but I get an error when I compile this...
Any help always appreciated!!!
View 2 Replies
View Related
Apr 4, 2008
Hi Guys
Got this problem, ive setup some new roles and users through asp.net configuration and it creates ASPNETDB.MDF in the App_Data folder when ive got a DB in SQL server 2005 express.
Basically how do i change it so the tables created for the users and roles are created in the SQL Server 2005 Db? p.s I'm very new to sql server 2005
Thanks
View 5 Replies
View Related
Mar 4, 2004
Hi,
I've been having trouble getting this tool to work with my server/database. I'm using XP Pro, MSDE, .NET 1.1.4322 and IIS 5.1. IIS has been installed and tested with the help of others. .asp pages are being served locally and to another PC on the network. No obvious problem here. When I installed MSDE I used the SAPWD='strongpassword" command to create a password. I also built a new database. I had positive feedback on both of these actions.
Now to the tool - when I activate the administrator it pops with the little window saying Web Server Configuration. It lists the following options:
Desired Port:
IIS
Cassini Personal Web Server
Restore Faults
and Start and Stop buttons.
If I leave the setting on IIS and click Start it opens a webpage entitled Web Data Administrator. Moving further down I have a welcome message and then the line -
Please enter a SQL Server name:
Username: [Desktopob shadowed entry]
Password
Server (local)
Authentication Method:
Windows Intergrated
SQl Login
and then a Login button
If I select SQL Login and enter "sa" as the username and then my password leaving (local) as the server I get the following error:
Invalid username and/or password, you are using a windows login that is not your own, or server does not exist.
I have tried all combination of \desktopsa, sa, you name it. The password I selected is correct because I made a point of writing it down so I wouldn't forget it. I have also tried all the combos using Cassini as well.
This is where I'm stuck at this point. All suggestions are appreciated.
Thanks,
Bob
View 3 Replies
View Related
Jan 6, 2005
Hello,
For management that doesn't have the software I do, I've been using a PDF postscript utility to print the model and give it to them. However, I have to fit the model for the specific page (which is really small, and you have to zoom in), or you have to view it on multiple pages. Well multiple pages can be a pain, because the table you are looking at is on page 1, but the relationships are on page 12, 22, or 50, for example. So I was looking for a tool that is cheap enough to implement that we could look at the entire model on one "surface" (not necessary to print), that the model could be viewed in.
Thanks,
Brian
View 2 Replies
View Related
Dec 3, 2005
Does anyone knows a good FREE tool for documenting a DB?
so far i'm with ApexSQL but its not free, wondering if anyone knows a free tool.
Thanks,
View 1 Replies
View Related
May 1, 2001
Hi All,
I have to load data from from a text file ( approx 3-5 million rows) into a table .
I can use straight bcp with the format file , but the loading of data has to follow some business logic
Let me know if there are any third party tool to load the data instead of regular bcp
I have SqlServer 7.0 on Windows NT 4.0
Thanks in advance
View 2 Replies
View Related
Jul 24, 2001
Dear all,
Is there any tool to convert Oracle's sql, sp.. to SQLServer's sql, sp. vice versa?
View 1 Replies
View Related
Sep 19, 2000
Has anybody see a tool can draw a Tree diagram for Parent-Child(PK-FK) relationships for SQL 7.0 DB.
Appreciate your help.
Thanks
View 3 Replies
View Related
Oct 7, 2002
Does anyone know of a T-SQL Reporting Tool that can be attached to either SQL 7 and/or 2000 databases.
I am looking for something similar to the Oracle 'Discoverer' tool.
The requirements are that users be able to create there own reports using a GUI with the minimum of prior knowledge of T-SQL.
Thanks in advance for any replies.
David Goodman.
View 3 Replies
View Related
Jun 27, 2002
1) Is there a tool out there that formats sql code for (SQL 2000) ?
2) if you have a date split into year = 2002, month =1 and day= 22
how can l combine it or rather concatenate it so it looks like
'2002-01-22'
View 1 Replies
View Related
Jan 23, 2007
Hi all -
I'm looking for a free or shareware database design tool that can connect to Sql Server 2000. Anyway know of one? Thanks
Joe
View 2 Replies
View Related
Jan 28, 2005
Hi all,
I am new to the db world so please let me know if I don't make any since.
I was wondering if there is an open source SQL database monitoring tool out there that I can run on my desktop that would give me a log of what is going on on the database (or the server).
Thanks.
View 2 Replies
View Related
Aug 6, 2007
I'm trying to find out the best O/R mapping tool for .NET. A perfect tool would begin with the database model and would be capable of generating 2 layers: the Domain layer (normal classes) and the Persistence layer (methods for storing and retrieving the domain layer objects from the database.
This tool would also be capable of generating the corresponding Visual Studio solution and projects.
I've already tried NetTiers, but it generates so many layers that the code is very dificult to manage and understand.
So any of you know a tool similar to the described above?
View 1 Replies
View Related
