Some Guidance For An SSIS Newbie Wanted
Apr 27, 2006
Hi!
I'm new to SSIS (and quite new to SQL Server). I have a process which I'd like to automize via SSIS - just don't know how and couldn't figure it out yet by playing around with the program. Shouldn't be too difficult though.
First of all, that's the process as I do it now:
1) Load several flatfile sources (dumps of SQL tables) into an SQL database.
2) Add identifier rows (to some tables), set the primary and foreign keys so the database is "recreated" and I can work on it.
3) Do several simple transformations, aggregations and selects across tables and finally write a new table containing information for reporting stuff.
I succeded in loading flatfiles within the data flow view, doing some transformations and saving the output to a flatfile. What I didn't find out: how can I "recreate" the database enabling me to perform "SELECT/FROM/WHERE" statements across tables? Will I have to write the imported files to tables within a db (how?) or can I avoid this step?
A little guide (newbie friendly) would be great help!
View 11 Replies
ADVERTISEMENT
Apr 10, 2007
im trying to convert a sql stored procedure to a ssis project as part of a learning experience, however, im not sure where to even begin. im looking to see if someone can help my find my footstep.
basically, the sproc looks through a table and pulls two fields into a cursor, there is a third field, the where clause is if say that field is greater than 1.
basically the first field it pulls is the name of another sproc to run, and the this field says if i want to execute that sproc or not. so i want to rewrite this sproc in ssis to go through the table, and execute each sproc name based off of the first table.
my initial ssis thought was to add a for each loop, but i couldnt figure out how to go through the table. anyone offer some help on how this could be done? thanks
View 3 Replies
View Related
Apr 25, 2008
Hello
I've been given the task of migrating a DTS package to SSIS (neither of which I am particularly familar with). The first job in the DTS package is to read a .ini file and set a bunch of variables. These variables are then used throughout the DTS package. After running the DTS package through the SSIS migration wizard this job turns into an execute script task and I can't see if it is still reading the .ini file. However, the only real purpose of this step is to allow different parameters to be passed in development, test, production etc. So I am thinking this whole step can be removed and effectively replaced with a package configuration (I'll probably use an XML file). My understanding is that by selecting the name/value pairs as appropriate in the XML package configuration file means this values will be passed in at runtime and achieve the same functionality. Is this the correct way to do this in SSIS, or do I still need the .ini file and variables?
Thanks for any advice on this issue.
Regards, John
View 7 Replies
View Related
Oct 10, 2007
Hi Gurus,
I am new to SSIS. I am trying to install just the SSIS in one machine("SSIS Machine") and just the DB Engine ("SQL Server Machine") in another machine. What I am trying to do is, separating the SSIS service and packages from the Database Engine and trying to run in in another machine. I have few questions on this topic. I searched on this forum but I couldn't find a concrete answer to those questions. Forgive me if it already answered/asked multiple times.
1. When I install SSIS in "SSIS Machine", do I need to install client components also in the same machine?
2. I already established this setup (SSIS with client components in one machine and SQL Server in another) but when I tried to connect to the SSIS thro' Management Studio from Sql Server machine, I keep getting "Access Denied" error. Is it possible to connect to SSIS server from another machine (using Management studio)? I tried the DCOM security permission options I found in the internet(I don't have domain id so I gave "Everyone" full access) but still I get the same error. Any help would be appriciated.
3. Do I need 2 SQL Server Licenses (Enterprise) if I go with this environment?
4. Is it possible to configure SQL Job to run SSIS installed in another machine?
I would appriciate any help on this issue.
Thanks
View 4 Replies
View Related
Jan 2, 2008
Please forgive my ignorance in advance, as I am just becoming familar SSIS. I am sure this is possible, but I would like a definite answer and perhaps a good tutorial on this. I have data in SQL 2005 that needs to reach a third party Oracle DB. The party doesnt give us direct write access, however they have provided a webservice to insert data. Is this feasible using SSIS... SQL 2005 ----> 3rd party web service -----> Oracle? If so can someone point me to a tutorial that explains this process? Thanks a bunch.
View 1 Replies
View Related
Jan 26, 2007
I am tasked with loading the data from one SQL Server 2005 database into another of the same type and schema on the same server. I'd like to use this task to get my feet wet in terms of using SSIS. Can someone suggest a way of doing this?
Thanks in advance
View 15 Replies
View Related
Mar 12, 2006
Following is what I would like to do, so I can keep updating my central SQL Server database with latest updates from the field. I like to use SSIS 2005 to create a package that could do this. Any help to get me started would be appreciated. I need some help soon, pls give me something to get started. APpreciate it. Thanks.
Open connection and read client location table on the local SQL Server database called PODO
For each location id in the table do the following:
Store locationid/clientid in a variable called CLLOC_ID
Construct file name with mdb extension and store in a variable MDB_FILE
Establish connection to the data import folder
Search for that MDB_FILE in the folder on the file system
If there is a file where match = true then do this:
1) Open the access database
2) Read and import the data from the customer experience table
3) Write that data to the SQL Server tables where location = CLLOC_ID
4) Exit process
IF there is no match, exit process
Keep looping until all the client ids/loc ids are read from the SQL Server client location table.
MA
View 4 Replies
View Related
Jan 9, 2008
I used the SSIS import/export wizard to create a single data flow of about a dozen source/destination tables. The tables however have some foreign key constraints between them. Executing the wizard-created package causes foreign key violations. Is there a way to control the order of the table load process within a single data flow? Or is the preferred way to set up several data flows that are ordered via the control flow tab?
View 6 Replies
View Related
Apr 11, 2008
Hello ALL,
My source columns are:
Customer Record DESTINATION TABLE
Name Address_line_1
sex city
PID state
legal_street zip_code
legal_state ID
legal_city
mail_street
mail_city
mail_state
mail_zip
My client needs. All Unique Addresses from the Source.
.
That is the duplicates should be removed and uniques should be available.
Please let me know first what he wants then also tell me what to use in the data flow and how to start. I want to understand the logic. Note: Two persons might have the same address or One address might belong to 2 or more people.
This is what my client said.
Please give me what should i have in the output
View 2 Replies
View Related
Aug 25, 2006
Hello, I have a file I need to retrieve nightly from a web server, it's not a SOAP request. It's just a wget call to a remote php script that returns a csv file.
I can't seem to find a way to do this, I see ftp, and web service, but no web get type of function. Is this not available in IS as provided from microsoft, and something I'm going to have to write? Seems like this type of thing would be allmost as common as ftp.
If anyone has any suggestions, any help would be be greatly appreciated. Right now, it looks like I'm going to have to use an external wget.exe to get the file.
Thanks
View 2 Replies
View Related
Mar 29, 2006
Hi all,
I've exposed my data (that exists in a proprietary format) with the ADO.NET provider interfaces (IDbConnection, IDataReader, IDbDataAdapter and IDbCommand). I can't seem to find any examples of how to get Integrated Services to hookup to this .NET code in my class library. Is it possible? My goal is for this provider to be both a destination and a source and for others to be able use IS to manipulate the data however they want.
Some links or examples would be great.
Thanks,
Dave
View 5 Replies
View Related
Aug 23, 2007
HI All. I'm trying to tweak the Transfer Logins task to exclude Windows Logins that are local to the Server (e.g. servernameusername) which obviously can't be transferred off the server. Annoying that we have a couple of local Logins on this system instead of all Domain Groups, but we're stuck with them due to firewall issues, and a policy excluding SQL Logins.
My idea is to create a text file as part of my Package that lists Logins to be Excluded From the Transfer - I think I then need to create a New File Connection to the Text File as a Connection Manager, then somehow get that data into a Variable, and then use an Expression to populate the 'LoginsList' Collection from syslogins where loginame not equal to logins in my textfilevariable?
Or maybe I'm over complicating this, and there's an easier solution? Lots of info in Books Online about Expressions and Variables, but having trouble finding examples that I can use. As a DBA, this is my first foray into SSIS, and as you can possibly tell, I'm floundering....
View 1 Replies
View Related
May 10, 2007
The following is a list of questions that I have not been able to obtain concrete answers. I am probably missing something:
1) ReadWriteVariables -- can the updated value for a ReadWriteVariable be accessed within the same data flow? It appears not as I think the PostExecute() fires at the completion of the data flow not the end of the Script Component. Secondarily, the Script Component is a non-blocking transformation so the component does not "see" the end of the pipeline prior to sending data down stream.
2) Record Count -- Because of #1 above, How could you calculate a record count for a data stream? It does not appear that one can calculate the number of records for a data stream within a data flow and then access the count from within the same data flow.
3) FinishOutputs() -- Is the concept of FinishOutputs() applicable to Script Component Destinations? Asked another way, is FinishOutputs() executed at the end of the data stream regardless of whether there are "real" outputs for the component? I can create a "Dummy" output to create FinishOutputs() but is this ok?
4) Script Component -- It appears that the Script Component Source, Transformation or Destination are really defined based on the columns defined in "Inputs and Outputs". Can you convert an Source script component to a transformation script component by simply adding an Output?
Sorry for these basic questions but I am not getting it completely. As you can tell...
View 12 Replies
View Related
Jul 14, 2004
I have been trying to rescue my company's helpdesk database because the company moved premesis and we lost that particular server.
It was originally running SQL Server 6.5, and my line manager simply copied the database.DAT files straight out of the MSSQLData directory without running a backup through Enterprise Manager.
Does anyone know how i can rescue these files so that we can get our helpdesk up and running again?!
thanks in advance for any help you can provide.
View 14 Replies
View Related
Jul 20, 2005
I have a HUGE project (at least for me) and need some guidance.I am trying to create a database for a local university movie clubthat allows users to input there basic personal information (name,address, telephone number) as well as movies in there collection. Themovies will be categorized by genre (comedy, romance, horror, etc.)and title. I want to be able to let the users add and remove moviesto their list of movies they own, I'll call it "MOVIES I OWN".The user will also need to be able to create a SECOND list of moviesthey would like to see, again they can choose by genre and title.They need to also be able to add and remove from this list also, i'llcall it "MOVIES I WANT TO SEE".The last part of the project will be to match the users of "movies iwant to see" with "movies I own" users. It will be displayed on thelocal university website when the user logs in and will alert the userto the match. If there is a match at a later time, maybe the user canbe emailed? Also if there is a match, perhaps the two movies can betaken off record after the user acknowledges the match. I would needit to be able to handle a small amount of users now logged in at thesame time , but would like for it to eventually handle several hundredusers logged on at the same time in the distant future with outperformance problems.I am not sure if sql 2000 is the best to get this done or perhapsoracle. I am currently trying this on mySQL with PHP but currently amlost in a forest of data. Any guidance suggestions will be greatlyappreciated. I am pretty new to this so please be kind...
View 3 Replies
View Related
Oct 9, 2007
Hi Guys
I'm new to this Forum and I downloaded SQL express advanced today, I'm more a network secuirty but I wanna save my company some money in create and quoting system and will work in the travel arena. However I'm not sure what querying program to use to pull quotes out and have options to email them as well. This would be the client side part, which records data.What program could I use to do this???
View 3 Replies
View Related
Sep 6, 2006
Hi,
We've decided to use SSIS for an ETL type project. We are going to convert some files to our special flat file format. Our special flat file format will not change, but we will need to convert about 40 different style flat files into this format. We will receive any of these files at random intervals. We will likely have to add more file formats to the 40 at some point, too.
I'd like to use a flexible design, but I just started with SSIS and its going to be hard to test all the possible combinations, so I hope I can find some guidance in this forum.
What would be a good approach to allow for the most reuse of work to support these various file formats?
View 4 Replies
View Related
Sep 20, 2007
Hello Dear Forum! I am new to this web. I am now days working with .NET Tecs especailly ASP.NET. I have developed a small school system in which i used SQL MDF file. I want to know about its approach if they want to make it online. and one more question, is MDF file is best for more than 10,000 Students? Can i convert the MDF into normal table format like SQL Server 2005 , i am using SQL Server Express.
Take Care.Wating for reply.
Developer
View 4 Replies
View Related
Aug 24, 2007
I am the IS manager for a medium business. I am also the DBA and the programmer and help desk and trainer etc etc.
In 2003 I embarked on a project which went live in 2005. We contracted to have our order processing system rewritten into SQL. I went from a character based 4GL language called Progress running in a UNIX enviroment to a knock down, drag out, full blown microsoft solution. I have been operating this new system for two years making modest enhancements and improvements as my skills with microsoft are increasing and improving (retraining my brain to think objects)and hiring contractors to do the really cool fun stuff.
I have to learn Server 2003, SQL Server 2005, SQL Express, T-SQL, Replication, Reporting Services, SSIS, IIS, DTS, Visual Source Safe, Visual Studio, VB, C#, ADO.NET and the .NET Framework fits in here too although I don't really know where.
I hope you are starting to understand my feeling of being 'in over my head'. Right now everything is running perfect. I do backups, restore them and create new reports. Basic stuff.
On top of this our company has merged with a larger one and this bigger company does not have a microsoft solution. They have an AS400 home grown application that I do not want to learn. I will not go backwards!!!!! I will loose my job/position in the next couple of years. I want certifications (to get that interview).
I just don't have the skills to back up the certification (I don't want to be one of those exam cram people) I do a plethora of unrelated taks all day long. Not because I have to do them I choose to do them. I like both DBA and Programming.
So, if you were me, and could get certifications and have experience to back them up what would you do first?
View 4 Replies
View Related
May 2, 2007
Greetings All,
I have a SQL2005 table that I'm needing to export to a 'formatted' csv flat-file to emulate a mainframe output. I'm new to SSIS so don't beat me up to bad... The current output of my ssis csv file is as follows:
95752,95725,0001,0, ,N
11473,8704,0001,0, ,N
1279215,436612,0001,0, ,N
43595,43592,0038,0, ,N
and I need to get to here:
95752 ,95725 ,1 ,0 ," ","N ",0
11473 ,8704 ,1 ,0 ," ","N ",0
1279215 ,436612 ,1 ,0 ," ","N ",0
43595 ,43592 ,38 ,0 ," ","N ",0
Basically I need to pad the first 2 columns with trailing spaces to a fixed width of 11 characters, 3rd column convert number format and pad to fixed 5, 4th column pad to fixed 3, 5th column add double quotes around 20 character fixed. 6th column add double quotes around and pad to fixed 2. Finally adding a , 0 to each row which is used a LF/CR.
Any feedback in regards to getting pointed in the right direction would be greatly appreciated...
View 5 Replies
View Related
Jul 12, 2007
I have a large (4 GB) xml file. I use the SSIS XML Source, generate an xsd and conversions/destinations and this does a great job of creating the multiple outputs including cross reference tables with autogenerated id's (I have no script task or component, so they must be auto). These load nicely into the six tables.
The guidance I seek is what should I do for the next xml source file as it will be an incremental data addition to the base file. I don't see any obvious way to keep the package from autogenerating id's that may conflict with the existing ids.
Am I missing the obvious? It may be, as I am enjoying SSIS, but I am not an expert at all. I can post pertinent items if needed.
Thank you for any guidance.
View 6 Replies
View Related
Mar 17, 2007
What some good reasons to select 64-bit rather than 32-bit for SQL Server 2005?
We are preparing to install a very large SQL Server 2005 cluster. Initially, the primary database is estimated at 1 Terabyte, with 5 more databases that will add up to another Terabyte. After a few years, the estimate jumps up to 20 Terabytes for the largest DB and another 20 for the other five, combined.
We've made the assumption that a database this large, with up to 1000 concurrent users, that this is a good candidate for a 64-bit installation. And that we're better off starting out with 64-bit, rather than migrating later.
Can someone provide some guidance?
View 1 Replies
View Related
Apr 4, 2008
I am gonig into interview for a junior developer position. The role involves a lot of SQL based work. Training is on the job, and they know I am new to this, but they want to know what I can do with SQL server by wednesday, and obviosuly I stand a better chance if I can do a reasonable amount by then.I am assuming I can practise with offline databases, so I would like to do that. Also I was wondering if there were any simple example databases I can load up.
I have downloaded and installed the software, and would like to know how to connect and create a test database.
There will be a small test in the interview and the questions are:
1. SQL Management and Data Extraction
For this task you will need to be familiar with database tables, data types and constraints. There will be some administration work using SQL Management Studio along with some T-SQL queries. You will need to show use of the SELECT, INSERT, UPDATE and DELETE statements. If you do not have SQL Server, you can download the express edition for free at the following URL. http://msdn2.microsoft.com/en-us/express/bb410791.aspx
2. XSLT
For this task I will be asking you to produce some HTML output displaying the data from an XML file. The concept is similar to ASP.
3. Database Design
You will be given a scenario for a company that requires some database software for the smooth running of their organisation. You will need to plan and design a data structure to accommodate these requirements. You will be allowed to spend as much time on this part of the test as you wish. Key information here is going to be database normalization.
These questions dont make a whole lot of sense to be at the moment, so would appreciate a breakdown in simpler terms.
This job will be a fantastic opportunity for me to get into development, and would appreaciate any help that you guys have to offer, thanks in advance.
View 21 Replies
View Related
May 20, 2008
alter table consumers
add check (zipcode like( '[0-9][0-9][0-9][0-9][0-9]'))
the datatype is char(5)
error message:
The ALTER TABLE statement conflicted with the CHECK constraint "CK__Consumers__Zipco__4316F928".
Based on this message, i would think that there is another constraint, however the build script does not have any constraints. So i am somewhat confused.
View 6 Replies
View Related
Apr 5, 2008
I know I already have an open thread and this is naughty, but just digging my other into a hole.
As stated in my other thread I am going into interview next week for a junior developer role, and they want to know as a test what I can find out and achieve with SSMS.
I would really appreachiate it if someone could give me some of their time to answer a few questions and guild me though the very basics. Either via MSN or Skype or something, sometime this weekend, as I really want this job on wednesday!
Thank You!
View 3 Replies
View Related
May 30, 2006
Is there any guidance on using Xp_Cmdshell from the SSIS team at Microsoft? We're needing to remotely execute SSIS packages on the server where MS SQL Server 2005 is installed.
FYI, I've researched this heavily and know about the common ways--re: Using SQL Agent or the Microsoft.Dts.Runtime library in .NET with a web service etc.
View 10 Replies
View Related
Aug 2, 2006
Ok guys I have read everything in my previous post but unfortunately can not seem to get it to work properly. Would anyone like to do this 10 minutes work (I am sure). Obviously I dont expect it to be free but if I continue I am going to get a sledge hammer to this now.
Thanks in advance
View 20 Replies
View Related
Feb 25, 2005
Hi
I'm learning by going through the tutorials and such and modifying the code to try and learn how to do more and different things.
I have a MSDE Database, and I'm building a query from the WebMatrix Code Builder.
I can get a Select and Where to work on a "Category" colum as String; to return from a text box control (where I enter the name) and on button click.
What I would like to learn how to do and am having trouble is do a select Where "Category" (string) and "Status"(Int) are the same.
I have two Seperate Textboxes after i make the Where / And Query, but I'm not sure i have the button click code right.
I have included the code below - Sorry if this is so basic ;-) - The Sub_Button1 Click is at the end.
Function ShowRecords(ByVal category As String, ByVal status As Integer) As System.Data.DataSet
Dim connectionString As String = "server='(local)'; trusted_connection=true; database='SalesContacts'"
Dim dbConnection As System.Data.IDbConnection = New System.Data.SqlClient.SqlConnection(connectionString)
Dim queryString As String = "SELECT [Contact].* FROM [Contact] WHERE (([Contact].[Category] = @Category) AND ("& _
"[Contact].[Status] = @Status))"
Dim dbCommand As System.Data.IDbCommand = New System.Data.SqlClient.SqlCommand
dbCommand.CommandText = queryString
dbCommand.Connection = dbConnection
Dim dbParam_category As System.Data.IDataParameter = New System.Data.SqlClient.SqlParameter
dbParam_category.ParameterName = "@Category"
dbParam_category.Value = category
dbParam_category.DbType = System.Data.DbType.String
dbCommand.Parameters.Add(dbParam_category)
Dim dbParam_status As System.Data.IDataParameter = New System.Data.SqlClient.SqlParameter
dbParam_status.ParameterName = "@Status"
dbParam_status.Value = status
dbParam_status.DbType = System.Data.DbType.Int32
dbCommand.Parameters.Add(dbParam_status)
Dim dataAdapter As System.Data.IDbDataAdapter = New System.Data.SqlClient.SqlDataAdapter
dataAdapter.SelectCommand = dbCommand
Dim dataSet As System.Data.DataSet = New System.Data.DataSet
dataAdapter.Fill(dataSet)
Return dataSet
End Function
Sub Button1_Click(sender As Object, e As EventArgs)
DataGrid1.DataSource = ShowRecords(CStr(TextBox1.Text)&(CInt(TextBox2.Text)))
DataGrid1.DataBind()
End Sub
View 4 Replies
View Related
Jan 2, 2008
Hi Can someone give me proper guidance .I am a data warehouse architect with 13 years in IT.
What are the prospects if I move to data mining.Will it be wise decision to add data mining.
Please guide.
View 3 Replies
View Related
Nov 30, 2006
Hi, does somebody recognize the problem with my code? -Thanks!
Description: An error occurred during the processing of a configuration file required to service this request. Please review the specific error details below and modify your configuration file appropriately. Parser Error Message: This is an unexpected token. The expected token is 'NAME'. Line 58, position 52.Source Error:
<sessionState mode="SQLServer"stateConnectionString="Jensen"sqlConnectionString=
"datasource=Database;userid=username;password=pass"cookieless="false"timeout="60"/>
View 1 Replies
View Related
Sep 23, 1999
Always looking for people who are strong with microsoft technology for new oppurtunities in the Seattle area. If you are good, we can help you find your ideal next position.
don't be shy. shoot me an e-mail.
thanks
Pat Copeland
View 2 Replies
View Related
Sep 16, 1999
I am looking for strong SQL developers and/or DBA's for some really sweet companies here in Seattle. Anyone interested that is good and wants to hear about new oppurtunities? I would love to help you out.
don't be shy just shoot me an e-mail.
View 2 Replies
View Related
May 8, 2002
Please help to find an example of the code written on VC++ for an extended stored procedure (dll) for SQL Server 2000.
Thanks
View 1 Replies
View Related
