I have downloaded an evaluation copy of a SQL Server performance tool (a fancy version of the Profiler) called Speed Coefficient from Imceda. Pretty interesting so far, but some questions are forthcoming (and probably will continue to come as I drill down and learn more about performance thangs).
It tells me I have a recompilation that I did not expect, it says the reason is "object not found at compile time, deferred to run time", but doesn't do too well at specifically telling me which object it is complaining about (yeah, not REALLY a complaint, but perhaps more a "mention", but I digress...)
I thought originally it was, perhaps, an object that I had not referenced correctly, but as it turns out, it is, I believe, referring to a global temporary table one of my procs creates. Upon further reflection/introspection, it makes sense to me that this is the case, since it won't HAVE the temp table object to kick around until it is created at run time.
Does this make sense? If so, I guess this is one of those times where the tool just makes reference to a possible issue, but it's up to the user to understand what the underlying cause of the "mention" is, and to determine if it is "OK" to have the recompilation occur.
I currently doing microsoft's hands-on training tutorial for SSIS. I realized that its not an comprehensive ETL tool as it was advertised to me. I see it more of a package tool for all imports, exports which include DML scripts.
In my scenario I have to convert 10 source data tables to destination schema tables. i normally use DML scripts to convert the data. Once I have mapping between source and destination tables it does not take more than hour to write DMl scripts. But when I try to use SSIS its taking me more time to create data flow tasks and create a package. And does SSIS support all the sql server 2005 string functions.
We have a web application using SQL Server 2005 to store and serve the data to forms made using .Net 2. The forms are used by customers to enter fifteen or so items if data. An important part of what the app does is to validate these 15 items of data against data brought in from the main application which is in Access 2003 (for example, did the customer enter a valid part #, did they actually purchase that part number in the qty stated, etc).
I want to periodically (nightly) schedule a copy of the lookup data from Access (2 different databases) into tables in SQL Server. The copy process would simply overwrite the existing lookup data in SQL server with the new data each night. My guess that about 250 mb of data would be copied. The two different databases are identical in structure, but different in data. They are used in two different countries.
The process as I see it would be to:
1. Delete existing lookup data. 2. Copy in data from database 1 3. Copy in data from database 2
I need to use something with enough programming power to resolve issues with primary keys, eg not copy Access table columns which contain them, and perhaps populate a column in the SQL Server lookup tables which indicate which Access database a given record came from.
If we use SSIS, then I will need to learn it, which is great as long as I am going down the right path in the first place.
Would SSIS be the right tool? Is there something more appropriate? Is there a good tutorial?
At our company we are considering building an architecture for file imports and processing and considering both Biztalk and SSIS at this time.
My understanding from reading the material out there regarding this subject that Biztalk is more suited for integrating applications and real time communication of information and SSIS is more suited for bulk loads into databases/data warehouses and data manipulation.
Currently we are somewhat along the lines described above, but there is a desire to use one technology over the other for importing files and data manipulation, and I am not sure that is practical. Also there is a debate currently as to whether Biztalk does better at handling logic than SSIS.
I have read through the article on microsoft site that outlines the above - http://www.microsoft.com/technet/prodtechnol/biztalk/2004/whitepapers...
However, I would like to get some input from people who have actually used both these tools in the real world for ETL process and could provide some insight so as to help us make an informed decision.
Hi all, I am looking for any information about using SSIS as a workflow automation (job engine) tool. My company is looking into buying a 3rd party app to do our job scheduling and I think that SSIS could do all that we need. The only issue they found with SSIS is the lack of a GUI/dashboard to view all jobs at once. We need more flexibility than the job scheduler in SQL Server Agent will allow. I have heard that it is possible to build a C# GUI app that can serve as a job engine front page.
What we need is a way to view all jobs in the system and be able to start, stop, pause all jobs manually in a graphical interface. I know of a few companies that are doing this but I am unable to find anything on line about it. My bosses are ready to give SSIS a shot if I can prove that we can build such an interface. Does anyone have any first hand knowledge of such an application or have any tips on where I should look?
I am using SAS Dataset and loading in to SQL server 2005.Now i want to delete columns which have 0 values and null values.Can anyone guide me how to do this?thank you.
I was wondering if anyone know how to do Dynamic Properties Tasks in SSIS? And, does anyone know how to create a global database in SSIS so I won't have to change all of the database names in my package? I'm sorry, I'm new to SSIS and I'm just really frustrated. Thank you.
I realize that I am confused about SSIS and security.
In BIDS, I work on and modify my packages. That part I understand. Then, I want to build my project, then deploy to SQL Server 2005. I know how to do that too (for the most part, please see below):
My confusion arises around the "Protection Level" options in the package properties. Right now I have everything set to "EncryptSensitiveWithUserKey". My understanding is that I need to change this in order to run my packages from SQL Server jobs, because only the creator of the package can currently run the job.
So my question is, since I want to deploy to SQL Server, don't I want to change the "Protection Level" to "ServerStorage"?
However, it will not let me change the protection level to ServerStorage. It says "The protection level, ServerStorage, cannot be used when saving to this destination"... presumably because it is attempting to save to the file system.
However, even when I built out my packages and saved to SQL Server, I could not change the protection level either.
Which is why I am really confused... Once you save your packages to SQL Server, how do you make changes to the packages, so that the changes are reflected in the packages stored on SQL Server?
There is some concept I am not understanding here.
Please write YES/NO against point number in your answer i.e. for example:
1) YES, 2) YES, 3) NO, 4) YES etc.
We have source SQL Server 200 databases in one instance and target SQL Server 2005 in another instance both being on different windows 2000 server systems.
Target OLAP DB design is derived from source OLTP.
Most master & transaction tables are as is but some target tables are a result of merging source tables.
We need to do SSIS packages based migration using data flow task based on column mapping & queries using Business Intelligence Development Studio (BIDS). There are no transformations required.
1) Can I group packages under project into sub-groups i.e. super packages?
2) Can I execute a set of packages together which are logically/physically grouped?
3) Can we execute set of ordered packages/individual packages from command line and even stored procedure?
4) Can we maintain transaction integrity across the entire set of ordered packages we execute?
5) Can we send a mail to mail id(s) in case of events like OnError etc.
6) Can we schedule the packages?
7) Does SSIS provide via BIDS a way to compare data in source with target to decide what got inserted/deleted/updated and based on that do the necessary?
8) Can SSIS packages be developed including code for data encryption?
9) Can we pass parameters to packages that get called from stored procedure?
10) Can we call stored procedure/function in SSIS package?
I have been working with SSIS and find myself stopped on what seem like very basic tasks.
Am I right in thinking that, if you want to import a flat file into a SQL datatable, in most PC based systems, that you have to manually cast EVERY SINGLE COLUMN to unicode string, if you don't want to get the 'cannot convert between unicode and non-unicode string data types'? I mean, this is surely a VERY COMMON task, taking a CR/LF delimited ASCII file and dumping it into a database tables? What if you have 50 fields to import?
Am I the only one who finds it very difficult to cast a string from a textfile to a date format? At this point, I always just dump the date to a 8 character string and save it that way, and use a SQL view to cast it to a date. At least SQL Server can do 'basic date conversions'. Is there any way to freeze execution and find out what data was provided to a derived column, if you're trying to do some substring operations and hodge together a date? I couldn't find any. I usually put a extra column in a datatable and plunk some data there to find out 'what it looked like', how else can you debug an SSIS program?
Is there a planned service pack or update to SSIS that will make this more usable? Does Microsoft ever intend to implement an UPSERT type module? Or a File Move operation that handles directories that fill up with log files, each of which have to be added to a datatable and then moved to an archive directory (surely, another very common task - I figured out how to accomplish this but it is far from intuitive, the way it currently works). Sorry for grumbling, but honestly, I find turnaround time to develop a very simple import program is RIDICULOUS. It's quicker to write a COBOL program to issue a bunch of formatted SQL UPDATE statements than using this application.
I am using SSIS with Transaction, and I met a lot of questions as below:
1.IF DTC doesn't support ADO.NET Connection?
Please check the ADO.net Connection's property, it has a property named like: DTCTrancactionSupport. But it's always set to False and unable to edit.
IF the answer is NO, HOW could I bound all my Tasks into one Transaction? You know, I have some Tasks with SQL connection, and I have also some Script Tasks, witch need the ADO.net connection.
2.IF DTC required to run in both the Destination Server and the Source Server?
I have a Dataflow Task, and take some data from a table in a Source Server to a Destination Server. The DTC doesn't run on the Source Server. I found the Dataflow Task hangs when I use the SSIS Transaction(just set the Task's transactionOpion to Required, Only one Task in the Package).
3.IF DTC doesn't support RetainSameConnection?
Many people say that. Really?
4.When a DTC transaction is running, if the table in it could not be modified by process outside of the transaction? For example, a package with a DTC transaction, and there is a SQL Task inside, when it is running, could I modified the same table's data manually(in the Sql Sever Management Studio)?
I am trying to migrate our processing from command line based scripts and foxpro to SQL so I need to run the SSIS packages using dtexec. I copied the dtexec file and a few dll's that are missing to our production servers but i cant execute the packages. I dont want to install the full client tools (particularly managment/business inteligence studio) on our production servers due to the overhead and limited system disk space.
Can somebody tell me what the minimum install would be so I would be able to run SSIS packages using the dtexec or dtexecui tools? I would also like to install some of the other command line client tools like osql etc.
Hi, I am less of a technical but more of a analyst professional and right now investgating on various tools / options for the new conversion project I will be leading in insurance client. One of the tools that client want to use is SSIS but the source and target database is not on SQL server but plans are to build a staging SQL server database for transformation. Does SSID supports this kind of ETL process where both source and target system are non SQL servers.
Hello all, this is my first post, I have a couple of basic questions regarding the use of SSIS. I have used SSIS for a couple of years for on off table loads but never implemented production jobs with it so please bear with these questions.
1. When making scripts that connect to one or more databases, is there a way to store the userid and password somewhere besides embedded within the package, such as a web.config file, this would make routine password changes much more manageble.
I guess it was only one question but any help would be greatly appreciated.
The following is a list of questions that I have not been able to obtain concrete answers. I am probably missing something: 1) ReadWriteVariables -- can the updated value for a ReadWriteVariable be accessed within the same data flow? It appears not as I think the PostExecute() fires at the completion of the data flow not the end of the Script Component. Secondarily, the Script Component is a non-blocking transformation so the component does not "see" the end of the pipeline prior to sending data down stream.
2) Record Count -- Because of #1 above, How could you calculate a record count for a data stream? It does not appear that one can calculate the number of records for a data stream within a data flow and then access the count from within the same data flow.
3) FinishOutputs() -- Is the concept of FinishOutputs() applicable to Script Component Destinations? Asked another way, is FinishOutputs() executed at the end of the data stream regardless of whether there are "real" outputs for the component? I can create a "Dummy" output to create FinishOutputs() but is this ok?
4) Script Component -- It appears that the Script Component Source, Transformation or Destination are really defined based on the columns defined in "Inputs and Outputs". Can you convert an Source script component to a transformation script component by simply adding an Output?
Sorry for these basic questions but I am not getting it completely. As you can tell...
Hi all.I'm looking for a tool which should act like some kind of middleware/logical layer bewtween the SQL server and the webbased user interface.- It should be possible to easily create simple web forms (only datainput and output) without programming effort by "clicking" the fieldsand their order on the web mask within an admnistrative interface.- It should also be bossible to add "new fields" to the database,including field type, permissions, value lists etc. (excluding anybusiness logic) without programming effort by administration.I know, that most workflow tools or "Trouble Ticket Tools", based ondatabases have this feature to easily configure new masks and add newfields to the database, but i need it as an tool-unspecific layer forthe MS SQL Server.Thank you very much for your feedback, any ideas are welcome!Best regards, Heiko.
1. Is it legal and OK to use a MSDN SQL copy on a production environment or is it strickly for test environments ??
2. If I own a legal copy of SQL 7 with 5 cals, can I legally use SQL MSDE and have more than 5 people access my SQL server or am I also limited to 5 users as my original ??
hiis there any tool to write stored procedurei have long queries to write instead of that i want a tool or editor where i can click or drag and drop to write procedurei make my work to complete fasterif any please tell me the site to downloadthanx byeramesh
Hello,I'm looking for recommendations for SQL tools used for comparing schemas from different databases. I would like the tool to be able to compare schemas for at least two databases and list the difference in tables, sps, functions, and view. Even if the both schemas contain the same structures I would like to know what differences exist. Also, I would like to be able to type in a table field name and get a list of every place in the db that field is referenced. On another note, how do others go about maintaining a version history of db structures? Any tools available for this functionality? Currently, I use visual source safe for my app. code. I could place copies of my SPs in visual source safe and edit them within visual studio. I'm not sure how I would easily get the edited version into my sql database without having to create a script or copy the edited version back to the db. Any thoughts?Any information is appreciated.Oscar
Hi guys, Im trying to use Bulk copy to transform a csv file into a table
BULK INSERT [BULK_INSERT_TEST] FROM 'c:stock.csv' WITH ( FIELDTERMINATOR = ',', ROWTERMINATOR = 'N'
)
--*STOCK LOOKS LIKE THIS stockcode,barcode,qty 10002,"10002087",0
TABLE Looks like this
id, stock,barcode, qty
can anyone give any guidance on performing this - at present this doesn't seem to work. Worth mentioning id is autoincrementing. I would also need to strip quotes around the second field. Note. Stock file has a header - do I need to strip this to use BCP?
There are 4 columns in total, however some rows have missing values for Column 4, and but given there are no field separators (all columns have fixed width and data is truncated if needed), how should the format be written to define a row terminator to apply to Column 3 if Column 4 is null.
I'm in the process of delivering a SQL Server BI solution to a customer. I have the cube built, SSIS working and an SSRS package with several hundred reports. They would like to roll their own reports.
I've looked at ProClarity (or whatever MS calls it now) and Report Builder. Both seem to have their pros and cons.
I think Service Broker is the right tool for what I want to do but need confirmation. Here is what I want to do.
Our company sells small business consulting services. When a client signs up with us, we put a server at their site that runs our app (using SQL Server 2005). Once the server is setup at their site and operational we constantly send certain information from that server to our central corporate server. There could be thousands of records every day being sent from each of these servers to our corporate server. We're using webservices now to transfer this data from each server but I'm afraid we're reaching a point where webservices will no longer work because of the volume of data.
1) I assume that Service Broker was created, in part at least, to address these types of situations?
2) If #1 is correct, is there any step by step examples/instruction that shows me how to use Service Broker to communicate between two SQL Server databases on two different servers?
Hi, I'm not sure this is the right place to ask this question (please re-direct me if it isn't). I would like to learn about SubSonic tool:1. do you recommend this tool ? if so what are its best uses?2. is its license allows me to work with it for free for commercial uses? any license limitations I should be aware of for using it as a tool? thanks,
