Newbie Questions About SSIS Script Components And Data Streams
May 10, 2007
The following is a list of questions that I have not been able to obtain concrete answers. I am probably missing something:
1) ReadWriteVariables -- can the updated value for a ReadWriteVariable be accessed within the same data flow? It appears not as I think the PostExecute() fires at the completion of the data flow not the end of the Script Component. Secondarily, the Script Component is a non-blocking transformation so the component does not "see" the end of the pipeline prior to sending data down stream.
2) Record Count -- Because of #1 above, How could you calculate a record count for a data stream? It does not appear that one can calculate the number of records for a data stream within a data flow and then access the count from within the same data flow.
3) FinishOutputs() -- Is the concept of FinishOutputs() applicable to Script Component Destinations? Asked another way, is FinishOutputs() executed at the end of the data stream regardless of whether there are "real" outputs for the component? I can create a "Dummy" output to create FinishOutputs() but is this ok?
4) Script Component -- It appears that the Script Component Source, Transformation or Destination are really defined based on the columns defined in "Inputs and Outputs". Can you convert an Source script component to a transformation script component by simply adding an Output?
Sorry for these basic questions but I am not getting it completely. As you can tell...
Everything I've read says that custom data flow components are built by inheriting from the Microsoft.SqlServer.Dts.Pipeline.PipelineComponent class.
But the stock components such as the Derived Column data flow transformation must each be implemented by their own class. So how do I base my custom components on those classes? The documentation for the PipelineComponent class doesn't list any such subclasses.
We are post-deployment with a serious reporting issue that's causing us to rethink our reporting solution. We're considering moving from what we have to SQL Server Reporting Services (client side).
I have spent the past couple of days getting up to speed on this feature and seeing if we can easily migrate it into our existing application. However, I've hit a couple of stumbling blocks and was hoping perhaps someone here could either tell me the solution or point me in the right direction.
I'm not having any problems creating basic reports (e.g. flat data).
However, I am having problems creating reports where there are related tables in a dataset.
The way it works with our existing solution is that I get a dataset (which contains several data tables) and point it to the report's datasource. That report expects those tables and I have defined table-relationships in the report which process and display the information correctly.
I'm not having as much luck with RDLC.
I can go into futher detail about how I'm creating the report, but let me just ask these general questions first: 1. Can I set a dataset containing multiple datatables equal to a property on an RDLC report and that RDLC report know how to treat and display the data? 2. Is there a better/smarter than this to get a field selection from my datasource (remember, this information is coming from a stored procedure so connecting directly to the database is not an option): a. In code, populate dataset b. In code, write dataset schema to xml (e.g. an xsd file) c. In Visual Studio, add the XSD file to project d. Use fields from XSD file to drag and drop fields on report
A fancy example would be nice too. I've googled like crazy the past couple of days and downloaded as many samples as I can find (including the ones on ftponline.com, gotreportviewer.com, "Tudor's WebLog", and several others). However, I have yet to find one that uses grouping and related datatables.
Thank you so much (if, for nothing else, reading this post )
I have a SSIS Package which I would like to modify using SSIS API. I need to put new component between some two existing data flow's components. During this process I need to disconnect two data flow's components using SSIS API. How can I do that?
Hello All, I am migrating data from one database to another. I am using Multicast to seperate (legal street,legal mail and legal city) and (mail_street,mail_state,mail_zip,mail_city) also later after UNION of the above I am doing two lookups as I had to get contact ID and Customer ID from other two tables. In UNION i am matching (Mail street legal street) and so on.
I am getting double the data in the output. my input data is 1000000 and im gettin 2000000.
We're experiencing a problem where intermittently our SSIS packages will hang. There are no log errors or events in the event viewer. It will happen whether the package is executed from the SQL Job Agent or run from BIDs. When running from BIDs it appears to hang inside one of the data flows (several parallel pipes with sorts, merge joins etc...). It appears to hang in multiple pipes within the data flow component. The problem is reproducable, we just kill it and re-run, and it appears to hang in the same places.
Now here's the odd thing: as we simply open and close some of the components in the pipe line after the place it hangs, a subsequent run will go further in the pipeline before hanging. If we open and close all the components after the point it initially hung, the data flow will run fine, from there on out. When I say "open and close" I mean no changes are made, we simply double-click the component, like a merge join, then click 'close.'
To me this does not seem like a memory problem but likely something is wrong with the metadata, where opening a component and closing it somehow alters the metadata to "right it".
This seems to occur intermittently after we make modifications to the package. It's like if you make any mod, even unrelated to the data flow, you then have to go through and open and close every component in your package to ensure it will work. Again, no errors or warnings are fired.
There is 1 error when i tried to install msde into my window 2003 server web edition. 1. I use command prompt to enter c:\sql2ksp3>setup SAPWD=aA1234567 the errors is "go to the control panel to install and configure system components" How do i install ?
Sql client for 2003 server web edition 2. i tried to install sql client in window 2003 web edition but the msdn cd could not start after i double click on the setupsql.exe. I thought the sql client can be install but not the server on web edition ?
I am a Sys Admin and I am studying SQL to obtain a dba. I am currently running through the 70-228 book and I have a couple questions maybe someone can set me straight.
For SQL Server backups we use Tivoli Storage Manager.
I am currently reading up on DTS, I have used DTS to move databases before. It is pretty straightforward. My question after reading about the usefullness of DTS packages is what are they used for commonly by DBA's? Are they used for backups or are they for pushing transformed data to secondary applications/db's. I see how it works but I want to connect the dots with real world uses.
Another question is about how much you tweak a SQL 2000 db. I have been told by people smarter than me, that very little is done to the DB's after they are created because SQL does a excellent job of optimizing itself. If I have a DB server that is suffering from a lot of I/O overhead what is a way that I can diagnose what is causing the issue. How could I tell if it was hardware growing pains or poor design?
I am pretty new to databases and was wondering whether you guys can help me out or point me in some sort of direction. I am working on my senior design project for college and my group decided to use SQL Server for our databasing needs. So far we installed SQL Server and created out database with permissions and all that stuff. So here is my question:
As of right now we are talking with a company (sniffer software) to get access to some of there code. This program provides information such as host ip and mac address, destination ip and mac address, protocol, port number, bits in, bits out, etc. What we wanted to do was get access to where this information is stored in there program and then push it to our database so we can do analysis on the data. But aside from getting access to this information in the program (we looked into writing an application that does all this ourselves but we are on a time limit) I am confused how to set the database up so that we can have automatic updates to the database using the information we pull from the program. I did a search here and did not find anything and i have been looking on google and the such and haven't had any luck finding anything that will point me in the right direction. If any of you guys can help us out with information it would be greatly appreciated
Hi all, ive just started on SQL server 2005 and Visual website developer. I have managed to start up a database and display it on the server.
But for my website, i need place 6-8 links to a catagory,and then to a sub-catagories. Im not sure on how to start such a thing and how to place links really - any chance of possible tutorial on the net? Thank you.
I have a fair bit of experience in designing database apps using VBA with MS Access for use on a single PC or for multiple users on an office LAN. I also have some limited web design skills using FrontPage 2000 and Adobe GoLive. However, I am now trying to find the best way (for me) to create a website with an associated online database - something I have never done before. Ideally, I want cheap or free software. I would also prefer visual design tools, like the Access interface.
I have some newbie questions:
SQL Server Express looks like it might be part of a solution. However, I would appreciate some guidance on the limitations of using this free software. What are the situations that would create a need for upgrading to one of the paid for versions of SQL Server?
Would the free webspace facilities provided via many ISP broadband services be suitable for hosting an SQL Server Express database or does this normally only come at extra cost?
What web design tools enable visual design of a website using SQL Server Express?
I have no experience of either SQL Server Express or MySQL, but are there any good reasons why one might choose one over the other?
Any guidance would be much appreciated. I realise I have much to learn!
just finished installing SQL Express, and now, dont know what to do next :) - I have installed FULL. Does it have any GUI tool for me to create a database? - how can I login into system? - how can I restore a backup file? I have a backup file from SQL 2005 standard - how can I create backup for databases?
I am a PHP/Delphi MySQL programmer for some years, now I am going to create some code in MS SQL and VB. So I downloaded and registered MS SQL 2005 Server Express Edition and some tools like SQL Server Studio Management Express, and VB 2005 Express.
As I have some knowledge of MySQL design and programming, I have installed MS software, rather without problems. Then, using Studio Management I created new database, new table in it with some four fields (int, varchars and smallint). Then from the same tool I opened table and inserted some data, and closed this application.
Next I opened Visual Basic and in Database Explorer I set up connection to my SQL database indicating the file it is stored in (path*.mdf). I ordered not to copy data to new file but to use original one.
Then in Data Sources I created new data source based on previously set connection. In the end I dragged and dropped my table from Data Sources into my new Form of my new Windows Application, nice grid with panel appeared. So I ran the app, I tried to put some data and then after trying to save (using floppy disk icon on panel) i got error: Update requires a valid UpdateCommand when passed DataRow collection with modified rows. which appears in event handler for saving in line i've indicated.
Me.Validate() Me.LudzieBindingSource.EndEdit() Me.LudzieTableAdapter.Update(Me.TestDataSet.ludzie ) ' <-- here highlights
I tried then to add to ID field identity specification, nothing changed. Then in query editor of management console I put some code: exec sp_configure 'user instances enabled', 1. Reconfigure.
As some sources mentioned. I restarted Service. Nothing Changed. So this is my problem. What to change.
I use Win XP Pro SP2.
And next one question, even more important for me is the way I connect to database. I found that in Visual Basic Creating new connection in Database Explorer I indicate the file on my local drive!!! And for example if I try to open my table alongside in Management Studio Express and in my App in VB and the error raises Cannot open user default database. Login failed. Login failed for user 'FS7120Mm227'. which is not a surprise for me as in VB i am using file which cannot be shared.
In MySQL I simply put address, port and so on and I could use my database in Delphi, PHP, and another management tools with no hassle. Is it possible in VB? Or it only can be used to exclusive using database?
Please inform me if such questions shall be directed to VB forum, I am not sure, and thank you for _any_ answer.
MS goes right way giving people great possibility to use and even sell small apps using their Express Editions of SQL and VB, but it is still too confusing for people who encouraged start using their apps and have such basic problems.
I've been learning SSIS and the BIDS for a few weeks now and there are 2 things that really annoy me. I'm hoping that there is a setting or option or something that I'm missing.
1. I place objects on the Control Flow surface, get everything arranged the way I want it, constraint lines all nice and tidy and then save it. When I open it things aren't the way they were when I closed it. Any way to make them stay the same?
2. If I select multiple objects and copy them when I paste them they are in really interesting places. Any way to have them stay in the same realitive positions?
This is one of the most helpful forums I've ever posted in so thanks to you all for the assistance.
Hi, I have a report with 3 parameters. One of which is Company name. To get the available list for this I use a query: select distinct name from companies, etc. However, I also want the user to be able to select "All" or an equivalent to get all the companies. I tried putting * and % in the default values (non-query), but the drop down list displays "<select a value>". Also I've enabled drill-down, is there a way to auto-expand the results?
I've exposed my data (that exists in a proprietary format) with the ADO.NET provider interfaces (IDbConnection, IDataReader, IDbDataAdapter and IDbCommand). I can't seem to find any examples of how to get Integrated Services to hookup to this .NET code in my class library. Is it possible? My goal is for this provider to be both a destination and a source and for others to be able use IS to manipulate the data however they want.
Running SQL Server 2000 on various servers. On my desktop when I open Enterprise Manager I notice that some of the instances show a green arrow and some show a blank circle. I imagine that might have to do with either permissions or the account that the SQL Server Service is running on. Is that correct?
I am using the sa account on these instances and they are all using mixed mode authentication.
Also, I can not stop or start a SQL Server Service from my Enterprise Manager. When I try to start the service from EM I get an 'Access Denied - Error 5 ' message.
Shouldn't I be able to stop/start these services without going to the box they are on?
I am using Merge Replication for my scenario (POS). I have made the publication (articles are a set of tables in DB ABC) and its subscription (in same DB and diff tables xyz1, xyz2). I have scheduled the push subscription as run continuously between Dates(e.g current date to 3 days ahead, just for testing). Now, Can I change this schedule? How can I edit/remove the artilce from the above publication?
I am setting up a development server with SQL 2005 Dev Ed beta 2. I have been going around in circles trying to find out how to create a database diagram. Microsoft really knows how to make things difficult! Would someone mind telling me how to build a diagram of my new database?
Well I have some queries as a newbie to SRSS 2000 reports deployment.
1) I have to build a VB .NET application with a web interface to display reports in that program. Is it possible? Is there any example codeing floating around which I can take a look? Or how to do it?
2) I have to pass the parameters for the report via a VB text box into the report and the report should be displayed. So the parameter bar in the report as it comes in IE7 should not come. How can I do that?
Or in simpler terms
- How to turn the SQL 2000 reporting features off to allow a simpler report display - How to display the report in a Microsoft Windows Form application.
Sql 2005 I want to grate execute permissions on my stored procedures to a role. While creating the role, it asks for "schemas owned by this role". To me, the schema is merely a namespace that allows you to group objects, but arent schemas such as db_datawriter roles that are central to the db and only admin type users should have ownership of these, correct ?
In a nutshell, I want to: create a new role and assign a user to that role with a stored procedure, grant execute permissions to this role
I was confused by the sql 2005 dialog that asks me to take ownership of roles such as db_datawriter, db_datareader etc, wouldnt that mess up other things with the database ?
help ...
I think its time I review all of the above items role user login schema
1) I have the Beta Preview of Roger Wolter's book on the subject. Has anything major changed from the beta version to the full version (of the book as well as the product)?
2) I want to build a front-end that queues requests for processing that is ultimately done by a standalone console application. The console app knows nothing about SQL Server 2005, so the database will have to fork/execute this app, pass it arguments on its command line, and pick up the result (written to either standard output or standard error) when it finishes. Is this possible with Service Broker? If so, how?
I'm finding that the standard components often just don't quite meet my needs, but would only need some fairly minor changes to save me and my team a lot of work (and produce more elegant solutions). So I was just wondering whether the source code was available for the standard components that come with SSIS, or if there is anyway to extend their functionality? Or do you just have to start form scratch?
hello all -- I'm an SSIS newbie who is exploring (custom) programming functionality through third party components, to be plugged into SSIS, for SMTP, FTP, PGP, TAR, WinZIP, basically for a scheduling & automation utility.
Is there such a concept of third party components (and marketplace for this?) or is VB a much better choice for this vis-a-vis components variety, $$ and robustness ?
Hi, As a hobby, I am creating custom SQL Server Integration Services components, which are installed as an add-on to out of the box components. They are mainly used to reduce the development time and minimize the maintance overhead. I would like to know, how I should market them? What are the mediums to use? Since this is the first time, I am creating a software for mass markets, I would really appreciate your help while marketing this baby.
Just FYI, the first commercially available package that I develop is for creating text based files. My package connects to any OLE DB datasource. Executes the query and stores the output in a text file. It is very useful while developing database driven custom feeds. Instead of working with multiple dataflow tasks, data flow sources, you just need to configure one single component and the rest is taken care for you. Since, the package implements the standard Microsoft Task Editor, it is very easy to learn.
Installing the .msi just creates a project folder in my Visual Studio directory. I'm unclear how to get from this point to being able to choose this component from my Toolbox items in SSIS. There was a readme file that talked about gacutil.exe and .snk files that was a above my head.
How can we install the SSIS runtime components alone.
We are calling SSIS packages from .NET code. We would like to know what are the minimum components we need to install in the machine in which the .NET code is running.
Is it possible to use versioning at all with SSIS components?
My experience so far is that I have not figured out how version them at all, because if I change the version number, then apparently the <property name="UserComponentTypeName" ...> has to be changed in every single reference to the component, in every dataflow, in every single dtsx file that uses the component at all -- this will be a tremendous headache as some of this stuff is in source control. It seemsl clearly infeasible to check out and substitute and change all these references, and check them all back in, except very infrequently -- surely not feasible to do it for each small change.
I am in the habit of versioning dlls and assemblies, and actually changing at least a minor build number for each actual change, so I am a bit unhappy to find that apparently I cannot feasibly do this for a custom SSIS component :(
(I just find a dislike at apparently being forced to leave the version number invariant when I make code changes.)
What I would be very happy with, is if there were a way to adjust all the references in all the dtsx files, to refer to only the major and minor build numbers say, specifying that I am not picky about the revision numbers below those, so that it will find the latest component in the GAC matching the major and minor numbers.
That way I could still update the component version with each code change, which would make me happy, and not have to edit a zillion dtsx occurrences (not having to do that would also make me happy).
Anyone know if that is possible?
I'm hoping it is, and that it is only my ignorance at component references preventing me from doing so.
What SSIS components must be loaded onto the server?
To run an SSIS ETL program, exactly what executable, components, programs, etc. must reside on the server that will be running the program. Our developers write an test their ETL programs locally, on their desktops or laptops, but must upload the scripts to the production server for execution. Obviously, they are not going load Visual Studio and SQL 2005, etc. up to the server just to run the scripts (tho SQL 2005 may be on the same box as one of the DB engines used for ETL).
