Issues For Embedded SSIS Solutions

Mar 16, 2006

My warehouse app employs a distributed architecture. Extractions from disparate (wildly so) systems, and transformations and loads into a standardized schema are performed at various locations close to the source systems (both physically and "logically" speaking). There are security and other reasons for this. However this causes some related design and implementation challenges for the ETL processing.

For one, the ETL processes must be successfully operated by non-technical medical administrators, who actually have little interest in the application and sometimes even the analytics produced by the system, who have other more pressing day to day work they want to be doing, in organizations where turnover is high,training is spotty, and LANs are fragile and often congested.

So, real-time feedback to the operator during processing is pretty dern important. I have built a fairly sophisticated GUI (using .Net forms inside a script component) for the operational interface-input boxes just wouldn't cut it).

But that interface is lacking real time feedback as to processing progress at runtime.

Anyone got that T-shirt yet? I'm thinking I need progress bars and real-time task and component progress reports. Also. is there a way to capture the built in logging output in real-time?

View 5 Replies


ADVERTISEMENT

When Should Ssis Package Use Embedded Sql Query?

Jun 27, 2007

On another thread a poster suggested that you can experience performance gains in Ssis by not using stored procedures.



I asked if I could have some Microsoft articles that articulated this position, but received no response.



Does anyone have links to Microsoft supported articles articulating when/if it is appropriate from Ssis to Not exec Stored Procedures for data manipulation or data collection?

View 24 Replies View Related

Any Embedded Tool In Win2003 Or SSIS For Extracting The Files From ZIP

Jun 20, 2007

Who knows whether exists embedded tool in win2003 or SSIS for extracting the files from ZIP?



Thanks in advance.





View 4 Replies View Related

Please Help With Oltp Solutions

Aug 28, 2007

hi. i dont understand what they mean when they say developing oltp solutions. can anybody pls explain it to me. also does anyone know what ways there are to develop sql oltp solutions using SQL 2005 reporting services, OLTO, Excel Services. as well as any good tutorials for it?

thanks for the help.

View 6 Replies View Related

Replication Solutions?

Jul 20, 2005

Hello everybodyI work at a company in Iceland and we have developed a 3-tier solutionwritten in ASP - Visual Basic - MSSQL2000, 4 companies are using thesolution almost constantly and accessing it through a browser. Theconnection has never gone down (yet) so that it has affected ourclients but we are thinking of how be able to run the solution locallyat every place and then create a replication to a main server that ishosted at our place.My question is: Does it affect speed for the clients that are usingthe solution or is there a better way of doing this?The solution is a ticket sale system and our clients use it every dayand people that sit at home should be able to order ticket online.Because of that we can't update the database every 5 minutes or 15minutes because we don't want a double booking in the same seat.Any help appreciated!- Sindri

View 1 Replies View Related

Solutions And Projects: What Are They And How Do They Help The DBA

Mar 10, 2006

I guess the Subject line sums it all up, but I need some experienced explanation of what do a solution and a project represent, and how do I use them to my advantage.

Is a solution an entire database? If so, how can I create a solution from my existing databases?

Are either of them a way to collect together scripts etc which will be run against a production database when the solution is rolled out?

What is a project? Is it a single set of scripts related to an upgrade to a database? If so, can it be executed as a single entity? How is the sequence of execution controlled?

And so on and so on....

2005 is such a step backwards for DBA's with all the features we used to have and now don't. If it wasn't for the fact that MS will eventually stop supporting 2000, I frankly see no incentive to upgrade myself.

Ok, OK, flame off.

Can anybody suggest some resources which might give me some insight into these questions?

Thanks,

-Rob

View 3 Replies View Related

Replication--Too Many Questions---Too Little Solutions

Dec 15, 1998

1) Do I have to install publishing on both servers (A and B) even though one will be publisher and the other will be subscriber.

View 1 Replies View Related

Replication--Too Many Questions---Too Little Solutions

Dec 15, 1998

1) Do I have to install publishing on both servers (A and B) even
though one will be publisher and the other will be subscriber.


2)a. Can named pipes be used for communication between these two servers
which are on the same domain but not on the same network. Why or why
not, whatever the answer may be?
b. If I use TCP/IP, it the connection set up using the client
configuration utility? How is the connection string set up in this
case?
c. Suppose the publishing server was not using Net-Beui. Could this pose
any problems for communication. (is using lmhosts sufficient in this
scenario)


3) I have set up a (remote) SQL Server to be a Publisher/ Distributor.
Both SQL servers have been configured to be remote servers relative to
each other. Following are the steps I have carried out to set up
replication:

______On the Publication Server (a remote server)
I went to Server --> Replication Configuration ---> Install Publishing

Next, I chose a local distribution server. I think that the
instdis.sql script ran fine because the distribution database was
installed successfully.

___Next, I went into Manage Publications from Server menu to set up
the publications.


_____________When I went the subscription server to subscribe to the
published articles, I got the following error message:


Error 14093: [SQL Server] You must be System Administrator (SA)
or Database Owner (dbo) or Replication Subscriber (repl_subscriber)
to execute the stored procedure.




PS
Please Help

View 1 Replies View Related

Bidirectional Replication Solutions

Apr 9, 2008

According to this article from last year:http://searchsqlserver.techtarget.com/tip/0,289483,sid87_gci1251149,00.htmlThese are the main options:* Merge Replication* Bi-directional Transactional replication* Immediate Updating* Queued Updating* Peer to Peer* RDAAre there any new alternatives that have popped up over the last year? Are all of six above still good options based on needs?We currently have a three server topology using merge replication.ServerA (App1DB) <--> ServerB (App2 DB)ServerA (App1DB) <--> ServerB (App3 DB)ServerA (App1DB) <--> ServerCServerA supports 1 intranet application using 1 DBServerB supports 2 extranet applications using 2 DB's (1 per application)ServerC is our DW server that we have installed a Search DB which is used by all applicationsPrior to our "upgrade" to merge replication we were using 1-way Transactional Replication so our topology looked like:ServerA --> ServerB (App2 DB)ServerA --> ServerB (App3 DB)We also had linked servers between ServerB and ServerA as well as between ServerC and ServerA to update data on ServerA. We would simultaneously update/insert the tables on ServerB/C and create custom stored procedures to handle the data already processed from the subscribers.With our new implementation we are seeing more latency as well as locking since merge replication is not running off of transaction logs anymore.My main question is would we see an increase in performance and less locking as a result of a topology like this:Master <--> ServerA (App1 DB)Master <--> ServerB (App2 DB)Master <--> ServerB (App3 DB)Master <--> ServerCWhere Master is a server and DB supporting no applications (hence no OLTP). Would latency be the same/better/worse? Should we stick with our current implementation and just performance tune it?A secondary question I have is given the bidirectional replication options above did we choose the best one for us? These servers are all on the same network hosted by the same provider over Gigabit Ethernet (I assume). I think we have the polling interval set at 5 seconds and we are thinking of moving it to 10 seconds at most. Real-time latency is not critical to our business but it would be a "nice to have". For conflict resolution we are keeping it simple, whichever was inserted/updated last "wins". It looks like Bi-directional Transactional replication might be a better option for us. Would it give us the autonomy we are looking for? Any major "cons" to using Bi-directional Transactional replication over merge replication (beside scalability). Scalability may come into play a few years down the road but for now it is not a high priority. Also would the Master model described above using Bi-directional Transactional replication be a successful implementation?ETA - One thing merge replication gives us is autonomy between our application servers, particularly when ServerA needs to come down for upgrades, the applications on ServerB can still function without any dependencies like we had before with 1-way transactional replication with linked server calls.

View 2 Replies View Related

Meta Integration Solutions

Apr 27, 2007

Hi,



Has anyone experience with Meta Integration Solutions and converting to SSIS ?

and converting to SQL ?

and converting to Analysis Server ?

and converting to Reporting Server ?



You can find their website http://www.metaintegration.net/Products/Overview/Solutions.html



Constantijn Enders



View 1 Replies View Related

SQL Northwind End To End Database Solutions (examples)

Apr 30, 2004

Does anyone know where I can find a Northwind end to end database solutions (examples) written in ASP.NET (VB). I would like to reverse engineer this project to learn more about ASP.NET?

Thanks.

View 1 Replies View Related

Solutions To Large Access In SQL 2005

Dec 17, 2007

I am on a project to develope an route finding system that search for the optimal route to stick with for users of the system. The current version that i've done and successfully run is using normal database access in MS SQL 2005. I stored nodes information in the database and the application will query them using normal "select" clauses and return a datatable object to the application. The result is rather slow cause by the multiple access to database server to query. The application used 8 second to look for a short route withour cosidering lots of calculation of traffic information that i will use later. Any comments on the architecture or approach to switch my algo to T-SQl?

View 5 Replies View Related

Need Links For Multiple Keyword Search Solutions Please

Jun 6, 2008

I have been informed that all my keyword search solutions are susceptible to SQL injection attacks.  Does anyone have links discussing basic ' multiple ' keyword search solutions?  I would think this is a very common routine (perhaps so much so than only newbies like myself do not know it).  I have read the posts about escaping ', doing replace " ' ", " '' ", using parameters and yet every multiple keyword solution I come up with is said to be injection prone.
Example: visitor enters:  Tom's antiquesinto a TextBox control and the C# code behind securely generates the below call to the database.
SELECT L_Name, L_City, L_State, L_Display FROM tblCompanies WHERE L_Kwords LIKE '%' + 'Tom's' + '%' AND L_Kwords LIKE '%' + 'antiques' + '%' AND L_Display = 1 RETURN
I understand that concantenting string parts using an array and then passing the sewn together string to a stored procedure exposes it to injection.  I hope that my single keyword routine below is secure, if it is not then I am not understanding how parameterized SP are supposed to be constructed to protect against injection.string CompanyName;CompanyName = TextBox1.Text;PROCEDURE CoNameSearch @CompanyName varchar(100)AS SELECT DISTINCT L_Name, L_Phone, L_City, L_State, L_Zip, L_Enabled, L_Display FROM tblLinksWHERE (L_Name LIKE @CompanyName + '%') AND L_Enabled = 1 AND L_Display = 1 ORDER BY L_NameRETURN
 

View 5 Replies View Related

What Solutions Make Returning From A Disaster Easier?

Apr 18, 2007

I'm trying to figure out what solution (replication, mirroring, clustering) would work best for me.

I have been reading many articles in BOL and in this forum. Most talk about getting data TO a backup/standby/subscriber, but I can't find a lot of info regarding getting the data BACK after a disaster is over.

We have a main office and a disaster recovery facility. Most of the time there are no data updates at the disaster location. So, I need to get data to the disaster facility via WAN (latency is not a huge issue - end of day syncing is fine) for backup purposes. In the event of a disaster, the main office will be offline and data changes will happen at the disaster site. When the disaster is "over" and we return to the main office, what's the best scheme to reverse the data back to the main office to start business again? We are a financial company, and have gigabytes of relatively static data. Most changes are current day. So, to snapshot a 100GB database when I know only a few hundred MB changes a day doesn't seem feasible to me.

Most replication scenarios (at least from what I see) can't easily "reverse" the replication after a disaster situation. I'm looking at merge replication on a schedule which seems to look good, but was wondering if anyone else has any ideas or suggestions?

View 5 Replies View Related

Best Practice For Report Projects Related To Application Solutions

Feb 20, 2007

Hello Reporting Services Gurus!

I'm about to start on my first reporting services project, but before I mess it up, I'm looking for some guidance on how best to achieve my mission. Here's what I'm looking to achieve:

I have a datacentric application (SQL Server 2005 Express w/ Advanced Services backend) in which I want to build about 50 "canned" reports for the end users. I want to build the reports utilizing server mode so I can take advantage of some of Reporting Services advanced features. I'm not sure what the best practice would be to build the reporting services project. Is it better to include the report project as another project within the application solution? Or, should I build the report project independent of the application solution? What are the pros and cons of doing it either way? How does including the report project build if it's included in the application solution? How would a ClickOnce deployment deploy the report project to the report server?

My ultimate goal would be to have an "off-the-shelf" software solution that includes an installation package consisting of the application project and report project. Is it even possible due to the Reporting Services architecture to achieve an install in this manner with ClickOnce, Windows Installer, or Installshield? Or, is building the report project indepedent of the application project and deploying the reports to the report server "manually" (i.e. deploy within the report server project) the only solution?

Any help would be greatly appreciated!

Tony

View 1 Replies View Related

SQL Server 2005 Non-ansi Joins: Any Easy Solutions?

Jul 20, 2006

My company wants me to research and flags or registry tricks that would allow non-ansi joins '=*' and '*=' in SQL Server 2005 with a compatiblity mode of 90 to be allowed.

The way I understand the situation is that in SQL Server 2005 with the database compatiblity set to 90, non-ansi join SQL such as the following would not work.

Select * from
Customer, Sales
Where Customer.CustomerID *= Sales.CustomerID

To work, the SQL above would have to be converted to ansi join SQL such as the following:

Select * from
Customer LEFT OUTER JOIN Sales
On Customer.CustomerID = Sales.CustomerID

Many hours would be spent browsing through millions of lines of code to find the non-ansi SQL and have changes made.

Does anyone know of any trace flaqs or registry entries that would allow SQL Server 2005 work in 90 compatiblity and still allow non-ansi =* and *= joins in SQL?

Thanks,
AIMDBA

View 3 Replies View Related

Any Good Whitepapers On Security/deployment For Entire SQL Server BI Solutions?

Aug 1, 2007

At my current employer we are struggling with the best way to manage security and deployment of a project that contains databases, SSIS, SSAS and SSRS components, using configurations.

Environment (Dev):
3 SQL Server databases, all using mixed-mode security, using SQL Server security credentials.
12 SSIS packages; one master package, eleven child packages, 3 shared data sources
1 SSAS database; one cube, 15 dimensions, three referenced data sources from the SSIS project (in same solution)
6 SSRS reports, one data source to cube (not shared- doesn't appear SSRS can share datasources among other projects in the solution? Why?)

Everything runs fine in development. Now comes the tricky part.

Deploying SSIS and SSAS into production environments:

-Packages use XML config files for connection strings to three relational data sources.
-Deploy to SQL Server storage. Deploy wizard copies package dependencies (including XML config files) to default location set in INI file. When I do this, no config file shows up in remote server (remote server not set up identical to local, so directory does not exist. Need UNC path?) So, being a developer with no "special" permissions on the PROD server, what security permissions is allowing the deployment wizard from copying files to this location on a production server?
-Using a deploy script using dtutil doesn't copy the SSIS dependencies. Is this matter of using COPY or XCOPY to copy the configuration files to the dependency location? Again, in real-world practice, do developers typically change this location in the INI file to another location, or stick with the default. In either case, how does security work that allows files to get copied to the remote folder? (i.e. manual, or SQL Server manages this file folder permission through some other magic)
When using SSMS and running the package after being deployed on the remote server, if the config path is the default (e.g. C:program filesMicrosoft SQL Server90DTSPackages...) it appears to be read from the local machines directory rather than the remote machines directory path (do I need to use UNC paths? The wizard doesn't give this option it seems)
-When scheduling the job from SQL Agent, does the proxy account need permissions to the folder the config files sit in?
-What about the roles security on the packages themselves? Where does the server roles come into play (dtsltuser, dtsadmin)
-Because the SSAS project uses connection references to the SSIS project in BIDS, and SSIS project uses configurations, will SSAS pick up on these connections?
-What about impersonation levels for SSAS? Leave all data sources set to default, and set the database impersonation level to "UseServiceAccount"? What if the developer is not the same as the OLAP administrator on the production server? In this case, Use Service Account isn't an option, and neither is the current users credentials.
-SSAS database also has security for Full Control, but still doesn't prevent security at the data source level within the database (talking about impersonation level, not source db credentials)
-How can SSRS connections leverage other shared connections?


As you can see, there are a ton of security considerations, none of which are intuitive and can be configured multiple ways and actually work (and a ton of ways that won't work).

I need a simple cheat-sheet about each step to take to configure this so multiple developers can work without interruption, hot-deploying SSIS, SSAS, and SSRS changes into different environments (QA, PROD).

-Kory

View 2 Replies View Related

DTS & Embedded CR/LF

Sep 18, 2000

I have a DTS package that transfers some columns defined as varchar(8000).

Sometimes, the values in the columns defined this way have embedded CR/LF in them. When they do, DTS does not always transfer the values to the target.

Any idea why? Workarounds? TIA.

View 1 Replies View Related

Embedded Select Vs. UDF

Jul 8, 2004

Why is using an embedded SELECT statement faster than using an embedded UDF that has the same exact SQL code within?

Example (not syntax corrected)
Ex1:
SELECT myValue1, myValue2 FROM myTable WHERE aValue = (SELECT Value FROM someTable WHERE myIndex = 800)

is much faster than something like

Ex2:
SELECT myValue1, myValue2 FROM myTable WHERE aValue = (dbo.FN_myUDF(@vmyIndex))

Given that dbo.FN_myUDF has the same code as the embedded select in the first example.

TIA,

KB

View 4 Replies View Related

Embedded Queries?

Feb 18, 2006

I'm looking into a problem a friend is having, and I'll say right offthe bat that I work with with php and MySQL, and not MS SQL.What he is attempting to do (in MS SQL) is take two database fieldsfrom a table (string fields), multiply them together, and put them intoa third field. This third column in the table has not yet been createdthe time of running the query.If it needs to be multiple queries, that is fine. My first thought isto use a simple ALTER query to add the column to the table, then tocall a UPDATE function which uses a select statement inside of it. I'mnot sure if something like this can even be done.// ------------ Suggested queryUPDATE chrisslu SET 'discquantity' = '(SELECTchrisslu.quantity*chrisslu.nr_of_discFROM chrissluWHERE (str(period,6)>=? AND str(period,6)<=?))' WHERE(str(period,6)>=?Andstr(period,6)<=?)// ------------ End Suggested queryIt starts with an UPDATE, but replaces the value to be set with aSELECT statement. I honestly don't even think this query issyntactically correct, I'm just trying to get the general concept down:).So, question the first: Is this type of query possible? The reasonI'm doing this is because I was told MS SQL has no way of storingtemporary variables... otherwise I would just call a SELECT statement,store the variable, and UPDATE the new field from the variable afterthe ALTER statement.Second question: If it is possible, am I on the right track, or doesit need to be entered in completely different than what I have?Third: Regarding the 'type'. Do I need to do any kind of typecastingor conversion of the fields? Both chrisslu.quantity andchrisslu.nr_of_disc are string fields (that is what I was told, theymay be varchar of some kind). In order to use them in a mathstatement, do they have to be floats, or doubles, or something similar?I appreciate any response, I know this was a long winded question.Chris

View 9 Replies View Related

Embedded Video?

May 22, 2007

Hey everyone,



Is there any way to embed a video in a report? It doesn't seem like it but I thought I may be missing something. Maybe directly editing the xml code? Thanks.



-Keith

View 1 Replies View Related

Embedded Tab Character

Dec 4, 2006



I have embedded tabs in a text field that I want to import to a destination table.

I was thinking I need to replace the tabs with spaces. REPLACE(character_expression,searchstring,replacementstring) Anybody know how to specify ascii in the character expression.If there is a better way I am open to suggestions, however I do not way to remove this in the raw data but handle at transformation time. Thanks,Larry

View 9 Replies View Related

Embedded Image Does Not Appear

Jul 11, 2007

Hi there!



I've an embedded image (logo) in my report, and in the preview it looks wonderful, but after deploy the logo doesnt appear in the report (report server).



The url of the dead-link-image is http://hamsql1/Reports$BI/Reserved.ReportViewerWebControl.axd?.



I thought that an embedded image is stored "in" the report, but it seems to be an other place. Is it nessecary to deploy an embedded image too? Is it a problem of permissions (i read it in another threat)? How to set these image-see-permission? Or is there an other problem?



Every idea is very welcome!



Thanks,

Torsten

View 6 Replies View Related

SQL In The Embedded Code

Mar 21, 2007

Is it possible to have a sql select statment in the embedded code in a report?

View 5 Replies View Related

Embedded IF EXISTS(SELECT * ...) 1 ELSE 0

Jun 11, 2008

I have a Select statement that was working just fine:
string sSqlCmd = "INSERT INTO SiloKeywords (Silo_ID, Keyword, UserName) SELECT Silo_ID,'" + Keyword + "', '" + strUsername + "' FROM SiloNames WHERE Silo_Name = '" + Uri + "'";
the PM now wants to capture the internal User, which I have done but I am having real problems with the syntax if the internal User exists:
 
string sSqlCmd = "INSERT INTO SiloKeywords (Silo_ID, Keyword, UserName, IsInternal) SELECT Silo_ID,'" + Keyword + "', '" + strUserName + "', if EXISTS(SELECT * FROM InternalUsersList WHERE Alias = '" + arrResult[1] + "') 1 ELSE 0 FROM SiloNames WHERE Silo_Name = '" + Uri + "'";
I am completely lost here.
 
Thanks for any help

View 3 Replies View Related

Embedded And Distribute MSDE On CD

Mar 18, 2004

Hi,

I am trying to find something like a light version of SQL Server 2k to store data for an application that will be distributed through a cd.

Since we also have a web front end for this application when the user is online, we prefer that we can reuse the code for connection on both the cd and the web.

Any idea what will work?

I have come accross MSDE 2k but was not able to find any documentation on the pros and cons for this product regarding its capacity, the type of data it can handle, security, ...

Please give me some pointer.

Thanks very much.

Baoha.

View 1 Replies View Related

Can Sql Server Express Be Embedded In .net?

Aug 25, 2005

I was wondering if anyone had tried to embed sql server express in a .net application. Specifically, I want to use it from within the app only, similar to embedded mysql (libmysqld) or firebird. I've seen some articles on sql server express saying this can be done, however there is no info on how do to this. Anyone know how it might be done?

View 1 Replies View Related

Embedded Control Characters

Oct 10, 2005

Beyond my control: I am finding control characters (likely tab) ismaking its way into address fields of our operational system. This ismessing me up when I load the data into our warehouse w/ BCP (fieldsget shifted).Is the a nifty way to strip control characters from data?TIARob

View 1 Replies View Related

Embedded Select Query?

Jul 20, 2005

What I'd like to do is use the result of one query as the input foranother. eg:INSERT INTO foo VALUES ((SELECT id FROM people WHERE name = "bar"),10,'foobar') WHEREid = 1;Problem is, MSSQL wants a scalar value. Is there a way around this?Can someone tell me the correct syntax for what I want to do, or is thissomething that will have to be done outside SQL?Err. Hope I've been clear. Thanks for any help you folks can give.

View 2 Replies View Related

Embedded Text Qualifiers

Feb 16, 2007

We have text files that are comma delimited, use double quotes as text qualifiers and sometimes have embedded double quotes. The embedded double quotes are escaped with an additional double quote like: below.

"123","product q"

"124","product ""a"""



DTS 2000 had no problem with this- it correctly parsed the files. The 2005 SSIS file connection manager correctly parses this in preview mode. But when the task is executed the task fails with the message "The column delimiter for column X was not found".

What is the recommended approach for this - we have alot of files in this format.



thanks

View 3 Replies View Related

Problem With Embedded Image

Jul 31, 2007



Hi,

I have a report with background embedded image. I have placed few textboxes on top of them to display data. Every time I set the location of the textbox, it is behaving strangely by rendering at one location one time and changing location next time when it is rendered. I am viewing it in PDF format.

How can I fix the location of textboxes that renders at one location all the time ?

Thanks,
Shyamkumar123


View 2 Replies View Related

Windows XP Embedded Support

Jun 7, 2007

Hi,

documentation states, that Windows XP Embedded (XPe) is not supported by any SQL Server 2005 Edition (as I intend to use the Epress Edition on that platform, I post to this forum).
So, why is that? Is it just not tested or are there technical reasons for it? What are these reasons? What works and what doesn't on XPe? May XPe be supported in any later version?

thanks,

Matthias

View 1 Replies View Related

SQL Mobile Embedded XP Support?

Jun 8, 2006

I am building a mobil application using Windows XP embedded. I need a database that will run on xp embedded as follows:

Client mode: Sometimes connected to server(local) sometimes disconnected.

Server Mode: Ready access to database from client and system running server.

Client must be able to switch to server mode in event of failure.

Does SQL Mobile support running Windows XP embedded on the Server?

Does SQL Mobile support running Windows XP embedded on the Client?

Does SQL Mobile support running Windows XP embedded in a failover mode?

If it does not support this type of operation then what mobile database will support XP Embedded in all modes stated above?

View 4 Replies View Related







Copyrights 2005-15 www.BigResource.com, All rights reserved