Distributing DTS Workload
Apr 14, 2008
Suppose I create a DTS package (I am an old timer from SQL Server 2000 days but I suppose with 2005/2008 integration services there is a similar concept of package) which essentially reads data from a set of csv files, transforms it, and loads it into a SQL Server database. There are two scenarios:
i) The transformation involves applying functions on the columns before loading to the database.
ii) The transformation involves joins of data from multiple files.
My question is, can I run this DTS package on a machine which does NOT have SQL server installed for scenario i) and scenario ii) above? The idea is to distribute the actual processing across a set of computers to take off load from the main database server. If it can be done for either case then can u please explain how? Thanks in advance for your help.
View 1 Replies
ADVERTISEMENT
Dec 16, 2004
Hi - does the workload governor kick in when more than 8 concurrent operations take place on MSDE as a whole, or does it kick on each database.
Ie. if I have 8 databases within MSDE - and there is 1 operation happening on each one at any given time, will one additional operation on any of the databases invoke the workload governor across ALL databases?
Thanks,
Mark
View 1 Replies
View Related
Jun 13, 2006
Hi, I need some help about a project I'm working on.
I'm working on a project (based on MS SQL Server) that will involve a lot of clients, each one constantly querying the server.
Just to have an idea of the project, I'm talking of about 2000 clients, each one queries the database 3-4 times per second: a total of 6-8000 queries per second.
The database is reached through Internet and every client uses a dedicated DSL connection.
My questions are:
- What is the maximum workload of a Microsoft SQL Server?
- What happens when the server workload reaches 100%? is there a queue? and what happens when the queue is full? is it possible that the server goes out of service due the excess of work?
- What are (more or less) the hardware requirements for such a server?
can anyone help me, at least telling me how to find these answers (web sites, books, ...)
thank you for you support
View 11 Replies
View Related
Jul 23, 2005
Hello,- Is any body aware of a random workload/query generator such as theTPC-H query generator (QGEN) ? I am looking for a query generator thattakes a schema as an input, and produces several queries.I need it mainly for performance evaluation. the generator I am lookingfor should produce SELECT queries, including joins, and not only INSERTINTO, DELETE, UPDATE queries such as the "SqlQueryGenerator" athttp://www.tucows.com/preview/297930ThanksRegards,Abdur-Rahman
View 2 Replies
View Related
Jan 8, 2007
Hello,
I am in the process of porting over an application from Access To SQL implementing SQL Server 2005 Express. My intention is to implement this database on a full time server and upgrade to a full blown version of SQL later. Am I correct in assuming that there is not limit on the number of concurrent connections to SQL Server Express since it was stated that the "Workload Governor" has been removed? Or does something else control the number of users that can be simultaneously connected to the server.
My reason for asking is I have 7 machines that need to access the server. I also have 2 databases that need to be accessed from each machine. If there is no limit, I will keep my databases seperate. However, if there is a limit, I will most likely merge the tables into 1 db.
Thank you,
Robert
View 3 Replies
View Related
Nov 6, 2007
I have a specific job that should be run with a decreased memory usage when the workload is heavy on the SQL Server.
This is a heavy job that has no specific requirement when it comes to response time.
It is important that the rest of the application shouldn’t be affected with longer response time when this job is running.
How can this job bee handled from the application, without having to create a separate batch job.
View 1 Replies
View Related
Aug 27, 2014
We are using Sql Server 2008 R2. We are planning to move to 2014 either through in-place o side-by-side. Mostly side-by-side.
Here first we want to test that 2014 and availability groups. First we are doing in dev box to test. Also we need asynchronous replica because just we need to use that as reporting server.
We want to take work load. For the dev boxes, the applications doesn't connect. Then in that case what the people will do.
If We take database backup from prod to that dev and run some queries in the loop and run the trace for some time and do either in-place migration or side-by-side migration and use the same back to restore and doing the running the same queries and compare this trace to previous trace will work?
View 0 Replies
View Related
Mar 19, 2015
I am trying to create a table that would represent a workload for each shop. In order to do that I need to have WorkLoad table and ShopWorkLoad table which is actually just aggregation of WorkLoad.
WorkLoad contains a list of following items:
current orders that are in the process (one select statement)
scheduled orders (another select statement)
expected orders (third select statement) that come through a third-party system
All of this needs to be live. So, for example, as soon as order is added to Order table it should be included in WorkLoad if certain conditions are met. Same goes for scheduled orders (which come from another table). Expected orders will be loaded on a daily bases (based on historical data).
ShopWorkLoad table is aggregation of WorkLoad table.
Currently I did it this way:
Added after insert/update trigger on Order table: when order is created/updated, if it meets certain conditions, it should be inserted in WorkLoad, otherwise remove it from workload if it's in there and doesn't meet conditions
Added after insert/update trigger on Schedule table: when order is scheduled, if it meets certain conditions, it should be inserted in WorkLoad, otherwise remove it from workload if it's in there and doesn't meet conditions
Running daily job that populates WorkLoad table with expected orders based on historical values
Final step is to create an indexed view vShopWorkLoad
My biggest concern is usage of triggers which call pretty complex logic to determine whether item should be added to workload or not.
One other option was to create vWorkLoad view and somehow make it an indexed view but currently I don't see a way of doing that because the query consists of 4 union select statements, below is pseudo example. But even if doing it that way, how to build aggregated indexed view on top of vWorkLoad indexed view?
Third option is to use sql agent job which would run every x seconds (maybe 20) and it would execute all of these queries to populate WorkLoad table with delay of 10-20 seconds, but I am still not sure if this is acceptable to the client.
Fourth option is to create 3 or 4 indexed view where sum of them makes a workload. Then, ShopWorkLoad view would be built on top of these 3 or 4 indexed views, but in this case I don't know how this would affect performance since ShopWorkLoad query would be often queried.
Example of workload pseudo query:
select
WorkLoadType = 'Order in process',
OrderId,
ShopId,
...
from
Order
[Code] ....
View 1 Replies
View Related
Mar 22, 2001
I want to know if there is anyway to distribute jobs to a non system
administrator. In other words do they have to have System Admin rights, to be able to modify a job and run the job.
View 1 Replies
View Related
Jun 21, 2006
Sorry if this something beaten to death already...
What is/should be required for a target client machine to use and connect to a local copy of a SqlExpress database? Can't seem to get a db connection established to a different machine through a VB6 app. Installed the sql native client on the machine, but still can't connect. What am I missing, or do I have to distribute/install the entire SqlExpress client? Working fine from my development machine.
Rick
View 11 Replies
View Related
Aug 18, 2000
I'm looking into adding OLAP Cubes as part of our software to be distributed with our OLTP and eventually OLAP databases.
Is there any books that deal with distributing OLAP Cubes and or security.
Our clients will have SQL Server with our databases.
Thanks
View 1 Replies
View Related
Aug 24, 1999
Hi
I had replication set up and working but had to unbind the Pub/Sub and re add. Now on the ditribution task in Executive I am getting the following error
The last distributor job id and the last subscriber job id do not match. No jobs were available with a job id > 2147.
This is SQL Server 6.5 by the way.
Does anyone know what has caused this and what the solution is?
Steve
View 2 Replies
View Related
Jun 26, 2007
Can reports be set up to run and print automatically to specific printers on a nightly basis. What I'm trying to accomplish is, I have 6 users who need to see specific transaction reports for thier department each day. Of course I don't want them to see the other departments reports.. What options do I have....
Thank You!!
View 1 Replies
View Related
Mar 27, 2007
Good Day!
I want to know how to distribute a database in SQL Server from one computer to another?
Thnx!
Kapalic
View 1 Replies
View Related
Jun 25, 2007
Hi,
I am new to SQL Server. I want to email my database and the diagram to a friend of mine so he can review my work.
I noticed that the sql server databases that I have downloaded had an extension of .msi. I don't know if I need to do something to make my database have the .msi extension before I can email it to my friend.
I have been able to copy and paste both the database and the log file to a folder on the desktop. I can't seem to open these though. I have right and left clicked on the
the databse file that is on my desktop. When I do both of these it prompts me to select a program to open it with. This is where I have problems. I go to the programs folder and then to the sql server folder hoping to be able to open it. I haven't been able to find anything to open it with. I have looked at every file in the folders but none seem to be appropriate.
I am using sql server 2005 express edition. Is this a limitation of the database that are made with this free edition?
thanks,
Krist
View 3 Replies
View Related
Jul 15, 2004
We have a large SQL database, and we need to send out updated records to many clients' sites which are not connected.
We currently have a tool which looks at the audit log of changes we made, creates a file based on this, which is then emailed to our clients. They then run a tool we created to remerge the changes.
I suspect SQL server replication might make all this possible. Am I right? Can SQL server produce a file automatically which can be applied to a remote database to update the tables as appropriate? From looking at some replication stuff it looks to me like you have to have the servers on the same network.
View 1 Replies
View Related
Nov 4, 2006
I'm writing an application that uses Sql Express 2005 with Advanced Services (for fulltext indexing) and will be needing to deploy the database server. I just realized that the install is 230+MB. Is there a way to trim this down to just the components/features I need?
Thanks,
Eric
View 1 Replies
View Related
May 3, 2007
Hello all,
Thanks for all your great forum topics; it gives me a nice break from coding all day!
I know the answer to this question is in the documentation that comes with the SQL Server Express [9] download, but I've been unable to find it. What I was wondering is what the restrictions were if you were to incorporate the "free" SQL Server Express for selling or general distribution? Any help would be greatly appreciated!
Thanks,
P.
View 5 Replies
View Related
Apr 4, 2006
If I want to distribute a VB database programm using MSDE I have to install MSDE seperately which can be a challenge for some users. It there a way I could deploy SQL Express with my VB program so the user would not have to do a seperate install? Is this possible in Visual Studio 2003 or do I need 2005?
Thanks
View 1 Replies
View Related
Feb 4, 2008
I have written a Visual Studio 2005 vb application the accesses SQL Server 2005 Express. How do I create a distributable CD with everything necessary on it. Can I use One-Click?
View 3 Replies
View Related
May 16, 2006
Hi every body
I use Sql server 2000 .and dial up comunication with weake connection.
i have around 4700 records for distribution,the distribution agent starts and at the same time
I select count(*) from Destination table with (nolock).
after distributing 3700 records it retry again from record one and it distribute agai
thanks for your kindness
View 4 Replies
View Related
Jul 2, 2006
Hi all,
I need to find all information about distributing SQL Server XE and applications using it . Is it legal ?? should it be used just for educational use?!!! could it be used to develop a commercial software !! what limitations are controlling this ??
thanks in advance !!
View 6 Replies
View Related
Apr 16, 2007
Hi
I would like to know, if it is at all possible for Setup files of SQL Server 2005 to be distributed. We have clients who currently use our software which makes use SQL Server 2000. We are looking ahead at distributing our new version with SQL Server 2005 and would like create an Installation package that will upgrade the database engine to the Workgroup or Enterprise editions of SQL Server 2005. Are there any suggestions around this topic, we are looking at ways in making this task of upgrading the engine as streamlined and automated for our Client. Please include advise with regards to licensing.
cheers
Kevin
View 1 Replies
View Related
Dec 14, 2005
hi i am new to mssql db stuff. I just developed my first VB.NET application and it uses a mssql database which is hosted locally on my development machine. I built the deployment msi file for my application which can be installed easily on client machine, but have no idea how to move the mssql db with it.
View 1 Replies
View Related
Apr 29, 2014
Why shrinkfile empty file does not redistribute data evenly in the primary file group with multiple files:
Please run the script attached to see what the end result is.
This is what I set up last night on my test machine.
1) Create database [FGTest] size 200MB
2) Create table called TEST on primary
3) Insert 40MB of data into test
4) Create another file group called temp in primary size 200MB
5) Shrinkfile('FGTest',emptyfile) so that all data is transfered from FGTest into temp file group.
6) Add another 2 files called DATA2 and DATA3. Both are 200MB.
7) We now have 3 empty files that I want data distributed evenly on. FGTest, DATA2 & DATA3
8) Shrinkfile('temp',emptyfile) to move all the data from temp over the 3 file groups evenly
I would expect at this stage to have the following:
FGTest = 13MB,
DATA2 = 13MB,
DATA3 = 13MB
(40MB of data over 3 files should be about 13 MBish in each file)
What I actually end up with is this:
FGTest = 20MB
DATA1 = 10MB
DATA2 = 10MB
It looks as though SQL Server is allocating 50% of all data to the original file and then 50% evenly over
the remaining files in PRIMARY.
View 3 Replies
View Related
