Being Able To Stop Parallelism In A C++ Program Calling SQL Server
May 7, 2007
Hi All,
I have written ETL software that runs on SQL Server. We are running it for the first time on a 4cpu (2 x dual core) machine on sql server 2005.
One of the things this software does is perform a 'select * from tablename' to validate that the tables passed to it as parameters exist. This has worked fine on previous releases and on single cpu machines because what the optimiser decides to do is to return just the first page of data and then fetch more. I guess it even works in 2005 standard edition.
However, 2005 enterprise edition allows parallelism. And what the optimiser is deciding to do with such a query is to parallelise it and fetch all rows and then give the result back to the program. So, instead of seeing a fraction of a second to return the first page of data we are seeing up to 90 seconds and the database goes and fetches 15M rows in parallel.
Obviously, what we would like to do is to somehow tell the optimiser that this set of programs should not perform any parallel queries. Or, we would like to turn parallelism off on the specific tables we are dealing with for the period of running these ETL programs....they have no need of parallel processing at the database level for virtually all the calls that are performed.
Would someone please be so kind as to advise us if we can do something like pass a parameter to ODBC to stop parallelism or if we can issue commands against specific tables to stop parallelism for a period and then turn it back on?
Thanks in Advance.
Peter Nolan
www.peternolan.com
View 4 Replies
ADVERTISEMENT
Mar 17, 2008
I am reading a guide telling how to create a private server and it says I need to restore the database. So I go to File->Open->File->My Computer->Local Disk->Program Files->Microsoft SQL Server->MSSQL.1->MSSQL->Data->GunzDB. Then the following message comes up: "The file cannot be opened because it is being used by another process. Please close all applications that might access this file and try again."
The question is, how do I get past this? I need the answer as soon as possible, I am a newb with this program so I don't entirely know what I am doing. I have tried searching for the answer, but I cannot find anything.
Thank you.
View 3 Replies
View Related
May 19, 2004
Hi,
Is there a stored proc or a function I can call from Query Analyzer to execute another .EXE and return when it's finished?
I'd like something like
SET @RET = sp_CallEXE('c:mypathmyprogram.exe myparameters')
(Except for the name of the function, there should be something that does this right? I mean, DTS does it already so it must be referring to a function or a SP.)
I want an answer to so that I can validate that everything went well (hence the @RET variable)
Thanks,
Skip.
View 1 Replies
View Related
Jul 9, 2007
hi,
i am new to SQL. i am trying to retrieve a set of data(i.e. set of rows which satisfies my condition given in the select statement). i want to output this recordset to my calling program i.e C#.net. how could i do this. i was adviced to use temp tables. but i am not sure how to do it. can any one say that using the temp table is the correct way. if so can u give me some sample code.
my proble is
Table name: ProductDetails Table
It has following columns
ProductID Name UnitPrice Category Stock
when i give a category value from front end it should bring me the rows which comes under the given category. there will be many rows under this category. how could i send this set of rows back to my calling program.
regards,
Johny
View 1 Replies
View Related
Aug 16, 2006
Hey guys,
Have you ever tried to call the SqlDependency.Stop() method in a class destructor (C#)? It seems like the finalization process hangs after the call to the SqlDependency.Stop() method (for example the assignment after the SqlDependency.Stop() method call is never executed).
~Program()
{
if (!_finalized)
{
SqlDependency.Stop(NocConnectionString);
_finalized = false;
}
}
I tried to use the ADO.NET 2.0 tracing, and it shows that the SqlCommand.Cancel() method call throws an exception during finalization, but it€™s not possible to intercept it. Do you have any clue about it or have you ever experinced the same problem?
Regards,
Dmytro Kryvko
View 10 Replies
View Related
May 23, 2007
why we use sql exceptions ...
what the program will do if we caught that exception .. i need some suggestions ... i got this exception(String or binary data would be truncated.
The statement has been terminated.).. will it affect the functionality of the program...
hiow can i avoid this exception..
View 1 Replies
View Related
Jun 12, 2015
I have got a question on max degree of parallelism and CPU cores.
If max degree of parallelism = 1, this signifies that sql will use serial execution plan (unless u change it in query level with MAXDOP hint). In serial plan, will the query use all CPU cores (say in my server I have 16 core processors)?
If in serial execution plan only one thread works, then what the other threads doing ? Idle (I may have a defined max server worker thread = 32767(by default)
Unable to create a relationship between this parameters.
View 3 Replies
View Related
Jul 2, 2015
Referencing an article regarding MAXDOP and cost threshold for parallelism from Brent Ozar's website: [URL] .....
We have a 2 physical CPUs that are 4 cores each with hyper threading enabled. When looking through the task manager, under the performance tab, I see 16 CPU threads.We have set the MAXDOP value is set at 4.
Reading further, cost threshold for parallelism setting is recommended at 50 to start with.
Our setting is at the default 5.
View 2 Replies
View Related
May 29, 2014
I am running a query on SQL 2012 Server with the Resource governer setup for my account to have Max DOP option set to 1.
The query still runs in about 1 minute and the execution plan still considers parallelism.
When I explicitly mention the OPTION (MAXDOP 1) , the query runs in 6 seconds.
How can i tell by querying DMV's whether my query is using parallelism or not?
View 2 Replies
View Related
May 21, 2007
Can I return a value in a variable from a SSIS program back to C# after the SSIS program is run from C#?
View 1 Replies
View Related
Apr 20, 2006
I know you can change the max degree parallelism server wide, but can you do it on the fly for one query? I know... trust the query processor but when I turn it off for this one sp, my query goes from 3 seconds to 0 and I got this ex-MS guy in here telling me there is a way, but he does not remember how.
I want him to simplify the sp or have his project's DBA do it, and I even offered to take a hack but.... you know.
View 2 Replies
View Related
Oct 26, 2006
I am about to build a program that will require the use of DB. In the pass I used Access as my back end, but this time I would like to use Sql Server. I just have a few questions before I begin.
1. After deploying and installing my program, do I need to have sql server installed on users system or can the DB file exist on its own and just use ADO.NET to access the file? (As I done in the pass to access my Access mdb file)
2. If I can deploy a sql file -- can it be done with Sql Server Express?
3. If it can't be done, what db program should I use?
View 3 Replies
View Related
Oct 29, 2001
Does anyone know about sqlserver's Parallelism.
a query without parallelism takes much less time as the one with parallelism, in my case it's 6 times faster without parallelism. If that's the true.
What do we need parallelism for?
Any ideas
Thanks
View 2 Replies
View Related
Jul 23, 2005
I have a function that returns a table of information aboutresidential properties. The main input is a property type anda location in grid coordinates. Because I want to get only acertain number of properties, ordered by distance from thelocation, I get the properties from a cursor ordered by distance,and stop when the number is reached. (Not really possible todetermine the distance analytically in advance.) The cursor alsoinvolves joins to a table of grid coordinates vs. postcodes (theproperties are identified mainly by postcode), and to a tablethat maps the input property type into what types to search for.Opening the cursor typically results in the creation of six toeight parallel threads, and takes approx 1 second, which is abouthalf of the total time for the function.Recently the main property table grew from 4 million to 6.5million records, and suddenly the parallelism is lost. Takingthe identical code and executing it as a script gives parallelism.Turning it into a SP that inserts into a #temp table and thenselects * from that table as the last statement also givesparallelism. But when it's in the form of a function, there isonly one thread -- and the execution time has gone from ~2 secto ~8 sec. I updated the statistics on the table, but stillno parallelism.I could turn it into a SP easily enough, but that would involvea change to the C++ program that calls it, which takes a whileto get through the pipeline. In the meantime, is there some wayto induce the optimizer to use parallelism? It used to.
View 3 Replies
View Related
Dec 15, 2006
hi,i've set 'max degree of parallelism' to 1 because some sql request hanged.Now when i connect, how can i set the parallelism to 4 for a session.Is there a command like this :'alter session set max degree of parallelism 4' ?ThanksPaul
View 6 Replies
View Related
Jul 20, 2005
If SQL Server is designed for multi processor systems, how can runninga query in parallel make such a dramatic difference to performance ?We have a reasonably simple query which brings in data from a few nonecomplex views. If we run it on our 2x2.4Ghz Xeon server it takes 6minutes plus to run. If we run this on the same server withOPTION(MAXDOP 1) at the end of the same query it takes less than asecond.Examining the execution plan, the only difference I have been able tosee is that parallelism is taking up 96% of the run time when usingtwo processors. This drops when using the one so a sort takes up thevast majority of the time for the query to run.OK, so running in parallel should mean that it's run in various partsand then 'joined up' later for performance gains, but how can it getit so wrong (timewise) ?If this is the case, will I see a significant difference changing ourserver to use a single processor, which seems completely the wrongapproach (or should I do this on each query in each app - eek) ?Do we have a problem that we don't know about that causes it to takethis long ?What can we do ? Ideally, using both processors would seem to bepreferrable.
View 2 Replies
View Related
Oct 16, 2006
Hi,
I would just like to confirm something with you guys...
Am I correct in saying that you dont need multiple connections to the same DB in a SSIS package in order to achieve parallel processing across multiple SQL tasks. In other words, I have 2 SQL tasks executing different stored procedures on the same DB that I want to run in parallel. They should be able to share one connection and still process in parallel, correct?
With that in mind, would the processing be faster if they each had their own connection?
Thanks in advance.
View 1 Replies
View Related
Mar 16, 2007
after running query at first time working all processes
but later 2-3 sec. working only one
SQL 2005
Hewlett Packard DL580 (16 processes)
What is ideas?
View 6 Replies
View Related
Sep 2, 2015
I have SQL Server Version:-
Microsoft SQL Server 2008 R2 (SP2) - 10.50.4000.0 (X64) Â Jun 28 2012 08:36:30 Â Copyright (c) Microsoft Corporation Express Edition with Advanced Services (64-bit) on Windows NT 6.1 <X64> (Build 7601: Service Pack 1) (Hypervisor)
Â
This is just an UAT server which has OS and hardware detail below:-
OS :- Windows Server 2008 R2 Standard
SP:- SP1
Processor :- Intel(R) Xeon(R) CPU Â X5650 @2.67GHz 2.66 GHz
RAM : - 4 GB
Bit - 64 bit
I want to set the value to max degree of parallelism, what value should i configure for the same?
Below is the snap property of SQL instance >> Processor
View 3 Replies
View Related
Jun 27, 2001
Hi,
My company has recently migrated our Informix database to SQL server 2K. The data is successfully migrated and the client are able to connect to the database. However, when trying to run a VB application to access the data, it keeps on prompting for SQL server login whenever an sql statement is executed. What could be the problem? We are using SQL server 2k on Windows 2000 and on the client Windows 95. Thanks!
View 2 Replies
View Related
Mar 15, 2008
Hi. First 3 paragraphs are about problem in general - this is for those who consider them well experienced and may have better ideas about how to solve this whole thing. In forth paragraph is the tehnical question.
Hi. First 3 paragraphs are about problem in general - this is for those who consider them well experienced and may have better ideas about how to solve this whole thing. In forth paragraph is the tehnical question.
Situation:
So me was trying to find out why files are saved in database incorrectly. First idea was to create a MD5 checksum of a file in my application and then create one with SQL script that saves data in database (data being the file, by the way).
On software side it worked fine 'cuz i'm developing in Visual Basic .NET 2005 which has a function that return a 32 byte array of hash from a data that you pass to it (like file) - GREAT!
SQL Server 2005 side - not so great. The only function that I have found so far that creates hash is HashBytes that is supposed to do it's job only there's a limintation to data size which means i cant make hash of say 4mb big file.
Problem:
So I figure maybe i could create a vb.net program that return a hash from a data passed to it and call it with SQL script (because hashbytes function doesnt work for me in case you didnt read the first 3 paragraphs)... but I don't know how:(
Any ideas?
View 4 Replies
View Related
Mar 15, 2008
Hi. First 3 paragraphs are about problem in general - this is for those who consider them well experienced and may have better ideas about how to solve this whole thing. In forth paragraph is the tehnical question.
Situation:
So me was trying to find out why files are saved in database incorrectly. First idea was to create a MD5 checksum of a file in my application and then create one with SQL script that saves data in database (data being the file, by the way).
On software side it worked fine 'cuz i'm developing in Visual Basic .NET 2005 which has a function that return a 32 byte array of hash from a data that you pass to it (like file) - GREAT!
SQL Server 2005 side - not so great. The only function that I have found so far that creates hash is HashBytes that is supposed to do it's job only there's a limintation to data size which means i cant make hash of say 4mb big file.
Problem:
So I figure maybe i could create a vb.net program that return a hash from a data passed to it and call it with SQL script (because hashbytes function doesnt work for me in case you didnt read the first 3 paragraphs)... but I don't know how
Any ideas?
View 14 Replies
View Related
Jul 23, 2005
We're experiencing a large number of deadlocks since we began runningSQL Server 2000 Enterprise Edition SP3 on a Dell 6650 with hyperthreading intel processors. We don't have the same problem on Dell6650's w/o the hyper threading. If I turn off the parallel queryprocessing option the deadlocks stop. I've tried all of the suggestionsfrom the Microsoft Knowledge Base under the following link -http://support.microsoft.com/?kbid=837983The only suggestion that actually yielded results was turning offparallel query processing but I don't want to give up what should be aperformance advantage if it wasn't for the deadlocks. Query tuning andindex tuning hasn't helped. Any suggestions? I haven't applied SP4yet. I'm wondering if anyone has seen the same problem resolved withSP4.*** Sent via Developersdex http://www.developersdex.com ***
View 4 Replies
View Related
Jul 23, 2005
Hi,I have a sql 2000 server with 8 processors, server settings are asdefault. I read on Technet that it is good practise to remove thehighest no. processors from being used for parallelism, correspondingto the no. of NICs in the server. One of our 3rd party developers hasrecommended only allowing one processor to be used as there is aperformance hit by the server working out which processor to use. Doesanyone have a definitive answer to this? I suspect he's wrong but I'dlike some hard evidence if possible, thanks.Kev.
View 3 Replies
View Related
Jun 16, 2006
Hi,
Is it possible to achieve partition parallelism in SSIS? What I am asking is, In DataStage, if I load some data like 'data reader -> trans1 -> trans2 -> destination' (and assume that I have 4 nodes configured), the tool divides the data into 4 different datasets and executes the package as 4 instances. This way the data load is very fast. Is it possible in SSIS?
Of course we can divide the dataset and load them thru multiple instances? But then dividing the dataset will differ for every load and so we need to modify the package all the time. Even if we divide the dataset, I am not sure 4 instances will run in 4 different nodes or in a same node? So anybody has any idea about it?
Thanks.
View 4 Replies
View Related
Jul 7, 2006
Hi,
In my package I have a source, a script component to make some changes to that and a destination. To speed up the process, within a data flow, I have created 6 copies of the above components and running them in parallel. Each source takes different set of data. I have divided the data using the record no such that, each set will read 1million records.
Now, my question is, though each pipleline is supposed to process exactly 1million records, they are not running at the same speed. For example, 1 pipeline completes processing all 1million records whereas another pipeline processed only 250000 records in that time. I don't see any reason for why one should run slow while another is running fast considering that both are doing the same thing?
Do you have any idea about this?
Thanks.
View 6 Replies
View Related
May 6, 2008
Hi,
actually a sever has a parallelism of 4 I would like to set the parallelism for a specific user to 2 without changing the code of the users application.
Is this possible.
As far as I understand with plan guides you just provide sql statements.
Need I to find all queries from the user, and add plan guides for all the queries, or tis there a more elegant way to do it.
br
fari
View 1 Replies
View Related
Mar 5, 2007
Where do I go in my computer to correct this problem so that I can complete the installation?
View 3 Replies
View Related
Feb 23, 2004
Hi,
I have a VB program to sit somethere. I like to call that program with passing two parameters from a trigger on the SQL Server. The parameters are the contents of the table which the trigger is created on.
Is there any way to achieve that?
Thanks a lot!
Regards,
Kevin Jin
View 1 Replies
View Related
Aug 1, 2000
Hi,
I am working on sql server bulk copy program. I am getting data files from
our vendors for shares and stocks. The data files are pipe separated values.
for ex the Ascii file format is
8388182|"ACC consultanats"|"rating for the current financial year"|23
My doubt is i have four columns in my sql server table named stocks.
table structure
---------------
serialno numeric
caption nvarchar(255)
memo1 ntext 16
sno int
In this the third column named memo1 in the data file would be a large
volume data. That is it may be upto one full A4 size page.
One important thing is, data in the third column is not formatted.
Since it is very urgent, let me know what would be the format file for
this type of data file and the bulk copy program utility.
Kindly let me know as early as possible.
Regards
Sivaramakrishnan
View 1 Replies
View Related
Apr 21, 2006
I found a search-and-replace program for SQL server that works GREAT, but I have a list of about 200 words to search for, with corresponding replacements. Rather than editing the code below for *each* word and running it 200 separate times, I'd like to iterate through the list, but my MS-SQL programming skills are...light (to say the least). Anyone have any ideas how I can create a list or hash-type variable and use the code to loop through and do the replacements all at once?
/*
*
* Search & Replace
*
* Use Ctrl+Shift+M to replace template values
*
*/
set xact_abort on
begin tran
declare @otxt varchar(1000)
set @otxt = '<string1, text, text to be replaced>'
declare @ntxt varchar(1000)
set @ntxt = '<string2, text, replacing text>'
declare @txtlen int
set @txtlen = len(@otxt)
declare @ptr binary(16)
declare @pos int
declare @id int
declare curs cursor local fast_forward
for
select
id,
textptr(<field_name, sysname, target text field>),
charindex(@otxt, <field_name, sysname, target text field>)-1
from
<table_name, sysname, target table>
where
<field_name, sysname, target text field>
like
'%' + @otxt +'%'
open curs
fetch next from curs into @id, @ptr, @pos
while @@fetch_status = 0
begin
print 'Text found in row id=' + cast(@id as varchar) + ' at pos=' + cast(@pos as varchar)
updatetext <table_name, sysname, target table> .<field_name, sysname, target text field> @ptr @pos @txtlen @ntxt
fetch next from curs into @id, @ptr, @pos
end
close curs
deallocate curs
commit tran
View 1 Replies
View Related
Sep 25, 2007
Hi,
I've written a piece of database software in Visual Basic (Visual Studio 2005) to use in my office. My office has networked computers all connected to a server (windows server 2003). I have an old-ish version of SQL server on the server computer (version 7). The software works perfectly within the office environment - each computer on the network connects to the SQL server on the server computer and works very efficiently.
I now want to be able to access the database from home by running the software on my home PC. I want the software to somehow connect to SQL server in the office over the internet. Having read many of the threads about remote SQL server connections, I feel totally out of my depth. I vaguely understand that SQL server needs to be somehow configured to receive connections from the 'outside world'. I don't know how to do this. I also don't know what I need to do on the Windows server to allow this to work. I thought I might be able to use the computer's IP address in the connection string, but using IPconfig on the server I only get network IP addresses - the server may be behind a router... How do I get the proper internet address? Once I've set up the server side to work with remote connections, how do I get my software to communicate with that server? Do I just need to change the connection string and that's it? What would the format of the connection string be?
I'm very sorry for asking so many questions in one go! I have ended up a bit frustrated after reading many threads and articles about SQL remote connections - it's all rather advanced and don't understand most of it!
Thank you very much in advance for any help you can provide.
Sam
View 1 Replies
View Related
Jun 15, 2006
I have a vb6 program running fine on a domain on SBS 2003 with the backend connected to sql express. I have added a third computer to the domain and I am unable to run the program on the third computer. I am almost certain that it is not a user, login, or role issue. I would appreciate ideas on how to localize this problem.
Thank you
View 5 Replies
View Related