Performance Problem, Lots Of Disk Activity, Running Out Of Memory
Jul 20, 2005
Fellas!!
This is a very complicated one and it took me a few days to figure out
exactly what's going on, but here's the final story:
I have a production environment running on .NET with a SQL Server
(2000, SP3). The SQL Server is on a dedicated Proliant computer with
2GB RAM (the actual SQLServer.exe process has dynamic memory
assignment and can reach up to 1.6GB RAM). Nothing else is running on
that specific computer.
Once the SQLServer is started, it hits 300MB RAM (the minimum that was
set in the configuration of the server - remember, it is dynamically
aquired).
Then there is a .NET program that requests just about all the data the
SQL Server contains (apart from a single table that contains roughly
1.6 million rows and another table that contains about 10000 rows
which are all of type IMAGE).
Once all the data is retrieved, the RAM is at about 400MB. From there
on, every update I make to the data on the server causes the RAM to go
up by a bit (that updates are done in a Transaction which of course is
committed at the end). It seems that BLOB updates are the major
problem in all of this. For some reason, uploading a blob of size 9MB
causes the RAM to go up by roughly 20MB and after commit it gose down
10MB (total gain of roughly 10MB RAM). Eventually the SQLServer
process hits its upper limit (1.6GB) and at this point it starts
slowing down.
Some performance checks showed me the SQLServer has a lot of disk
activity, it seems it is reading and writing pages of data from/to the
HD all the time (which causes the queries to be much much much
slower).
We have a development environment running the exact same code (it is
the exact same in everything, except for the amount of data stored in
the DB). This does not happen there at all.
I have a few questions:
1. Why is the RAM going up after BLOB updates?
2. Why is the RAM going up at all?
3. How can I tell the DB which tables should remain in the RAM at all
time (never swapped back to the HD?) - DBCC PINTABLE does not seem to
do the job.
It does not seem to have anything to do with the .NET code.
Thank you very much,
M Yamo.
View 4 Replies
ADVERTISEMENT
Jan 30, 2007
Hi all,
I have a strange situation. Performance monitor shows that SQLServer:Transactions Transactions value is 125, but SQL Server Profiler does not show any activity.
I ran sp_who2 and I have a bunch of processes with SUSPENDED status. Would those be counted in Performance monitor?
When does a transaction get suspended?
Thanks.
Alec
View 1 Replies
View Related
Apr 19, 2004
Hello All,
I am trying to perform a performance audit on a SQL Server where I am positive my most likely bottle neck is the disks. I assume the primary metrics I should be interested in are %Disk Activity and Disk Queue Length, right?
What are the actually guidelines for these metrics? Also, Are there other perf mons I should be running against the disks?
Thanks In Advance.
Ryan Hunt
View 2 Replies
View Related
May 25, 2004
I am planning an application where ~1000 companies will be accessing data. Should I use a key to identify the company and place all data in one table i.e (WHERE company =123) or should the application create company specific tables i.e should I have 1000 small tables with 100 records in each, or one table with 100,000 records?
View 2 Replies
View Related
Mar 28, 2001
What to check on NT or Sql server it self
if
1. NT performance shows that sql serve.exe -CPU 93% memory 49000
2. Sql server current activity only sa connected to master.
Thanks
View 2 Replies
View Related
Oct 29, 2007
i'd like to block all other processes that interact with my database while my archive process is running on that database. Can this be done?
View 4 Replies
View Related
Apr 14, 2015
Is there anywhere in the SSMS to see the start time of a currently running job, currently running it's first step, without needing to run a query?
If not, why would Microsoft decide to not show it in the job activity monitor?
SSMS = Sql Server Management Studio
View 1 Replies
View Related
May 16, 2007
Please forgive the simplicity of this question - I am not the dba type. When I connect to a server and look at my connection attributes in activity monitor, the user column shows the correct information for my domainusername. When I run a certain stored procedure in that connection, the domainusername changes to another person. We are not using execute as, setuser, or anything special to explicitly change the user. The stored procedure is in a schema that is owned by dbo (principal_id = 1 - I verified by checking sys.database_principals.)
Does anyone have any suggestions?
Thanks in advance,
John Hennesey
View 4 Replies
View Related
Aug 10, 2007
***cross-posted to MSDN SQL Server forum***
Hi All. Hopefully I posted this to the right section.
I would like help identifying if I have a disk performance issue or not. First the background: we have a j2ee application using the MS SQL 2005 JDBC driver and Hibernate on 4 application servers, and an active-passive SQL Server 2005 cluster. All of the servers reside in the same physical rack and switch.
Our application is typically bounded by CPU on the app server, or throughput from the database. Several months ago we were using SQL 2000 and would often max out the CPU on the database server before anything else, but often the database could keep up and we would max out the app servers CPU. Now we have 2005 on a much more powerful machine and more app servers, but we seem to be running up against a problem with throughput from the database.
The issue is not CPU. The total cpu average on the database server, as monitored in perfmon on 30 second intervals, stays consistently below 40%. The app servers stay well below 30%. But what concerns me is the Average Disk Queue Read Length on the database server, particularly for our E: drive. On this db server, the transaction log, the system and temp dbs, and our application's database are all on separate EMC SAN shares, connected via fibre chanel. The E drive houses the app data and is a 15-way meta device (fifteen 10GB logical devices striped at 960k for a 150GB device) in a RAID-S configuration, EMC Symmetrix array located in the same rack. The database is roughly 30GB.
I have read various articles online describing how to interpret the Average Disk Queue Read Length performance counter with regards to SQL Server. Some have said this should not exceed the number of physical spindles * 2. We are seeing values of 32 consistently, averaging over 60 during peak processing hours, and spiking to well over 100 on a scale of 1.0. (3-second sample interval).
So since our application servers seem to be waiting on their database calls (a lot of inserts with frequent, but small-resultset selects) and do not show I/O issues either with their local storage, memory, or network interface. The database server again has no CPU, network, or memory issues. I should add the the Average Disk Queue Write Length counter does not have any issues; its always below 1 (on a 1.0 scale). The EMC array has both read and write caching. The indexes of the application database are rebuilt weekly and defragmented every day, with stats rebuilt after the defrag.
So how can I further determine where my performance problem lies? All thoughts appreciated! Thanks!
-tuj
View 2 Replies
View Related
Oct 25, 2007
I am running SQL server 2000 SP4 on a server with 2 Dual core 4G processors with data attached via a SAN>
I have a 70G database with 10 users that is giving attrocious performance. I have just tried to run a count(*) accross a couple of tables and am still waiting for the results 15 mins later. When I look at the disk queue it is around 50/60. I thought the target for this was around 2. I am sure that the hardware that we have in place is capable of running this db. However I`m not sure how to fully analyse what is going wrong here.
Any tips would be greatfully received.
Si
View 2 Replies
View Related
Aug 9, 2007
Hi All:
I would like help identifying if I have a disk performance issue or not. First the background: we have a j2ee application using the MS SQL 2005 JDBC driver and Hibernate on 4 application servers, and an active-passive SQL Server 2005 cluster. All of the servers reside in the same physical rack and switch.
Our application is typically bounded by CPU on the app server, or throughput from the database. Several months ago we were using SQL 2000 and would often max out the CPU on the database server before anything else, but often the database could keep up. Now we have 2005 on a much more powerful machine and more app servers, but we seem to be running up against a problem with throughput from the database.
The issue is not CPU. The total cpu average, as monitored in perfmon on 30 second intervals, stays consistently below 40%. But what concerns me is the Average Disk Queue Read Length, particularly for our E: drive. On this machine, the transaction log, the system and temp dbs, and our application's database are all on separate EMC partitions, connected via fibre chanel. The E drive houses the app data and is a 15-way meta device (fifteen 10GB logical devices striped at 960k for a 150GB device) in a RAID-S configuration.
I have read various articles online describing how to interpet the Average Disk Queue Read Length performance counter with regards to SQL Server. Some have said this should not exceed the number of physical spindles * 2. We are seeing values of 32 consistently, averaging over 60 during peak processing hours, and spiking to well over 100 on a scale of 1.0. (3-second sample interval).
So since our application servers seem to be waiting on their database calls (a lot of inserts with frequent, but small-resultset selects) and do not show I/O issues either with their local storage, memory, or network interface. The database server again has no CPU, network, or memory issues. I should add the the Average Disk Queue Write Length counter does not have any issues; its always below 1 (on a 1.0 scale). The EMC Celerra array has both read and write caching. The indexes of the application database are rebuilt weekly and defragmented every day, with stats rebuilt after the defrag.
So how can I further determine where my performance problem lies? All thoughts appreciated! Thanks!
-tuj
View 9 Replies
View Related
Jun 22, 2004
We've got an internal database that replicates with another database server for our website.
Not all tables are replicated, some use merge and the others are snapshot based and published regularly to the public website facing server.
However, there's a lot of data (well, large textual data) that's being transferred and it seems to be generating massive log files that continue to grow and grow.
I'm fairly new to adminning an SQL Server box, so was wondering if anyone can tell me what the best way to keep it under control is? I've heard its possible to truncate the logs, effectively deleting any data that has already been processed by subscribing servers etc.?
As I said, I'm very much new to this and would really appreciate some guidance, if only to the right part of the SQL Server Books Online :)
Thanks,
Paul
View 2 Replies
View Related
May 7, 2015
In SQL Server 2014, how big for the block size is better for performance? 64 KB? 4 KB?
For normal database files, best practise is 64 KB disk block size. Not sure if it is same for memory-optimized filegroup.
View 12 Replies
View Related
Aug 4, 2015
How to fix this SQL error event logg 17204 and 17207 ?
View 8 Replies
View Related
Aug 2, 2006
Hi
I did a load testing and found the following observations:
1. The Memory:Pages/sec was crossing the limit beyond 20.
2. The Target Server Memory was always greater than Total Server Memory
Seeing the above data it seems to be memory pressure. But I found that AvailableMemory was always above 200 MB. Also Buffer Cache HitRatio was close to 99.99. What could be the reason for the above behavior?
View 1 Replies
View Related
Jun 22, 2015
We're having some issues with where our backups write to, so I've been watching and monitoring the performance, when I noticed today that restore labelonly from disk has been running almost non stop for the past few hours.
The account running the query is the SQL Server's service account, and the program is "Microsoft SQL Server".
Every minute or so the SPID changes which made me think it was related to the transaction logs, the "restore labelonly" runs for as long as each database in the transaction log backup.
Example: Database A transaction log backup takes 1 minute and the SPID XX for restore labelonly runs 1 minute
Database B transaction log backup starts and there is a new SPID for restore labelonly.
I hope this makes sense because I normally don't see this restore labelonly running.
View 2 Replies
View Related
Oct 3, 2006
Hi all,
I'm running out of disk space when running SSIS package. Is there any way to select where temp files are saved during package execution ?
View 8 Replies
View Related
Mar 2, 2004
Have a sql 2000 db which I have no say in design, just make it run. My typical sql counters such as system queue, and buffer cache and cache hit ratio are all good. If I need to monitor disk activity (mainly how fast my data is being read, how long the user is waiting for that data for both reads and inserts), what are the best counters for this, and what value should throw up a red flag.
View 1 Replies
View Related
Feb 11, 2006
We have an application that is experiencing I/O contention,particularly in tempdb but also in two other databases. The data isstored on mirrored PowerVault 220's, each with 10 of 14 possible disks.The PowerVaults are JBOD devices, not true SANs. The current config hasfour separate groups of physical drives assigned to distinct logicaldrives for log files, tempdb, and the two app dbs. This means, forexample, that tempdb resides on one mirrored drive. The standard advicewhen faced with disk contention is to add spindles if possible. With 4empty slots, we would presumably assign the new physical disks to themost stressed db, e.g. tempdb.An alternative arrangement would be to combine all the physical drivesinto one logical drive, and put all the files, log and data, onto thesingle logical drive. The hope for this configuration is that thePowerVault would automagically distribute the data among the drivessuch that all drives were in use, all spindles reading and writing atmaximum capacity when necessary. It is my understanding thatfull-featured SANs, like NetApps and EMC models, do this. My questionis whether this configuration is best for the PowerVault, as well. Oris this the essential difference between JBOD and a true SAN?Has anyone tried both arrangements?Advice is much appreciated.
View 2 Replies
View Related
Jul 20, 2005
I 'inherited' a group of SQL Server server class machines. They aretrue server technology but the disk sub-systems are lacking. There isone hot-swap backplane that all the drives share (with one SCSIchannel) thusly even though there are three logical drives (composedfrom 6 to 8 hard drives), they all go through one channel. This iscreating a performance issue that is noticable and can be seen invarious performance counters that Microsoft recommended one shouldmonitor in terms of disk I/O. For a cheaper 'fix', I can add aseperate two drive bay (with its own SCSI channel) with mirroreddrives. I would then mostly likely place the transaction log files onthis new channel. Or I could place the indices filegroup files onthis new channel for DBs with mainly searching going on (not muchupdating). If I went this route I would be leaning towards thetransaction log move since the second method would require me movingDBs around quite a bit. Any input on this solution (besides spendingmore money)?What I would prefer to do is get a better server class machine or addan external drive bay solution (not a SAN). I would try to get threeor four SCSI channels in the new hardware to split the differentfile/filegroups out (i.e. transaction logs files, data filegroup,indices filegroup, etc.). My only concern here is: would this moreexpensive solution be worth the money? As far as replacing servers, Ihave only two kinds of experience...replacing somewhat underpoweredservers with slightly less underpowered servers and replacing overkillservers with even more overkill servers. In both cases, the disksub-systems were fairly equivalent from the old system to the new one.Will going the three/four channel route really get data moving along?We have one server in particular that hosts a database (one of many onit) for a web application that gets decent traffic (it is a privatelogin based system for internal use and external use by our clients'agents). Periodically throughout the day, there are 2-5 minute burstswhere performance slows to a crawl. I want to spend more timeprofiling queries and such before recommending we spend more money,but the folks I am working for want quick results and there is quite abit of stored procedure logic to profile and investigate. I know thedisk sub-system is definately in need of an overhaul, but I would liketo get an idea of peformance gains from adding either one additionalchannel over the existing single channel as well as going thethree/four channel route over the existing single channel setup.Any information would be greatly appreciated.Regards,Tony
View 2 Replies
View Related
May 14, 2007
Summary: Started replication April 1 of 4M xact / day publishing system to subscribing system.
Performance was good. Latency was ~ 5-7 seconds.
May 10 we noticed that the DB was behind (latency was 12 hours).
All performance counters seem good with the exception of the disk.
. Performance spikes are 8 minutes apart and last from 30 - 60 seconds.
. During this period, Disk % Busy (1 - Disk % Idle) is 100%
The publisher DB publishes about 50-52 xacts/sec.
Rate of distribution (distribution DB to Subscriber DB) is ~ 47 xacts / second, so latency is increasing (currently at 33 hours). Previously my Subscriber system's "capacity" was 150 xacts / sec.
I know this because several weeks ago, the network went down, we were 24 hours behind.
When the network came back up the replication subscriber system was able to catchup at around 150 xacts / sec, or 3X the production system rate.
What has changed between then and now? Not much. We did install Tivoli Service Manager (IBM's backup system) a couple of weeks ago. It seems to run fine on a nightly basis, but I don't see any periodic heavy Disk I/O from that. Just to be sure, I've had them shut the TSM services down just to be sure.
We've also eliminated all extraneous processes other than those I need for performance monitoring (there was a RTVscan, virus scan process).
I've eliminated Autogrowth's as an issue as I've bumped the growth so that they are very infrequent (several days at this point. When we resolve the problem, I'll dial this down to something more reasonable.
My disk configuration is not ideal I realize (single Raid-5 disk with 3 partitions), however, this has not changed in the 6 weeks.
Thanks for any help on this!
Jack Griffith
Configuration:
Subscribing System:
SQL Server: 2000, SP4 - 8.0.2039
CPU - 2.8GHZ Xeon, Quad Dual-core
Memory - 3.5GB RAM
Disk: 3 partitions on a single RAID-5 disk with 1118 GB of space:
C: 39GB System and Programs
D: 97GB Log space
E: 982 GB Data space
Replication configuration:
- nosynch, continuous Transactional Replication
- Distribution db is on Subscription system
- distribution - Publication of approx. 50 transactions / second
Subscriber DB configuration:
DB size: 64458 MB
Logging: Simple (at this point)
distribution
DB size: 3111 MB
Logging: Simple (at this point)
View 1 Replies
View Related
Aug 19, 2015
If I'm on a remote machine, meaning a computer not in the WSFC cluster, and I open SSMS 2014, point it to a SQL Instance, and open activity monitor:
1. I get all the panes and charts except % Processor Time.
2. Then, if I authenticate to the cluster's domain by mapping a drive with valid domain credentials, I'm free to put performance counters in the Perfmon - - - but SQL Activity Monitor shuts down with“The Activity Monitor is unable to execute queries against server SQL-V01INSTANCE1..Activity monitor for this instance will be placed into a paused state.Use the context menu in the overview pane to resume the activity monitor.
Additional information: Access is denied. (Exception from HRESULT: 0x80070005 (E_ACCESSDENIED))(Mscorlib)”
3. Of course, the Activity monitor can't be resumed via the context menu. Removing counters and closing the perfmon do not work. I dropped the mapped drive and rebooted the machine. That brought back 95% of the information in the Activity monitor.
4. Further experimentation showed that any mapping of drive shares present on the SQL Server to the computer running SSMS cut off functionality of the 'overview' pane in the remote machine's SQL Activity monitor -- the monitor that had been trying to watch the server offering the shares.
View 4 Replies
View Related
May 29, 2008
Depending on the way I write a query, I come up with these 2 stats.
Is there a sure winner in this race, keeping in mind the overall health of the server?
(I'm not sure of the specs of the server, as I can't log on to it :/ but are there any sql variables that would show cpu speed and # of cpus?)
I almost am leaning towards the single cpu query because of lower resources used -
or are most of the "reads" in the parallel'd query not read directly from the HD, but using the Table Spool created internally (query plan shows it)?
CPU Reads Writes Duration
Parallel: 200k 3.2m 2400 62s
Solo: 79k 1.1m 600 79s
View 9 Replies
View Related
Jul 23, 2005
Hi-We've got an ASP.Net web app that runs off a Microsoft SQL Server2000 backend. After a few days the SQL server is completely out ofmemory and crawls. It looks like there could be some connections thataren't being closed or something. Is there a good way to figure outwhere the problem is. Looking at the current activity in EnterpriseManager there are a lot of threads sleeping and a few that arerunnable. Any ideas?Thanks
View 1 Replies
View Related
Dec 28, 2006
hello,all
I am new to Sql 2000,I installed sql 2000 database in C disk,but Now I found my C disk space is smaller than before,So I want to move my databse(include data and structure) from C Disk to D Disk(its space is very large) .
is it possible to do it ?
if its can be done ,do I need to change my asp.net program source code (exp: chaneg my crystal report connectstring ) ?
thanks in advanced!
View 1 Replies
View Related
Dec 4, 2007
Hello
We have a problem with MS SQL 2005 Standard on a Windows 2003 x64 Box.
Server MS 2003 Server x64 R2
Quad Core 2.13GHz
Memory: 12 GB
MS Sql 2005 Standard, Sp2
The Sql process uses only 80MB of RAM (from 8 GB) so this machine is very slow. We set the min and max Memory in SQL without success. SQL ist very slow. Has anyone here some hint's to solve this problem? Thank'.
rainbow1
View 5 Replies
View Related
Oct 31, 1999
Just two questions actually. We have built a ColdFusion based forums package. Currently we have it on two beta sites, and we are usin sql 7 for the db. Firstly the forums are serving 200-300 people at any given time, about 16,000 unique people a day. SQL & seems to stay at around 50% cpu usage on a dual p3 with 512mb ram. Is that normal? Seems like alot of cpu usage. The other thing is it takes 500mb of ram and just about drains the server of all of its ram even though in the memory properties for the sql server its set to 255mb maximum. Any insight is appreciated.
View 2 Replies
View Related
Mar 17, 2008
Hi,
I hope somebody here can help on my problem.
I wrote a MFC application using sql ce and is running prefectly smooth on wince and PPC 2003.
But when i deploy it to Windows Mobile, the database performance drop dramatically.
I knows the drop on performance is due to the I/O speed on the flash memory.(The previous mobile os are using ram instead).
Is there any solution or work around which I can solve this problem.
Recently i solved the performance issues on "insert" to DB by using a commit buffer(Instead of commit to the sdf instantly).
But how about the "select" performance? It's too slow which take about 3 sec to select a record from the DB.
Does Microsoft provide any suggestion on the sqlce perofrmance on windows mobile?
Thanks
Keith
View 3 Replies
View Related
Jan 10, 2008
Hi Experts,
here we are having a very very serious problem,
in SQL 2005, we are having 8 CPU's and in the task manager CPU usage is showing 100% and the performance of server is very very poor and last night the server has got rebooted, still CPU Usage is showing 100%, So how can i improve the performance? it is very very urgent for me, can any one please let me know what is going on and what is the solution.
-Thanks N Regards,
Kanthi.
View 3 Replies
View Related
Sep 24, 2007
Hi -
I am facing 2 problems :
PROBLEM 1 :
We have a few packages that run pretty fast on a desktop server with 2 Gig RAM, Dual processor (approx 4-5 hours). But the same packages run very very slow on the another server containing 8 CPU and 12 Gig RAM (ran for 24 hours without completing).
PROBLEM 2 :
The CPU% ranges from 40-80% and the PF usage is stagnant at 2GB on desktop server for the same package. But in the 8CPU server, the CPU % ranges from 0-10% but the PF Usage raises from 750 MB to 8 GB.
This has become critical to our application.
TIA,
Shabs
View 4 Replies
View Related
May 31, 2007
Hi All,
I'm new at SQL Server, I decided to use it in order to get all the advantages of using Vb.net and Sql. My server is a SQL standard version. I'm using a relational DB most of the time for complex select queries, everytime the server is used it performs 30 or 40 queries at the time, and I have recently realized that server consumes a lot of memory after one or two days of beign up.
Let's say that if I restart the SQL Server memory usage is about 650 Mb, but after two days memory is 1.4Mb, I have used Sql Profiler and Tunning Assistant where it recommended me to create some indexes, which indeed I created them, but that did not solved the memory problem, although some queries run faster.
My questions are:
is this memory usage is normal ?
if not, what should I check out to reduce memory usage?
Thanks in advance
George
View 12 Replies
View Related
Oct 5, 2006
Hi,
I know the SSIS memory problem has probably been covered quite a bit, but being a newbie in using the SSIS, I'm not quite sure how to improved the performance of my SSIS package.
Basically, I've a package that loops through all the subdirectories within a specified directory, and it then loops through each file in the subdirectory and with the use of the Data Flow, process each file (according to their filenames) with a Script Component to insert data into a SQL DB.
Each subdirectory has up to 15 different csv files, but each is less than 5kB. I probably have about 100 subdirectories.
When I run the package, it functioned properly, but the package stalled (no error but just stuck in one Data Flow) after a while, and when I checked my CPU memory, it was running at 100%.
I'm not sure how I could fix it or improved the memory allocation. I was not expecting to have any memory problems as the file size is small and the number of rows of data going into and out of the Script Component is no more than 20.
Any advice? Thanks.
View 1 Replies
View Related
Jul 23, 2005
Hi,i am having a strange problem running memory intensive queries on SQLserver.I am doing an update on a table with 9 million records from anothertablewith 50 records.the query i am running isupdate table1set var1 = b.var2from table2 bwhere key1=b.key1this query hanges for ever. I had thought that there was a problem withmy machine...but once out of the blue it ran in 16 minutes.I am running a 1 Ghz PIII with 512 MB of memory.Any ideas as to what could be the issue ?RegardsRishi
View 11 Replies
View Related
