Is There A SQL Server API To Flush All Buffers To Disk?
Jul 20, 2005
I am looking for an API to flush all data in memory held by SQL Server
to disk. Also, is there a tool for SQL Server like eseutil for
Exchange that lets you correct a SQL database?
A long time ago I posted this: http://blogs.conchango.com/jamiethomson/archive/2005/06/09/1583.aspx explaining all the different buffer types in the pipeline.
I have to admit I'm still not clear on the difference between a private buffer and a flat buffer though.
Are flat buffers a subset of private buffers. If so, if a private buffer is not a flat buffer - what is it? If not, can a buffer be a private buffer AND a flat buffer?
Some descriptions from BOL are:
Private buffers: A private buffer is a buffer that a transformation uses for temporary work only Flat buffers: Flat buffers are blocks of memory that a component uses to store data
That sounds like two ways of saying the same thing to me! It certainly doesn't distinguish them anyway!
Just seeking some clarification here. If you could whip of a demo package that explains the difference between the two (with reference to the Performance Counters) then that would be great.
While waiting for the fax of instructions to contact MS Support, I thought I would post here (tried several times and no fax...)
We get this message in the log file and then all hell breaks loose until it resets memory. The SQL Service continues working but nobody can connect for about 5 minutes and then is seems to reset itself. This has happened three times over the past two weeks. Only one time it did create the SQLDUMP files but all three occurences have practically the same entries.
We are running SQL Server 2005 x64 SP2 under Windows 2003 x64 SP1. We have 4GB RAM and SQL is configured to use 2GB of it. We have a large number of databases (about 400) on this one instance that experiences this problem. The server itself is not under a tremendous load. All of the databases were recently upgraded from SQL 2000 SP4 32 bit instance. The first occurence happened just days after the migration.
----- Log Entries -----
LazyWriter: warning, no free buffers found.
2007-06-14 14:15:56.18 spid3s Memory Manager VM Reserved = 4415288 KB VM Committed = 4398048 KB AWE Allocated = 0 KB Reserved Memory = 1024 KB Reserved Memory In Use = 0 KB
2007-06-14 14:39:56.82 Server Resource Monitor (0x1180) Worker 0x000000008000C1C0 appears to be non-yielding on Node 0. Memory freed: 148160 KB. Approx CPU Used: kernel 125 ms, user 62 ms, Interval: 65000. 2007-06-14 14:40:56.84 Server Resource Monitor (0x1180) Worker 0x000000008000C1C0 appears to be non-yielding on Node 0. Memory freed: 218536 KB. Approx CPU Used: kernel 328 ms, user 93 ms, Interval: 125046. 2007-06-14 14:41:56.84 Server Resource Monitor (0x1180) Worker 0x000000008000C1C0 appears to be non-yielding on Node 0. Memory freed: 288960 KB. Approx CPU Used: kernel 515 ms, user 125 ms, Interval: 185046. 2007-06-14 14:42:56.84 Server Resource Monitor (0x1180) Worker 0x000000008000C1C0 appears to be non-yielding on Node 0. Memory freed: 366008 KB. Approx CPU Used: kernel 718 ms, user 171 ms, Interval: 245046. 2007-06-14 14:43:56.84 Server Resource Monitor (0x1180) Worker 0x000000008000C1C0 appears to be non-yielding on Node 0. Memory freed: 435992 KB. Approx CPU Used: kernel 968 ms, user 296 ms, Interval: 305046. 2007-06-14 14:44:56.84 Server Resource Monitor (0x1180) Worker 0x000000008000C1C0 appears to be non-yielding on Node 0. Memory freed: 505160 KB. Approx CPU Used: kernel 1203 ms, user 390 ms, Interval: 365046. 2007-06-14 14:45:56.84 Server Resource Monitor (0x1180) Worker 0x000000008000C1C0 appears to be non-yielding on Node 0. Memory freed: 572488 KB. Approx CPU Used: kernel 1468 ms, user 468 ms, Interval: 425046. 2007-06-14 14:46:56.84 Server Resource Monitor (0x1180) Worker 0x000000008000C1C0 appears to be non-yielding on Node 0. Memory freed: 639056 KB. Approx CPU Used: kernel 1703 ms, user 500 ms, Interval: 485046.
hello,all I am new to Sql 2000,I installed sql 2000 database in C disk,but Now I found my C disk space is smaller than before,So I want to move my databse(include data and structure) from C Disk to D Disk(its space is very large) . is it possible to do it ? if its can be done ,do I need to change my asp.net program source code (exp: chaneg my crystal report connectstring ) ? thanks in advanced!
I have a three tier system using SQL server 2000, we are currently experiencing IO bottle necks on our SCSI Raid 10 array, which holds the Data and the logs in separate partitions.
So my options as I understand it are:
Get Enterprise edition
or
Get another physical raid 10 array and separate the logs and data i.e. data on one array and logs on the other array.
I would like to try the latter but I am totally unsure how much difference this will make or whether it will make any difference at all.
Does anyone know how much performance increase I will get from using two arrays as opposed to one?
Any other advice on this scenario would be greatly appreciated.
Someone told me they closed there window and lost their query and wanted to know if there was some way to get it back from a buffer or something. Does anyone know if that is possible? I was not aware of anything like that, but the way that SQL caches and buffers things I thought it might be possible extract a query from a buffer or something...?
Occasionally on my SQL 2K SP3 Standard servers. The servers lock up withthe error.LazyWriter: warning, no free buffers found.After that I have to restart the SQLServer service to get things up andrunning. How can I prevent this from happening?Thanks,Ray
I have a long running (48 hour+) stored proc. I've added PRINT statements that print "1%", "2%", etc to provide progress so I can get a ballpark idea of how far along the process is. I ran the stored proc and realized that this won't work. I don't see the output of the print statements until the stored proc is completed. I'm running this from Query Analyzer (SQL Server 2000 SP3a).
For example, the following will wait ten seconds and then print both statements; it doesn't print one then wait, then print the other. Is there any flush command to make the print statements take effect immediately?
SQL Server has encountered 1 occurrence(s) of cachestore flush for the 'Object Plans' cachestore (part of plan cache) due to some database maintenance or reconfigure operations.
I want to trace the user logins by using a stored procedure. This script (sp_login_trace) is created by the SQL Profiler tool. (Once this procedure works well, I will use sp_procoption to run it automatically everytime the SQL Server startup.)
After I successfully created sp_login_trace, I run it (exec sp_login_trace). The trace process is started and TraceID is 1. (I use select * from ::fn_trace_getinfo(default) to verify it). However the file size of login_trace.trc is always 0 even after I use Query Ananlysis or Eneterprise manager to let some users to login into the SQL Server instance. (when I use SQL Profiler to start a trace, the trace file size will increase along with users continaully login in). At that time if I use SQL Profiler to open the login_trace.trc file, the system will give me an error message: No data since Empty File.
After I stop and delete the trace process, I find that the file size of login_trace.trc becomes 128K and I can see the login records caught by sp_login_trace if I use SQL Profiler to open this file again.
How can I flush the buffer to trc file frequently without need of stopping trace process?
Could anybody help me on how to do buffer flush before program exit? I am using OLE DB to insert record to SQL CE database, but cannot control data commit, each time if I exit too quick, then all of inserted record cannot stored into database .sdf file. Thanks a lot!
If I return the Average, Minimum, and Maximum values for the counter Physical Disk: Avg. Disk Queue Length, and those values are 10, 0, 87 respectively, which value do I use to compute the Avg. Disk Queue Length for a 4 disk array(RAID 10): Average, Minimum, or Maximum? The disk(lun) is on a SAN.
I have detached a SQL Server 2005 database from one server and attached it to another SQL Server 2005 and I now get the following in the error Log and in the event viewer evry 10 - 20 minutes or so.
SQL Server has encountered 1 occurrence(s) of cachestore flush for the 'Object Plans' cachestore (part of plan cache) due to some database maintenance or reconfigure operations.
2007-03-16 12:37:14.64 spid17s SQL Server has encountered 1 occurrence(s) of cachestore flush for the 'SQL Plans' cachestore (part of plan cache) due to some database maintenance or reconfigure operations.
2007-03-16 12:37:14.64 spid17s SQL Server has encountered 1 occurrence(s) of cachestore flush for the 'Bound Trees' cachestore (part of plan cache) due to some database maintenance or reconfigure operations.
Starting up database 'DBName'
It appears under different SPID's 18, 20, 24 ...... and consistently has the 4 errors in a row Full Text indexing is running for both servers but I dont know if this is the cause of the error.
I would greatly appreciate any help to get rid of this as I have trawled the net and not found anything of use.
-- Initialize Control Mechanism DECLARE@Drive TINYINT, @SQL VARCHAR(100)
SET@Drive = 97
-- Setup Staging Area DECLARE@Drives TABLE ( Drive CHAR(1), Info VARCHAR(80) )
WHILE @Drive <= 122 BEGIN SET@SQL = 'EXEC XP_CMDSHELL ''fsutil volume diskfree ' + CHAR(@Drive) + ':'''
INSERT@Drives ( Info ) EXEC(@SQL)
UPDATE@Drives SETDrive = CHAR(@Drive) WHEREDrive IS NULL
SET@Drive = @Drive + 1 END
-- Show the expected output SELECTDrive, SUM(CASE WHEN Info LIKE 'Total # of bytes : %' THEN CAST(REPLACE(SUBSTRING(Info, 32, 48), CHAR(13), '') AS BIGINT) ELSE CAST(0 AS BIGINT) END) AS TotalBytes, SUM(CASE WHEN Info LIKE 'Total # of free bytes : %' THEN CAST(REPLACE(SUBSTRING(Info, 32, 48), CHAR(13), '') AS BIGINT) ELSE CAST(0 AS BIGINT) END) AS FreeBytes, SUM(CASE WHEN Info LIKE 'Total # of avail free bytes : %' THEN CAST(REPLACE(SUBSTRING(Info, 32, 48), CHAR(13), '') AS BIGINT) ELSE CAST(0 AS BIGINT) END) AS AvailFreeBytes FROM( SELECTDrive, Info FROM@Drives WHEREInfo LIKE 'Total # of %' ) AS d GROUP BYDrive ORDER BYDrive
I am trying to setup a test cluster and am having an issue. When I try to create the resource of a physical disk it takes both the drive e: and drive q: and doesn't seperate them into two physical disks as resources. This means when I try to associate the quorum disk it links the to physcial disk resource of drive e and q. Then when I try to install SQL2k5 I get the warning about installing SQL on the quorum disk. Am I missing something? Is there a way to seperate e and q onto two physical disk resources so I can specifically associate the quorum to q and the sql to e or should I be setting the quorum disk to a majority node set? Thanks in advance.
I have corrected the problem with named pipes versus TCP/IP but I have found a reference to lrustats stating that page flushes should be less than 100 and free avg less than 10 but my numbers are extremely high. One of the things I noticed upon assuming this job was that many settings were off a bit bit LogLRU buffers were never set.This setting was introduced with service pack 2 with a readme.txt on how to configure. All my references and the ones that I have skimmed through in bookstores do not contain this configuration setting.
Microsoft no longer offers SP2 for download so that I could get the readme.txt. If any of you have this file or service pack laying around, please email it to me so that I can configure. It is suppose to take care of some bufwait errors. I even download the French version to see if it was there and maybe I could find someone with a french background but it did not contain the file. It was in last years Technet CD but my company tossed it when they got the new one. Can anyone help?
I'm working on a stored procedure, and I'm nearing completion. The procedure takes a while to run (it's a very large dataset), but that's not my issue. The problem is that I include several print statements, but I don't get to see any of them until the entire procedure has finished. Is there anything I can do to be able to see them as they are printed? Using sql server 2000 with management studio for 2005.
hi We have use SQL Server 2005 on 64bit windows 2003 server Cluster. SQL instance stop responding for some leaving application interrupted. In the Sql Err log I noticed the following information memory errors just before cluster issued stop command to SQlsrvres. Lazywriter gave warning that no free buffers found .
Anyone seen this SSIS error when importing data? I have a 64bit quad processor with 8gb and am importing from Oracle 9 using 32bit DTExec.exe from the command line.
OnInformation,Myserver,MyDomainSQLAdmin,J001OracleDimExtract,{CEB7F874-7488-4DB2-87B9-28FC26E1EF9F},{1221B6EB-D90A-466E-9444-BA05DBC6AFD8},6/29/2006 10:58:08 AM,6/29/2006 10:58:08 AM,1074036748,0x,The buffer manager detected that the system was low on virtual memory, but was unable to swap out any buffers. 2 buffers were considered and 2 were locked. Either not enough memory is available to the pipeline because not enough is installed, other processes are using it, or too many buffers are locked.
I am working on my own data flow source component. Here is a fragment of this component code:
public override void PrimeOutput(int outputs, int[] outputIDs, PipelineBuffer[] buffers) { PipelineBuffer selectedBuffer = buffers[0]; string message; while ((message = GetMessage()) != null) { selectedBuffer.AddRow(); selectedBuffer.SetString(0, message); // how to flush data here? } selectedBuffer.SetEndOfRowset(); } private string GetMessage() { // we are retrieving some message here, this is a long-term process }
When a new row is added by this component to the buffer then this row is not immediately available to the next component in data flow. It is possible to configure SSIS in that way that each row is immediately sent to the next data flow component? If no also please inform me about that.
[DTS.Pipeline] Information: The buffer manager detected that the system was low on virtual memory, but was unable to swap out any buffers. 12 buffers were considered and 12 were locked. Either not enough memory is available to the pipeline because not enough is installed, other processes are using it, or too many buffers are locked.
We are using sql server 2005 Enterprise Edition with service pack1
I got the following error messages in the SQL log
The operating system returned error 21(The device is not ready.) to SQL Server during a read at offset 0x00000000090000 in file '....mdf'. Additional messages in the SQL Server error log and system event log may provide more detail. This is a severe system-level error condition that threatens database integrity and must be corrected immediately. Complete a full database consistency check (DBCC CHECKDB). This error can be caused by many factors; for more information, see SQL Server Books Online. fcb::close-flush: Operating system error 21(The device is not ready.) encountered.
I got these errors for about 2 hrs and after that I see these messages in the sql log
Starting up database ' ' 1 transactions rolled forward in database '' (). This is an informational message only. No user action is required. 0 transactions rolled back in database ' ' (). This is an informational message only. No user action is required. Recovery is writing a checkpoint in database ' ' ( ). This is an informational message only. No user action is required. CHECKDB for database '' finished without errors on (local time). This is an informational message only; no user action is required.
Can anyone please help me in troubleshooting this issue. Why this migh have happened.
I have been tasked with moving our SQL server estate onto new 64bit SQL 2008 Virtual servers on a VM base. Each Virtual server will be attached to our SAN that i will have no control over. Do i ask for multiple LUNs pretending that there is a COS), Etemp), FData) and Glog) disk structure or do I just present a very big space as a single C: drive and let it go.We are consolidating lots of old physical servers onto fewer (more powerful) virtual servers (according to the VM and SAN administrators)
Hi, I have a 250 GB database and not much space left on the disk drive. I want to run SQLMAINT to do optimization and integrity checks on this db. My question is : How much work space does SQLMAINT need to perform these tasks?. Thanks in advance for your help. F.
I try to find some feedback regarding setting the TempDB files on a RAM disk.Specifically I am looking for "production results" that could show the difference/benefit of such an usage.The tests on physical server and VM I already made have shown a boost in overall SQL Server 2012 performance on SQL Server instances housing data for SharePoint 2013 and Dynamics AX 2012 R2.Graphic below show differences between 5 different configuration on the same physical server:
- Physical HD: Server with local HD - Physical SANEX1PRD: Server with TempDB files stored on a low-end SAN - Physical SAN1: Server with TempDB files stored on a high-end SAN (around 100000 IOps) - Physical SAN1 Jumbo: same setup with Jumbo Frame activated on NIC and DB engine - Physical RAMdrive: with TempDB files stored on a 16 GB soft RAM drive within OS memory
Results were really impressive for the DB engine housing Dynamics AX data. My colleagues from the SharePoint team told me it also boosted a bit overall SharePoint performances but they did not have any baseline comparison to show.If you have some feedback, results, links, whatever I am interested.Indeed before setting this to all our SQL Server 2012 instances I y rather collect some *real world* feedback.
I'am running SQL Server 2005 Express Advanced Services on a Windows 2003 Server in a hosted Environment. Some times SQL Server is beginning writing entries into C:ProgrammeMicrosoft SQL ServerMSSQL.1MSSQLLOGERRORLOG until the disk is full.. After that, I have to delete the error log file (some GB of size), restart the server and everything is running fine until the log file runs amok again.
I have installed SQL Server Management Studio.
With SQL server 2005 Standard I can configure or disable Error logging in the Management Studio. But with the Express Edition it seems that is not possible.
What I want to do is (maybe with system stored procedures)
limit number of error log files by cycling it, e.g. 5 files and delete the old ones limit the size of one log file. e.g. 100 MB
Is there an option to configure this in the Express edition of SQL Server 2005?
I have a question concerning where to put certain database files for the followinig RAID configurations. The server has 2 RAID configs: 2 hds in a RAID 1 and 4 hds in a RAID 10. The server will host 4 database instances: A replicated db, a Reporting Services db (which technically constitutes 2 db instances) and an application db. In order to get the best performance, should I put the OS, SQL binary and log files on the RAID 1 config with the data and tempdb on the RAID 10? If not, please explain the best solution. Thank you!