We are noticing a lot of blocking when performing bulk inserts.
Sometimes the lead blocker(s) are blocking inserts into the same tables, but most of the time, it's insert to other tables. So, while bulk insert into table X is running, bulk insert into table Y is blocked from the insert into X.
- We turned off transactions from the application performing the bulk inserts - no change.
- We enabled table locking from the application - no change
- We enabled lock_on_bulk_load on the tables - this seems to have worked; however still see some blocking.
According to the lock_on_bulk article - When you specify table locking for a bulk import operation, a bulk update (BU) lock is taken on the table for the duration of the bulk-import operation, shouldn't we be seeing this BU lock? Instead, we see nothing but LCK_M_X (exclusive table locks).
Just found out that in order get a BU lock, no indexing can exist on the table ... sure be nice if that was in the article!
Also, we saw the exclusive locks happening before we made these changes, meaning it wasn't using the row locks that it should be by default. Assuming we're missing something here with lock escalation. I am profiling just for lock escalation, and it never happens ...
So I guess my pending question at this time is that why when inserting into table X do inserts into table Y get blocked?
SQL Server 2008, Tempdb ("ReportServerTempDB")of the report server is growing enormously over 20gb in 3 days. All our reports drive from stored procedures which is a different server from Reportservertempdb. We have no report subscriptions.
We store no snapshots. All we do is run the reports.
I don't know where I start to investigate the issue.
If you have followed a link to this page (i.e. the URL contains "SearchTerms=xxx") please scroll down to see items of interested highlighted
This list of SQL Server Frequently Asked Questions (FAQ) was originally compiled in August 2005 based on the most frequently "read" posts on SQL Team. Since then new links have been added as topics become popular - so it might be better referred to as FGA - Frequently Given Answers!
NC=Indicates that the Article has NOT been checked for relevance. As time goes on these are getting fewer and fewer The latest change is in Red Recent changes are in Teal
URL to link to this post: 17Nov2005 http://www.sqlteam.com/forums/topic.asp?TOPIC_ID=55210&SearchTerms= Followed by either "Keyword1,Keyword2" or "PhraseWord1+PhraseWord2"
The icon allows you to RightClick-CopyShortcut to provide a link to this topic with appropriate keywords - I got bored of having to make them manually! 07Mar2006 Stats
Homework I put this one at the top, in case you don't read any further. To find out if your homework will be answered on this forum you should see if it qualifies: Requirements for homework questions 07Mar2006 Example of Failure to meet the minimum requirements 07Mar2006 Learning SQL www.sql-tutorial.net 07May2007 www.firstsql.com/tutor.htm 07May2007 www.w3schools.com/sql/default.asp 07May2007
Why? Why is my LDF Log File so big / getting bigger / bigger than my MDF 14Oct2005
How to
How to ask a question on SQL Team and get a quick answer - i.e. the information you need to provide 06Nov2006 How to format your question or a reply (in particular see the [code] option 18Sep2007 how to encrypt and decrypt my password 07May2007 Rounding up/down to nearest multiple (e.g. order in dozens, buy at least 30) 28Sep2006 Replace multiple spaces, or other character, with single space 28Sep2006 Generate a Random String 13Sep2006 String comparison 24Jul2006 Article: Images and SQL Server How to get Images or blobs into SQL table (NC) Inserting Images as BLOB in SQL server table (NC) How to use DeBUG window in QUERY Analyzer (NC) Use a Parameter in the Top clause Dynamic SQL or How do I SELECT TOP @var records? Can't insert NULL into table with unique key (NC) Efficient paging of recordsets with T-SQL Article: Paging in SQL 2005 07May2007 Execute dynamic SQL that is longer than varchar(8000) 05Oct2005 Convert multiple spaces to single spaces 14Oct2005 Making Upper/Lower Case Sensitive comparison 27Oct2005 Convert to Proper Case (each word starts with capital) 19Feb2006 Deleting a massive number of rows 04Mar2006 Combine values from rows into a column 05Oct2005 i.e. FOO 1 FOO 2 becomes: FOO 1,2
CSV / Splitting delimited lists WHERE IN @MyCSV Article: Passing a CSV or Array to a Stored Procedure (NC) Best split functions and Concatenating data from rows with same GroupBy column Triggers Triggers to audit changes trigger to keep tables matching Set Update/Modified date column using a trigger 02Nov2005 Deactivating a trigger 07Feb2006
SQL Team Articles Article: Identity and Primary Keys - and Surogate Keys See also Article: Sending SMTP Mail using a Stored Procedure (NC) Article: Dynamic ORDER BY (NC) Article: More Trees & Hierarchies in SQL (NC) Article: Using SELECT to INSERT records (NC) Article: Deleting Duplicate Records Article: Returning a Row Number in a Query (NC) Article: Calculating Running Totals (NC)
Datatypes / Collation
GUIDs (NC) IsGUID check GUID is valid 01Aug2007 DATALENGTH() and LEN() 21Sep2005 Compare BIT columns and a BitMask column 05Jan2006 Collation conflict in Assignment or WHERE clause 05Feb2006 (See "Developer / Techie stuff" below for some other Collation links)
Datatypes - Dates
Time Only Function: F_TIME_FROM_DATETIME 29Apr2006 Date/Time Info and Script Links (including UNIX time) 16Apr2006 Date formatting ( Format Date/Time (NC) last date of the month (NC) Getting the date portion of a datetime value or Round a date to Year, Quarter, Month, Day, Hour or Minute How to get date in t-sql (NC) compare date with current date (NC) convert varchar to datetime (NC) Compare dates regardless of time? (NC) Convert varchar to date (NC) Time in Analog (NC) Start of Week Function (NC) Earlier date than 1/1/1753
Techniques
Prevent Administrators accessing a database 14Sep2006 SELECT vs SET 26Sep2005 What is a cursor, Why is it bad? 23Sep2006 Cursor or Not (NC) TSQL/ SP Best Practices (NC) case-sensitive (NC) User-Defined string Functions Transact-SQL (NC) Passing an array into a stored procedure (NC) Comparing two tables or views easily (NC) All Tables in All Databases (NC) Better Phonetic Matching Algorithm (A better Soundex) Levenshtein Edit Distance Algorithm (A better Soundex) Fuzzy matching 01Aug2007 Find tables with no PK (NC) Function: HTMLDecode (NC) How to protect errorlogs to tables from rollbacks (NC) Distance of Zip/Postcodes Great Circle Formula 07May2007 Haversine Formula 07May2007 Accuracy v. Speed 07May2007 See also here, here, here, here and here and here Upsert - combined Update/Insert Procedure 01Aug2007 and here Display all columns but only for rows with the most recent date 01Aug2007
Developer / Techie stuff
Changing column names on referencing objects 07May2007 When was DB object last updated 07May2007 How to clear the whole database 07May2007 Move / Script Development Database Changes to Production Database / Ship Application updates 04Nov2006 Script to analyze table space usage 11Oct2006 What version, and Service Pack, of SQL Server am I using? 23May2006 SQL Beautifier 13Apr2006 Performance / System Monitor / PerfMon - preferred values for counters 04Apr2006 SQL Server 2000 Knowledge Base Articles - updated every 3 days 21Jan2006 Concerns about SQL 2000 SP4 08Jan2006 Support stops for SP3a AND installing sp4 on cluster 26May2006
How to read Mssql transaction log file Compare data in two tables / two databases 31Oct2005 Recommended books 07Oct2005 Good tools for DBAs 06Oct2005 Best SQL Security Method (NC) Some query optimalisation (NC) Requirements for Horizontal Partitions Multiple SQL Server 2000 Instances (NC) Primary Key v. Unique Index (NC) HOWTO Run SQL Profiler without sysadmin rights (NC) upgrade from SQL7 to SQL2000 (NC) xp_cmdshell execute right (NC) Monitor object usage over time (NC) size of user tables in a database (NC) Changing Column Length (NC) Add, Sub, Mul, Div With Really! Big Numbers (NC) So you're a DBA eh?! Why I'm not a DBA (NC) Do you S-Q-L or Sequel (NC) Interview questions (NC) More Interview questions 11Sep2006 Best steps to becoming a DBA (NC)
Developer / Techie stuff - Collation
Bulk Delete of Records 07May2007 and locking issues 01Aug2007 Deleting lots of Records from a large table 01Aug2007 Synchronising data using staging tables 07May2007 Version Control 07May2007 how to change Collate on the server 20Sep2005 How to change default collate in SQL 20Sep2005 Cannot resolve collation conflict for equal to OP 20Sep2005 different Collation Settings between Databases 20Sep2005
Database Design and Structures
Codes - Common Lookup Tables CLT 04Jun2006 Good table naming convention 09May2006 Naming Store Procedures 09May2006 Recursion / Hierarchy 29Apr2006 Hierarchies (NC) Normalisation (NC) NULL opinion? (NC) Finding evidence of sql injection attack Security Testing: SQL Injection 09May2006
Locking
Using the NOLOCK hint 07Jun2006 how does row locking work (NC)
EMail
sp sending mail (NC) CDOSYS Send Email (NC)
XML
SQL Server 2000 XML (NC) Adding an XML Root Tag Handling tricky namespaces in OpenXML
Backup / Restore / Replication / Jobs / Houskeeping
Backup How Backup works What type of backup should I use? - Full, Differential, Transaction? Should I shrink the log? 22Jan2006 Automating Backups - Scripting v. Maintenance Plan / Wizard 08Mar2006 Backup to Disk, or direct to Tape? 05Dec2006
Disaster Procedure Checklist I think I have a database corruption, how do I mitigate any further damage and get upright again? 07May2007 How I Rescue Data from Corrupted Databases 07May2007 Disaster Recovery Plan 07May2007 Quiesce the SQL Service to enable low-level copy 01Aug2007
Restore RESTORE syntax / example Restore SQL 2005 to SQL 2000 via Downgrade 07May2007 Restore Full and all TLogs backups based on files in disk folder 07Dec2006 Move database to a new server (with minimal downtime) 16Feb2006 Moving to a new sql server 25Jun2006 Fix Orphaned Users (after a Restore or moving DB to a different server) Script User Logins after a Restore 01Aug2007 and Scripting users & logins from one server to another (e.g. after moving DB to a different server) 25Jul2006 how long will take to restore a 10 gb? All I have is the .bak file Restoring .BAK file to a different server? Unable to restore/attach a MDF file (single file attach / sp_attach_single_file_db) See also: This and this and this
Shrink Database / Files
Shrinking TEMPDB 19Apr2006 Database Shrinking 09Jan2006 Log Shipping -Shrinking Log and Data files (NC) MSDB Massive msdbdata.mdf
Housekeeping / Maintenance
Tara Blog Houskeeping Routines 06Oct2006 Index Tuning Wizard 07May2007 User Rights for SQL Service 07May2007 When was SQL Server last restarted? 07May2007 Optimising your indexes with DBCC INDEXDEFRAG 06Oct2006 Detach Move Db & Log File Reattach 17Mar2006 Rebuilding Indexes DBCC CHECDB, CHECKTABLE, NEWALLOC (old) Suspect Databases after Power Cut? Jobs failing (NC) Reindex All Tables in All Databases (old) deleting replication (NC)
Export / Import / DTS
Copying/backing up DTS Packages (NC) Export to Excel (NC) Generate INSERT statements for N rows of data Tool to script database records? (i.e. generate INSERT statement script) How to convert DATE Format from DD/MM/YY to MM/DD/YY in DTS (NC) Importing an Excel File with DTS (NC) Is DTS Secure for credit card backup to Access? (NC) Winzip in scheduled DTS Package (NC) Article: Using BULK INSERT to Load a Text File (NC)
Output / Reporting / Formatting
Pivot table (NC) Article: Dynamic Cross-Tabs/Pivot Tables (NC) which led to: More Dynamic CrossTabs using T-SQL 04Aug2006 (Jeffs Blog)
Error Messages
SQL Server does not exist or access denied (NC)
Connections
Linked Server Newbie Question (NC)
Configuration / Hardware
/3GB in boot.ini on Win2003 standard 19Oct2005 SQL memory /3gb /PAE AWE 19Sep2005 SQL Server Standard & 4GB memory Using AWE option in SQL Increasing SQL Server Memory
MSDE
MSDE Setup fails (NC) MSDE Install Issue (NC) Installing MSDE on XP Pro (NC) Backing up MSDE database (NC)
SQL 2005 / Yukon
Migrating to SQL 2005 Hints and Tips 01Aug2007 SELECT not allowed in Yukon/SQL2005? (NC) Download Yukon/SQL2005 (NC) SQL Server Management Studio (NC)
MySQL, Oracle, and the others
MySQL not as good as MS SQL? Migrating from Oracle to SQL Server
Other
SQL Reporting Service SP2? (NC)
Got an hour .. or ten ... to spare?
Would you work at Elwoo's office? Simple Quiz What is a Yak? 07May2007
We are receiving following alerts frequently about 1:40 AM in the morning. We have backups running on 11:00 PM everyday and rebuild job running at 2:00 AM. Not sure the exact cause of this error.
Error:
The file group "PRIMARY" for the database "tempdb" in SQL instance "MSSQLSERVER" on computer "XYZ" is running out of space.
tempdev Initial size : 133,100 MB Growth: By 10 percent, Limited to 140000 MB templog Initial Size : 5,475 MB Growth: By 10 percent, Unlimited
At one of your client sides we have configured Always on with synchronous mode.Also we have schedule rebuild index and update statistics job which runs in night every alternate day. the issue is there are more then 100 sleeping queries which is blocking update statistics job.
I have to stop update statistics job manually once i come to office manually.
Once I have killed blocking sleeping query but then other sleeping query blocked it and so on.
I'm working on databases where statistics of some indexes (tables) are changing too frequently. Once I update them manually, one minute after they get 10-20% change, and five minutes after they get over 100% change. Tables get updated very frequently (multiple times in a second).
When I run a query to read from sys.stats, sys.dm_db_stats_properties and other dynamic views, I see that they were last updated when I did it manually, but the change rate overpassed the 500+20% (tables have multiples of 10K rows). Auto create and update statistics are set to true on all databases, and I don't know why sql server does not do that automatically.
When I launch Outlook, it takes forever for the program to finally open. With any inbound email, it stops processing whatever is underway at the time....and frequently there is a 2-3 second lag between keyboard input and what appears on the screen. SQLserver is usually consuming upwards of 1-gb of memory....help. Mike
It is possible that Data Collection can cause massive increasing MB/sec to tempdb ? I cannot find connection with tempdb and I set cash file, but on same disk.
Or it can be something different? Last two weeks what I checked was Read/Write MB/s to tempdb increasing progressively.
One time it was about 20MB/sec
After it was reseting and again 1MB/sec..
What I checked , External company which install SQL Server made one file for tempdb, next week or during breaktime(it will be possible), I would like make 8files next weekend work.
Now I saw that TempDB mdf was still increased, but using was just 8-10%
Error:- (1 row(s) affected) DBCC execution completed. If DBCC printed error messages, contact your system administrator. Msg 5042, Level 16, State 1, Line 1
The file 'tempdev1' cannot be removed because it is not empty.
Note: =>I restarted SQLServer from SSMS and then ran same commands mentioned above ,......and getting same error... => I executed above commands and restarted services...no change...
I'm having an argument with our infrastructure architect who has just gone and bought lots of SSD drives to use for our tempdb data and log files, sounds great doesn't it? There is a catch though, his plan is to add the disks to the two available slots in each blade in a RAID0+1 configuration, effectively giving you one usable drive, and adding both data and log files on to one disk.
I then pointed out that SQL Server best practice is to host tempdb data and log files on two separate drive to reduce contention. The architect then basically said that because this isn't spinning disk the issue of drive, r/w contention isn't an issue I don't agree with this and wanted to get some opinions from the community, I'm still advising that two separate disks should be used but someone just went and spent £80k ($150k) on SSDs and doesn't want to back down...
I can get a snapshot of tables in tempDB, but I would like to track which procs are causing the load in the tempDB.
I think I can sample and record objects in the tempdb, but I would like to record the proc creating the most tempDB usage, and disk read/writes associated with those procs.
The DMV's give usage in the individual DB's, but what's a good way to correlate procs in the DB's to tempdb usage?
I have a server which is not running optimally and I checked the default trace. I have around 600 entries in the default trace which are all Missing Column Statistics and the database is tempdb.is_auto_create_stats_on and is_auto_update_stats_on are both 1 for tempdb.
So we have new servers that are going to be installed with SQL 2012 and I'm debating the wisdom of splitting tempdb with multiple files.
I know it's a myth that performance automatically improves if you split it into a number of files based on processors, but I'm debating the wisdom of putting a file on each of my data / log file drives.
For instance, I have a server with a C: drive (OS), D: drive (Data for system DBs and install of programs - 458 GB), an F: drive for user DB data files (767 GB), and a J: drive for log files (255 GB).
Obviously no files are going on C:. I'm debating on whether or not we should even leave system DBs on the D: drive given in our current 2k8 servers, we end up with Memory.dmp files over flowing the D: drives as well as .cabs and other install / update files that tend to collect on that drive over the years.
But if we leave the system DBs on D:, I'm wondering if adding a second tempdb file to F: and a third to J: will improve query performance or not.
Some file names listed could not be created. Check related errors.
[code]
I did not have remote connections enabled yet, so the resolutions I have found that include sqlcmd or starting in single user configuration are not working. Any way that might allow me to restore the usual tempdb settings, which I think would allow SQL to start again?
I have a tempdb split into 4 files (5 if you include the log).
Autogrowth is disabled on the mdf/ndf files so that they can be used round robin (1 file per logical CPU).
Is there a way to be alerted when there is x% of free space left?
I know hwo to check the free space via t-sql but want to be able to be alerted. I could run a sql job that reports the free space and send a database mail message if under x% but wondered if there was a built in (or better) method?
We are seeing very high Average Disk Queue Length numbers in one of our clusters (both nodes of the cluster are Virtual, but have their own dedicated virtual environments). Our main data drive also houses TempDB, which I would like to move.
Each node in the Active/Passive cluster are running Windows Server 2012 Standard 64bit and SQL Server 2012 Enterprise 64bit. There is a separate drive for Log files and data files.
The data files also have TempDB on them as previously mentioned. I am reading that you can set up a local disk on each node of the cluster, with the same drive letter and path and then move tempdb as you would with a stand alone SQL Server.
I am in the process of formulating recommendations with respect to the purchase of additional storage for our current SQL 2012 SharePoint (2013) instance. My recommendation is to purchase separate storage (i.e, 15k disks) for the TempDB and Tlogs respectively (two sets of raid 10 disks). Currently, this server is hosting several instances, including SP, using two arrays (one for database and the other for Tlogs).
I am attempting to find information/recommendations on how to go about projecting the amount of storage for each of these while factoring in for growth.
Additional Details:
how to best formulate a reasonable estimate. Our largest content database belongs to IT and is currently ~80GB. That said, this is currently an outlier. The remaining content databases are less than 10GB (most are less than 2-3 GB). However, SharePoint will be used for digital document imaging in addition to, eventually, replacing file shares as our primary document storage medium once we roll it out.
Our current tempDB is ~400MB, but the instance was recently started a few days ago, as we had to failover to our backup server for hardware maintenance. I do not have any historical data on TempDB growth at this time. Also, I don't know how useful this would be given we have not fully deployed yet.
I was in the process of creating additional TempDB.ndf files, and received an error saying they already exist. I checked the location and it was empty, nothing to see here. So I looked in sys.master_files and there are several tempdb files listed in various locations, all of which come up empty.
So the files are listed as online in sys.master_files, but they do not exist on the server. I restarted SQL services but it did not change anything.
I receive Error: 3967, Severity: 17, State: 1. Insufficient space in tempdb to hold row versions. We have 8 data files for temp db of 10210 GB size and given 10240 GB as max size.
As MS suggest to calculate the temp db file size and growth rate we need to monitor the perform counters Free Space in Tempdb (KB) and Version Store Size (KB) in the Transactions object.
basic formula: [Size of Version Store] = 2 * [Version store data generated per minute] * [Longest running time (minutes) of your transaction
My report disk utilizations says tempdb is full ? I thonk I need a shrink for the file .
Still I am confused in calculating the size , My perform counter gives me data as such
Free Space in tempdb (KB) 279938496 Version Generation rate (KB/s) 53681040 Version Cleanup rate (KB/s) 53422320 Version Store Size (KB) 258720 Version Store unit count 22 Version Store unit creation 774 Version Store unit truncation 752
I am currently investigating aa high avg write time ms issue (145ms) which seems to be only occuring on the tempdb data files.I have followed the recommended setup of TEMPDB in that
1. Data files = number of physical cores 2. Data files and logfiles are on separate partitions away from the other databases. 3. Tempdb is presized and no incremental file increases look like they are happening with frequency.
We have sharepoint 2012 setup on other sql servers and with TEMPDB setup following the same guidelines, with far more Sharepoint activity on a similary specified hardware which is why its confusing.FileIO auditing on the partitions themselves shows that the FileIO is very fast on the partitions that the tempdb data file which leads me to beleive that Sharepoint may be the culprit perhaps due to excess use of tempdb with operations taking a long time to resolve.
