Keeping Entire Table In Memory
Aug 17, 1999How can I instruct SQL Server to keep entire table in memory? ie the memory pages should not be swapped to HD.
View 2 RepliesHow can I instruct SQL Server to keep entire table in memory? ie the memory pages should not be swapped to HD.
View 2 RepliesIs it possible to keep a Cached Lookup in memory when executing multiple Data Flows? Executing DFT€™s in parallel will cache and use the same LOOKUP statement. But what if I€™m executing the DFT sequentially, can I keep the LOOKUP from the first DFT in memory for the second DFT? For example, in my case, I€™m caching a lookup against the Customer dimension for invoices. The second DFT then processes credits and again does a lookup against the Customer dimension. I want to use the cached Customer records from the first DFT.
View 1 Replies View RelatedHi
We need to keep track of all changes that are made to our tables.
The changes will be saved in a table that records:
- the table in which the change was made
- the name of the field that was changed
- the old data for the field
- the new data for the field etc..
I've seen a few examples that record the name of the table that was
modified but none that record done to the field level.
Can anybody give some guidance?
Thanks..
Wayne
I need to update a row but keep a lock on the table (so no one else can update it) while I do run some more code. In Oracle, it always locks whatever you update until you hit commit, but sql server works opposite. How do I tell it not to commit a statement, or how would I explicitly get a lock and then release it later?
View 4 Replies View RelatedHi, I am trying to find a way to capture all the status (Start time, execution time, Status messages etc) from executing a DTS package in to a table I will create in a database, does anyone know, where those information being kept?
When I excute the DTS package manually, a window will come up and show the status of each step within the DTS package. I am hoping to capture these information and load it to my log table.
Thanks in advance.
I am looking to test this feature - and the "Transaction Performance Collector" has recommended me a table to port to In-Memory OLTP.Â
I have now tried the "Table Memory Optimization Advisor" tool.
After a couple of tweaks to the table design - the tool is now passing validation but the tool is not allowing to progress to the next step:
Could it be down to not having enough memory? But would this not show in the advisor?
Guys,
First off, I'm not very familiar with SQL Server. I need some guidance on what the best path to take is for this as it may not even be table partitions.
I have a huge table (155 million rows) and it's gotten so large than I can't even delete a large set of rows from it (i.e. delete everything older than 6 mo, which would be ~100 million rows). When trying to run a delete like this, it just goes for a LONG time and then just eventually runs out of memory.
The current data in this table can actually be completely cleared out soon (after Feb 1st) and I plan to do this with TRUNCATE TABLE, or just DROP and recreate. Once I do this, I want to create a way to keep this table moderately sized so it never grows that large again and it seems table partitions may be the way to go for this?
I'd like to keep the last 6mo of data in it (I have a datetime column to keep track of this). Anything older I'd like automatically removed. Can I do this with table partitioning? Create 6 partitions that store the 6 most recent months of data and everything older automatically gets dropped off?
If not partitions, what do you suggest to keep this DB modest size?
Thank you.
Hi! I've some data in a table, and I want to add these records to another table, but I tried yet with select ... Into ... and I lose all the data from this one. I need to ADD the data from table to table. Any suggestion?
Thx
Hi,I need to lock a table so that Inserts are prevented as well as deleted andupdates. At present I'm thinking this might do it:SELECT * FROM myTable WITH(UPLOCK)but then again I'm not sure whether this will cover the insert case.Thanks,Robin
View 5 Replies View RelatedHow can I do in order that a table show all its rows in a page?And if the table can only show a number of rows cause arrive to the end of the page show the table in the next page?
Thanks!
I am giving the following statement outside the begin tran block in a stored procedure:
select * from table1 with (holdlock) where column1 = @colValue
I have the following two questions regarding above situation:
Being outside the begin tran block, will the select statement cause a lock to be held or it has to be within the begin tran block for the lock to be held?
Will the select statement hold a lock only on the row or rows matching the where clause, or it may lock a page or even the entire table in this situation?
Thanks
I'm a newbie to script writing. I'm trying to write a script to copy alldata from a table to the same table in a 2nd database. Both databases areon the same server and are identical in design. I can do this with DTS butwanted a script I could email to a user to run in Query Analyzer.Example:Copy entire table called 'Customers' in the 'Data01' database totable 'Customers' in the 'Data02' databaseI want to overwrite all data in the destination table.Thanks
View 1 Replies View Related
I have table in my SQL database, I want to copy(or insert) one record(one entire row using primary key, which is auto)
I am thinking something like this
Insert Into Table1 (a, b, c,d,e) values(select a,b,c,d,e from Table1 where Primarykey=1 or any number)
telll me how do I do that
maxs
The problem that I am having is that I am not getting the results from an entire table, just a small subset. The sql command is "Select * from login", which has always implied to me, get everything, to end of table, or, until I tell you to stop. I am getting exactly, 10 items as input, and then it stops. I am including the code below in the hopes that someone can spot what it is that I am doing wrong.
The purpose of the routine is to convert a user login database into a new, expanded format.
TIA, Tom
Imports Common.Utility.Utils
Partial Class UserDatabaseConversion
Inherits System.Web.UI.Page
Dim sqlConn As SqlConnection = getSQLConnection(ConfigurationManager.AppSettings("utilityDbName"))
Dim sqlConnN As SqlConnection = getSQLConnection(ConfigurationManager.AppSettings("utilityDbName"))
Dim sqlStringOld As String = "Select * from login"
Dim sqlStringNew As String = "usp_UT_AddNewUser"
Dim buildCnt As Integer = 0
Dim oldUserCnt As Integer = 0
Dim newUserCnt As Integer = 0
Dim oLocation As String = Nothing
Dim errors As Integer = 0
Dim mesg As String = Nothing
Protected Sub Page_Load(ByVal sender As Object, ByVal e As System.EventArgs) Handles Me.Load
getOldUsers()
End Sub
Sub getOldUsers()
Dim odate As Date = Date.Today
Dim sqlCmdOld As SqlCommand = New SqlCommand()
sqlCmdOld.CommandText = sqlStringOld
sqlCmdOld.CommandType = CommandType.Text
sqlCmdOld.Connection = sqlConn
Dim dbReader As SqlDataReader = Nothing
Try
sqlConn.Open()
dbReader = sqlCmdOld.ExecuteReader()
If dbReader.HasRows Then
Do While dbReader.Read()
oldUserCnt += 1
Select Case IsDBNull(dbReader.Item("reg_date"))
Case True
buildNewUser(dbReader.Item("name"), dbReader.Item("password"), dbReader.Item("email"), dbReader.Item("type"), dbReader.Item("org"), dbReader.Item("occupation"), dbReader.Item("location"), odate)
Case False
buildNewUser(dbReader.Item("name"), dbReader.Item("password"), dbReader.Item("email"), dbReader.Item("type"), dbReader.Item("org"), dbReader.Item("occupation"), dbReader.Item("location"), dbReader.Item("reg_date"))
End Select
Loop
End If
Catch ex As Exception
Finally
sqlConn.Dispose()
End Try
If errors = 0 Then
lblMesg.Text = "Conversion completed succesfully.<br />Users In: " & oldUserCnt & " , Users Out:" & newUserCnt
Else
lblMesg.Text = "Conversion comleted with errors.<br />Users In: " & oldUserCnt & " , Users Out:" & newUserCnt & "<br />"
lblMesg.Text += mesg
End If
End Sub
Sub buildNewUser(ByVal oname As String, ByVal opassword As String, _
ByVal oemail As String, ByVal otype As String, _
ByVal oorg As String, ByVal oocc As String, _
ByVal oloc As String, ByVal oregdate As Date)
Dim sqlCmdNew As SqlCommand = New SqlCommand()
sqlCmdNew.CommandText = sqlStringNew
sqlCmdNew.CommandType = CommandType.StoredProcedure
sqlCmdNew.Connection = sqlConnN
Dim userName As String = Nothing
Dim firstName As String = Nothing
Dim middleName As String = Nothing
Dim lastName As String = Nothing
Dim title As String = Nothing
Dim emailAddress As String = Nothing
Dim passWord As String = Nothing
Dim userType As String = Nothing
Dim org As String = Nothing
Dim occ As String = Nothing
Dim loc As String = Nothing
Dim rdate As Date = Nothing
oLocation = oloc
' user name will be the first part of the email address
emailAddress = oemail
Dim uname() As String = oemail.Split("@")
' extend the length of the user name to at least 5 characters
Select Case uname(0).Length
Case Is > 4
userName = uname(0)
Case Is = 4
userName = uname(0) & "1"
Case Is = 3
userName = uname(0) & "12"
Case Is = 2
userName = uname(0) & "123"
Case Is = 1
userName = uname(0) & "1234"
End Select
' check for suffixes
Dim suffix() As String = oname.Split(",")
Select Case suffix.Length > 1
Case True
title = suffix(1)
End Select
' split names out
Dim names() As String = suffix(0).Split(" ")
' possibly 3 components to the name
Select Case names.Length
Case Is = 1
firstName = names(0)
Case Is = 2
firstName = names(0)
lastName = names(1)
Case Is = 3
firstName = names(0)
middleName = names(1)
lastName = names(2)
End Select
' setup password. must be at least 6 characters.
' if less, extend to 6.
Select Case opassword
Case Is > 5
passWord = opassword
Case Is = 5
passWord = opassword & "1"
Case Is = 4
passWord = opassword & "12"
Case Is = 3
passWord = opassword & "123"
Case Is = 2
passWord = opassword & "1234"
Case Is = 1
passWord = opassword & "12345"
End Select
' user type, organization and occupation
userType = otype
org = oorg
occ = oocc
' registration date
Select Case IsDBNull(oregdate) Or IsNothing(oregdate)
Case True
rdate = Today.Date
Case False
rdate = oregdate
End Select
' try to extract the location
' get the country
Dim country As String = getCountry(oloc)
' get the state
Dim state As String = getState(oloc)
' only thing left is the city
Dim city() As String = oloc.Split(",")
' add parameter values
sqlCmdNew.Parameters.AddWithValue("@NamePrefix", System.DBNull.Value)
sqlCmdNew.Parameters.AddWithValue("@FirstName", replaceSingleQuote(firstName))
Select Case IsNothing(middleName)
Case True
sqlCmdNew.Parameters.AddWithValue("@MiddleName", System.DBNull.Value)
Case False
sqlCmdNew.Parameters.AddWithValue("@MiddleName", replaceSingleQuote(middleName))
End Select
Select Case IsNothing(lastName)
Case True
sqlCmdNew.Parameters.AddWithValue("@LastName", System.DBNull.Value)
Case False
sqlCmdNew.Parameters.AddWithValue("@LastName", replaceSingleQuote(lastName))
End Select
sqlCmdNew.Parameters.AddWithValue("@UserName", replaceSingleQuote(userName))
Select Case IsNothing(title)
Case True
sqlCmdNew.Parameters.AddWithValue("@NameTitle", System.DBNull.Value)
Case False
sqlCmdNew.Parameters.AddWithValue("@NameTitle", title)
End Select
sqlCmdNew.Parameters.AddWithValue("@Password", replaceSingleQuote(passWord))
sqlCmdNew.Parameters.AddWithValue("@EmailAddress", emailAddress)
sqlCmdNew.Parameters.AddWithValue("@PasswordHint", userName)
Select Case isEmpty(org)
Case True
sqlCmdNew.Parameters.AddWithValue("@Organization", System.DBNull.Value)
Case False
sqlCmdNew.Parameters.AddWithValue("@Organization", replaceSingleQuote(org))
End Select
Select Case isEmpty(occ)
Case True
sqlCmdNew.Parameters.AddWithValue("@Occupation", System.DBNull.Value)
Case False
sqlCmdNew.Parameters.AddWithValue("@Occupation", replaceSingleQuote(occ))
End Select
sqlCmdNew.Parameters.AddWithValue("@Address1", System.DBNull.Value)
sqlCmdNew.Parameters.AddWithValue("@Address2", System.DBNull.Value)
Select Case isEmpty(city(0))
Case True
sqlCmdNew.Parameters.AddWithValue("@City", System.DBNull.Value)
Case False
sqlCmdNew.Parameters.AddWithValue("@City", replaceSingleQuote(city(0)))
End Select
sqlCmdNew.Parameters.AddWithValue("@ZipCode", System.DBNull.Value)
sqlCmdNew.Parameters.AddWithValue("@PostalCode", System.DBNull.Value)
Select Case IsNothing(state)
Case True
sqlCmdNew.Parameters.AddWithValue("@State", System.DBNull.Value)
Case False
sqlCmdNew.Parameters.AddWithValue("@State", state)
End Select
sqlCmdNew.Parameters.AddWithValue("@Other", System.DBNull.Value)
Select Case IsNothing(country)
Case True
sqlCmdNew.Parameters.AddWithValue("@Country", System.DBNull.Value)
Case False
sqlCmdNew.Parameters.AddWithValue("@Country", country)
End Select
sqlCmdNew.Parameters.AddWithValue("@GeneralEmail", False)
sqlCmdNew.Parameters.AddWithValue("@FundRaiseEmail", False)
sqlCmdNew.Parameters.AddWithValue("@GeneralUSPSMail", False)
sqlCmdNew.Parameters.AddWithValue("@FundRaiseUSPSMail", False)
sqlCmdNew.Parameters.AddWithValue("@DateAdded", rdate)
sqlCmdNew.Parameters.AddWithValue("@DateUpdated", rdate)
sqlCmdNew.Parameters.AddWithValue("@Active", True)
sqlCmdNew.Parameters.AddWithValue("@UserType", userType)
Dim iRows As Integer = 0
' try to insert the row into the table
Try
sqlConnN.Open()
iRows = sqlCmdNew.ExecuteNonQuery()
If iRows <> 1 Then
errors += 1
mesg += "<br />User: " & firstName & " " & lastName & " was not added.<br />"
End If
newUserCnt += 1
Catch ex As Exception
Dim exMsg As String = ex.Message.ToString()
writeApplicationLog(exMsg, ConfigurationManager.AppSettings("UtilityDBName"))
errors += 1
mesg += "<br />User: " & firstName & " " & lastName & " was not added due to " & exMsg & "<br />"
Finally
sqlConnN.Close()
sqlCmdNew.Dispose()
End Try
End Sub
Function getState(ByVal loc As String) As String
Dim state As String = Nothing
If loc.Contains("NY") Then
state = "NY"
ElseIf loc.Contains("WA") Then
state = "WA"
ElseIf loc.Contains("PA") Then
state = "PA"
ElseIf loc.Contains("AZ") Then
state = "AZ"
ElseIf loc.Contains("MI") Then
state = "MI"
ElseIf loc.Contains("WI") Then
state = "WI"
ElseIf loc.Contains("CT") Then
state = "CT"
ElseIf loc.Contains("NC") Then
state = "NC"
ElseIf loc.Contains("CA") Then
state = "CA"
ElseIf loc.Contains("NY") Then
state = "NJ"
End If
Return state
End Function
Function getCountry(ByVal cntry As String) As String
Dim country As String = "US"
If cntry.Contains("Canada") Then
country = "CA"
End If
Return country
End Function
End Class
Somebody suggested I use the slowly changing dimension object to keep a table current between servers and databases.
I have two databases servers. One 2000 and the other 2005. I was told because security reasons I should not link them
The two tables are not identical. The destination table only has a handle of the columns and in some cases columns are not the same size.
Once a day. When new rows are introduced in the source table, I'd like to introduce those rows into the destination table. When rows are modified, I'd like to apply changes to the distination table.
I started playing with the slowly changing dimensions object, but am frankly confused connected the dots. Will I have an output oledb object for each command object?
Am i using the right the object for objective?
Are there any good tutorials or videos out there that will cover this?
Thanks for any help or information!
Hello,
I have a table that I want to cache. So, if this is my query "SELECT * FROM myTable" that I create the cache from, how can I filter the data?
I really need to cache the whole table, since there are a myriad of different statements being executed against the table, so just caching a specific query won't do.
I've found the best approach to make a question is writing it in code, so how can I do the following:
SELECT ColNames FROM MyTableWhichShouldNowBeCached WHERE whereColumns=someParams.
ColNames and someParams make up for a variation of about to 120 different queries. I use SQL 2005 express edition with advanced services.
Cheers!
/Eskil
Hi Guys!
This SQL statement, though carefully written to delete only selected rows, deletes the entire A_Shift_Times table:
DELETE FROM A_Shift_Times
WHERE EXISTS (
SELECT 1
FROM
Users
WHERE
(A_Shift_Times.time_in >= CONVERT(DATETIME, '2000-05-29 00:00:00', 102)) AND
((Users.User_Password LIKE N'mrr%') OR (Users.User_Password LIKE N'work%')))
What gives? What am I doing wrong here?
Hi everybody,
We are using SQl-2000 Sp2 clustured on two compaq servers
We have to modify our database shema every month.
We are using sp_repladdcolum and sp_repldropcolum to
modify table without snapshot.
Owever I would like to add/delete entire table in shema
database without using snapshot ?
Is it possible ?
How to do ?
Thanks.
Fabio
I convert EBCDIC file to text file at C:orderorder.txt. There are about 1400000 rows. The length of line is 1500.
How to insert entire line into one column of the table? (only one column, named as [LineDetail])
I'm trying to create a query that searches an entire database for keywords inside of the columns for each table within the database. For instance my tables have 2 columns one named ID and the other Permission, I'd like it be able to return all the lines that are associated with that keyword. So if I search "Schedule" it returns all the lines containing that word in it within that database.
View 6 Replies View RelatedI usually go for the largest datafile and then query in-there...But now I need to automate it for several instances... I need to be able with one script quickly retrieve the top 5 largest tables for the entire instance,not by database...
View 3 Replies View RelatedHi,
I have a table which has about 1144 records i want to know how much memory it is using. how do i find this ?
Any Help will be appreciated,
Thanks
ran.
How can I load a table into memory in SQL Server 2000 or SQL Server 2005 ?
How can I keep the table always in memory in SQL Server 2005 ?
with all my thanks
I have a table with several million rows and it gets queried every day by an application. When this happens, it either ties up the server or crashes SQL 6.5 in general. Because of the thoroughness of the query, its understandable. An idea to get around the locking up was to run DBCC CHECKTABLE on this table to put it in cache and run the query.
Will this speed up the process or prevent the locks? Has anyone done this?
Alan
I just need some clarification on using temporary tables. When I create a table, 2 columns and insert 1million records in it, is the temp table I created #STUFF in memory or in the tempdb?
Thanks
Gene Laoyan
I'm not an expert in SQL administration, but I can't figure out why this should be a complicated issue. I have a really big table in SQL Server (2005 Dev. Ed.), that has lots of rows, but only a few columns. One of the columns is an INT, and I'm running a single query against the table dependant only on that column, ex. SELECT A,B FROM tbl WHERE C = 7, C being the INT column in question.
Naturally I figured that with >100M records it would help to put a clustered index on that column C. It took a long time to create the index, but it worked and it did seem to help; the query now completes in short order instead hours or days. But the memory on my system still maxes out (I have 2GB of DDR2 and the physical memory usage hits 1.95 immediately), and doesn't get returned after the query. The only way to get that memory back seems to be restarting the MSSQL process, but the next time I run the query it happens again.
I need to run this query quite a few times in my code and can't help but think that, even though it's like 1000% faster now that I have the index, it could maybe go a lot faster if it weren't using all my ram. I can't figure why it would need all that memory; a clustered index should enable a binary search right? The query mentioned above usually only returns a few hundred or few thousand records, which, given my record size should only be a few hundred MB at most.
Any ideas why it's spiking like this, or why the memory isn't being reclaimed?
thanks
What should i do when i want to save a select resultset,but not in a newtable,in memory is the best?thanks!
View 1 Replies View RelatedHello. I have received the follwoing error upon an attempt to Browse the Cube. All other tabs are functional, including the Calculations tab. We are running Windows Server 2003 SP2 and SQL Server 2005 SP2. Any suggestions would be greatly appreciated!
**EDIT** - Have confirmed SP1 for VS2005 is installed both locally and on server, also.
Attempted to read or write protected memory. This is often an indication that other memory is corrupt. (Microsoft Visual Studio)
------------------------------
Program Location:
at Microsoft.Office.Interop.Owc11.PivotView.get_FieldSets()
at Microsoft.AnalysisServices.Controls.PivotTableFontAdjustor.TransformFonts(Font font)
at Microsoft.AnalysisServices.Browse.CubeBrowser.UpdatePivotTable(Boolean translate)
at Microsoft.AnalysisServices.Browse.CubeBrowser.UpdateAll(Boolean translate)
at Microsoft.AnalysisServices.Browse.CubeBrowser.InitialUpdate()
at Microsoft.AnalysisServices.Browse.CubeBrowser.SupportFunctionWhichCanFail(FunctionWhichCanFail function)
Hi,
We are seeing memory issues with a simple C# based TVF that splits an input string into a table based a delimiter. Entries such as the following show up in the SQL log:
AppDomain 8 (DBName.dbo[runtime].7) is marked for unload due to memory pressure.
AppDomain 8 (DBName.dbo[runtime].7) unloaded.
These entries only show up after the TVF has been called many times (~ half million times or more).
We encountered the same memory issues with June CTP, but they appeared to be fixed in Sepetmber CTP. Somehow the issues come back for us in the SQL Server 2005 RTM version. With June CTP after these errors show up many many times, the SQL server had to be re-started.
I'd appreciate any comments/ideas regarding these error messages in the SQL log?
We are using the RTM version of SQL2005: Microsoft SQL Server 2005 - 9.00.1399.06 (Intel X86)
Oct 14 2005 00:33:37
Copyright (c) 1988-2005 Microsoft Corporation
Enterprise Edition on Windows NT 5.2 (Build 3790: Service Pack 1)
Thanks,
Wenbin Zhang
I've been researching AWE to determine if we should enable this for our environment.
Currently we have a quad core box with 4 gb of RAM (VMware). OS: Windows 2003 std, SQL Server 2005 std. 3GB is not set but will be as soon as we can perform maintenance on the server.
I have read mixed feedback on AWE, either it works great or grinds you to a hault. I would assume that the grinding to a hault is due to not setting the min/max values correctly or not enabling the lock page in memory setting.
We only have one instance of SQL on the server and this box won't be used for anything else aside from hosting SQL services. We do plan on running SSRS off of this server as well.
1. Will running SSRS and enabling AWE cause me problems? Will I have to reduce the max setting by the SSRS memory usage or will it share and play nice?
2. How do I go about setting the Max value? Should it be less than the physical RAM in the box? Right now its set to the default of 214748364, even if I don't enable AWE should this default value be changed?
3. It seems that even at idle the SQL server holds a lot of memory and the page file grows. If I restart the process in the morning, memory usage in taskmon is at 600mb or so. By the end of the day, its up around 2gb. How can I track down whats causing this, should this even concern me?
4. The lock Page in memory setting worries me. Everything I've read on this seems to give a warning about serious OS and other program support degradation. In some cases to the point where they have to restore the settings on the server before they can bring it back up. What are your thoughts on this.
SQl7, sp3, NT4
How do I keep th job history of a job, say if I re-create the job?
We recreate the jobs often as part of a code move, but I'd like to retain the history of the previous jobs?
** sp_help_jobhistory -- only shows the jobs that exist, and not old jobs that no longer reside on the server.
Thanks,
AJ
Hi all, here are my goals: Have the same DB on two different stand-alone computers, and keep them up-to-date from each other.
Basically a user would input to a DB for a week. Then every week or two, update the other stand alone DB with the new input. The DB would be exactly the same.
What are my options for this? I'd like it as easy as possible! Are there any software packages that deal with this type of transfer, etc.? Thank you!
Hello
my table :
Report :
R_id (PK)
RName
RDate
i am having a few 10.0000 lines and i want to keep the last 10 (or less if not in the table) rows maximum for each name
i can have 100 report by name (100 rows with the same name and of course R_id and RDate are different)
how can i do it ?
thanks a lot for helping