Database Script Creates Huge Log File
Mar 20, 2008
I am creating a database for an application through script. After the tables, views, and sp's are created, the database is populated with data. After all of this (and before the application is even run), the log file is about 700MB. If I shrink the database, it takes the log down to 1MB. The mdf file is about 165 MB before and after it has been shrunk.
I have two questions:
1. Is there something I should look for in my database scripts or is there a setting that could prevent this from being created so large.
2. Is there a script I can run in my sql code after the database has been created and populated to shrink it.
Thank you for your help
View 2 Replies
ADVERTISEMENT
Feb 1, 2005
I am upgrading a .mdb to MSSQL. The .mdb is 17MB, but the resulting MSSQL is 72MB. Tried using both the Access Upsizing Wizard and Enterprise Manager DTS. I have done this a number of times before, but never ran into this problem. Any ideas what coule be going on, and how to fix it?
View 2 Replies
View Related
Feb 1, 2005
I am upgrading a .mdb to MSSQL. The .mdb is 17MB, but the resulting MSSQL is 72MB. Tried using both the Access Upsizing Wizard and Enterprise Manager DTS. I have done this a number of times before, but never ran into this problem. Any ideas what coule be going on, and how to fix it?
View 1 Replies
View Related
May 20, 2006
The log file for my database in SQL Server 2005 is huge. How do I get empty it or in effect shrink it or start it over?Thanks
View 1 Replies
View Related
Nov 3, 2015
Have a database that's in "Simple" recovery mode whose .ldf has grown to 270GB. This database is a data warehouse so "full" is not required. I put it in simple mode a month ago and shrunk the log down and now it's filled up the disk.
What steps can I take to mitigate this in future? I've read that this is caused by long running transactions which fill the log for DR purposes. Should I put the database back into full mode and backup/truncate daily.
The auto-growth is set to 128MB which is very low.
View 3 Replies
View Related
Mar 22, 2007
I have several SSIS packages which are run in sequence. But some strange reason sometimes a SQL Dump is created. If I restore database and restart the same ETL is works, or it creates another SQL Dump in a different place. Any idea on how to debug this problem?
SQL Server 2005 Developer Edition SP1 ( should I upgrade to SP2? )
Windows 2003/r2 64b ( latest patches )
Eventlog
Bucket 09193120, bucket table 5, EventType sql90exception64, P1 dtsdebughost.exe, P2 2005.90.1399.0, P3 0000000043500d1e, P4 ntdll.dll, P5 5.2.3790.1830, P6 0000000042438b79, P7 0, P8 000000000003d5a8, P9 0000000000000000, P10 NIL.
For more information, see Help and Support Center at http://go.microsoft.com/fwlink/events.asp.
SQL Dump file
03/21/07 17:55:11, ACTION, SQLDUMPER_UNKNOWN_APP.EXE, Input parameters: 4 supplied
03/21/07 17:55:11, ACTION, SQLDUMPER_UNKNOWN_APP.EXE, ProcessID = 5928
03/21/07 17:55:11, ACTION, SQLDUMPER_UNKNOWN_APP.EXE, ThreadId = 0
03/21/07 17:55:11, ACTION, SQLDUMPER_UNKNOWN_APP.EXE, Flags = 0x0
03/21/07 17:55:11, ACTION, SQLDUMPER_UNKNOWN_APP.EXE, MiniDumpFlags = 0x0
03/21/07 17:55:11, ACTION, SQLDUMPER_UNKNOWN_APP.EXE, SqlInfoPtr = 0x000000000100FAF8
03/21/07 17:55:11, ACTION, SQLDUMPER_UNKNOWN_APP.EXE, DumpDir = <NULL>
03/21/07 17:55:11, ACTION, SQLDUMPER_UNKNOWN_APP.EXE, ExceptionRecordPtr = 0x0000000000000000
03/21/07 17:55:11, ACTION, SQLDUMPER_UNKNOWN_APP.EXE, ContextPtr = 0x0000000000000000
03/21/07 17:55:11, ACTION, SQLDUMPER_UNKNOWN_APP.EXE, ExtraFile = <NULL>
03/21/07 17:55:11, ACTION, SQLDUMPER_UNKNOWN_APP.EXE, InstanceName = <NULL>
03/21/07 17:55:11, ACTION, SQLDUMPER_UNKNOWN_APP.EXE, ServiceName = <NULL>
03/21/07 17:55:11, ACTION, SQLDUMPER_UNKNOWN_APP.EXE, Callback type 11 not used
03/21/07 17:55:12, ACTION, SQLDUMPER_UNKNOWN_APP.EXE, Callback type 7 not used
03/21/07 17:55:12, ACTION, SQLDUMPER_UNKNOWN_APP.EXE, MiniDump completed: C:Program FilesMicrosoft SQL Server90SharedErrorDumpsSQLDmpr0053.mdmp
03/21/07 17:55:12, ACTION, DtsDebugHost.exe, Watson Invoke: Yes
03/21/07 17:55:12, ACTION, DtsDebugHost.exe, Watson Invoked: C:PROGRA~1COMMON~1MICROS~1DWDW20.EXE dw20.exe -d "C:Program FilesMicrosoft SQL Server90SharedErrorDumpsSQLDmpr0001.mft"
Versions
Microsoft Visual Studio 2005 Version 8.0.50727.762 (SP.050727-7600)
Microsoft .NET Framework Version 2.0.50727
Installed Edition: IDE Standard
Microsoft Visual Studio 2005 Premier Partner Edition - ENU Service Pack 1 (KB926601)
This service pack is for Microsoft Visual Studio 2005 Premier Partner Edition - ENU.
If you later install a more recent service pack, this service pack will be uninstalled automatically.
For more information, visit http://support.microsoft.com/kb/926601
SQL Server Analysis Services Microsoft SQL Server Analysis Services Designer Version 9.00.2047.00
SQL Server Integration Services Microsoft SQL Server Integration Services Designer Version 9.00.2047.00
SQL Server Reporting Services Microsoft SQL Server Reporting Services Designers Version 9.00.2047.00
View 3 Replies
View Related
Jul 11, 2006
After much work and thanks to all of you who helped on this here is a code sample that can be adapted. From the dataflow task add an OLEDB source component, a row count component and finally a Script Destination Component.
On the Script Destination Component rename the Input node of the imports and outputs tree view to "ParsedInput"
The readonly User: variables that start with gs can be read in the PreExecute method
The readwrite User: variable giSuccessCount can only be used in the post execute task because it is populated by the Row Count Component which is the previous object in the Dataflow
The xml code is adapted from an idea in Donald Farmers book
enjoy
Dave
Now if someone can make a Script Source Component that can read a file with a header , data body and trailer that would b egreat!
' Microsoft SQL Server Integration Services user script component
' This is your new script component in Microsoft Visual Basic .NET
' ScriptMain is the entrypoint class for script components
Imports System
Imports System.Data
Imports System.Math
Imports System.IO
Imports Microsoft.SqlServer.Dts.Pipeline.Wrapper
Imports Microsoft.SqlServer.Dts.Runtime.Wrapper
Imports System.Xml
Public Class ScriptMain
Inherits UserComponent
Dim sw As StreamWriter
'In addition to using the Imports System.Xml statement a reference must be added to the
'System.Xml assembly (Select Project-Add Reference from IDE)
Dim xWriter As XmlTextWriter
Dim OutputFileType As String '.csv or .xml
Public Overrides Sub PreExecute()
'Read Only variables
Dim gsPickUp As String = Me.Variables.gsPickUp 'D:ftprootOutAvid'
Dim gsPickUpFilename As String = Me.Variables.gsPickUpFilename '1_AVID_'
Dim gsPickUpFileExtn As String = Me.Variables.gsPickUpFileExtn '.csv'
Dim gsMemoText As String = Me.Variables.gsMemoText 'Memo Text : credit adjustment'
Dim gsStatementText As String = Me.Variables.gsStatementText 'Statment Text : credit adjustment'
Dim gsRunMode As String = Me.Variables.gsRunMode 'UPDATE'
Dim fileName As String = gsPickUp & "" & gsPickUpFilename
fileName = fileName & (Format(Now(), "yyMMdd").ToString)
'MsgBox(fileName)
OutputFileType = gsPickUpFileExtn
If OutputFileType = ".csv" Then
fileName = fileName & gsPickUpFileExtn
sw = New StreamWriter(fileName) 'connection to dest file
'Header records
sw.Write(gsRunMode)
sw.Write(Environment.NewLine) ' end of line
sw.Write(gsMemoText)
sw.Write(Environment.NewLine)
sw.Write(gsStatementText)
sw.Write(Environment.NewLine)
sw.Write(Environment.NewLine) 'Spacer
End If
If OutputFileType = ".xml" Then
fileName = fileName & gsPickUpFileExtn
'xWriter = New XmlTextWriter(Me.Connections.XMLConnection.ConnectionString, Nothing)
'xWriter.WriteStartDocument()
'xWriter.WriteComment("Customer file parsed using script")
'xWriter.WriteStartElement("x", "customer", "http://some.org/name")
'xWriter.WriteAttributeString("FileName", Me.Connections.XMLConnection.ConnectionString)
xWriter = New XmlTextWriter(fileName, Nothing)
xWriter.WriteStartDocument()
xWriter.WriteComment("Customer file parsed using script")
xWriter.WriteStartElement("x", "customer", "http://some.org/name")
xWriter.WriteAttributeString("FileName", fileName)
End If
End Sub
Public Overrides Sub ParsedInput_ProcessInputRow(ByVal Row As ParsedInputBuffer)
If OutputFileType = ".csv" Then
Dim delim As String = ","
sw.Write(Row.ProjectID.ToString + delim)
sw.Write(Row.TransactionRefNum.ToString + delim)
sw.Write(Row.CustomerNum.ToString + delim)
sw.Write(Row.AccountNum.ToString + delim)
sw.Write(Environment.NewLine) ' end of line
sw.Flush() 'send the stream to file
End If
If OutputFileType = ".xml" Then
xWriter.WriteStartElement("CUSTOMER")
xWriter.WriteStartElement("ProjectID")
xWriter.WriteString(Row.ProjectID.ToString)
xWriter.WriteEndElement()
xWriter.WriteStartElement("TransactionRefNum")
xWriter.WriteString(Row.TransactionRefNum.ToString)
xWriter.WriteEndElement()
xWriter.WriteStartElement("CustomerNum")
xWriter.WriteString(Row.CustomerNum.ToString)
xWriter.WriteEndElement()
xWriter.WriteStartElement("AccountNum")
xWriter.WriteString(Row.AccountNum.ToString)
xWriter.WriteEndElement()
End If
End Sub
Public Overrides Sub PostExecute()
If OutputFileType = ".csv" Then
'Create the trailer
sw.Write(Environment.NewLine) ' blank line
sw.Write("RECORD_COUNT: " & Me.Variables.giSuccessCount.ToString) 'ReadWrite Varible
sw.Write(Environment.NewLine)
sw.Flush() 'send the stream to file
'Close file
sw.Close()
End If
If OutputFileType = ".xml" Then
xWriter.WriteStartElement("RecordCount")
xWriter.WriteString(Me.Variables.giSuccessCount.ToString)
xWriter.WriteEndElement()
xWriter.WriteEndElement()
xWriter.WriteEndDocument()
xWriter.Close()
End If
End Sub
End Class
View 6 Replies
View Related
Apr 18, 2008
Hi,
I want to run a stored procedure in Database A that will create a View on a table in database B. Is that possible?
Example (Creating a Stored Procedure in Database A):
create procedure test
as
begin
use B
create view Vjobs
as
select * from jobs
end
This does not work because you are not allowed "USE" in a Stored Procedure.
Please help.
View 1 Replies
View Related
Oct 11, 2004
Hi all,
I found my database log file is 26GB and the database file is just about 280MB. We are doing full backup everyday. However, my sql server seems running very slow now and please advise:
1. How can I decrease/truncate my log file?
2. Would the huge size of the log file be the reasons slowing up my sql server?
3. Would anyone give me direction knowing more on the transaction log?
Thank you and appreciated!
View 4 Replies
View Related
Oct 30, 2007
Hi guys, its my first post! Its also like my first time really diving into sql. We are using sharepoint on site here along with sql server 2005, one of our log files is 255 GBs and needs to be made smaller very fast!! We are almost out of disk space and the log is growing fast.
I am very new to sql and dont even know where to go to enter commands, so youll have to bear with me here. I've read about truncating and shrinking and some other things, I am just worried and dont want to mess anything up. I know this is probably a simple task, but like I said, with the truncate command I was reading about, I dont even know where to go to type it in!!! If someone could please help it would be much appreciated. Thanks so much.
View 14 Replies
View Related
Jan 28, 2008
I have a .bak file of 72gb. But my database size is only 32gb, I got this value from sp_spaceused?
Anyone know why the .bak file is so big?. Is it possible to reduce the size? How could i reduce it?
http://www.sqlserverstudy.com
View 9 Replies
View Related
Nov 13, 2007
Hi
this is regarding SQL Sever 2000. ( it was upgraded form sql7). its log file is increasing in very high manner. say 40 gb, 50 gb and now 57 gb. Mdf file is around 15 mb. we created back up and tried to restore to another system. its asking 57 gb free space. how to proceeed with file recovery. we have backups but it askes more space for log file. how to retrieve the data.
rgds
Pramod
View 3 Replies
View Related
Feb 9, 2007
I have a solution with a couple of ssis projects in it. Everytime I open the solution, Visual Studio creates an extra .database file for the project's existing xxx.database file. The solution is under VSS control and VS2005 checks out the project and shows the file as a newly added file.
What causes this and how can I prevent this from occurring?
The files contain this:
<Database xmlns:xsd="http://www.w3.org/2001/XMLSchema" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns:dwd="http://schemas.microsoft.com/DataWarehouse/Designer/1.0" dwd:design-time-name="d3ce9653-3ac5-4ee5-85c3-7d60b2e5f109" xmlns="http://schemas.microsoft.com/analysisservices/2003/engine">
<ID>Publish</ID>
<Name>Publish</Name>
<CreatedTimestamp>0001-01-01T00:00:00Z</CreatedTimestamp>
<LastSchemaUpdate>0001-01-01T00:00:00Z</LastSchemaUpdate>
<LastProcessed>0001-01-01T00:00:00Z</LastProcessed>
<State>Unprocessed</State>
<LastUpdate>0001-01-01T00:00:00Z</LastUpdate>
<DataSourceImpersonationInfo>
<ImpersonationMode>Default</ImpersonationMode>
<ImpersonationInfoSecurity>Unchanged</ImpersonationInfoSecurity>
</DataSourceImpersonationInfo>
</Database>
There are 2 of us working on the solution and the other fellow does not see this behavior.
The files do not show up in the VS2005 solution explorer. If they are user specific as I suspect (impersonation info?), then they should not be added to the .dtproj project file.
Thanks!
View 3 Replies
View Related
Sep 25, 2004
One of my databases is approxiamtely 1.5 GB's but when I run a backup the backup file explodes to 38GB's.
What is the proper usage of Shrink Database, is this safe or is there another method to reduce the size?
View 2 Replies
View Related
May 12, 2000
I am using SQL Server 7 and have about 5 databases. One of them has a data file of about 10 Meg, and most of the others are larger. I do a nightly backup to both a local and mapped drive. On both, the size of the backup file for this database is more than 500 Meg, but the rest appear to be an appropriate size. Does anyone know why this would be happening? The database works fine, it does not get a lot of insert/delete activity and I run DBCC every weekend. If anyone has any ideas I would sure like to hear from them.
View 1 Replies
View Related
Feb 21, 2001
I have a huge log file (285M) on SQL Server 7.
The database itself is about 10M.
How can I reduce the log file ?
Is it possible to build it again from scratch ?
I tried the Truncate Transaction Log but it didn't help.
View 2 Replies
View Related
Mar 13, 2002
I have a 2 GB text file(semicolon delimited), which I need to
pump into SQL 2K. What is the best way to achieve this in shortest time?
I tried with DTS BCP, it tooks 1 minute 14 secs to transfer 13457 records. This is only 1/2000 of the record i need to transfer. Please help.
Thanks.
regards,
Terry
View 2 Replies
View Related
May 22, 2013
I have an xml file of 44 Gb (Not Meg, its really GB) . Delivered by the Danish custom authorities.
My problem is simple - How to import such a beast? I have seen a limit of 2.1 Gb everywhere.
View 9 Replies
View Related
Feb 15, 2006
how to delete a very huge log file, to free up some harddisk space
View 3 Replies
View Related
May 19, 2007
Hello all,
I have inherited a SQL 2000 server, and am therefore an absolute beginner of SQL2000.
I know this has been covered before, but I don't know how to use the KB as I don't know how to run the commands/script.
http://support.microsoft.com/kb/272318/
I can no longer backup the SQL database because the 'transaction log backup' file is about 17GB. The SQL database is only about 2GB! The partition that it is backed up onto fills up every day.
Can anyone help me?
Thanks
Puskas
View 4 Replies
View Related
Nov 15, 1999
Help,
Have a user who needs many data changes to a huge table with over 2 million rows.
He hopes to do this by running a series of queries.
Last time our log grew gargantuan. I know 7.0 is wonderful & dynamic, but after
an hour of rambling through books online I'm more confused than ever.
Is there a way we can code into his query at intervals any hard coded checkpoints
or log backups? Do we need to dbcc shrinkfile the logfile?
Thanks,
Mark Blackburn
mark@mbari.org
View 1 Replies
View Related
Dec 16, 2004
I am writing a .NET service, and I need to insert files in a SQL DB for temporary storage.
I have never inserted a file into a SQL database before. I am thinking about using the image field type. Has anyone done this before? How did you do it?
View 9 Replies
View Related
Oct 31, 2015
I have a database data file almost at 2tb maxing out a windows drive. Only 16gb left. Should I just add another data file on another Windows drive for growth? Or just move current huge data file to a new GPT drive? Or do both adding another data file and moving existing to its own new GPT drive?
Primary objective is to make do for now.
View 1 Replies
View Related
Oct 31, 2005
I have the next question, and i would like to hear what do you thinkabout, and if is there a better solution for "my problem"here is the question, I have a huge table with 60GB of data (imagefiles). The problem happen always when i try to ALTER the structure ofthe table. For example I change a field char(3) to char(4)...thesqlserver then performs the "alter table" command...that must besomething similar than "insert into the new table + drop the actualtable" and for that I need about 60GB o space for my LOG file, andtakes hours to complete the operation.Is this the only way to alter a single field in my table??I would like to heard you opinions...Thanks..ALberto
View 2 Replies
View Related
Oct 22, 2015
I am copying data from database to an excel file through SSIS. database is MS SQL 2005 and BIDS is also 2005.However, the job doing this task fails every time.As per investigation, the result of the query is more than 100,000 rows and we know that excel has a limit of 65000 rows of data.Is there a way of setting the limit up? or something? a better approach maybe so that everything will be copied to the excel file successfully.
The PrimeOutput method on component "Source - Query" (1) returned error code 0xC02020C4. The component returned a failure code when the pipeline engine called PrimeOutput(). The meaning of the failure code is defined by the component, but the error is fatal and the pipeline stopped executing. There may be error messages posted before this with more information about the failure. End Error Error: 2015-10-22 04:34:58.05 Code: 0xC0047021
Source: Data Flow Task
Description: SSIS Error Code DTS_E_THREADFAILED.
Thread "SourceThread0" has exited with error code 0xC0047038. There may be error messages posted before this with more information on why the thread has exited. End Error DTExec: The package execution returned DTSER_FAILURE (1). Started: 4:30:00 AM Finished: 4:35:05 AM Elapsed: 304.844 seconds. The package execution failed. The step failed. "
View 1 Replies
View Related
Sep 20, 2007
Hi all,
I have a 400MB Excel file that I consume from another automated process (don't ask). I copy this file down locally to my server, and I am attempting to create an SSIS package that points to this file via a connection manager. My computer starts gobbling up massive amounts of memory (devenv.exe gets up to about 800MB or so, then drops back down to 100MB) even when I attempt to rename the connection in the connection managers tab.
I have set all BypassPrepare to TRUE and ValidateExternalMetadata properties to FALSE, and still it can take up to 3 to 6 minutes for BI Dev Studio to respond. My specs:
Intel Centrino Duo 2.00 GHz
2GB RAM
XP Pro SP2
There MUST be a way for me to work effectively on a file of this size. Please help! Thanks much for any assistance.
Sincerely,
Brian Pulliam
View 4 Replies
View Related
May 21, 2007
I have an 80 gig server. My client has sent me a backup of their db. The actual data file is 47 gigs. The client didn't truncate the transaction log, and on the backup , it is 37 gigs.
Question : when restoring this to my dev box, do I need to restore the transaction log, or, is there a way to truncate it so the 37 gigs isn't transferred to the server?
Thanks
View 11 Replies
View Related
Apr 3, 2000
SQL 7 SP1 NT4 SP5
I have a TRANSACTION table with 150 million rows.
I have a USER table.
Each user has about 600 records in the TRANSACTION table.
The TRANSACTION cluster index is on USERID + RECID . The second index is on USERID + Fieldx + Fieldy.
The TRANSACTION table gets about 1.4 million inserts in a normal day and about 40,000 updates.
I want to go through the USER table and delete all users who have not visited me in a while.
I want to do this without substantially hindering performance in a production environment. I can perform this over a week period or two if needed.
The best way I thought of doing this was to grab x amount of users in a cursor and loop through deleting their corresponding TRANSACTION records.
Does anyone have any ideas on a better way. What is going to happen to my indices during this time ?
Thanks !!!
View 3 Replies
View Related
Sep 2, 2007
Greetings, I have just arrived back into the country (NZ) and back into ASP.NET.
I am having trouble with the following:An attempt to attach an auto-named database for file (file location).../Database.mdf failed. A database with the same name exists, or specified file cannot be opened, or it is located on UNC share.
It has only begun since i decided i wanted to use IIS, I realise VWD comes with its own localhost, but since it is only temporary, i wanted a permanent shortcut on my desktop to link to my intranet page.
Anyone have any ideas why i am getting the above error? have searched many places on the internet and not getting any closer.
Cheers ~ J
View 3 Replies
View Related
Dec 16, 2007
Hello friends,
I am new to the SQL Server 2005 development.
From last 1 week or so, i have been facing very strange problem with my sql server 2005s database
which is configured and set on the hosting web server. Right now for managing my sql server 2005 database,
i am using an web based Control Panel developed by my hosting company.
Problem i am facing is that, whenever i try to modify (i.e. add new columns) tables in the database,
it gives me error saying that,
"There is already an object named 'PK_xxx_Temp' in the database. Could not create constraint. See previous errors.
Source: .Net SqlClient Data Provider".
where xxx is the table name.
I have done quite a bit research on the problem and have also searched on the net for solution but still
the problem persist.
Thanks in advance. Any help will be appreciated.
View 5 Replies
View Related
Jul 14, 2015
I have transnational replication setup on two environments, on one server distribution database is tiny, but on the second server the distribution database is 5 times bigger, and taking up lot of space, both environments have almost same size of data.
View 15 Replies
View Related
Mar 13, 2008
I am actually a newbie to asp.net and i m using ASP.net 3.5 i.e, VWD 2008. i am using it for the first time as my tool to develop a website for my final year project. i am planning to develop an online job recruitment site like www.monster.com. Rigth now i am confused how will i manage my database. i've learned to use databinding concept of SQL SERVER in VWD 2008 but will it be enough to handle such huge # of Job postings and employers and as well as Resumes in pdf of word format? or do i have to create a separate databse in SQL Server and to connect it with my website? i am confused at the moment. please help me in this matter.
Regards,
Jigzy
View 6 Replies
View Related
Jun 14, 2006
Hi.
I am trying to put a hugh chunk of text into my database for example information to a particular product which has more than 2000 characters. I had saw this datatype "nvarchar(MAX)" in SQL Server 2005 and was wondering if i can use this to store my text.
Thanks
View 1 Replies
View Related
