0

The Issue

I've been running a particularly large query, generating millions of records to be inserted into a table. Each time I run the query I get an error reporting that the transaction log file is full.

I've managed to get a test query to run with a reduced set of results and by using SELECT INTO instead of INSERT into as pre built table. This reduced set of results generated a 20 gb table, 838,978,560 rows.

When trying to INSERT into the pre built table I've also tried using it with and without a Cluster index. Both failed.

Server Settings

The server is running SQL Server 2005 (Full not Express). The dbase being used is set to SIMPLE for recovery and there is space available (around 100 gb) on the drive that the file is sitting on.

The transaction log file setting is for File Growth of 250 mb and to a maximum of 2,097,152 mb.

The log file appears to grow as expected till it gets to 4729 mb.

When the issue first appeared the file grow to a lower value however i've reduced the size of other log files on the same server and this appears to allow this transaction log file grow further by the same amount as the reduction on the other files.

I've now run out of ideas of how to solve this. If anyone has any suggestion or insight into what to do it would be much appreciated.

SQL_Deadwood
  • 491
  • 5
  • 14
MKC
  • 64
  • 6
  • 1
    check out https://technet.microsoft.com/en-us/library/dd425070(v=sql.100).aspx. there are things you can do to force SQL Server to bulk insert, which should decrease the number of T-log entries. – Jeremy Jan 03 '17 at 18:28

1 Answers1

0

First, you want to avoid auto-growth whenever possible; auto-growth events are HUGE performance killers. If you have 100GB available why not change the log file size to something like 20GB (just temporarily while you troubleshoot this). My policy has always been to use 90%+ of the disk space allocated for a specific MDF/NDF/LDF file. There's no reason not to.

If you are using SIMPLE recovery SQL Server is supposed manage the task of returning unused space but sometimes SQL Server does not do a great job. Before running your query check the available free log space. You can do this by:

  1. right-click the DB > go to Tasks > Shrink > Files.
  2. change the type to "Log"

enter image description here

This will help you understand how much unused space you have. You can set "Reorganize pages before releasing unused space > Shrink File" to 0. Moving forward you can also release unused space using CHECKPOINT; this may be something to include as a first step before your query runs.

Alan Burstein
  • 7,770
  • 1
  • 15
  • 18
  • Thanks for the advice. I've already had a go at Shrinking the log file to 0, with no luck. I've run CHECKPOINT (twice in a row) which I saw should reset it, also with no luck. The issue is mainly that the log appears not to grow further than 4 - 5 Gb for some reason. I'm thinking that although I have around 100Gb showing free, it is being used in someway so the log can take it. – MKC Jan 04 '17 at 11:29
  • No, I read that there was no advantage if they were on the same drive, which in my case they would be. I may have a go though. Thanks for the suggestion. – MKC Jan 05 '17 at 15:40