2

This may be a very basic question, but how can you determine beforehand whether a large operation will end up using database log or tempdb space?

For instance, one large insert / update operation I did used the database log to a point where we needed to employ SSIS & bulk operations just so the space wouldn't run out, because all the changes in the script had to be deployed at one time.

So now I'm working with a massive delete operation, that would fill the log 10 times over. So I created a script to check the space used by the database log file and delete the rows in smaller batches, with the idea that once the log file was large enough, the script would abort and then continue from that point the next day (allowing normal usage to continue till the next backup, without risk of the log running out of space).

Now, instead of filling the log, the latter query started filling up tempdb. Tempdb data file, not log file, to be specific. So I'm thinking there's a huge hole where my understanding of these two should be. :)

Thanks for any advice!

Edit:

To clarify, the question here is that why does the first example use database log, while the latter uses tempdb data file, to store the changes? And in general, by which logic are DML operations stored to either tempdb or log? Normally log should store all DB changes while tempdb is only used to store the processed data during operation when explicitly requested (ie, temp objects) or when the server runs out of RAM, right?

Kahn
  • 1,630
  • 1
  • 13
  • 23
  • Does your table has index? Before any bulk operation you should drop index then recreate after the bulk operation. – vasin1987 Jan 23 '14 at 11:08
  • Hey, edited the question. The bulk operations work fine and the indexes there aren't a concern here. The current problem is the deletion that cannot be handled with a bulk operation, which for some reason is using up tempdb space instead of db log as I thought it should. :) – Kahn Jan 23 '14 at 11:16

1 Answers1

2

There is actually quite a bit that goes on behind the scenes when deleting records from a table. This MSDN Blog link may help shed some light on why tempdb is filling up when you try and delete. Either way, the delete will fill up the transaction logs as well, it just sounds like tempdb is filling up before it gets to the step of logging the transaction(s).

I'm not entirely sure what your requirements are, but the following links could be somewhat enlightening on your transaction logging issues. These are all set for SQL Server 2008 R2, but you can switch to whatever version you are running.

Recovery Model Overiew

Considerations for Switching from the Simple Recovery Model

Considerations for Switching from the Full or Bulk-Logged Recovery Model

You also have the option of truncating the table, but that depends on a few things. If you don't need the operation to be logged and you're deleting all the records from the table you can truncate. If you are doing some sort of conditional delete, but you're deleting more than you're keeping, you could always insert all of the records you want to keep into another "staging" table and then truncate the original. Then you can re-insert the records into the staging table. However, that really only works when you have no foreign key relationships on that table.

Michael
  • 599
  • 6
  • 11
  • Thanks for the comment! And yeah, truncation isn't an option here. The table is insanely large, and we'll only be deleting like 30% of it. Looks like what I need to do is to include tempdb and check them both as I'm deleting the data. – Kahn Jan 27 '14 at 08:00