How do I reduce transaction log backup size after a full backup?

Question

I have three maintenance plans set up to run on an Sql Server 2005 instance:

• Weekly database optimisations followed by a full backup
• Daily differential backup
• Hourly transaction log backups

The hourly log backups are usually between a few hundred Kb and 10Mb depending on the level of activity, daily differentials usually grow to around 250Mb by the end of the week, and the weekly backup is about 3.5Gb.

The problem I have is that the optimisations before the full backup seem to be causing the next transaction log backup to grow to over 2x the size of the full backup, in this case 8Gb, before returning to normal.

Other than BACKUP LOG <DatabaseName> WITH TRUNCATE_ONLY, is there any way to reduce the size of that log backup, or prevent the optimisations from being recorded in the transaction log at all, as surely they will be accounted for in the full backup they precede?

Answer 1

Some interesting suggestions here, which all seem to show misunderstanding about how log backups work. A log backup contains ALL transaction log generated since the previous log backup, regardless of what full or differential backups are taken in the interim. Stopping log backups or moving to daily full backups will have no effect on the log backup sizes. The only thing that affects the transaction log is a log backup, once the log backup chain has started.

The only exception to this rule is if the log backup chain has been broken (e.g. by going to the SIMPLE recovery model, reverting from a database snapshot, truncating the log using BACKUP LOG WITH NO_LOG/TRUNCATE_ONLY), in which case the first log backup will contain all the transaction log since the last full backup - which restarts the log backup chain; or if the log backup chain hasn't been started - when you switch into FULL for the first time, you operate in a kind of pseudo-SIMPLE recovery model until the first full backup is taken.

To answer your original question, without going into the SIMPLE recovery model, you're going to have to suck up backing up all the transaction log. Depending on the actions you're taking, you could take more frequent log backups to reduce their size, or do more targeted database.

If you can post some info about the maintenance ops you're doing, I can help you optimize them. Are you, by any chance, doing index rebuilds followed by a shrink database to reclaim the space used by the index rebuilds?

If you have no other activity in the database while the maintenance is occuring, you could do the following:

• make sure user activity is stopped
• take a final log backup (this allows you to recover right up to the point of maintenance starting)
  • switch to the SIMPLE recovery model
  • perform maintenance - the log will truncate on each checkpoint
  • switch to the FULL recovery model and take a full backup
  • continue as normal

Hope this helps - looking forward to more info.

Thanks

[Edit: after all the discussion about whether a full backup can alter the size of a subsequent log backup (it can't) I put together a comprehensive blog post with background material and a script that proves it. Check it out at https://www.sqlskills.com/blogs/paul/misconceptions-around-the-log-and-log-backups-how-to-convince-yourself/]

Answer 2

You could shrink them, but they will just grow again, eventually causing disk fragmentation. Index rebuilds and defrags make very large transaction logs. If you don't need point-in-time recoverability, you could change to Simple recovery mode and do away with the transaction log backups entirely.

I'm guessing you're using a maintenance plan for the optimizations, you could change it to use a script that does index defrags only when a certain fragmentation level is reached and you would not likely suffer any performance hit. This would generate much smaller logs.

I would skip daily differentials in favor of daily full backups BTW.

Answer 3

Your final question was: "Other than BACKUP LOG WITH TRUNCATE_ONLY, is there any way to reduce the size of that log backup, or prevent the optimisations from being recorded in the transaction log at all, as surely they will be accounted for in the full backup they precede?"

No, but here's a workaround. If you know that the only activities in that database at that time will be the index maintenance jobs, then you can stop transaction log backups before the index maintenance starts. For example, some of my servers on Saturday nights, the job schedules look like this:

• 9:30 PM - transaction log backup runs.
• 9:45 PM - transaction log backup runs for the last time. The schedule stops at 9:59.
• 10:00 PM - index maintenance job starts and has built-in stops to finish before 11:30.
• 11:30 PM - full backup job starts and finishes in under 30 minutes.
• 12:00 AM - transaction log backups start again every 15 minutes.

That means I don't have point-in-time recoverability between 9:45 and 11:30pm, but the payoff is faster performance.

Answer 4

Easy answer: Change your weekly optimization job to run in a more balanced manner on a nightly basis. i.e. re-index tables a-e on Sunday night, f - l on Monday night etc... find a good balance, your log will be roughly 1/6th of the size on average. Of course this works best if you aren't using the built-in ssis index maintenance job.

The downside to this and it's significant depending on the load your db experiences is that it wreaks havoc with the optimizer and the re-use of query plans.

But if all you care about is the size of your t-log on a weekly basis, split it up from day to day or hour to hour and run the t-log backups in-between.

Answer 5

You might also look into a third party tool (Litespeed from Quest, SQL Backup from Red Gate, Hyperbac) to reduce the sizes of the backups and logs. They can pay for themselves quickly in tape savings.

Answer 6

It can probably be assumed that your "optimizations" include index rebuilds. Only performing these tasks on a weekly basis may be acceptable on a database that does not encounter a great deal of updates and inserts, however if your data is highly fluid you may want to do a couple of things:

1. Rebuild or reorganize your indexes nightly if your schedule permits and if the impact is acceptable. When performing these nightly index maintenance tasks target only those indexes that are fragmented beyond say 30% for rebuilds and in the range of 15-30% for reorgs.

2. These tasks are logged transactions, so if you're concerned about log growth then I would advocate what Paul recommended. Final transaction log backup prior to index maintenance, switch to Simple recovery, followed by the maintenance process and then switch back to Full recovery followed by a Full data backup should do the trick.

I take a zen-like approach to my log files: they are the size they want to be. So long as they've not endured abberant growth due to poor backup practices in comparison to database activity that is the mantra I live by.

As for scripts that perform the discretionary index maintenance look online: there are a ton out there. Andrew Kelly published a decent one in SQL Magazine about a year ago. SQLServerPedia has some scripts from Michelle Ufford, and the latest issue of SQL Magazine (July 2009 I believe) has a full article on the topic as well. Point is to find one that works well for you and make it your own with minimal customizations.

Answer 7

Can you specially backup your transaction log at various points during your database optimization? The total size of the t-logs would be the same, but each one would be smaller, possibly helping you in some way.

Can you do more targeted database optimization so fewer transactions are created (someone mentioned this but I'm not sure the implications were spelled out). Such as tolerating a certain amount of fragmentation or wasted space for a while. If 40% of your tables are only 5% fragmented, not touching them could save quite a bit of activity.