Announcement

Collapse
No announcement yet.

Exchange log files are very large - 45GB total - Is this bad?

Collapse
X
  • Filter
  • Time
  • Show
Clear All
new posts

  • Exchange log files are very large - 45GB total - Is this bad?

    I have a sbs 2011 environment and the log files for exchange are massive - the database itself is around 45GB and the log files are around the same size.

    I run two backups on the server - windows backup which runs every night, and also backup assist backup which backups up the exchange database once per week.

    I have had a problem recently where backup assist keeps failing when trying to back up exchange - it produces an error that pertains to the VSS writers timing out.
    Backup assist have investigated and asked me to increase the VSS timeout in teh registry - I have done this to no avail.

    They said its a windows problem of some sort.

    I have been trying to dig deeper into the issue which is when I came across the large log files, and wondered if that might be something to do with the problem.

    I have read a microsoft tech note on circular loggin and wonder if thats what I need to impliment, but I dont fully understand the implications.

    http://msdn.microsoft.com/en-us/libr...XCHG.140).aspx

    Can someone give me some pointers?
    thanks
    David Silvester
    Systems Administrator

  • #2
    Re: Exchange log files are very large - 45GB total - Is this bad?

    Exchange server has a number of logs. I'm assuming that you mean the transaction logs, correct?

    If so, then you need to perform a successful backup of Exchange with an Exchange aware backup program that can flush the transaction logs. I'm fairly certain that a Full VSS backup with Windows Server Backup will do this as long as you have the volume where the Exchange mailbox database and log files are located selected in Windows Server Backup.

    As for whether or not it's "bad" to have 45GB of log files, that's dependent upon your server. My Exchange server regularly creates 45GB worth of log files every day and that's normal for us. Our logs get flushed with our daily backups. There shouldn't be any performance impact of having that volume of log files but the risk is that you'll run out of disk space on the log file drive if you're not regularly flushing the log files.

    As for enabling circular logging, in smaller environments where they don't want to deal with transaction log management I often see circular logging enabled. It allows Exchange to automatically flush the transaction logs in order to "re-use" the logs negating the need to flush them with your backup. This avoids the possibility of running out of log file disk space, which will shut Exchange down. The risk of using circular logging is that you can only restore Exchange from the latest Full backup. Restoration from incremental (log) backups that occurred since the last full backup are not possible, so some data loss is possible.

    Comment


    • #3
      Re: Exchange log files are very large - 45GB total - Is this bad?

      Originally posted by joeqwerty View Post
      Exchange server has a number of logs. I'm assuming that you mean the transaction logs, correct?

      If so, then you need to perform a successful backup of Exchange with an Exchange aware backup program that can flush the transaction logs. I'm fairly certain that a Full VSS backup with Windows Server Backup will do this as long as you have the volume where the Exchange mailbox database and log files are located selected in Windows Server Backup.

      As for whether or not it's "bad" to have 45GB of log files, that's dependent upon your server. My Exchange server regularly creates 45GB worth of log files every day and that's normal for us. Our logs get flushed with our daily backups. There shouldn't be any performance impact of having that volume of log files but the risk is that you'll run out of disk space on the log file drive if you're not regularly flushing the log files.

      As for enabling circular logging, in smaller environments where they don't want to deal with transaction log management I often see circular logging enabled. It allows Exchange to automatically flush the transaction logs in order to "re-use" the logs negating the need to flush them with your backup. This avoids the possibility of running out of log file disk space, which will shut Exchange down. The risk of using circular logging is that you can only restore Exchange from the latest Full backup. Restoration from incremental (log) backups that occurred since the last full backup are not possible, so some data loss is possible.
      Thanks for the info, its not something I fully understood.

      So in its default mode of operation, log files will be kept UNTIL a full backup is done, then they will be pruned?

      I am doing full windows backups (not SBS backups) so not sure why this isnt happening.

      As it happens, right after I posted this, the backup assist backup completed successfully!

      Anyway, I will log in tonight and do some investigations based on what you have told me, then I will post back.
      David Silvester
      Systems Administrator

      Comment


      • #4
        Re: Exchange log files are very large - 45GB total - Is this bad?

        Correct, log files will build up until an Exchange Aware backup is done
        (not all backup software is suitable)

        Full windows backups should do this, as should Backup Assist - but IIRC you need to make sure you are specifying the Exchange stores for backup (their support is very good if you have problems)
        Tom Jones
        MCT, MCSE (2000:Security & 2003), MCSA:Security & Messaging, MCDBA, MCDST, MCITP(EA, EMA, SA, EDA, ES, CS), MCTS, MCP, Sec+
        PhD, MSc, FIAP, MIITT
        IT Trainer / Consultant
        Ossian Ltd
        Scotland

        ** Remember to give credit where credit is due and leave reputation points where appropriate **

        Comment


        • #5
          Re: Exchange log files are very large - 45GB total - Is this bad?

          Originally posted by Ossian View Post
          Correct, log files will build up until an Exchange Aware backup is done
          (not all backup software is suitable)

          Full windows backups should do this, as should Backup Assist - but IIRC you need to make sure you are specifying the Exchange stores for backup (their support is very good if you have problems)

          Within Backup Assist you can (and obviously have to) specify to clear the logs on a successful backup. Im assuming that if the OP is doing a full exchange backup with backup exec they couldn't have stipulated to flush the transactional logs.

          Comment


          • #6
            Re: Exchange log files are very large - 45GB total - Is this bad?

            Originally posted by 5habbaranks View Post
            Within Backup Assist you can (and obviously have to) specify to clear the logs on a successful backup. Im assuming that if the OP is doing a full exchange backup with backup exec they couldn't have stipulated to flush the transactional logs.
            Thanks, thats led me to a possible problem - we only have the Rsync version of backup assist, it does still backup exchange, but I have a feeling that it does not give the option to prune the exchange logs, does that sound right?

            Also, windows backup is set to do a full server backup, which includes the exchange drive, how come that is not pruning logs? Any ideas?
            David Silvester
            Systems Administrator

            Comment


            • #7
              Re: Exchange log files are very large - 45GB total - Is this bad?

              Hi guys,

              I spoke to backup assist and they said there is an option to prune the exchange logs by selecting full VSS instead of copy.

              I understand that also windows backup does a VSS copy by default and doesnt prune logs, I found this article about it:

              http://social.technet.microsoft.com/...=windowsbackup

              I followed that on the server, and got the option to select Full VSS backup, however, it advises that this should be done only if no other backup programs are being used - which is not true in my case, as Im using windows backup and backup assist.

              Just wopndering if there are any implications in what I am going to do - basically, it will be as follows:

              backup assist - backs up data and exchange every night at 8pm - this is off site backup.

              windows backup - backs up full server, data exchange etc every morning at 4am - this is local backup, just for failsafe, and easy restore - Im going to set this to do full VSS backup and prune logs.

              Will that be ok?
              David Silvester
              Systems Administrator

              Comment


              • #8
                Re: Exchange log files are very large - 45GB total - Is this bad?

                I think I buried my question in too much fluff!

                Is there any danger in pruning the logs with Windows Backup? It warns me I shouldn't do it if I have other backups running, not sure why??
                David Silvester
                Systems Administrator

                Comment


                • #9
                  Re: Exchange log files are very large - 45GB total - Is this bad?

                  What I used to do is cleared Exchange log file manually. If you know your full backup is successfull then you can copy the log file with anything older than 1 week to a backup location then delete the logs file.

                  You can keep the copy logs for as long as you need or delete the oldest when you know its safe.

                  Comment


                  • #10
                    Re: Exchange log files are very large - 45GB total - Is this bad?

                    For <deity> sake, do NOT delete or move log files manually - Exchange will be very unhappy!

                    A backup which prunes the logs will be fine - the warning is because it will set audit bits and generally confuse other backup software and make recovery more awkward, but it will not in itself cause any problems
                    Tom Jones
                    MCT, MCSE (2000:Security & 2003), MCSA:Security & Messaging, MCDBA, MCDST, MCITP(EA, EMA, SA, EDA, ES, CS), MCTS, MCP, Sec+
                    PhD, MSc, FIAP, MIITT
                    IT Trainer / Consultant
                    Ossian Ltd
                    Scotland

                    ** Remember to give credit where credit is due and leave reputation points where appropriate **

                    Comment


                    • #11
                      Re: Exchange log files are very large - 45GB total - Is this bad?

                      Originally posted by Ossian View Post
                      For <deity> sake, do NOT delete or move log files manually - Exchange will be very unhappy!

                      A backup which prunes the logs will be fine - the warning is because it will set audit bits and generally confuse other backup software and make recovery more awkward, but it will not in itself cause any problems
                      I agreed with you on this post...
                      The reason I did that on my old exchange server because we had an unrealiable backup server etc..

                      I used Symantec Backup.. It h as the option to flush Exchange logs after a complete full backup...

                      Comment


                      • #12
                        Re: Exchange log files are very large - 45GB total - Is this bad?

                        Thanks guys.
                        I just read through this Microsoft article on Exchange 2010 log files so that I could understand it a bit better:

                        http://technet.microsoft.com/en-us/l...xchg.141).aspx

                        Humannetwork, in the article it does say that after you successfully discount the databases, the log files are no longer needed and can be safely deleted, so thats an option, but I gather that its probably not the recommended route.

                        Ossian, thanks for the heads up, that makes sense, I will configure SBS to do a full backup AND prune the log files.

                        Out of interest, how are the logs pruned - ie how does the backup software know that all of the logs have been replayed already?
                        I'm just wondering, for example, if the backup failed mid way through, would it matter that it had already pruned the logs?
                        David Silvester
                        Systems Administrator

                        Comment


                        • #13
                          Re: Exchange log files are very large - 45GB total - Is this bad?

                          logs get pruned at the end of the backup process, using the checkpoint file to indicate which logs are no longer needed.

                          This is done automatically by an "Exchange Aware" backup application
                          Tom Jones
                          MCT, MCSE (2000:Security & 2003), MCSA:Security & Messaging, MCDBA, MCDST, MCITP(EA, EMA, SA, EDA, ES, CS), MCTS, MCP, Sec+
                          PhD, MSc, FIAP, MIITT
                          IT Trainer / Consultant
                          Ossian Ltd
                          Scotland

                          ** Remember to give credit where credit is due and leave reputation points where appropriate **

                          Comment


                          • #14
                            Re: Exchange log files are very large - 45GB total - Is this bad?

                            Originally posted by Ossian View Post
                            logs get pruned at the end of the backup process, using the checkpoint file to indicate which logs are no longer needed.

                            This is done automatically by an "Exchange Aware" backup application
                            Thanks very much.
                            David Silvester
                            Systems Administrator

                            Comment


                            • #15
                              Re: Exchange log files are very large - 45GB total - Is this bad?

                              if you have two backup products pruning logs, they'll confuse the logs if you need to recover.

                              it's ok to use two programs - you just need to make sure only one is responsible for truncating/pruning the logs.
                              Please do show your appreciation to those who assist you by leaving Rep Point https://www.petri.com/forums/core/im.../icon_beer.gif

                              Comment

                              Working...
                              X