Announcement

Collapse
No announcement yet.

LTO3 throughput rate varies between jobs

Collapse
X
  • Filter
  • Time
  • Show
Clear All
new posts

  • LTO3 throughput rate varies between jobs

    Hi Folks

    I recently installed a Quantum LTO3 HH stand-alone tape drive on our W2k3 file server. It is used to backup data via Arcserve 12.

    Usually, the average throughput will be 800MB per minute but other times it will be 400MB per minute. For example, I backup all our data to tape every night. The same job is run through Arcserve and does not change from one day to the next. Two days ago the throughput was 800MB, last night it was 400MB.

    Has anyone else experienced this where throughput has halved?

    There is nothing to configure in Arcserve, it just detects the drive and sends data to it. I've had a look on Quantum's support site for information about this but cannot find anything relating to variable throughput between jobs.

    I've been using the drive for less than two weeks and this has happened twice now. The problem is that the time taken to complete the backup doubles. Usually a full backup will complete in 6.5hrs. When it takes 13hrs it begins to affect server performance as the backup is active while staff are accessing files.

    Any help or suggestions will be appreciated.

    Thanks!
    A recent poll suggests that 6 out of 7 dwarfs are not happy

  • #2
    Re: LTO3 throughput rate varies between jobs

    Fragmentation, other backup jobs running, other services/events happening at the same time would be where I would start.
    cheers
    Andy

    Please read this before you post:


    Quis custodiet ipsos custodes?

    Comment


    • #3
      Re: LTO3 throughput rate varies between jobs

      What's the size of the backup job when the time doubles? Does the amount of data being backed up double as well?

      Comment


      • #4
        Re: LTO3 throughput rate varies between jobs

        Thanks to both of you for your replies.

        I have looked at the events that are recorded during the backup and there is nothing logged during the 4 hours after the backup starts. The backup starts at 10pm. WSUS3 is also installed on the machine and is scheduled to poll Microsoft's site for updates between 2 and 3am. It is scheduled to deliver updates at 6am each Thursday morning. The only other program running is our mail server: Mercury/32 which uses negligible resources.

        There are no other backup jobs running. I run a single GFS job that backs up all our data every night. The size of the backup is 315GB and this size is fairly consistent over the backups that have completed so far including the two occassions when throughput has dropped to 400MB per minute.

        I have run a Windows defragmentation analysis over the data drive (4 drives in a RAID 5 configuration), and it reports that the drive does not need to be defragmented.

        I don't understand why the throughput would change. Both times this has happened, it has dropped by one half.

        Thanks again.
        A recent poll suggests that 6 out of 7 dwarfs are not happy

        Comment


        • #5
          Re: LTO3 throughput rate varies between jobs

          I would suspect WSUS. Can you monitor the job and if it drops to 400Mb per minute try shutting down WSUS to see if it picks back up?

          Comment


          • #6
            Re: LTO3 throughput rate varies between jobs

            Originally posted by joeqwerty View Post
            I would suspect WSUS. Can you monitor the job and if it drops to 400Mb per minute try shutting down WSUS to see if it picks back up?
            Thank you.

            What I will do is I will stop the WSUS web site before I leave work and then re-start it after the job finishes the next morning. I will re-schedule the polling period for mid-day so I don't miss any updates. If the throughput changes when WSUS is stopped I'll post back (and post back if it doesn't...)

            This will probably take a couple of weeks to test.

            Cheers!
            A recent poll suggests that 6 out of 7 dwarfs are not happy

            Comment


            • #7
              Re: LTO3 throughput rate varies between jobs

              Sounds like a plan. Keep us posted.

              Comment


              • #8
                Re: LTO3 throughput rate varies between jobs

                I had stopped the WSUS web sites (Administration and Shares) and also stopped the Update Services service, but the issue occured again last night.

                I have gone through Arcserve's logs and have checked the Windows event logs but cannot see anything that might cause this to happen.

                If anyone has any ideas they will be gratefully received. In the meantime, I will contact Quantum tech support and Arcserve tech support and see if they can help resolve this.
                A recent poll suggests that 6 out of 7 dwarfs are not happy

                Comment


                • #9
                  Re: LTO3 throughput rate varies between jobs

                  Anti-virus?

                  One thing I liked about Symantec was it had a "ignore scanning files on backup"

                  ESET didn't have this so I had to turn it off at times. It made a 40-50% difference on how long it took the backup.

                  One other thing to look at... I don't' know Arcserve but in Backup Exec the logs would show how long each volume, database or share took to backup. You could check to see its a particular set of data that is causing the slowdown.

                  Comment


                  • #10
                    Re: LTO3 throughput rate varies between jobs

                    Originally posted by Garen View Post
                    Anti-virus?

                    One thing I liked about Symantec was it had a "ignore scanning files on backup"

                    ESET didn't have this so I had to turn it off at times. It made a 40-50% difference on how long it took the backup.

                    One other thing to look at... I don't' know Arcserve but in Backup Exec the logs would show how long each volume, database or share took to backup. You could check to see its a particular set of data that is causing the slowdown.
                    Hi Garen, thanks for your reply.

                    I have Sophos installed on the system and it is configured to scan executable and infectable files only. I don't think it is the cause of the problem but I will have a good look at the configuration settings and see if there is anything I can change.

                    Thanks.
                    A recent poll suggests that 6 out of 7 dwarfs are not happy

                    Comment


                    • #11
                      Re: LTO3 throughput rate varies between jobs

                      Is Sophos set for on-access scanning or doing a scheduled scan? Also is this a recent issue? We had a Sophos update apply a month or so back that caused havoc with a customers backup. It enabled on-access scanning and killed the job rates all over the place.
                      BSc, MCSA: Server 2008, MCSE, MCSA: Messaging, MCTS
                      sigpic
                      Cruachan's Blog

                      Comment


                      • #12
                        Re: LTO3 throughput rate varies between jobs

                        Originally posted by cruachan View Post
                        Is Sophos set for on-access scanning or doing a scheduled scan? Also is this a recent issue? We had a Sophos update apply a month or so back that caused havoc with a customers backup. It enabled on-access scanning and killed the job rates all over the place.
                        Thanks for the reply.

                        We've been using Sophos for the last 2 years with the same settings. On-access scanning is set to normal and zip files are ignored, HIPS is not active as that throws up too many false positives. Scheduled scans occur every night - they scan the entire computer and start 3 hours before the backup starts.

                        I've just checked the scan log for the scheduled scan and discovered it was taking *hours*. I suspect that part of the reason may be due to the fact that we have lots of password protected zip files - unfortunately, Sophos is not the brightest of players when it comes to encrypted files and will constantly try and try and try to open them, spamming the logs as it does so with an event for each error about not being able to access the same file because it is encrypted. I've just turned zip file scanning off for the scheduled scan. I digress....

                        What is strange is that the MB/per second is see-sawing:



                        This is the first time I have used an LTO3 drive, and it was installed 2 weeks ago.

                        I will be contacting Quantum/CA today to see if they have any suggestions.

                        Cheers!
                        A recent poll suggests that 6 out of 7 dwarfs are not happy

                        Comment


                        • #13
                          Re: LTO3 throughput rate varies between jobs

                          Two things occur to me, the second of which is only slightly related:

                          1. I'm assuming you're rotating your backup tapes. Could the tapes be the problem? Do you keep the backup job logs? If so, you could check to see if the slow backups are occurring when certain tapes are used.

                          2. If you have a clean system, and your AV software has some type of "resident shield" component for "real time" scanning, why do you run a nightly AV scan? Theoretically your systems should never get infected. Have you thought about scheduling a weekly instead of a daily scan? Maybe you could push the scan out to the weekend at a time that backups are not running and where a scan that takes hours won't run into the backups.

                          Comment


                          • #14
                            Re: LTO3 throughput rate varies between jobs

                            Originally posted by joeqwerty View Post
                            Two things occur to me, the second of which is only slightly related:

                            1. I'm assuming you're rotating your backup tapes. Could the tapes be the problem? Do you keep the backup job logs? If so, you could check to see if the slow backups are occurring when certain tapes are used.
                            Good thought. Unfortunately, I just checked and the lower throughput is not restricted to particular tapes - each of the tapes have been used when both speeds have been seen.

                            Originally posted by joeqwerty View Post
                            2. If you have a clean system, and your AV software has some type of "resident shield" component for "real time" scanning, why do you run a nightly AV scan? Theoretically your systems should never get infected. Have you thought about scheduling a weekly instead of a daily scan? Maybe you could push the scan out to the weekend at a time that backups are not running and where a scan that takes hours won't run into the backups.
                            Quite right. I will change that. This system has 'evolved' and nightly scanning is a left-over from less 'security conscious' times.

                            Thanks again for the help and suggestions.
                            A recent poll suggests that 6 out of 7 dwarfs are not happy

                            Comment


                            • #15
                              Re: LTO3 throughput rate varies between jobs

                              Well this is a tough one. As Garen stated earlier, if the job logs give you details on each server that's being backed up maybe you can look at them to determine which server or servers it's hanging up on.

                              Comment

                              Working...
                              X