Announcement

Collapse
No announcement yet.

Server will hesitate for breif periods then OK

Collapse
X
  • Filter
  • Time
  • Show
Clear All
new posts

  • Server will hesitate for breif periods then OK

    I have a Windows Server 2003 on a Dell Poweredge SC1430. Current enviorment is as follows:

    Single server enviorment
    Windows Server 2003 R2 SP2
    Updates applied
    Dell PowerEdge SC1430
    RAID 1
    SAS5/iR
    Gigabit switch - 24 port
    About 17 users

    Up to about 2 weeks ago, everything was running fine. Files were flowing, system seem to be running as expected. However, since then, there are these random delays when in or opening a file (on server). Basically, you could be running fine, then all of a sudden your stuck with the "Windows ciricle" waiting for a response. Doesn't matter what you do, opening or saving a file. We also have a "core" program that runs on Borland Db, that could quickly lock up or maybe just a 2 minute wait til I'm ready.

    Not sure where to go. Here is what I have tried.....

    Though a disk failure - replaced hard drives. Both drives and setup new RAID 1
    Made sure all updates are applied to OS
    Updated Firmware and driver for SAS Controller
    Updated Firmware and driver for NIC
    Replaced network switch with new Gigabit 24 port.
    Talked to Dell support and ran online tests - all hardware seem to pass.
    Checked task manager for CPU utilization when slow - usage was low
    Check Disk I/O when slow - suggestions welcome but it didn't look out of wack to me.
    Full virus Scan using Symantec End Point - both server and workstation
    Full Malware scan using Malware bytes and Spybot - on server

    Not sure what else to look at. to many more parts and I may as well just replace the whole thing.

    Thanks in advance

  • #2
    Re: Server will hesitate for breif periods then OK

    During these slow times have you also run a perfmon?

    I would check average disk queue length, % idle time, disk transfers/sec. If the idle time drops to 0 then the disk is working very hard. If the queue rises but the transfers are still low then there's probably something wrong with the disks or controller.

    Also, instead of task manager use process monitor from www.sysinternals.com as this will give you a lot more information to work with.

    It sounds like you've done a good bit of hardware updates but I would also check for BMC, chipset, and BIOS updates as well as hard drive firmware updates.

    But the SC1430 is getting long in the tooth. If possible I would get a new server and migrate to at least Windows 2008 R2. If your LOB applications require a 32-bit OS then I would look at maybe running a few VMs. One for your main server running 2008 R2 and one for your LOB app running Windows 2003 R2. I would also use Server 2012 for the host OS as this will give you the best performance for the VMs. Also, with a Server 2012 standard license you get rights for two VMs without getting additional licenses.
    Regards,
    Jeremy

    Network Consultant/Engineer
    Baltimore - Washington area and beyond
    www.gma-cpa.com

    Comment


    • #3
      Re: Server will hesitate for breif periods then OK

      Thanks,

      I'll check this out and report back. I want to replace the server, but they would like to see if I can fix it. Tight cash flow and all.

      I'll let you know shortly.

      Comment


      • #4
        Re: Server will hesitate for breif periods then OK

        Sounds good.

        I mentioned Process Monitor in my post above and I meant to say Process Explorer. (both are part of the Sysinternals suite)
        Regards,
        Jeremy

        Network Consultant/Engineer
        Baltimore - Washington area and beyond
        www.gma-cpa.com

        Comment


        • #5
          Re: Server will hesitate for breif periods then OK

          Disable the antivirus - I bet it's doing some real-time scanning of some files that it shouldn't be.
          Regards,

          Lee

          Comment


          • #6
            Re: Server will hesitate for breif periods then OK

            Thanks for the suggestion, but I (for a brief period) fully uninstalled the Antivirus. Hope to get Process Explorer on the box today and post back later

            Comment


            • #7
              Re: Server will hesitate for breif periods then OK

              Ok, So I was watching Performance and compared it to a server at another site. All looks good. There were mainly short peaks of max disk use or buffer, etc. but nothing like an extended or seemed excessively high utilization.

              Also, I went out and pulled down the Dell 32 bit Diag, booted off CD and ran all tests. All hardware passed the test.

              Could this be something within Windows? An update or something off in the OS?

              Comment


              • #8
                Re: Server will hesitate for breif periods then OK

                It certainly could be something wrong with the OS.

                Have you checked for the firmware updates I mentioned?
                Regards,
                Jeremy

                Network Consultant/Engineer
                Baltimore - Washington area and beyond
                www.gma-cpa.com

                Comment


                • #9
                  Re: Server will hesitate for breif periods then OK

                  Yes, all firmware and drivers on the BMC, chipset, NIC, SAS, drives, and bios have been updated to the latest version per Dell support site.

                  Comment


                  • #10
                    Re: Server will hesitate for breif periods then OK

                    Ok, change from previous post. as of 4:12 CST, the Proformance Monitor now has spiked and staying quite high:

                    Duration 1:40

                    Pages/sec - avg = around 670.xxx
                    Disk read time - avg = 35.xxx
                    Processor - stays low

                    My Pages per sec, average queue length, %disk read time, and avg disk read queue length all hit the roof and stayed there for what I would guess would be at least 5 solid minutes. Of course this made the Disk Idle time drop to 0 while all this was going on. Now it is back to normal.

                    Am I back to the SAS5/iR card? I created a new RAID and imaged the drives. Hard Drives are brand new

                    -------------- update - just so all the info is there...

                    I went in and checked the Pagefile size for each drive and it was set pretty low. I changed it to system managed for C and D and although I get a few and far between spike, it seems much better. And by spike, I mean it occasionally hits between 40 - 60. The others have all seem to calm down as well. My most active is still Avg Disk Queue length - showing an average of +/- 0.44.
                    Last edited by jllkc3; 13th December 2012, 23:44. Reason: More Info

                    Comment


                    • #11
                      Re: Server will hesitate for breif periods then OK

                      If the average disk Queue Length is around 0.44 its fine.
                      Did you see what process was causing this issue in the first place?
                      Marcel
                      Technical Consultant
                      Netherlands
                      http://www.phetios.com
                      http://blog.nessus.nl

                      MCITP(EA, SA), MCSA/E 2003:Security, CCNA, SNAF, DCUCI, CCSA/E/E+ (R60), VCP4/5, NCDA, NCIE - SAN, NCIE - BR, EMCPE
                      "No matter how secure, there is always the human factor."

                      "Enjoy life today, tomorrow may never come."
                      "If you're going through hell, keep going. ~Winston Churchill"

                      Comment


                      • #12
                        Re: Server will hesitate for breif periods then OK

                        Process Explorer should show you which process is causing the high disk utilization. It sounds like some program is behind this.

                        Has anyone created any custom queries to the DB recently? Has there been any application updates recently? Can you check to see if there's any updates to the software on the server/clients?

                        If the disk transfers are high as well as the queue then the disks and controller are probably working fine but are just getting overwhelmed.
                        Regards,
                        Jeremy

                        Network Consultant/Engineer
                        Baltimore - Washington area and beyond
                        www.gma-cpa.com

                        Comment

                        Working...
                        X