Announcement

Collapse
No announcement yet.

Server 2008 unexpected shutdowns

Collapse
X
  • Filter
  • Time
  • Show
Clear All
new posts

  • #16
    Re: Server 2008 unexpected shutdowns

    So it pretty much happened again like clockwork except at a slightly different hour. The interesting thing to note is that I shutdown the VMs on Monday for some system maintenance and did a reboot on the host machine after applying a few updates. This "should" have reset the time period between reboots but it still went ahead and did it last night.

    As suggested, I removed the option to automatically reboot on error but when my boss came in this morning everything was already up and running. Still no memory dumps and the event logs are the same as before.

    I am pretty much convinced that a firmware update will work but my boss isn't. Given that this is our only server capable of housing our 3 VMs, the fear of something going wrong is hindering me from doing what needs to be done. Would someone provide a convincing argument to proceed with the firmware update? I was given the go ahead to do OS updates but I had to cancel in the middle of them on Monday.

    Comment


    • #17
      Re: Server 2008 unexpected shutdowns

      I don't understand why someone should provide you a "convincing argument'? It's not like this is a law debating class

      If you have a system whose BIOS is out of date, would you not want to bring everything current before you start to diagnose it? Maybe there was as issue that caused random reboots that was addressed in the update? You won't know until you try. It's akin to trying to fix Windows problems prior to doing any patches, service packs or updates.

      If you and your boss are that anal about trying to fix this, contract the work out to someone else that can assume the responsibility. In my case on our Dell PowerEdge, we have been OK for almost a month now on randon reboots which used to occur weekly. All we did was firmware and BIOS updates but I;m sure you read that already.
      Network Engineers do IT under the desk

      Comment


      • #18
        Re: Server 2008 unexpected shutdowns

        Believe me, I'm all for updating the firmware and while in the middle of composing a convincing email my boss walked in. Of course I took the opportunity to explain my reasoning and gave him a description of the fix I plan on applying. Here is an excerpt directly from the HP website regarding a critical Systems ROMPaq firmware upgrade for the Proliant DL180 G5 series

        Important Notes:
        NoneFirmware Dependencies:
        NoneProblems Fixed:
        Resolved a processor issue where an unlikely and complex sequence of conditions operating in 64-bit mode could result in an application-level error, blue screen, kernel panic, or other unpredictable system behavior. This issue affects platforms utilizing Intel Xeon 5400-series processors. This issue is not unique to HP ProLiant servers and could impact any system using affected processors. This revision of the System ROM contains an updated version of Intel microcode for affected processors that addresses this issue. Due to the potential severity of the issue addressed in this revision of the System ROM, this System ROM upgrade is considered a critical fix. HP strongly recommends immediate update to firmware revisions with required critical fixes.Known Issues:
        None

        I have been given the go-ahead but will have to come in on a day that nobody is here, basically a Sunday which will have to be on the 24th since I won't be in town this weekend. Until then I'll just keep on truckin'
        Attached Files

        Comment


        • #19
          Re: Server 2008 unexpected shutdowns

          Its been about a couple weeks since my last post and I finally got the go ahead to do the BIOS update. I did it this past Sunday and everything seems to be working out just fine. I did have a scare at one point when the OS wouldn't boot but that was because the machine was trying to boot from our external drives attached to it. I guess another couple weeks need to go by until I can really say it's been fixed.

          As a side note, I did notice our server did the unexpected restart 6 days after the previous one (usually its 14). That was pretty much the winning argument since it was starting to occur more frequently.

          Comment


          • #20
            Re: Server 2008 unexpected shutdowns

            Just an update, we had ANOTHER unexpected restart a couple of hours ago. It had been about 9 days since I went and did the BIOS update. We have about 30 OS updates but I wanted to refrain from doing them until I was sure they were necessary. I'm guessing the next steps would be to run a memory and hard drive test? I'm open to other suggestions.

            Comment


            • #21
              Re: Server 2008 unexpected shutdowns

              are you generating memory dumps ?
              Please do show your appreciation to those who assist you by leaving Rep Point https://www.petri.com/forums/core/im.../icon_beer.gif

              Comment


              • #22
                Re: Server 2008 unexpected shutdowns

                The option is selected but nothing is generated.

                Comment


                • #23
                  Re: Server 2008 unexpected shutdowns

                  Is there anything in the event logs???

                  Have you scanned for viruses and malware???

                  Have you tested the hardware in the server???

                  I would reckon if the OS was to blame then you would at least see something in the event logs.

                  Comment


                  • #24
                    Re: Server 2008 unexpected shutdowns

                    I would go back to my originial suggestion of a bad power supply failing the BIOS update. If the power is abruptly cut, there is no opportunity to write an event to the log. The HP has redundant power supplies, but only one is working at a time. If one PSU is failing on a temporary basis, maybe the alternate PSU is not working.

                    Do you have a good UPS with power conditioning like an APC SC-1500 for example? Are you using a 'Y' power cable to the UPS instead of individual power cords to a power bar?

                    Maybe you could see if the server will run on just one PSU. If it does, shut down and switch the PSU and see if it runs on the other one.

                    As far as I know, you can't silence the alarm on an HP if you remove a PSU so you will have to live through that. You just want to ensure both PSU are working.
                    Network Engineers do IT under the desk

                    Comment


                    • #25
                      Re: Server 2008 unexpected shutdowns

                      Don't remember if it has been asked, but is the beast under warranty -- if so, get HP onto it
                      Tom Jones
                      MCT, MCSE (2000:Security & 2003), MCSA:Security & Messaging, MCDBA, MCDST, MCITP(EA, EMA, SA, EDA, ES, CS), MCTS, MCP, Sec+
                      PhD, MSc, FIAP, MIITT
                      IT Trainer / Consultant
                      Ossian Ltd
                      Scotland

                      ** Remember to give credit where credit is due and leave reputation points where appropriate **

                      Comment


                      • #26
                        Re: Server 2008 unexpected shutdowns

                        Looks like I've got quite a bit of testing to do this weekend. I'll make sure to run a memory test, disk check, and swap the PSUs. I just checked the power connections and it looks like both of the PSUs are connected to the same battery. We currently have two battery backups, they are APC Back-UPS NS 1250, not entirely sure if this is enough to power everything on them.

                        On the first battery we have a total of 5 switches connected (one 8 port, one 16 port, and three 24 ports) and a CCTV PSU connected. On the second battery, we have two 24-port switches, VoIP system (Allworx 6x), and a DVR connected. I can post the voltage specs if anyone thinks that would be helpful.

                        On the first battery, I can see the "Building wiring fault" light is on but not on the second battery. Time to dig up the manual!

                        Comment


                        • #27
                          Re: Server 2008 unexpected shutdowns

                          Sorry for the delayed response but I tracked it down to the battery backup not being sufficient enough to power the server while on battery. Of course my boss knew the battery wasn't meant for the server but waited to tell me after I spoke with CDW techs.

                          Comment

                          Working...
                          X