Announcement

Collapse
No announcement yet.

Freezing PCs

Collapse
X
  • Filter
  • Time
  • Show
Clear All
new posts

  • Freezing PCs

    Hi all,

    I am not sure this is the right place to ask this, so please feel free to suggest where it should be.

    I have a problem with PCs freezing.

    I will try and give as much information as possible.

    I have a product that runs in around 500 vending machines.
    On two sites we have a problem with the pcís freezing randomly.
    We think other sites are running fine without this problem.
    I am unable to reproduce the problem with the same set up in the office

    The PCs are fan less +5v 500 GHz, 512GB SDRAM, with 8GB CF card
    (AVALUE VIA CX700M)
    The OS is WEPOS SP3
    There are 2 USB devices connected.
    The main app is a windows .net app running on MSDE SQL

    The PCs run continuously and can only be connected to remotely.

    We have had this problem for quite a long time now and we have tried lots of things but I am really running out of ideas.

    The PCs seem to freeze randomly, we have about 120 PCs in these two sites and get between 1 and 5 freezes a day, it would appear a lot of the freezes happen over night when there is not much use of the machines.

    I have rebuilt the image and reimaged the machines, I have run registry scans, and there are no viruses or ad ware.

    I have run CPU temp monitoring software and the CPU temp does not budge even when I run the CPU at max for a while.

    Although we run CF cards I have run it on a HDD and appear to have the same problem.

    I have put surge protectors on the power and also tried UPSís.

    I am now starting to think that it could be a USB problem, one of the devices is a HID card reader which has an LED powered by the USB, sometimes we notice that then the PC has frozen the LED is off and the other day an engineer was looking at a PC we believed to be frozen when he noticed the USB device was not in completely, he took it all the way out and pushed it back in and the PC appeared to unfreeze and we could once again remotely connect to it. Looking at the event logs it would appear that it restarted once the USB was removed.

    Sorry for such a long message but I really could do with your help, please just remember that I do not have direct access to these machines, we have to send someone to site if we need to physically do anything to it that can not be done via VNC.

    Many thanks

    Craig

  • #2
    Re: Freezing PCs

    I'm assuming you've scanned the Windows event logs?

    When the PC is "frozen", does it respond in any way whatsoever? Can you ping it? I've had servers frozen, but the TCP/IP stack was still responsive which indicated at what level the freezing was occurring.

    Does the HID card reader manufacturer still support their drivers or are those drivers specifically made for WEPOS SP3?
    Wesley David
    LinkedIn | Careers 2.0
    -------------------------------
    Microsoft Certifications: MCSE 2003 | MCSA:Messaging 2003 | MCITP:EA, SA, EST | MCTS: a'plenty | MCDST
    Vendor Neutral Certifications: CWNA
    Blog: www.TheNubbyAdmin.com || Twitter: @Nonapeptide || GTalk, Reader and Google+: [email protected] || Skype: Wesley.Nonapeptide
    Goofy kitten avatar photo from Troy Snow: flickr.com/photos/troysnow/

    Comment


    • #3
      Re: Freezing PCs

      Thanks for the reply,

      There is nothing in the event log, ping stops responding, when we go to the device and put a monitor on it the time to frozen at the time we lost communication with it.

      There are very few devices that produce WEPOS specific drivers, we just use XP drivers, we use this card reader in many other site with no problems, I need to check that we are using exactly the same driver version.

      Comment


      • #4
        Re: Freezing PCs

        Checking exact driver versions across your fleet would be a good thing. From there you can see if there is any correlation between freezing and driver version (correlation does not necessarily mean causation, but still... it's good info to start with). The ultimate test would be to see if reseating the USB device unfreezes a frozen machine like you said might have happened once already. If that's the case, it sounds most certainly like a driver problem.

        Do you have uptime reports that can show how long the PCs have been up before they froze? Is it commonly multiple days or can it vary from a matter of hours to a matter of months before a PC freezes?
        Wesley David
        LinkedIn | Careers 2.0
        -------------------------------
        Microsoft Certifications: MCSE 2003 | MCSA:Messaging 2003 | MCITP:EA, SA, EST | MCTS: a'plenty | MCDST
        Vendor Neutral Certifications: CWNA
        Blog: www.TheNubbyAdmin.com || Twitter: @Nonapeptide || GTalk, Reader and Google+: [email protected] || Skype: Wesley.Nonapeptide
        Goofy kitten avatar photo from Troy Snow: flickr.com/photos/troysnow/

        Comment


        • #5
          Re: Freezing PCs

          Thanks again for your reply, sorry for the late message i have been away for a few days.

          The card reader device is actually a Human Interface Device and the same driver is used across sites.

          we do have detailed external monitoring of the devices so we can tell that a freeze can happen with in a day or after many days/ weeks, we think we have devices on the same site that have not frozen at all, but we are unable to check each device every time it stops responding to a ping.

          some more information for you, last year we created a new clean image and changed a lot of the devices, this seemed to have fixed the problem, we did not see any problems on these machines for months and then they started creeping back in. Not sure if this means anything?

          I also did a registry clean the other day that seemed to make the problem worse, they all froze over the next week, I had to undo the changes, probably not related but could be.

          Also it appears to happen then the PC is idle, mostly over night.

          another idea is that these are 5v machines and we are running then in big city office block not on their regulated power circuit, the same socket block is also running the vending machine, we seemed to be OK during the winter, could it be a power issue, we have tried a surge protector and a UPS on some machines that appeared to still freeze, but I am not convinced that it is not an environmental issue.

          any other ideas very welcome

          Craig

          Comment


          • #6
            Re: Freezing PCs

            Wow, this has got to be one of the most seemingly random issues I've heard of. One thing I'd recommend is making a policy of creating a secured implementation of SNMP on all PCs that you deploy. Or you could create some kind of WMI script to get the information. Either way, it would be helpful to watch RAM, CPU, hard disk and etc. usage every 5 minutes or so (A pretty high polling schedule, but I think you could afford it if the PCs aren't being used for anything intense).

            With that much data coming back to you into a SNMP collector, you'd have even more data to try to correlate with. You could also get network enabled battery units and get SNMP information from those to monitor power information. The more information the better.

            The fact that a new image seemed to have some effect, but the issues came back eventually, seems to indicate that a program has some kind of memory or corruption issue.

            BTW, do the machines ever BSoD? If so, you could analyze the crash dump. I'm not sure if a hard lock without a BSoD leaves any kind of memory trail that can be analyzed.
            Wesley David
            LinkedIn | Careers 2.0
            -------------------------------
            Microsoft Certifications: MCSE 2003 | MCSA:Messaging 2003 | MCITP:EA, SA, EST | MCTS: a'plenty | MCDST
            Vendor Neutral Certifications: CWNA
            Blog: www.TheNubbyAdmin.com || Twitter: @Nonapeptide || GTalk, Reader and Google+: [email protected] || Skype: Wesley.Nonapeptide
            Goofy kitten avatar photo from Troy Snow: flickr.com/photos/troysnow/

            Comment


            • #7
              Re: Freezing PCs

              Run Memtest
              Change video card (if not onboard)
              Change power Supply.
              1 1 was a racehorse.
              2 2 was 1 2.
              1 1 1 1 race 1 day,
              2 2 1 1 2

              Comment

              Working...
              X