Announcement

Collapse
No announcement yet.

Network down

Collapse
X
  • Filter
  • Time
  • Show
Clear All
new posts

  • Network down

    I'm not after a solution here as I'm not sure how to go about troubleshooting this, but a funny thing has happened the last couple of days... so any thoughts would be appreciated

    Yesterday at 11:40am and today at 11:27am the network disappeared in a cloud of dust.

    Yesterday it was down for about 10mins then magically came back up.

    Today the outage lasted a lot longer.

    When looking at it today I saw no events in the clients or servers before this happened, apart from clients complaining about no DNS servers being available to resolve name blah blah

    Nothing in the servers. All scheduled tasks are set to run outside office hours.

    During today's outage I was unable to ping the clients either via name or IP.

    We have 3 DC's all running DNS. I restarted the DNS service on all of them but it made no difference. I replaced the cable between the wall socket and the switch to which the servers are attached but no help.

    Our oldest DC is a 2003 server. As soon as I restarted it (after 30mins) the network came back up.

    Nothing like switching it off and back on again...
    A recent poll suggests that 6 out of 7 dwarfs are not happy

  • #2
    Re: Network down

    the week before last, i had 3 outages, and had absolutely no clue why. i saw nothing in any logs, nor did i see and red lights in the wrong places...

    so i restart the modem (using comcast business class at this place) and it works again, but the other two times, nada. nothing. zip, zilch, goose egg...

    so i start monitoring via pings and telnet/SSH on the modem and firewall and DCs...

    turns out, comcast was doing upgrades in our area and nobody decided to call or let us know that they were going to stop all advanced routing functions, disable the admin page on the modem and restart the thing several times for fresh configs and firmware upgrades... of course i had to call and ask some backdoor questions. i asked if they were doing work, they say 'we dont think so...' so then i asked, 'can you get to my admin page or send my modem a refresh packet, im at IP x.x.x.x'. he says, 'well, i cant do that right now', which is there way of saying they are pushing updates...

    thanks comcast. good lookin out.

    Last edited by James Haynes; 20th August 2014, 16:46.
    its easier to beg forgiveness than ask permission.
    Give karma where karma is due...

    Comment


    • #3
      Re: Network down

      Interesting.

      My setup is slightly different. We use a router for the gateway, not any of the servers. I can switch the router off and network connectivity will not be affected.

      ADSL <--> Router <--> Network <==> Clients/Servers
      A recent poll suggests that 6 out of 7 dwarfs are not happy

      Comment


      • #4
        Re: Network down

        Are there any changes to the network which could cause this? When the network is down are all the lights on the switches solid? Do you have any monitoring on the switches that show utilization. Is spanning tree enabled?
        Could you connect a laptop running wireshark to a switchport to see what traffic is been generated at the time of the outage?
        It could be that someone is plugging in a switch, or plugging in a cable somewhere which is causing a L2 broadcast storm.
        Please remember to award reputation points if you have received good advice.
        I do tend to think 'outside the box' so others may not always share the same views.

        MCITP -W7,
        MCSA+Messaging, CCENT, ICND2 slowly getting around to.

        Comment


        • #5
          Re: Network down

          Originally posted by Blood
          Yesterday it was down for about 10mins then magically came back up.
          How long do your switches take to reboot if they pack a sad?
          1 1 was a racehorse.
          2 2 was 1 2.
          1 1 1 1 race 1 day,
          2 2 1 1 2

          Comment


          • #6
            Re: Network down

            Our network is very simple. We use unmanaged switches so they come up pretty fast. A couple of weeks ago one of the 24 port switches went in 'Segment 1'. We replaced it and within seconds of it being switched on the network was up again.

            Here's an expertly drawn diagram of our network:


            I'm the only IT person here so no one else unplugs cables etc.

            When this happened the first thing I did was check the switches and they were all on and blinking.

            It's really weird.
            A recent poll suggests that 6 out of 7 dwarfs are not happy

            Comment


            • #7
              Re: Network down

              So how many switches do you currently have??

              From what i can see there are 3 x 24 port switches and various others??

              Comment


              • #8
                Re: Network down

                Here's a better drawing: http://img539.imageshack.us/img539/2095/yxaiYN.jpg

                The 2nd segment was created when the building was extended. Purple triangles are desktop switches. Green squares are the patch panels and 24 port switches.

                [Edit]
                The servers are in the loft.
                A recent poll suggests that 6 out of 7 dwarfs are not happy

                Comment


                • #9
                  Re: Network down

                  rope in a deputy to help you out with this.

                  next time it happens, pulls the cable between segment 1 and segment 2.
                  then run ping tests (ip, not dns but you probably know that) against objects in their segment only. (this is why you need a deputy.. one person in each segment)

                  if one segment comes back quicker than the other, or if one comes back and the other doesn't, you'll have some ideas I suspect
                  Please do show your appreciation to those who assist you by leaving Rep Point https://www.petri.com/forums/core/im.../icon_beer.gif

                  Comment


                  • #10
                    Re: Network down

                    Aha - that's great. I'll remember to do that if it happens again.

                    Thanks!
                    A recent poll suggests that 6 out of 7 dwarfs are not happy

                    Comment


                    • #11
                      Re: Network down

                      Originally posted by Blood View Post
                      Here's a better drawing: http://img539.imageshack.us/img539/2095/yxaiYN.jpg

                      The 2nd segment was created when the building was extended. Purple triangles are desktop switches. Green squares are the patch panels and 24 port switches.

                      [Edit]
                      The servers are in the loft.
                      I can't see the image.

                      What is the need for a desktop switch rather than having them all patched to a core switch??

                      Comment


                      • #12
                        Re: Network down

                        Imageshack - I don't know what the hell is going on with their site

                        I don't understand your question. The desktop switches are used to provide for expansion of the existing network. A cable is connected to a wall socket and fed through to the other side or centre of the room and connected to a 4 port desktop switch. The computers on the desks in that area are then connected to the switch.
                        A recent poll suggests that 6 out of 7 dwarfs are not happy

                        Comment


                        • #13
                          Re: Network down

                          I've been onto Imageshack support - the image is now available.
                          A recent poll suggests that 6 out of 7 dwarfs are not happy

                          Comment


                          • #14
                            Re: Network down

                            Originally posted by wullieb1 View Post
                            I can't see the image.

                            What is the need for a desktop switch rather than having them all patched to a core switch??
                            ... on the fly expansion.. on-the-budget-expansion
                            in some instances, I'd normally say "poor support staff" but probably not in this case.

                            i suspect he's in an environment where it's not simple and easy to just run patch ports to every desk..
                            Please do show your appreciation to those who assist you by leaving Rep Point https://www.petri.com/forums/core/im.../icon_beer.gif

                            Comment


                            • #15
                              Re: Network down

                              hmmm

                              The building we occupy is a former Victorian school house. When we moved here in 1994 BNC cabling was installed for our fleet of computers. In about 2002/3 CAT5 cabling replaced the BNC cabling. This was all done 'in-house'. Over the years the number of computers we use has risen and has grown beyond the capabilities of the cabling as it now exists.

                              All the cabling, except for that which was installed when the building was extended is fed through the rafters (which are visible) and down the outside of the walls to RJ45 sockets. In some places there is no more room to feed new cables. We would need to 're-cable' everything in order to meet demand and that is not cheap.

                              If there is a better and cheap way of accommodating the surplus of computers we use I would be pleased to hear suggestions. Don't forget, I am not a trained IT person. Everything I know I have learnt as needed so there are gaps in my knowledge. One of the reasons I stay on this forum is because you guys have a far better understanding of professional IT than I do and I constantly learn from you.
                              A recent poll suggests that 6 out of 7 dwarfs are not happy

                              Comment

                              Working...
                              X