Announcement

Collapse
No announcement yet.

Weird Arp problem cisco 3750

Collapse
X
  • Filter
  • Time
  • Show
Clear All
new posts

  • Weird Arp problem cisco 3750

    I have a weird arp problem I hope someone can help me solve.

    I have 3 cisco switches in different buildings connected directly with a fiber link.
    All switches are running eigrp 10

    switch one runs vlans:
    30 - Lan 10.0.30.0 /24
    31 - Phones 10.0.31.0 /24
    100 - management 10.0.0.0 /24 (10.0.0.251)
    IP route is 0.0.0.0 0.0.0.0 10.0.0.252 10


    switch 2 runs vlans:
    20 - Lan 10.0.20.0 /24
    21 - Phone 10.0.21.0 /24
    1 - old network for servers 192.168.10.0 /24
    100 - management 10.0.0.0 /24 (10.0.0.252)
    IP route is 0.0.0.0 0.0.0.0 10.0.0.253 5

    switch 3 runs vlans:
    10 - Lan 10.0.10.0 /24
    100 - 10.0.0.0 /24 (10.0.0.253)
    IP route is 0.0.0.0 0.0.0.0 10.0.0.254

    We then have a watchguard firebox for our firewall
    Internal IP is 10.0.0.254

    I can reach any device on any vlan from any other vlan.

    How ever from my laptop (vlan 20 switch 2) I set a continuous ping to 10.0.0.254 (firebox)
    after a few minutes this ping will start to fail, if i then clear the arp on switch 2 the ping resumes within a few seconds. then after approx 5 minutes the ping will fail again until i clear the arp.

    I have been told this is probably a routing loop, However I am not sure where to start looking to fix this issue.

    If I plug into switch 3 on either vlan 10 or 100 I can get to the firebox (10.0.0.254) however from switch 1 or switch 2 I cannot so at leasr I know the problem is probably on switch 2.
    Below is the sh ip route from switch 2:

    Gateway of last resort is 10.0.0.253 to network 0.0.0.0

    C 192.168.10.0/24 is directly connected, Vlan1
    10.0.0.0/8 is variably subnetted, 7 subnets, 2 masks
    D 10.0.10.0/24 [90/3072] via 10.0.0.253, 02:27:19, Vlan100
    C 10.0.0.0/24 is directly connected, Vlan100
    D 10.0.0.0/8 is a summary, 21:57:36, Null0
    D 10.0.30.0/24 [90/3072] via 10.0.0.251, 06:16:01, Vlan100
    D 10.0.31.0/24 [90/3072] via 10.0.0.251, 06:16:01, Vlan100
    C 10.0.20.0/24 is directly connected, Vlan20
    C 10.0.21.0/24 is directly connected, Vlan21

    Below is the static routes I have in place on switch 2:

    router eigrp 10
    network 10.0.0.0 0.0.0.255
    network 10.0.20.0 0.0.0.255
    network 10.0.21.0 0.0.0.255
    network 192.168.10.0
    auto-summary
    !
    ip classless
    ip route 0.0.0.0 0.0.0.0 10.0.0.253 5 name To-DC-Switch-1
    ip route 192.168.10.0 255.255.255.0 Vlan1

    however internet conectivity still works ok.

    Can anyone point me in the right direction to find my routing loop please?

    Regards
    Shaun
    Last edited by shaun; 12th February 2009, 17:06.

  • #2
    Re: Weird Arp problem cisco 3750

    Any particular reason why you are using different gateways?
    (maybe I don't understand the mind drawing that quickly )
    Marcel
    Technical Consultant
    Netherlands
    http://www.phetios.com
    http://blog.nessus.nl

    MCITP(EA, SA), MCSA/E 2003:Security, CCNA, SNAF, DCUCI, CCSA/E/E+ (R60), VCP4/5, NCDA, NCIE - SAN, NCIE - BR, EMCPE
    "No matter how secure, there is always the human factor."

    "Enjoy life today, tomorrow may never come."
    "If you're going through hell, keep going. ~Winston Churchill"

    Comment


    • #3
      Re: Weird Arp problem cisco 3750

      When you say the ping fails what type of response do you get? Do you get "request timed out", do you get "destination net unreachable", etc.?

      A request timed out would indicate a layer 1 or 2 problem. A destination not reachable would indicate a layer 3 problem.

      Comment


      • #4
        Re: Weird Arp problem cisco 3750

        the way it works is, we have 3 buildings building 1 linked via fiber to building 2, building 2 linked via fiber to building 3 (data center)

        Our internet connection is at the data center.

        switch1 fa 1/0/24 is connected to swtich2 fa 1/0/48 switch2 1/0/47 is connected to switch3 fa 1/0/24 switch3 fa 1/0/23 is connected to the watchguard firebox trusted interface. Then the firebox'es wan link goes to our ether line.

        I am not sure if I have the gateways correct so if there is a better way to do this please let me know ( am not ccna certified yet tho looking to take the test in the next few months.

        Also we have discovered a problem with remote vpn users. They use the watchguard vpn software when connected they are not given the correct gateway. they get an ip address and then the gateway is one above the ip address. there is no way to set the default gateway for the dhcp scope on the vpn of the firebox (I dont know much about fireboxes as thats left to some else to configure)

        Hope this helps.

        Shaun

        Comment


        • #5
          Re: Weird Arp problem cisco 3750

          Originally posted by joeqwerty View Post
          When you say the ping fails what type of response do you get? Do you get "request timed out", do you get "destination net unreachable", etc.?

          A request timed out would indicate a layer 1 or 2 problem. A destination not reachable would indicate a layer 3 problem.
          Hi its a request timed out response.

          I am wondering if switch one is advertising a route to the 10.0.0.0 network
          Although the route I have is 0.0.0.0 0.0.0.0 10.0.0.252 from switch 1 to switch 2 with an admin distance of 10.

          However even tho ping fails internet works which is really throwing me off here.
          Also after i clear the arp the ping resumes within 5 seconds.

          Regards
          Shaun

          Comment


          • #6
            Re: Weird Arp problem cisco 3750

            Based on the fact that you get a "request time out" and the fact that clearing the ARP cache momentarily resolves the problem, I'd say you're dealing with a layer 2 problem and that you can rule out any routing issue as the problem. If there was a routing problem you would probably get one of two things:

            1. Unreachable - meaning no router knows how to get to the destination
            2. TTL expired in transit - meaning there is a routing loop and the ping is bouncing between routers until the TTL expires.

            Since you're not getting the above I'd say that the problem is at layer 2.

            Is the watchguard configured in drop in mode? How many watchguard interfaces are connected to the internal network? What is the ARP cache timeout value on switch 2?

            Comment


            • #7
              Re: Weird Arp problem cisco 3750

              Originally posted by joeqwerty View Post

              Is the watchguard configured in drop in mode? How many watchguard interfaces are connected to the internal network? What is the ARP cache timeout value on switch 2?
              I have no idea about the watchguard been in drop in mode (nor do i know what that means ) as its my manager who configured that. I do know there is only one connection on the trusted network. I will find out tomorrow about that

              The arp cache is what ever default value is as I havent changed.

              Regards
              Shaun

              Comment


              • #8
                Re: Weird Arp problem cisco 3750

                When the watchguard is configured in drop-in mode (configured in the Network|Configuration menu) all interfaces use the same ip address. This could cause problems if two or more of those interfaces connect to the internal network as you may wind up with two different "paths" to the ip address of the watchguard based on which of it's interface MAC addresses is in the ARP cache of the switch. It may also happen if you have a loop in your switches. Do you have the switches connected in such a way as there may be a "switching" loop? does sw1 connect to sw2 which then connects to sw3? Or does sw1 connect to sw2 & sw3 and sw2 connects to sw1 and sw3, etc? If so you may have a "switch" loop unless you're using the spanning-tree protocol on the switches to avoid loops.

                Comment


                • #9
                  Re: Weird Arp problem cisco 3750

                  Originally posted by joeqwerty View Post
                  When the watchguard is configured in drop-in mode (configured in the Network|Configuration menu) all interfaces use the same ip address. This could cause problems if two or more of those interfaces connect to the internal network as you may wind up with two different "paths" to the ip address of the watchguard based on which of it's interface MAC addresses is in the ARP cache of the switch. It may also happen if you have a loop in your switches. Do you have the switches connected in such a way as there may be a "switching" loop? does sw1 connect to sw2 which then connects to sw3? Or does sw1 connect to sw2 & sw3 and sw2 connects to sw1 and sw3, etc? If so you may have a "switch" loop unless you're using the spanning-tree protocol on the switches to avoid loops.
                  Hi,

                  Its switch 1 to swtich 2 then switch 2 to switch 3 then switch 3 to firebox.
                  When ever I look at the arp table for the firebox IP its always the same mac address.

                  Think I will look at switch 1's arp table tomorrow.
                  When this problem happens if I plug into switch 3 i can ping the firebox successfuly, so I assume the problem is lying between switch 1 and 2 ?

                  Regards
                  Shaun

                  I dont think the firebox is in drop in mode as we have had different IP's on the trusted side.

                  Comment


                  • #10
                    Re: Weird Arp problem cisco 3750

                    Also, look at the ARP table in the watchguard. It may have the ARP entry for the host in question registered on the wrong interface. This typically happens when it's in drop-in mode, but you never know.

                    Comment


                    • #11
                      Re: Weird Arp problem cisco 3750

                      well, A strange thing happened this morning. It seems to work ok now it never times out. Spoke to my boss and he had accidentaly pulled the power from switch.
                      Thats all he had done? so he says anyway Baffled me

                      Comment

                      Working...
                      X