Every network admin is going to have trouble with network links on a Cisco router, at one point or another. The best way to troubleshoot any networking issues is to use the OSI model and go layer by layer. In my article How to use the OSI Model to Troubleshoot Networks, we talked about the different troubleshooting approaches and how to use them to troubleshoot your network, in general. In this article, you will find out how to use the OSI model to troubleshoot, bottom up, using a Cisco router.
If you will recall, the OSI model starts with the physical layer (layer 1) and goes up to layer 7 (application). When troubleshooting with a Cisco router, much of your time will be spent working in layers 1-3. They are:
Because these layers build on each other, Layer 1 is most critical, without layer 1, layer 2 will not function. Without layer 1 & 2, layer 3 will not function, and so on. For this reason, I start troubleshooting at layer 1, physical, and move on up from there.
Remember, if Layer 1 isn’t up, nothing else will work so make sure you start here. Examples of layer 1 are your T1 circuit or your Ethernet cable – physical connectivity. I usually troubleshoot layer 1 and layer 2 in union because they are so closely paired. Examples of layer 2 – data link – are your line protocol (such as Ethernet, ATM, 802.11, PPP, frame-relay, HDLC, or PPP).
To troubleshoot at these layers, the first thing I would do on your router is a show interface. Here is an example of a LAN Gigabit Ethernet circuit:
Router# show interface GigabitEthernet0/0 is up, line protocol is up Hardware is BCM1125 Internal MAC, address is 0015.2b46.5000 (bia 0015.2b46.5000) Description: LAN Connection to Data center Internet address is 10.20.100.1/16 MTU 1500 bytes, BW 1000000 Kbit, DLY 10 usec, reliability 255/255, txload 1/255, rxload 1/255 Encapsulation ARPA, loopback not set Keepalive set (10 sec) Full-duplex, 1000Mb/s, link type is autonegotiation, media type is RJ45 output flow-control is XON, input flow-control is XON ARP type: ARPA, ARP Timeout 04:00:00 Last input 00:00:00, output 00:00:00, output hang never Last clearing of "show interface" counters never Input queue: 0/75/0/0 (size/max/drops/flushes); Total output drops: 0 Queueing strategy: weighted fair Output queue: 0/1000/64/0 (size/max total/threshold/drops) Conversations 0/2/256 (active/max active/max total) Reserved Conversations 0/0 (allocated/max allocated) Available Bandwidth 750000 kilobits/sec 5 minute input rate 3218000 bits/sec, 1715 packets/sec 5 minute output rate 1390000 bits/sec, 2129 packets/sec 1416888620 packets input, 15402720 bytes, 0 no buffer Received 0 broadcasts, 0 runts, 0 giants, 0 throttles 0 input errors, 0 CRC, 0 frame, 0 overrun, 0 ignored 0 watchdog, 1556005 multicast, 0 pause input 0 input packets with dribble condition detected 1666663097 packets output, 573841802 bytes, 0 underruns 19 output errors, 0 collisions, 3 interface resets 0 babbles, 0 late collision, 0 deferred 19 lost carrier, 0 no carrier, 0 pause output 0 output buffer failures, 0 output buffers swapped out
Here is what a WAN T1or T3 circuit might look like:
Routerl# show interface serial 3/0 Serial3/0 is up, line protocol is up Hardware is DSXPNM Serial Description: Sprint T3 Internet address is 10.2.100.2/30 MTU 4470 bytes, BW 9000 Kbit, DLY 200 usec, reliability 255/255, txload 77/255, rxload 26/255 Encapsulation HDLC, crc 16, loopback not set Keepalive set (10 sec) Last input 00:00:00, output 00:00:00, output hang never Last clearing of "show interface" counters never Input queue: 0/75/0/0 (size/max/drops/flushes); Total output drops: 18394 Queueing strategy: fifo Output queue: 0/40 (size/max) 5 minute input rate 927000 bits/sec, 1914 packets/sec 5 minute output rate 2752000 bits/sec, 1504 packets/sec 1560997932 packets input, 3254680247 bytes, 0 no buffer Received 255480 broadcasts, 1 runts, 1 giants, 0 throttles 1567 input errors, 1567 CRC, 976 frame, 496 overrun, 0 ignored, 908 abort 1303636803 packets output, 3737276508 bytes, 0 underruns 0 output errors, 0 collisions, 3 interface resets 0 output buffer failures, 0 output buffers swapped out 1 carrier transitions DSU mode 1, bandwidth 9000, real bandwidth 9000, scramble 0
Here is the quick version:
Router# show ip interface brief Interface IP-Address OK? Method Status Protocol GigabitEthernet0/0 10.20.100.1 YES NVRAM up up Serial3/0 10.2.100.2 YES NVRAM up up
Here is what you look for:
In general, verify that you have a good cable on each side, verify that line protocols match, and that clocking settings are correct.
If this is an Ethernet connection, is there a link light on the switch?
If this is a serial connection, do you have an external CSU/DSU? If it is an external CSU, check that the Carrier Detect (CD) light & data terminal ready (DTR) lights are on. If not, contact your provider. This also applies if you have an internal Cisco WIC CSU card. If that is the case, take a look at this Cisco link on understanding the lights on that card.
You can, of course, use the Cisco IOS test commands to test your network interfaces with internal staff and with your telecommunications providers.
Do not proceed to upper level layers until your Physical interface on the router shows as being UP and your line protocol is UP. Until then, don’t worry about IP addressing, pinging, access-lists or anything like that.
Once you have Layers 1 & 2 working (your show interface command shows the line is “UP & UP”, it is time to move on to layer 3 – the OSI Network layer. The easiest thing to do here to see if layer 3 is working is to ping the remote side of the LAN or WAN link from this router. Make sure you ping as close as possible to the router you are trying to communication with – from one side across to the other side.
Here are examples of successful & failed pings:
Router# ping 10.2.100.2 Type escape sequence to abort. Sending 5, 100-byte ICMP Echos to 10.2.100.2, timeout is 2 seconds: !!!!! Success rate is 100 percent (5/5), round-trip min/avg/max = 1/2/4 ms Router# Router# Router# Router# Router# ping 1.1.1.1 Type escape sequence to abort. Sending 5, 100-byte ICMP Echos to 1.1.1.1, timeout is 2 seconds: ..... Success rate is 0 percent (0/5) Router#
The easiest way to check the status of Layer 3 – the network layer – is to do a show ip interface brief, as I did above. Here is an example:
Router# show ip interface brief Interface IP-Address OK? Method Status Protocol GigabitEthernet0/0 10.20.100.1 YES NVRAM up up Serial3/0 10.2.100.2 YES NVRAM up up
Notice the IP addressing on each of these interface. Also do a show running-config, like this (you can even specify an interface, like this):
Router# show running-config int serial3/0 Building configuration... Current configuration : 225 bytes ! interface Serial3/0 description Sprint T3 bandwidth 9000 ip address 10.2.100.2 255.255.255.252 no ip proxy-arp no ip mroute-cache dsu mode 1 dsu bandwidth 9000 no cdp enable end Router#
I would recommend taking this interface configuration and comparing it, side by side, with the remote WAN connection to ensure they are the same. Ask yourself questions like:
Here is an example. Look at the two interfaces below. What is the real problem, causing these two to not communicate?
Router 1
interface Serial3/0 description Sprint T3 – TO ROUTER 2 bandwidth 9000 ip address 10.2.100.2 255.255.255.252
Router 2
interface Serial3/0 description Sprint T3 – TO ROUTER 1 bandwidth 1500 ip address 10.2.100.5 255.255.255.252
No, there is no problem with the bandwidth statement. Bandwidth statements are only used as comments and by routing protocols to select the best route. The real problem here is that the second router’s serial interface is not on the same IP subnet as router #1. Even though they have the same subnet, the 10.2.100.5 IP address will never be able to communicate to the 10.2.100.2 IP address because they are on different networks but directly connected.
Let’s say that you are now able to ping across the link, from one side to another. While that is a great sign, it doesn’t always mean that everything is “fixed”. You still may not be able to communicate from a client on the LAN of one router, to a client on the LAN of another router, due to things like improperly configured IP routing protocols.
For one LAN to communicate to another LAN, through routers (through a WAN, usually), you MUST have either static routes or dynamic routes configured. To ensure you have a route configured for the network you are trying to reach, do:
Router# show ip routes
and look at
Router# show ip protocols
For troubleshooting layers 3, all the way up, look at the output of this command:
Router# show ip interfaces GigabitEthernet0/0 is up, line protocol is up Internet address is 10.20.100.1/16 Broadcast address is 255.255.255.255 Address determined by non-volatile memory MTU is 1500 bytes Helper address is not set Directed broadcast forwarding is disabled Multicast reserved groups joined: 224.0.0.10 Outgoing access list is not set Inbound access list is not set Proxy ARP is disabled Local Proxy ARP is disabled Security level is default Split horizon is enabled ICMP redirects are always sent ICMP unreachables are always sent ICMP mask replies are never sent IP fast switching is enabled IP fast switching on the same interface is disabled IP Flow switching is enabled IP CEF switching is enabled IP CEF Flow Fast switching turbo vector IP multicast fast switching is disabled IP multicast distributed fast switching is disabled IP route-cache flags are Fast, Flow cache, CEF, Subint Flow Router Discovery is disabled IP output packet accounting is disabled IP access violation accounting is disabled TCP/IP header compression is disabled RTP/IP header compression is disabled Policy routing is disabled Network address translation is enabled, interface in domain inside WCCP Redirect outbound is disabled WCCP Redirect inbound is disabled WCCP Redirect exclude is disabled BGP Policy Mapping is disabled
Now, let’s say that you have made it to the point where you can ping from LAN to LAN, through your WAN. Congratulations – that is a very good sign. If you are still having trouble, it must be in OSI Layers4-7. Here are those layers listed out and possible issues you might experience in each layer:
Summary
In summary, using the OSI model to troubleshoot connectivity issues is the fastest and most efficient way to troubleshoot any network issue. Even if someone calls you to work on a Windows share problem, all of the same principles in this article apply to that troublesooting process. So remember, the next time you work on a network issue – remember the OSI model and how to use the bottom-up approach to troubleshooting! It could same you a while lot of time!