Cisco CCNP Route – BGP Troubleshooting

Every time you troubleshoot your BGP (Border Gateway Protocol), always follow a tried and tested troubleshooting model. That way, you can work more efficiently and probably even resolve the problem much faster. We usually follow an 8-point troubleshooting model like the one below:

  1. Define the problem
  2. Gather the facts
  3. Document the facts
  4. Consider the possibilities
  5. Create an action plan
  6. Implement the action plan
  7. Repeat until resolution
  8. Document the solution

Just like in most protocols, there are some very specific, common issues and problems, as well as their corresponding solutions. Here in our article on BGP troubleshooting, we’ll be looking at these four potential problems/issues:

  1. Neighbor relationship problems
  2. Route advertisement issues
  3. Routes missing from the table
  4. Address summarization problems


 

Neighbor Relationship Problems

There are a number of possible reasons that may prevent neighbors from establishing a relationship. Here are some of the most common causes of neighbor relationship problems: Possible Cause: Layer 2 or Interface is down

If the Layer 2 process or Interface is down, it may prevent a neighbor relationship from forming. The easiest way to determine that this is the problem when it’s stuck in the Active or Idle state is by executing the show interface <slot/port>. For example, execute: show interfaces fastethernet 0/0 or show interfaces gigabitethernet 0/0 When the output is displayed, look at the interface that’s facing the BGP peer. You should be able to see a line where it says something like:

Fast Ethernet 0/0 is up, line protocol is up.

The first part (Fast Ethernet 0/0 is up) is talking about the physical hardware portion, while the second part (line protocol is up) is talking about the Layer 2 process.

So if you see something like: Fast Ethernet 0/0 is up, line protocol is down, that means the physical interface is fine but ethernet keepalives are not sent/received on the interface. Sometimes, the reason is that something isn’t plugged-in on the other side. Another reason could be you simply have a bad cable.

If, however, the interface shows to be administratively down, then you can remedy that by simply issuing the no shutdown command on the interface.

In figuring out Layer 1 and Layer 2 problems, always start at Layer 1 and work your way up. Troubleshoot and solve the Layer 1 and Layer 2 issues on that particular interface. If it’s a hardware failure, if it’s a cable failure, or something more subtle, this method will be the best way to take care of this particular cause. Possible Cause: Incorrect IP address used in neighbor statement in BGP An incorrect IP address in the neighbor statement in BGP may also cause neighbor relationship problems.

Consider this diagram: BGP Troubleshooting Router Relationships So, for example, in router R1, the configuration should be: router bgp 65001 neighbor 10.99.99.6 remote-as 65006 (this is the neighbor statement for R6, for peering)

Conversely, on router R6, the statement in the configuration should be: router bgp 65006 neighbor 10.99.99.1 remote-as 65001 If, for example, the IP address on either R1 or R6 is wrong, the relationship will not form. So the easiest way to figure out if there’s a problem is to first execute the show ip bgp summary command. It will show you the state of the neighbor relationship, how many prefixes if any are being received, and whether the neighbor is stuck in Idle or Active state.

In addition to that, you can also execute the show running config command to verify that the peer address in the neighbor  statements are correctly configured in BGP.

One of the things that’s going to be helpful is to: go in using PuTTY, SecureCRT, Tera Term or whatever terminal services program you’re most comfortable with; highlight the BGP section; paste the configuration into Notepad; and then go into the neighbor router and do the same thing. See if the neighbor configuration statements are in fact correct. If they’re not correct, delete the neighbor statement and recreate them with the correct IP addresses.   Possible Cause: Configured eBGP peers are not directly connected Although eBGP peers that aren’t directly connected can still have neighbor relationships, an additional command must be used and a special path must be in place for the neighbor relationship to work.

You can determine whether the neighbor is more than one hop away by executing the traceroute <peer ip> command. You’ll be able to tell very easily by the number of different sections and hops shown in the output.

Once you discover that the neighbor is in fact more than one hop away, you can go back into the BGP configuration and put in the statement: neighbor <peer ip> ebgp-multihop <ttl>. ttl stands for time-to-live, which can be a value from 1 to 255 and which refers to the maximum number of hops that can be taken to reach the peer. If it’s 2 hops away, you put 2. If it’s 3 hops away, you put 3, and so on. You may also leave that parameter blank. If you do that, the default value of 255 will be applied.

Another thing you can use is the show tcp command, which you execute in the command-line. It will show you whether a TCP connection is up and in progress with that particular neighbor. Subsequently, it will allow you to verify establishment of a TCP session on port 179 toward a BGP peer. Possible Cause: Misconfigured neighbor authentication Another cause that’s very common for neighbor relationships not forming in BGP is the failure to configure the correct key in the password statement.

If you do a show ip bgp summary and you see that the neighbor relationships are indeed stuck in Active or Idle, the easiest way to resolve this is to execute the show running config command and then verify that the neighbor <peer ip> password <key> command is present on both peers and that the configured keys match.

Again, it would help if you copy those snippets and paste them into Notepad. That way, you can view the snippets all by themselves as you try to compare them and spot anything missing or mistyped. Remember that the key is case-sensitive. If that statement is incorrect, you don’t have to delete the entire configuration. Just recreate the neighbor <peer ip> password <key> statements on both BGP peers. Possible Cause: Unicast traffic being blocked between devices This possible cause for neighbor relationships not forming is not as common as the those mentioned earlier. However, it’s still a possible cause nevertheless. In this case, Unicast traffic between the peers is being blocked. Blocking can be caused by a firewall, an access list, or a variety of different things.

Most likely, it’s going to be something configured on one or the other peers that may be blocking it but if there’s an intermediate device, then you’ll have to take a good look at that device’s configuration as well.

But in the particular device in question, execute the show ip interface and show access list commands. Look at all the access lists, see if anything is being applied on the interface facing that particular neighbor, and if there is, either alter the access list to allow the traffic or you may have to alter it altogether.

Once you’re done with that, make sure everything is correct, make sure that the statements for peering are correct, and make sure show interface shows up/up. Possible Cause: Neighbor statement misconfigured under BGP There may be other things in the neighbor statement that may cause problems in neighbor relationships. For instance, timers may be mismatched.

To check whether something’s incorrect with the neighbor statement in BGP, start by executing the show ip bgp summary.  You’ll want to know first whether the neighbor relationships aren’t even shown. If they aren’t shown, then that means the neighbor relationship is not being established and it has not been configured at all.

If it has been configured, you’re going to see at least Active or Idle as a result of that command. At that point, you can also execute the show running-config in order to be able to go through and parse through the configuration. If there are only a few entries, this task is going to be very easy.

However, if it’s really long, then it can be difficult. If so, then paste it to Notepad so you can use the search function to find specifically what you’re looking for. If you know a specific IP that’s correct, you can go to that particular IP where you’ll find the entire string of numbers and commands, which will be listed in order. From that you can gather some information and determine possible causes.

Once you’ve pinpointed what’s wrong, you can delete and recreate the neighbor statements with the correct IP address combinations.

Route Advertisement Issues

Possible Cause: Automatic network summarization is in effect Although there can be a number of things that may prevent routes from being advertised correctly, one of the primary culprits is the auto-summary command.

Both BGP and EIGRP have automatic summarization enabled by default. If you don’t execute the no-auto summary command when you configure BGP, it’s going to automatically summarize into a classical boundary.

To check whether summarization is enabled, use the show ip protocols command. Look for the statement that says, Automatic route summarization is <disabled | enabled>. In EIGRP, it will say, “… is in effect” or “… is not in effect”.

So in BGP, you’ll want to see “disabled”. If it says something different, then you will need to go back into the BGP router configuration and execute the no auto-summary command. Possible Cause: The advertisement does not match a route in the IP routing table If the particular route being injected into BGP by the network statement is not matching a route in the IP routing table, then you can encounter route advertisement problems.

To remedy this, execute the show ip route command to discover whether the route exists in the current routing table. If it is not there, troubleshoot the local router and the IGP to determine the cause. See whether it was even configured in the router or check the IGP to see if there is a particular issue that’s preventing the route from being installed in the routing table.

Once you see the cause of the problem, correct it. Possible Cause: Route synchronization is enabled By default, route synchronization is enabled in BGP. Synchronization means that it has to exist in the IP routing table through an IGP, static routes, or other means, or BGP will not advertise the route.

To solve this problem, first execute the show ip protocols command and check whether the statement says IGP synchronization is enabled. If so, go back into the BGP configuration and put in the no synchronization command in order for that behavior to be turned off. Possible Cause: Route redistribution is not configured or is misconfigured  One more cause for a problem with route advertisement issues may have to do with redistribution. Basically, if redistribution has not been configured, BGP won’t pull those routes in. The table has to be populated either through direct injection into BGP using the network statement or through redistribution.

You may use the show ip protocols command and look to see if redistribution has been configured. If it has, then investigate some more to see where the problem lies. It could be that you may have put in an incorrect AS number or a process ID relating to OSPF.

Routes Missing from Table

Now, aside from routes not being advertised correctly in BGP, you may find that certain routes are not showing up in the routing table, which is a little bit of a different issue. They sound the same but route advertisement is actually the BGP process announcing something out, and whether or not something’s being installed in the routing table is a separate issue altogether. Possible Cause: Default route misconfigured If the default route is not showing up, then it’s probably simply because it was not configured correctly. To investigate, execute the show ip route and see whether there is a route to 0.0.0.0 network to begin with. If it’s missing from the table, it won’t be injected or advertised from BGP.

You may also execute the show running-config and verify whether the network 0.0.0.0 and the default-information originate statements are existing and correctly configured in BGP. Add those statements if you find them missing. Possible Cause: Route synchronization is enabled We discussed this earlier as a possible cause for route advertisement issues, so just review that section if you want. Possible Cause: iBGP neighbors not fully meshed You may have a problem with iBGP neighbors because these neighbors are not full meshed. Remember, when the routes are being advertised among iBGP peers, it has a ttl of 1. If they’re not all fully meshed, you won’t be able to propagate routes correctly.

To verify that peerings are happening correctly, use the show ip bgp neighbors command. You may also use the show running-config command and do a little bit more investigative work in order to determine whether or not the neighbors are in fact fully meshed.

If there are connections that are missing, configure them using the <peer ip> remote-as <ASN> command. For example, let’s bring back the diagram we showed you earlier. BGP Troubleshooting Router Relationships If these were IBGP routers and there were direct connections between the Internet Router and Router 1 as well as between Router 1 and Router 6, but there was no connection between the Internet Router and Router 6, that would be a missing leg to a fully-meshed environment.

So if you have that kind of missing connection, you can then use the <peer ip> remote-as <ASN> command to remedy the problem.

Finally, execute the show ip bgp summary command, make sure that all the other peers that you’re looking for are indeed there and that each one of the neighbors shows all the other peers that are required. That will be your best way of determining whether or not a fully meshed relationship has been configured. Possible Cause: iBGP next-hop is not reachable One of the ways for you to check whether this is an issue is to do the show ip route command and examine the IP routing table for the required static/IGP route. And this is true with even eBGP. If the next-hop route is not reachable, it’s not going to be advertised.

If a static route is required, configure it using the ip route <prefix> <mask> <next-hop> command. Now, if for some reason, the IGP is failing to install the route leading to the iBGP next-hop address, you may have a whole other set of issues just related to the IGP.

In that instance, you have to make sure you’re looking at BGP as one sort of troubleshooting domain and the IGP as another. They depend on each other but they can have different issues.

Address Summarization Problems

Possible Cause: No subnets in the routing table in the range of the aggregate address In terms of address summarization problems, one common cause for an aggregate not being advertised is because there isn’t some part of the IP address range within the aggregate that exists already in the routing table.

The easiest way to check that is to execute the show ip route. See if any networks within that range currently exist. If you were advertising an aggregate address 216.145.0.0 and the mask is 255.255.252.0 and there are no 216.145-any type addresses in terms of the routing table, it’s not going to advertise the aggregate.

Execute the show running-config to verify that the aggregate address statement is correctly configured. If it is not, delete and recreate the statement that need corrections.

Relevant Debug and Show Commands

The troubleshooting process in BGP requires the use of several Show and a couple of Debug commands that you should be familiar with. BGP Show Commands Show ip protocols This will show the BGP status of your neighbor, your Autonomous System Number (ASN), timer intervals, networks being advertised, and neighbors/gateways. Consequently, it will be helpful in troubleshooting neighbor relationships, missing routes, summarization issues, and filtering problems.

Here’s a portion of an output resulting from a show ip protocols command: BGP Troubleshooting Show IP Protocols Show ip route bgp This will show BGP routes advertised by neighbors, the administrative distance, MED value (if any), and the route source/interface. These information will be helpful in troubleshooting missing routes, summarization issues, filtering problems, and general troubleshooting tasks. BGP Troubleshooting Show IP Route Show ip bgp summary This will show your configured peer IP addresses, your ASN, the number of messages received and sent, how long the session has been up, the state (Active, Idle, etc.), and prefixes received. This will be helpful in troubleshooting neighbor relationship issues, local configuration issues, and routing problems. BGP Troubleshooting BGP Summary Show ip bgp neighbors There are two forms of this command. The first one is just show ip bgp neighbors, which will show you the IP address of neighbors, router ID of neighbors, state of a neighbor, peer-group (if configured), BGP messages/types received, and so forth.

However, you can also specify the peer ip address and then add either of these two optional parameters received-routes and advertised-routes. Received-routes is for routes being received by that particular neighbor, while advertised-routes is for routes advertised out to a particular neighbor.

Here’s the syntax: show ip bgp neighbors [peer ip address] {received-routes | advertised-routes} That will show you how many routes you are actually advertising or receiving and can be extremely helpful in attacking problems associated with routes missing and so forth. It can also be helpful in troubleshooting neighbor relationships, local configuration issues, and other issues associated with BGP routes.

Here’s part of what I got when I issued the command: show ip bgp neighbors 10.6.6.6 05 show ip bgp neighbors with ip Now, here’s part of what I got when I used show ip bgp neighbors 10.6.6.6 advertised-routes instead: BGP Troubleshooting Neighbors Advertised Routes Notice the default route that’s being advertised at the top, followed by a whole host of prefixes.

Lastly, here’s part of what I got when I used show ip bgp neighbors 10.6.6.6 received-routes BGP Troubleshooting Neighbors Received Routes It’s practically the same type of information but it’s showing what’s being received. Show ip bgp This shows the entire contents of the routing information base. It will show your network mask/prefix, next-hop router, MED value, Local Preference Value, Weight Value, AS path, and so forth.

In turn, that information is going to be helpful in troubleshooting missing routes, filtering problems, and general troubleshooting tasks.

Now we go to the Debug commands.

BGP Debug Commands Debug commands display results that are more real-time views of the situation compared to the results of the show commands, which mostly allow you only to look at settings.

debug ip bgp <peer ip> updates This will show you everything that has to do with neighbor adjacencies and relationships. So it’s good for troubleshooting neighbor relationships and routing update issues.

BGP Troubleshooting BGP Updates

debug ip bgp events

This is helpful in looking at different network events like interfaces that go up and down, processes that change, neighbor relationships that change, and so forth. It can help in troubleshooting routing update issues, convergence issues, and interface flapping.

 Conclusion

That wraps up our discussion on BGP troubleshooting. I hope you learned a lot and I look forward to having you here again.