The Myth of eBGP Multihop Between Loopback Interfaces
Overview
Introduction
External BGP sessions, as is well known, are referred to as eBGP sessions and have the characteristic of using a default TTL value of 1. Nevertheless, this behavior is not explicitly specified in the RFCs, starting with RFC 1105, which was the first to describe BGP and detail its core functionality and characteristics. For a comprehensive list of RFCs covering BGP and its associated features and updates, refer to the RFC Editor portal.
Therefore, when establishing an eBGP session with a peer that is more than one Layer 3 hop away, the neighbor <remote-peer-IP> ebgp-multihop command must be used. This command also allows specifying the TTL for BGP packets. If not explicitly configured, the TTL value used will be 255. The command used to specify the TTL value is neighbor <remote-peer-IP> ebgp-multihop <TTL-value>
Scenario Overview
Before we get into the focus of this post, let’s take a brief look at the topology.
In this topology, two routers, R1 and R2, are configured and belong to separate Autonomous Systems, AS 65100 and AS 65200, respectively. The initial configuration for R1 is shown below.
R1#show running-configuration | s r b
router bgp 65100
bgp log-neighbor-changes
neighbor 172.25.2.2 remote-as 65200
neighbor 172.25.2.2 ebgp-multihop 255
neighbor 172.25.2.2 update-source Loopback0
!
address-family ipv4
neighbor 172.25.2.2 activate
exit-address-family
The initial configuration for R2 is shown below.
R2#show running-configuration | s r b
router bgp 65200
bgp log-neighbor-changes
neighbor 172.25.1.1 remote-as 65100
neighbor 172.25.1.1 ebgp-multihop 255
neighbor 172.25.1.1 update-source Loopback0
!
address-family ipv4
neighbor 172.25.1.1 activate
exit-address-family
The configuration follows a standard setup. The session has been established between the Loopback 0 interface of R1, 172.25.1.1 and the Loopback 0 interface of R2, 172.25.2.2. As this is an eBGP session, the neighbor <remote-peer-IP> ebgp-multihop command has been applied to increase the TTL value. For illustrative purposes, the AFI/SAFI used here is IPv4 Unicast. To verify the status of a BGP neighborship, either show bgp neighbors <remote-peer-IP> or the more recent show ip bgp neighbors <remote-peer-IP> command can be used. Below, the latter command has been executed on R1 to display the state of the BGP session established with R2 (for completeness, the full output of the command has been included).
R1#show ip bgp neighbors 172.25.2.2
BGP neighbor is 172.25.2.2, remote AS 65200, external link
BGP version 4, remote router ID 172.25.0.2
BGP state = Established, up for 16:57:49
Last read 00:00:02, last write 00:00:03, hold time is 180, keepalive interval is 60 seconds
Neighbor sessions:
1 active, is not multisession capable (disabled)
Neighbor capabilities:
Route refresh: advertised and received(new)
Four-octets ASN Capability: advertised and received
Address family IPv4 Unicast: advertised and received
Enhanced Refresh Capability: advertised and received
Multisession Capability:
Stateful switchover support enabled: NO for session 1
Message statistics:
InQ depth is 0
OutQ depth is 0
Sent Rcvd
Opens: 1 1
Notifications: 0 0
Updates: 2 3
Keepalives: 1121 1122
Route Refresh: 0 0
Total: 1126 1128
Do log neighbor state changes (via global configuration)
Default minimum time between advertisement runs is 30 seconds
For address family: IPv4 Unicast
Session: 172.25.2.2
BGP table version 2, neighbor version 2/0
Output queue size : 0
Index 9, Advertise bit 0
9 update-group member
Slow-peer detection is disabled
Slow-peer split-update-group dynamic is disabled
Sent Rcvd
Prefix activity: ---- ----
Prefixes Current: 1 0
Prefixes Total: 3 0
Implicit Withdraw: 2 0
Explicit Withdraw: 0 0
Used as bestpath: n/a 0
Used as multipath: n/a 0
Used as secondary: n/a 0
Outbound Inbound
Local Policy Denied Prefixes: -------- -------
AS_PATH loop: n/a 1
Total: 0 1
Number of NLRIs in the update sent: max 1, min 0
Last detected as dynamic slow peer: never
Dynamic slow peer recovered: never
Refresh Epoch: 2
Last Sent Refresh Start-of-rib: 16:57:49
Last Sent Refresh End-of-rib: 16:57:49
Refresh-Out took 0 seconds
Last Received Refresh Start-of-rib: 16:57:49
Last Received Refresh End-of-rib: 16:57:49
Refresh-In took 0 seconds
Sent Rcvd
Refresh activity: ---- ----
Refresh Start-of-RIB 1 1
Refresh End-of-RIB 1 1
Address tracking is enabled, the RIB does have a route to 172.25.2.2
Route to peer address reachability Up: 1; Down: 0
Last notification 16:58:55
Connections established 1; dropped 0
Last reset never
External BGP neighbor may be up to 255 hops away.
External BGP neighbor NOT configured for connected checks (multi-hop no-disable-connected-check)
Interface associated: (none) (peering address NOT in same link)
Transport(tcp) path-mtu-discovery is enabled
Graceful-Restart is disabled
SSO is disabled
Connection state is ESTAB, I/O status: 1, unread input bytes: 0
Connection is ECN Disabled, Mininum incoming TTL 0, Outgoing TTL 255
Local host: 172.25.1.1, Local port: 179
Foreign host: 172.25.2.2, Foreign port: 53502
Connection tableid (VRF): 0
Maximum output segment queue size: 50
Enqueued packets for retransmit: 0, input: 0 mis-ordered: 0 (0 bytes)
Event Timers (current time is 0x4119FB9A):
Timer Starts Wakeups Next
Retrans 1123 0 0x0
TimeWait 0 0 0x0
AckHold 1125 1108 0x0
SendWnd 0 0 0x0
KeepAlive 0 0 0x0
GiveUp 0 0 0x0
PmtuAger 0 0 0x0
DeadWait 0 0 0x0
Linger 0 0 0x0
ProcessQ 0 0 0x0
iss: 3526559991 snduna: 3526581471 sndnxt: 3526581471
irs: 2269872968 rcvnxt: 2269894491
sndwnd: 15396 scale: 0 maxrcvwnd: 16384
rcvwnd: 15350 scale: 0 delrcvwnd: 1034
SRTT: 1000 ms, RTTO: 1003 ms, RTV: 3 ms, KRTT: 0 ms
minRTT: 1 ms, maxRTT: 1000 ms, ACK hold: 200 ms
uptime: 61069803 ms, Sent idletime: 2047 ms, Receive idletime: 2247 ms
Status Flags: passive open, gen tcbs
Option Flags: nagle, path mtu capable
IP Precedence value : 6
Datagrams (max data segment is 1460 bytes):
Rcvd: 2223 (out of order: 0), with data: 1126, total data bytes: 21522
Sent: 2249 (retransmit: 0, fastretransmit: 0, partialack: 0, Second Congestion: 0), with data: 1124, total data bytes: 21479
Packets received in fast path: 0, fast processed: 0, slow path: 0
fast lock acquisition failures: 0, slow path: 0
TCP Semaphore 0x7F193B70BCD0 FREE
As shown, with the configuration above, the session state is marked as Established.
BGP state = Established, up for 16:57:49
The TTL for this session is 255, as noted earlier.
Connection is ECN Disabled, Minimum incoming TTL 0, Outgoing TTL 255
The IP address of R2’s Loopback interface used to establish the session belongs to a network that is not locally configured on R1.
Interface associated: (none) (peering address NOT in same link)
Another relevant detail from the previous output is shown below.
External BGP neighbor NOT configured for connected checks (multi-hop no-disable-connected-check)
Examining the eBGP Multihop Claim
With the (simple) topology and its configurations reviewed, it is now possible to revisit the initial claim: for eBGP sessions established between two IP addresses of peers separated by more than one Layer 3 hop, the command neighbor <remote-peer-IP> ebgp-multihop must be used. In the case of directly connected routers, where an eBGP session is configured between Loopback interfaces, the applicability of this rule deserves further examination. It is often argued that, when considering traffic from R1 to R2, if R2 were to receive a message containing BGP information destined for its Loopback interface, the TTL would be decremented. From this perspective, the command would be required to adjust the TTL value and ensure correct delivery to R2’s Loopback 0 interface.
However, this assertion is incorrect. Without loss of generality, consider how router R2 handles the TTL field of a BGP packet received from R1. Since the destination IP address of the packet (172.25.2.2) is assigned to a local logical interface, the TTL value is never decremented. TTL is only decremented when a packet is forwarded through a physical interface to an adjacent device. Therefore, a TTL value of 1 remains sufficient to establish an eBGP session between the Loopback interfaces of two directly connected devices.
As a result, it can be reasoned that removing the neighbor <remote-peer-IP> ebgp-multihop command from the configuration would not prevent the session from being established. To validate this, the command can be removed on both R1 and R2 and the session behavior observed.
The command to enter on R1 is as follows.
R1(config)#router bgp 65100
R1(config-router)#no neighbor 172.25.2.2 ebgp-multihop
The same command, appropriately adjusted, should be applied on R2.
R2(config)#router bgp 65200
R2(config-router)#no neighbor 172.25.1.1 ebgp-multihop
The debug can then be enabled, for example on R1 using the debug ip bgp 172.25.2.2 command, to observe the behavior in greater detail. To expedite the process, the session can be reset using the clear ip bgp * command. Among the resulting messages, two specific errors provide clear insight into the observed behavior.
Dec 15 20:22:46.340: BGP: nbr global 172.25.2.2 Open active delayed 7168ms (35000ms max, 60% jitter)
Dec 15 20:22:46.340: BGP: nbr global 172.25.2.2 Active open failed - open timer running
If a packet capture is performed on the link connecting the two GigabitEthernet 1 interfaces of the routers, it becomes evident that neither R1 nor R2 sent any packets to their respective peer to establish a new eBGP session. The only packets present, shown below, are those generated by the session reset triggered using the clear command.
| No. | Source | Destination | Protocol | Info |
|---|---|---|---|---|
| 1 | 172.25.2.2 | 172.25.1.1 | BGP | KEEPALIVE Message |
| 2 | 172.25.1.1 | 172.25.2.2 | TCP | 179 → 28357 [ACK] Seq=1 Ack=20 Win=16209 Len=0 |
| 3 | 172.25.1.1 | 172.25.2.2 | BGP | NOTIFICATION Message |
| 4 | 172.25.2.2 | 172.25.1.1 | TCP | 28357 → 179 [FIN, PSH, ACK] Seq=20 Ack=22 Win=16134 Len=0 |
| 5 | 172.25.1.1 | 172.25.2.2 | TCP | 179 → 28357 [ACK] Seq=22 Ack=21 Win=16209 Len=0 |
| 6 | 172.25.1.1 | 172.25.2.2 | TCP | 179 → 28357 [FIN, PSH, ACK] Seq=22 Ack=21 Win=16209 Len=0 |
| 7 | 172.25.2.2 | 172.25.1.1 | TCP | 28357 → 179 [ACK] Seq=21 Ack=23 Win=16134 Len=0 |
Therefore, removing this command from the configuration did not allow the eBGP session to be established. What, then, is preventing the session from being successfully formed? To investigate further, the output of the show ip bgp neighbors 172.25.2.2 command was retrieved again from R1. This output can then be compared with the initial one, obtained when the neighbor <remote-peer-IP> ebgp-multihop command was still present in the configuration of both routers. From the entire output, only a single entry is needed.
This is the output obtained when the neighbor <remote-peer-IP> ebgp-multihop command was applied on both R1 and R2.
External BGP neighbor NOT configured for connected checks (multi-hop no-disable-connected-check)
This is the output obtained after removing the neighbor <remote-peer-IP> ebgp-multihop command from the configuration on both R1 and R2.
External BGP neighbor configured for connected checks (single-hop no-disable-connected-check)
Between these two outputs, two key differences can be observed:
- In the first case, the session is defined as multihop due to the applied command, whereas in the second case, it is classified as single-hop.
- In the first case, the session is not configured for the connected check, while in the second case, it is.
Between these 2 points, the second is the most important. Reviewing the fundamental steps taken by R1 (and R2) to establish an external BGP session, a crucial verification occurs: the device checks whether the configured remote peer IP belongs to a directly connected subnet. If this check fails, the eBGP session establishment is halted. For sessions between the Loopback interfaces of adjacent routers, these addresses cannot belong to the same subnet (unless misconfigured). Consequently, this check prevents the session from being successfully established.
Disabling the Connected Check Option
The solution is to disable this verification while keeping the TTL value set to 1. This is achieved by applying the command neighbor <remote-peer-IP> disable-connected-check within BGP configuration mode, as shown below.
R1(config)#router bgp 65100
R1(config-router)#neighbor 172.25.2.2 disable-connected-check
The same command must also be applied on R2.
R2(config)#router bgp 65200
R2(config-router)#neighbor 172.25.1.1 disable-connected-check
Once applied on both routers, the session is correctly established.
R1#show bgp neighbors 172.25.2.2 | include BGP state
BGP state = Established, up for 00:26:45
The updated configuration for R1 is presented below.
R1#show running-configuration | s r b
router bgp 65100
bgp log-neighbor-changes
neighbor 172.25.2.2 remote-as 65200
neighbor 172.25.2.2 disable-connected-check
neighbor 172.25.2.2 update-source Loopback0
!
address-family ipv4
network 10.100.1.0 mask 255.255.255.0
neighbor 172.25.2.2 activate
exit-address-family
The same applies to R2.
R2#show running-configuration | s r b
router bgp 65200
bgp log-neighbor-changes
neighbor 172.25.1.1 remote-as 65100
neighbor 172.25.1.1 disable-connected-check
neighbor 172.25.1.1 update-source Loopback0
!
address-family ipv4
neighbor 172.25.1.1 activate
exit-address-family
At this point, the output of the show ip bgp neighbors 172.25.2.2 command can be reviewed on R1 to examine the session with R2, filtering to show only the relevant information.
R1#show ip bgp neighbors 172.25.2.2 | include connected checks|same link|Outgoing TTL
External BGP neighbor NOT configured for connected checks (single-hop disable-connected-check)
Interface associated: (none) (peering address NOT in same link)
Connection is ECN Disabled, Mininum incoming TTL 0, Outgoing TTL 1
The session is still classified as single-hop, since the neighbor <remote-peer-IP> ebgp-multihop command has not been applied, although the connected check has been disabled. Moreover the TTL value remains at its default of 1.
Key Takeaways
In conclusion, we have confirmed that, for directly connected routers, a BGP session can be successfully established between their loopback interfaces even when using packets with a TTL of 1.
I hope this post has been helpful. If you have any additional information to share, feel free to reach out via social media. See you in the next one! 🙂