The Myth of eBGP Multihop Sessions Between Loopback Interfaces
Overview
Introduction
External BGP sessions, commonly referred to as eBGP sessions, use a default TTL of 1. Nevertheless, this behavior is not explicitly specified in the RFCs, beginning with RFC 1105, which was the first document to describe BGP and detail its core functionality and characteristics. Therefore, when establishing an eBGP session with a peer that is one or more Layer 3 hops away, the neighbor <remote-peer-IP> ebgp-multihop command must be used. This command also allows you to specify the TTL for BGP packets by appending the desired value. If not explicitly configured, the TTL value used will be 255.
Specificare che one hop away significa che il peering non + fatto su una network adiacente in cui ci sono entrambi gli IP
SIstemare la prima sezione. Therefore, when establishing an eBGP session with a peer that is one or more Layer 3 hops away, the neighbor ebgp-multihop command must be used. In Cisco therminology, this would require the use of neighbor ebgp-multihop command. E’ solo in csico che deve essere utilizzato questa tipolgoai di comando.
Aggiungere la nota che inserendo solamente il comando senza specificare il parametro nella show runn comapre il valore 255
For a comprehensive list of RFCs covering BGP and its associated features and updates, refer to the RFC Editor portal.
Scenario Overview
Before diving into the main topic, a brief overview of the topology is presented.
In this topology, two routers, R1 and R2, are configured and belong to separate Autonomous Systems, AS 65100 and AS 65200, respectively. The initial configuration for R1 is shown below.
R1#show running-configuration | s r b
router bgp 65100
bgp log-neighbor-changes
neighbor 172.25.2.2 remote-as 65200
neighbor 172.25.2.2 ebgp-multihop 255
neighbor 172.25.2.2 update-source Loopback0
!
address-family ipv4
neighbor 172.25.2.2 activate
exit-address-family
The initial configuration for R2 is shown below.
R2#show running-configuration | s r b
router bgp 65200
bgp log-neighbor-changes
neighbor 172.25.1.1 remote-as 65100
neighbor 172.25.1.1 ebgp-multihop 255
neighbor 172.25.1.1 update-source Loopback0
!
address-family ipv4
neighbor 172.25.1.1 activate
exit-address-family
The configuration follows a standard setup, with the session established between the Loopback 0 interfaces of R1 and R2. Since this is an eBGP session, the neighbor <remote-peer-IP> ebgp-multihop command has been applied to increase the TTL. For illustration, the AFI/SAFI used here is IPv4 Unicast. To verify the status of a BGP neighborship, either show bgp neighbors <remote-peer-IP> or the more recent show ip bgp neighbors <remote-peer-IP> command can be used. In the example below, the latter command is executed on R1 to display the state of the BGP session with R2; the full output is included for completeness.
R1#show ip bgp neighbors 172.25.2.2
BGP neighbor is 172.25.2.2, remote AS 65200, external link
BGP version 4, remote router ID 172.25.0.2
BGP state = Established, up for 16:57:49
Last read 00:00:02, last write 00:00:03, hold time is 180, keepalive interval is 60 seconds
Neighbor sessions:
1 active, is not multisession capable (disabled)
Neighbor capabilities:
Route refresh: advertised and received(new)
Four-octets ASN Capability: advertised and received
Address family IPv4 Unicast: advertised and received
Enhanced Refresh Capability: advertised and received
Multisession Capability:
Stateful switchover support enabled: NO for session 1
Message statistics:
InQ depth is 0
OutQ depth is 0
Sent Rcvd
Opens: 1 1
Notifications: 0 0
Updates: 2 3
Keepalives: 1121 1122
Route Refresh: 0 0
Total: 1126 1128
Do log neighbor state changes (via global configuration)
Default minimum time between advertisement runs is 30 seconds
For address family: IPv4 Unicast
Session: 172.25.2.2
BGP table version 2, neighbor version 2/0
Output queue size : 0
Index 9, Advertise bit 0
9 update-group member
Slow-peer detection is disabled
Slow-peer split-update-group dynamic is disabled
Sent Rcvd
Prefix activity: ---- ----
Prefixes Current: 1 0
Prefixes Total: 3 0
Implicit Withdraw: 2 0
Explicit Withdraw: 0 0
Used as bestpath: n/a 0
Used as multipath: n/a 0
Used as secondary: n/a 0
Outbound Inbound
Local Policy Denied Prefixes: -------- -------
AS_PATH loop: n/a 1
Total: 0 1
Number of NLRIs in the update sent: max 1, min 0
Last detected as dynamic slow peer: never
Dynamic slow peer recovered: never
Refresh Epoch: 2
Last Sent Refresh Start-of-rib: 16:57:49
Last Sent Refresh End-of-rib: 16:57:49
Refresh-Out took 0 seconds
Last Received Refresh Start-of-rib: 16:57:49
Last Received Refresh End-of-rib: 16:57:49
Refresh-In took 0 seconds
Sent Rcvd
Refresh activity: ---- ----
Refresh Start-of-RIB 1 1
Refresh End-of-RIB 1 1
Address tracking is enabled, the RIB does have a route to 172.25.2.2
Route to peer address reachability Up: 1; Down: 0
Last notification 16:58:55
Connections established 1; dropped 0
Last reset never
External BGP neighbor may be up to 255 hops away.
External BGP neighbor NOT configured for connected checks (multi-hop no-disable-connected-check)
Interface associated: (none) (peering address NOT in same link)
Transport(tcp) path-mtu-discovery is enabled
Graceful-Restart is disabled
SSO is disabled
Connection state is ESTAB, I/O status: 1, unread input bytes: 0
Connection is ECN Disabled, Mininum incoming TTL 0, Outgoing TTL 255
Local host: 172.25.1.1, Local port: 179
Foreign host: 172.25.2.2, Foreign port: 53502
Connection tableid (VRF): 0
Maximum output segment queue size: 50
Enqueued packets for retransmit: 0, input: 0 mis-ordered: 0 (0 bytes)
Event Timers (current time is 0x4119FB9A):
Timer Starts Wakeups Next
Retrans 1123 0 0x0
TimeWait 0 0 0x0
AckHold 1125 1108 0x0
SendWnd 0 0 0x0
KeepAlive 0 0 0x0
GiveUp 0 0 0x0
PmtuAger 0 0 0x0
DeadWait 0 0 0x0
Linger 0 0 0x0
ProcessQ 0 0 0x0
iss: 3526559991 snduna: 3526581471 sndnxt: 3526581471
irs: 2269872968 rcvnxt: 2269894491
sndwnd: 15396 scale: 0 maxrcvwnd: 16384
rcvwnd: 15350 scale: 0 delrcvwnd: 1034
SRTT: 1000 ms, RTTO: 1003 ms, RTV: 3 ms, KRTT: 0 ms
minRTT: 1 ms, maxRTT: 1000 ms, ACK hold: 200 ms
uptime: 61069803 ms, Sent idletime: 2047 ms, Receive idletime: 2247 ms
Status Flags: passive open, gen tcbs
Option Flags: nagle, path mtu capable
IP Precedence value : 6
Datagrams (max data segment is 1460 bytes):
Rcvd: 2223 (out of order: 0), with data: 1126, total data bytes: 21522
Sent: 2249 (retransmit: 0, fastretransmit: 0, partialack: 0, Second Congestion: 0), with data: 1124, total data bytes: 21479
Packets received in fast path: 0, fast processed: 0, slow path: 0
fast lock acquisition failures: 0, slow path: 0
TCP Semaphore 0x7F193B70BCD0 FREE
As shown, the session is established with the configuration above.
BGP state = Established, up for 16:57:49
The TTL for this session is confirmed to be 255, as noted previously.
Connection is ECN Disabled, Minimum incoming TTL 0, Outgoing TTL 255
R2’s Loopback IP used for the session is not on a subnet directly configured on R1.
Interface associated: (none) (peering address NOT in same link)
Last but not least, the following output plays a key role in this post.
External BGP neighbor NOT configured for connected checks (multi-hop no-disable-connected-check)
Examining the eBGP Multihop Claim
With the simple topology and its configurations reviewed, it is now possible to revisit the initial claim: for eBGP sessions established between two peer IP addresses separated by one or more Layer 3 hops, the neighbor <remote-peer-IP> ebgp-multihop command must be used. In the case of directly connected routers, where an eBGP session is configured between Loopback interfaces, the applicability of this rule warrants further examination. It is often argued that BGP traffic from R1 to R2 destined for R2’s Loopback interface would have its TTL decremented, implying the command is needed to adjust the TTL and ensure correct delivery.
However, this assertion is incorrect. Consider how R2 handles the TTL field of a BGP packet received from R1. Since the packet’s destination IP (172.25.2.2) is assigned to a local logical interface, the TTL is never decremented. TTL is only decremented when a packet is forwarded through a physical interface to an adjacent device. Therefore, a TTL of 1 is sufficient to establish an eBGP session between the Loopback interfaces of two directly connected routers.
Consequently, removing the neighbor <remote-peer-IP> ebgp-multihop command from the configuration does not prevent the session from being established. To validate this, the command can be removed on both R1 and R2, and the session behavior can be observed.
The following command is then applied on R1.
R1(config-router)#no neighbor 172.25.2.2 ebgp-multihop
The same procedure applies to R2.
R2(config-router)#no neighbor 172.25.1.1 ebgp-multihop
Debugging can then be enabled on R1, for example, using the debug ip bgp 172.25.2.2 command to observe the session behavior in greater detail. To expedite the process, the session can be reset with the clear ip bgp * command. Among the resulting messages, two specific errors provide clear insight into the observed behavior.
Dec 15 20:22:46.340: BGP: nbr global 172.25.2.2 Open active delayed 7168ms (35000ms max, 60% jitter)
Dec 15 20:22:46.340: BGP: nbr global 172.25.2.2 Active open failed - open timer running
If a packet capture is performed on the link connecting the two GigabitEthernet 1 interfaces of the routers, it becomes evident that neither R1 nor R2 sent any packets to their respective peer to establish a new eBGP session. The only packets present, shown below, are those generated by the session reset triggered using the clear command.
| No. | Source | Destination | Protocol | Info |
|---|---|---|---|---|
| 1 | 172.25.2.2 | 172.25.1.1 | BGP | KEEPALIVE Message |
| 2 | 172.25.1.1 | 172.25.2.2 | TCP | 179 → 28357 [ACK] Seq=1 Ack=20 Win=16209 Len=0 |
| 3 | 172.25.1.1 | 172.25.2.2 | BGP | NOTIFICATION Message |
| 4 | 172.25.2.2 | 172.25.1.1 | TCP | 28357 → 179 [FIN, PSH, ACK] Seq=20 Ack=22 Win=16134 Len=0 |
| 5 | 172.25.1.1 | 172.25.2.2 | TCP | 179 → 28357 [ACK] Seq=22 Ack=21 Win=16209 Len=0 |
| 6 | 172.25.1.1 | 172.25.2.2 | TCP | 179 → 28357 [FIN, PSH, ACK] Seq=22 Ack=21 Win=16209 Len=0 |
| 7 | 172.25.2.2 | 172.25.1.1 | TCP | 28357 → 179 [ACK] Seq=21 Ack=23 Win=16134 Len=0 |
Removing this command from the configuration did not allow the eBGP session to be established. What, then, is preventing the session from forming successfully? To investigate, the output of the show ip bgp neighbors 172.25.2.2 command was retrieved again from R1. This output can be compared with the initial one, obtained when the neighbor <remote-peer-IP> ebgp-multihop command was still present on both routers. From the complete output, only a single entry is relevant.
This is the output obtained when the neighbor <remote-peer-IP> ebgp-multihop command was applied on both R1 and R2.
External BGP neighbor NOT configured for connected checks (multi-hop no-disable-connected-check)
This is the output obtained after removing the neighbor <remote-peer-IP> ebgp-multihop command from the configuration on both R1 and R2.
External BGP neighbor configured for connected checks (single-hop no-disable-connected-check)
Between these two outputs, two key differences can be observed:
- In the first case, the session is defined as multihop due to the applied command, whereas in the second case, it is classified as single-hop.
- In the first case, the session is not configured for the connected check, while in the second case, it is.
Of the two points, the second is the most significant. During the fundamental steps R1 (and R2) perform to establish an external BGP session, a critical verification occurs: the device checks whether the configured remote peer IP belongs to a directly connected subnet. If this check fails, the eBGP session cannot be established. In the case of sessions between the Loopback interfaces of adjacent routers, these addresses cannot belong to the same subnet (unless misconfigured). As a result, this verification prevents the session from being successfully formed.
Disabling the Connected Check Option
The solution is to disable this verification while keeping the TTL at 1. This is achieved by applying the command neighbor <remote-peer-IP> disable-connected-check within BGP configuration mode, as shown below.
R1(config)#router bgp 65100
R1(config-router)#neighbor 172.25.2.2 disable-connected-check
The same command must also be applied on R2.
R2(config)#router bgp 65200
R2(config-router)#neighbor 172.25.1.1 disable-connected-check
Once applied on both routers, the session is correctly established.
R1#show bgp neighbors 172.25.2.2 | include BGP state
BGP state = Established, up for 00:26:45
The updated configuration for R1 is presented below.
R1#show running-configuration | s r b
router bgp 65100
bgp log-neighbor-changes
neighbor 172.25.2.2 remote-as 65200
neighbor 172.25.2.2 disable-connected-check
neighbor 172.25.2.2 update-source Loopback0
!
address-family ipv4
network 10.100.1.0 mask 255.255.255.0
neighbor 172.25.2.2 activate
exit-address-family
The same applies to R2.
R2#show running-configuration | s r b
router bgp 65200
bgp log-neighbor-changes
neighbor 172.25.1.1 remote-as 65100
neighbor 172.25.1.1 disable-connected-check
neighbor 172.25.1.1 update-source Loopback0
!
address-family ipv4
neighbor 172.25.1.1 activate
exit-address-family
At this point, the output of the show ip bgp neighbors 172.25.2.2 command can be reviewed on R1 to examine the session with R2, filtering to show only the relevant information.
R1#show ip bgp neighbors 172.25.2.2 | include connected checks|same link|Outgoing TTL
External BGP neighbor NOT configured for connected checks (single-hop disable-connected-check)
Interface associated: (none) (peering address NOT in same link)
Connection is ECN Disabled, Mininum incoming TTL 0, Outgoing TTL 1
The session is still classified as single-hop, since the neighbor <remote-peer-IP> ebgp-multihop command has not been applied, while the connected check has been disabled. Moreover, the TTL value remains at its default of 1.
Takeaways
In conclusion, we have confirmed that, for directly connected routers, a BGP session can be successfully established between their loopback interfaces even when using packets with a TTL of 1.
I hope this post has been helpful. If you have any additional information to share, feel free to reach out via social media. See you in the next one! 🙂