Commit 89285016bc for frr
commit 89285016bc6bcf8e85fd0d4800ba6744edc9b153
Author: Sougata Barik <sougatab@nvidia.com>
Date: Mon Jul 13 04:54:36 2026 +0000
bfdd: keep transmitting AdminDown after session/profile shutdown
Problem
-------
When a BFD session (or a whole BFD profile) is administratively shut
down, bfd_set_shutdown() moved the session to AdminDown, deleted the
transmit timer, and sent exactly ONE Control packet carrying
State=AdminDown. After that the session went silent.
If that single AdminDown packet is dropped, or simply loses the race
with the remote's detection timer (detect_mult * negotiated interval),
the peer never learns we went AdminDown. Instead it expires the session
with diag "Control Detection Time Expired", declares the session Down,
and tears down whatever protocol BFD protects. For BGP this means the
peer sends a Cease/BFD Down NOTIFICATION and our session resets.
This is nondeterministic and gets worse at scale and with short timers:
shutting a profile fans out an AdminDown burst across many sessions in a
few milliseconds, so a handful reliably lose the race. Observed on a
304-session box: 6-9 BGP sessions reset per profile shut/no-shut with
50ms/mult-3 (150ms detection) timers.
RFC recommendation
------------------
RFC 5880 Section 6.8.16 (Administrative Control):
"BFD Control packets SHOULD be transmitted for at least a Detection
Time after transitioning to AdminDown state in order to ensure that
the remote system is aware of the state change. BFD Control packets
MAY be transmitted indefinitely after transitioning to AdminDown
state in order to maintain session state in each system."
Sending a single packet transmits for ~0 time, violating this SHOULD.
Solution
--------
Do not delete the transmit timer on shutdown. Keep advertising
State=AdminDown, but slow the transmit rate to BFD_DEF_SLOWTX (1s) since
the session is no longer Up (RFC 5880 Section 6.8.3), mirroring the
existing ptm_bfd_sess_dn() DOWN handling. Echo and the receive/detection
timers are still disabled - we only need to keep informing the peer, not
detect it, while administratively down.
The remote now reliably receives AdminDown (diag "Neighbor signaled
session down", RFC 5880 Section 6.8.6) and holds the protocol session up
instead of timing it out. Cost is 1 packet/s per session while shut.
For offloaded (data-plane) sessions the same guarantee must be enforced
in the data-plane program (keep emitting AdminDown); a comment marks
that path.
Ticket: #5433819
Signed-off-by: Sougata Barik <sougatab@nvidia.com>
diff --git a/bfdd/bfd.c b/bfdd/bfd.c
index 2cdaa5024d..9604f987a3 100644
--- a/bfdd/bfd.c
+++ b/bfdd/bfd.c
@@ -1860,7 +1860,7 @@ void bfd_set_shutdown(struct bfd_session *bs, bool shutdown)
SET_FLAG(bs->flags, BFD_SESS_FLAG_SHUTDOWN);
bs->local_diag = BD_ADMIN_DOWN;
- /* Handle data plane shutdown case. */
+ /* Offloaded sessions must also keep sending AdminDown. */
if (bs->bdc) {
bs->ses_state = PTM_BFD_ADM_DOWN;
bfd_dplane_update_session(bs);
@@ -1868,19 +1868,21 @@ void bfd_set_shutdown(struct bfd_session *bs, bool shutdown)
return;
}
- /* Disable all events. */
+ /* Stop receive/echo timers, but keep transmitting AdminDown. */
bfd_recvtimer_delete(bs);
bfd_echo_recvtimer_delete(bs);
- bfd_xmttimer_delete(bs);
bfd_echo_xmttimer_delete(bs);
/* Change and notify state change. */
bs->ses_state = PTM_BFD_ADM_DOWN;
ptm_bfd_notify(bs, bs->ses_state);
- /* Don't try to send packets with a disabled session. */
- if (bs->sock != -1)
+ /* Advertise AdminDown at slow rate while shut. */
+ if (bs->sock != -1) {
+ bs_set_slow_timers(bs);
ptm_bfd_snd(bs, 0);
+ ptm_bfd_start_xmt_timer(bs, false);
+ }
} else {
/* Already working. */
if (!is_shutdown)
diff --git a/bfdd/bfd_trace.h b/bfdd/bfd_trace.h
index 0094a7e16d..160b3b54e4 100644
--- a/bfdd/bfd_trace.h
+++ b/bfdd/bfd_trace.h
@@ -319,6 +319,21 @@ TRACEPOINT_EVENT(
)
)
+/* BFD transmit timer scheduling tracepoint. */
+TRACEPOINT_EVENT(
+ frr_bfd,
+ xmttimer_schedule,
+ TP_ARGS(struct bfd_session *, bs, uint64_t, jitter, bool, scheduled),
+ TP_FIELDS(
+ ctf_integer(uint32_t, local_discr, bs ? bs->discrs.my_discr : 0)
+ ctf_integer(uint8_t, state, bs ? bs->ses_state : 0)
+ ctf_integer(bool, shutdown, bs ? CHECK_FLAG(bs->flags, BFD_SESS_FLAG_SHUTDOWN) : false)
+ ctf_integer(int, socket_fd, bs ? bs->sock : -1)
+ ctf_integer(uint64_t, jitter_us, jitter)
+ ctf_integer(bool, scheduled, scheduled)
+ )
+)
+
/*
* BFD control client event tracepoint
* is_connect: true=connect, false=disconnect
@@ -678,6 +693,7 @@ TRACEPOINT_LOGLEVEL(frr_bfd, packet_remote_discr_zero, TRACE_WARNING)
TRACEPOINT_LOGLEVEL(frr_bfd, packet_ttl_exceeded, TRACE_WARNING)
TRACEPOINT_LOGLEVEL(frr_bfd, echo_packet_error, TRACE_WARNING)
TRACEPOINT_LOGLEVEL(frr_bfd, timer_negotiation, TRACE_INFO)
+TRACEPOINT_LOGLEVEL(frr_bfd, xmttimer_schedule, TRACE_INFO)
TRACEPOINT_LOGLEVEL(frr_bfd, control_client_event, TRACE_INFO)
TRACEPOINT_LOGLEVEL(frr_bfd, echo_mode_change, TRACE_INFO)
TRACEPOINT_LOGLEVEL(frr_bfd, dplane_session_not_found, TRACE_WARNING)
diff --git a/bfdd/event.c b/bfdd/event.c
index 94ba975b11..6221329782 100644
--- a/bfdd/event.c
+++ b/bfdd/event.c
@@ -136,9 +136,11 @@ void bfd_xmttimer_update(struct bfd_session *bs, uint64_t jitter)
/* Remove previous schedule if any. */
bfd_xmttimer_delete(bs);
- /* Don't add event if peer is deactivated. */
- if (CHECK_FLAG(bs->flags, BFD_SESS_FLAG_SHUTDOWN) || bs->sock == -1) {
- bs->xmt_TO_actual = 0; /* Timer not scheduled */
+ /* Skip if inactive; allow scheduling while AdminDown. */
+ if (bs->sock == -1 ||
+ (CHECK_FLAG(bs->flags, BFD_SESS_FLAG_SHUTDOWN) && bs->ses_state != PTM_BFD_ADM_DOWN)) {
+ bs->xmt_TO_actual = 0; /* Unscheduled */
+ frrtrace(4, frr_bfd, xmttimer_schedule, bs, jitter, false);
return;
}
@@ -146,6 +148,7 @@ void bfd_xmttimer_update(struct bfd_session *bs, uint64_t jitter)
event_add_timer_tv(master, bfd_xmt_cb, bs, &tv, &bs->xmttimer_ev);
event_set_tardy_threshold(bs->xmttimer_ev, bs->xmt_TO / 2);
+ frrtrace(4, frr_bfd, xmttimer_schedule, bs, jitter, true);
}
void bfd_echo_xmttimer_update(struct bfd_session *bs, uint64_t jitter)