High-Performance XDP BNG · CGNAT · QoS · Low-Latency · Carrier-Grade Reliability

This cycle was about the control plane under stress: the connect/disconnect storms after an outage, and the slow drift that builds over weeks of uptime. We moved to a newer kernel base (7.1.1), removed a use-after-free crash class, and cut the per-subscriber control-plane work that piles up during churn — then proved each change under load that mirrors a real bad day. The sections below walk the threat, each fix, and the validation that backs it.

1 · The threat — the reconnect storm, not steady traffic

When the power comes back to an area, thousands of CPEs power on within seconds and all try to reconnect at the same instant. The BNG must tear down the dead sessions, then re-establish thousands of new ones — re-authenticating, re-allocating IPv4 and IPv6 addresses, re-installing routes — in a burst. This is the single most demanding thing a BNG does, and it is where weak implementations crash, leak, or stall.

2 · Crash-class eliminated — pool use-after-free under churn

The most important fix in this cycle removes an entire category of crash. Under heavy connect/disconnect churn — the exact reconnect-storm condition — the IPv4/IPv6 address-pool lease handling could touch memory that had just been freed (a use-after-free), and the service could crash. Three related defects were found and fixed across the lease-claim, pending-lease-wakeup and lease-create paths. This was the root cause of crashes seen on a production node.

3 · IPv6 made cheap — no routing-engine churn per subscriber

Standard practice assigns an address to every subscriber's virtual interface, which makes the kernel create a connected route and arm address-configuration timers — and floods the routing engine with add/remove events every time a subscriber connects or disconnects. The new IPv6-unnumbered mode installs the subscriber's /64 as a lightweight device route instead. The subscriber still gets standard IPv6 auto-configuration and prefix delegation — nothing changes on their side — but the per-subscriber routing churn disappears.

4 · Faster online — IPv4 the instant the session starts

Dual-stack bring-up used to gate the IPv4 service behind the slower IPv6 prefix-delegation handshake. Early IPv4 binding decouples them: the subscriber's IPv4 forwarding and accounting are armed at session start, not after IPv6 finishes negotiating.

5 · Robustness — survives a failed disk, smooth under storms

Two more changes harden the service against the messy realities of field hardware and storm load.

6 · Newer kernel base — 7.1.1, validated at 10k subscribers

The platform moved from the 7.0.x kernel series to 7.1.1, carrying forward staggered batch teardown of subscriber interfaces so that thousands of PPP/IPoE interfaces are cleaned up in groups instead of one-at-a-time. The new base was validated end-to-end — establish, hold, traffic, tear down — at 10,000 subscribers on both virtual (vmxnet3) and bare-metal (Intel ixgbe) hardware, with zero driver or module errors.

7 · Proven, not just shipped — the validation

None of the above relies on "it compiled". Every change was exercised under load that mirrors a real bad day, monitoring every counter across the whole stack — session service, XDP data plane, routing engine, and system memory / CPU.

Inside the soak — the counters we watch through every simulated outage

  phase         active  mem   fd    routes     crash  status
  -----------  -------  ----- ----- --------- ------  -------
  steady          8000  flat  flat  ~48k           0  OK
  power-cut    8000->0  flat  flat  48k->~0        0  DRAIN
  re-ramp      0->8000  flat  flat  ~0->48k        0  RECOVER
  steady          8000  flat  flat  ~48k           0  OK