Subscriber Edge on Commodity Servers
Broadband Gateway · CGNAT · QoS · Protection
Datapath Architecture · Conditions Attached To Every Number

The Benchmark Everyone Quotes Has No Subscribers In It

This page makes the case that an in-kernel XDP/eBPF datapath is the better foundation for a broadband gateway than the DPDK and VPP kernel-bypass designs most software BNGs are built on. It is written for a reader who already knows DPDK, so it argues from primary sources — the strongest of them published by the kernel-bypass camp about itself — with the measurement conditions attached to every figure and a date on every citation. Where DPDK and VPP are genuinely ahead, we name the objection and answer it rather than leave you to discover it.
Headline packet rates are quoted without the state a broadband gateway exists to hold. Load the state — using the other camp's own published number — and the case for staying inside the kernel is the stronger one.
6.9 Mpps
per core — VPP's own
published carrier-grade NAT
figure, at 1,000 users (2017)
6.3×
below the 43.5 Mpps headline
the industry quotes. Different rigs,
a year apart — conditions inside
0 cores
dedicated to polling. A poll-mode
driver reads 100% at any offered
load, by design, not by accident
4,400+
subscribers, zero packets
discarded — our own
measurement, live network

Software forwarding benchmarks are quoted in a currency that hides the thing a broadband gateway actually exists to do. 40-plus million packets per second per core is a real measurement, and it is 64-byte Layer 2 forwarding with no per-subscriber state whatsoever — no address translation table, no per-subscriber rate, no session lookup, no accounting. The moment you load the state, the published numbers from the kernel-bypass camp itself fall to single digits. That is not our claim about them. It is their table. And once both architectures are in single digits, the per-core rate stops being what separates them — what separates them is what the dataplane costs you to run, to watch, to upgrade and to trust.

The argument in eight lines
1
Their own number is the strongest argument we have.FD.io's One Terabit Software Router whitepaper (July 2017, VPP 17.04) publishes CGNAT44 at 6.9 Mpps/core and 359 cycles/packet — at 1,000 users with 15 ports each.
2
Stateless L2 in the same table: 19–27 Mpps/core.115–118 cycles/packet. Same whitepaper, same rigs, same zero-loss criterion. The state is what costs, and a BNG is nothing but state.
3
No core is dedicated to moving packets.Our processor cost tracks offered load. A DPDK poll-mode core reads 100% busy at any load, by design — and that core does nothing else, ever.
4
No hugepages are reserved before we will start.No boot-time memory contract to size, clear, mount or re-mount, and no memory carved out of the machine that nothing else may use.
5
The port never leaves the kernel.ip, ethtool, kernel routing, bonding and LACP all keep working on the same port that forwards traffic — where DPDK binds to vfio-pci or uio, they do not.
6
One forwarding table, not two.We read the kernel's own FIB from inside the forwarding program. VPP mirrors it into a second copy. A copy can diverge; a table you read cannot.
7
The forwarding program is replaced without restarting the dataplane.An atomic in-place swap, where link-based attach is available. A userspace dataplane code change is structurally a process restart.
8
The objections are named in section 3, and answered.Then the number nobody publishes: 4,400+ subscribers, ~2 Mpps, six cores of a 2016 processor, busiest core 72%, zero discards — one minute at evening peak, and not a maximum.

1 · Start with the number the other camp published about itself

In July 2017 FD.io published Vector Packet Processing — One Terabit Software Router. It is a good paper, and we are not here to dispute it: it states its conditions in footnotes, it reports zero-packet-loss throughput rather than a lossy peak, it uses 64-byte frames, and it disables TurboBoost so the per-core figures mean something. Its results come from Intel labs, Cisco labs and FD.io's own continuous-integration benchmark programme. We are pointing at one row in it.

Read down the table by cycles per packet — the frequency-independent measure, so no argument about clock speed can muddy it. The cost climbs steadily as the forwarding function acquires state, and the row that carries per-subscriber address-translation state is the most expensive forwarding function in the table. It is also the only row in the table that describes what a broadband gateway does for a living.

What state costs, in the kernel-bypass camp's own published measurements

Cycles per packet, from the FD.io VPP whitepaper of July 2017. Lower is better. Conditions are printed on every row because without them these numbers mean nothing.
L2 crossconnect2.2 GHz · 2 threads/core · no state
115 cycles/pkt19.1 Mpps per core
L2 crossconnect3.2 GHz · 2 threads/core · no state
118 cycles/pkt27.1 Mpps per core
IPv4 routing, 1M /32 routes2.3 GHz · 2 threads/core
188 cycles/pkt12.0 Mpps per core
L2 MAC switching, 100k MACs2.3 GHz · 2 threads/core
221 cycles/pkt10.4 Mpps per core
IPv4 routing, 1M /32 routes2.3 GHz · 1 thread/core
242 cycles/pkt9.5 Mpps per core
IPv6 routing, 0.5M /128 routes2.3 GHz · 1 thread/core
307 cycles/pkt7.5 Mpps per core
CGNAT44 — carrier-grade NAT1,000 users, 15 ports each · 2.3 GHz · 1 thread/core
359 cycles/pkt6.9 Mpps per core
Conditions, verbatim from the source: “All tests performed with VPP 17.04 release”; CPUs with “TurboBoost disabled”; “Packet non-drop throughput rate (zero packet loss) per physical CPU core for 64B Ethernet frame size”. Bar length is scaled against a 400-cycle reference so the rows are comparable to one another; each value is printed beside its bar rather than inside it, so a short bar is never inflated to hold its own label. Source: FD.io, Vector Packet Processing — One Terabit Software Router, July 2017. This data is nine years old and describes VPP 17.04 — it is the most fully-conditioned published CGNAT figure from the kernel-bypass camp that we could find, and we present it as what it is, not as current.
The gap, stated twice — once at its largest, once at its most defensible. Against the 43.5 Mpps single-core figure the industry quotes as DPDK's headline, 6.9 Mpps is a 6.3× fall, and that is the comparison worth confronting, because the headline is what gets put in front of you in a deck. Be clear about what it is: two different measurements, on different hardware, published a year apart. So here is the version that needs no such caveat, and it stays inside a single table — same 2017 whitepaper, same rigs, same methodology, same zero-loss criterion: stateless L2 crossconnect at 115–118 cycles per packet, carrier-grade NAT at 359. That is 3.1×, and it is unarguable, because nothing changed except the state. Take the smaller number. The argument still holds.

And 1,000 users is not a deployment

The CGNAT44 row's stated condition is 1,000 users with 15 ports each — a 15,000-entry translation table. A real carrier-grade NAT node holds tens of thousands of subscribers with port blocks of hundreds or thousands each, which is a translation table two to three orders of magnitude larger, with a working set that no longer sits comfortably in cache. The published 6.9 Mpps is therefore not a pessimistic figure. It is a generous one, measured at a subscriber count an operator would call a lab, and it is still 3.1× the cost of the stateless row everyone quotes.

Where that leaves the choice. Once the subscriber table is loaded, both architectures are in the same single digits per core, and the per-core packet rate stops being the thing that separates them. What separates them is everything around the packet: how much of the machine the dataplane takes before it moves anything, whether the port is still a port, whether there is one forwarding table or two, what happens when you upgrade, and what you can see when something goes wrong. On every one of those, running inside the kernel is the better answer — and that is the rest of this page.

2 · What running inside the kernel buys

Five arguments, each of which follows directly from the decision to run the datapath as verified eBPF inside the kernel rather than as a process that has taken the network card away from it. Each is sourced, and each is scoped as tightly as we would want a competitor's claim scoped.

01  No core is dedicated. Processor cost tracks offered load.

A DPDK poll-mode driver does not wait to be told a packet has arrived — it spins, continuously, asking. That is not a tuning choice or an implementation detail that a future release might soften; it is the model, and DPDK's own Programmer's Guide requires it: “to avoid any unnecessary interrupt processing overhead, the execution environment must not use any asynchronous notification mechanisms”, and “multiple logical cores should never share receive or transmit queues for interfaces”. A core assigned to packet processing is therefore spent, wholly, whether the network is at peak or asleep. The peer-reviewed head-to-head measured exactly this:

“Since DPDK by design dedicates a full core to packet processing, and uses busy polling to process the packets, its CPU usage is always pegged at 100%… In contrast, both XDP and Linux smoothly scale CPU usage with the offered load, with a slightly larger relative increase in CPU usage at a small offered load level.” — Høiland-Jørgensen et al., The eXpress Data Path, ACM CoNEXT 2018, §5.2. The authors are the XDP maintainers; the statement about DPDK is a measurement, not an opinion.
Processor utilisation against offered load 100% 50% 0% Processor idle Offered load → peak DPDK poll mode — 100% at every load, by design This area is bought, powered and cooled — and not used In-kernel XDP — cost tracks the traffic Shape of the curves is schematic and drawn to illustrate the behaviour quoted above; it is not a reproduction of measured data points. Source for the behaviour: ACM CoNEXT 2018 §5.2 (measured), and the DPDK Programmer’s Guide, Poll Mode Driver (24.03), which mandates the model.
SchematicILLUSTRATIVE — the two shapes, not the two magnitudes. What the shapes mean in a procurement conversation: a poll-mode design charges you its peak cost at three in the morning, and the reference measurement reserved one core of six purely for control before any packet moved. On a six-core box that is a sixth of the machine gone at the door.
We have not earned this jab cheaply, so here is ours. We have had a core pegged at 100% in production — not from a DPDK-style design decision but from our own interrupt-deferral tuning, which stopped the interrupt ever being re-armed and quietly took interrupt affinity out of the picture. We found it by measurement, fixed it by moving packet processing onto dedicated kernel threads, and interface drops on that node fell by a factor of 413. The defensible claim is architectural: XDP yields the core when there is nothing to do, and a poll-mode driver structurally cannot. The indefensible claim would be that we never have a busy core.

02  Nothing is reserved before the dataplane will start

DPDK's memory is hugepage-backed, and DPDK's own Getting Started Guide is candid about what that costs operationally: “some kernel versions may not allow reserving 1 GB hugepages at run time, so reserving them at boot time may be the only option”. Boot-time reservation means the memory contract is decided before the machine has finished starting, is fixed until it next reboots, and is unavailable to anything else on the box for the whole of that time. It is enough of an operational object in its own right that DPDK ships a lifecycle tool for it — dpdk-hugepages.py, which clears reservations, mounts and unmounts the backing filesystem and re-reserves, and which needs root to change anything.

NO RESERVED CONTRACT

Memory is ordinary kernel memory

0 GB
hugepages reserved before the datapath will start
  • Nothing to size in advance, nothing to get wrong at boot, nothing to clear and re-mount when it changes.
  • No block of RAM carved out of the machine that the rest of the machine may not touch.
  • Sizing a deployment is one conversation shorter, and a machine that has been re-purposed does not come with a stale reservation attached.
THE OTHER SIDE, FAIRLY

DPDK's own documented starting point

4 × 1 GB
the reservation in DPDK's own boot-parameter example
  • Hugepages reduce translation-lookaside-buffer pressure and DPDK's rationale for them is sound. We are not claiming ordinary kernel memory is faster — this is a claim about the operational contract, not about throughput.
  • We also will not claim that a crashed process strands its hugepages. We could not source it, so we do not say it.

03  The port never leaves the kernel

Where a DPDK deployment binds the network port to vfio-pci or uio, the port stops being a Linux network interface. DPDK states the consequence plainly: “any network ports or other hardware under Linux control will be ignored and cannot be used by the application”, and the reverse holds too — the kernel no longer has a device there. Everything an operations team knows how to do to a network port stops applying to the port that carries your subscribers, and is replaced by whatever the vendor chose to expose.

Our forwarding program attaches to a kernel network device rather than replacing it. Addressing, link state, ethtool counters and ring configuration, VLAN sub-interfaces, the bonding driver and LACP, kernel routing, and every SNMP and monitoring agent already pointed at that interface all keep working — on the same port, at the same time, while it forwards.

Kernel bypassport bound to a userspace I/O driver
port leaves kernel control hugepages reserved, often at boot core polls at 100% by design capture needs the framework compiled in
Kernel bypassbifurcated driver — the fair case
standard tools keep working hugepages still reserved core still polls at 100% capture still needs the framework compiled in
In-kernel XDPBNGSOFT
port stays in the kernel no reserved memory contract CPU tracks offered load capture tool attaches to the running program
The middle row is the one most comparisons leave out, and we put it in deliberately. DPDK's bifurcated drivers — notably on NVIDIA/Mellanox parts — keep the network device visible to the kernel, and DPDK documents this explicitly: the model “enables the user to use legacy linux tools such as ethtool or ifconfig while running DPDK application on the same network ports.” Any claim that DPDK costs you standard tooling must be scoped to vfio-pci or uio binding, or a reader running a bifurcated driver will correctly dismiss the whole page. Note what does not change in that middle row: the reserved memory, the pegged core, and the capture story.

04  One routing table, not two

A kernel-bypass forwarding plane cannot use the kernel's routing table, because it is not in the kernel — so it must build its own and keep it in step. VPP solves this properly, and we want to be exact about that, because the lazy version of this argument is false and would be rebutted in the first meeting: VPP does integrate with routing daemons. Its Linux control-plane plugin creates Linux network devices mirroring VPP interfaces, handles ARP and neighbour discovery in Linux, and mirrors netlink route events into VPP's own forwarding table, so FRR or Bird run unmodified on top of a VPP dataplane. FRRouting maintains documentation for VPP as an alternate forwarding plane.

The honest difference is narrower than “can or cannot”, and it is about failure modes. VPP holds a second forwarding table, synchronised from the first. Our forwarding program performs a full routing lookup against the kernel's own table, from inside the datapath, on the packet. There is no copy.

Kernel-bypass dataplane (VPP) Forwards from a synchronised copy Routing daemon — FRR / Bird Linux kernel routing table ▼  netlink events Linux control plane — mirrors them across Second forwarding table — the dataplane's own Two tables, held in step by a process Packets follow the copy, not the original In-kernel XDP dataplane (BNGSOFT) Forwards from the table itself Routing daemon — FRR / Bird Linux kernel routing table ▲  read on the packet Forwarding program reads it directly No second table Nothing here to build, populate or reconcile One table. There is no copy to diverge What routing decided is what forwarding does
This is a reliability argument, not a capability argument. Synchronisation between two authoritative-looking copies of the same state is a well-known source of the worst class of network fault: the one where the control plane is correct, every routing tool agrees, and traffic still goes the wrong way — because forwarding is reading a table that stopped matching some time ago and nothing is obliged to notice. Shadow state can drift. State you read cannot. We removed that failure mode by never creating it.

05  The forwarding program is replaced without restarting the dataplane

Replacing an attached XDP program is a kernel operation, not an application trick. The new program is pinned in the BPF filesystem, then atomically swapped for the running one using a netlink replace flag that names the exact program being replaced — so the operation either succeeds completely or does not happen. If it fails, the new program is unpinned and the operation restarts from a fresh read of kernel state. There is no window in which the interface has no forwarding program, and no process is restarted, because the dataplane is not a process.

WHAT WE STAND BEHIND

The claim, with both caveats attached

  • Zero downtime for logic and daemon updates — the great majority of what a release contains.
  • Data-structure changes still require a brief restart. When the shape of the state changes, the state has to be rebuilt.
  • The atomic swap requires link-based attachment and is unavailable where that is not present. We would rather write that sentence than have you find it.
THE OTHER SIDE, FAIRLY

What is structurally true of a userspace dataplane

  • A kernel-bypass dataplane is an ordinary process, so a dataplane code change is a process restart. That is architecture, not criticism.
  • Vendors build to survive it — control- and user-plane separation, spare user planes, session handover.
  • We will not claim DPDK gateways must drop traffic to upgrade. We could not source it, and it would not be fair.

And five more that follow from the same decision

OBSERVABILITY
Capture works against the program that is already running.
  • The XDP capture tool attaches to an already-loaded program on a live interface and records at entry and exit, including the packets the program drops, with no cooperation from whoever wrote it.
  • DPDK's equivalent runs as a secondary process and requires the primary application to have compiled the capture framework in; it exits when that process exits.
Both models need a purpose-built tool — plain tcpdump is blind to XDP verdicts. The difference is whose cooperation the tool requires, and whether you can start it on a box that is already misbehaving.
FAULT CONTAINMENT
The kernel will not load a program it cannot prove safe for itself.
  • A userspace dataplane holds direct device access; when it faults, the forwarding path goes with it.
  • Ours is statically verified before it is allowed to run, and cannot corrupt kernel memory.
Scope this exactly: that is a guarantee about the kernel surviving our program, not a guarantee that our program is correct. Correctness is our regression suite's job — see section 3.
SAME-NIC FORWARDING
Where the traffic pattern suits it, we are ahead of DPDK.
  • The same peer-reviewed head-to-head finds XDP “surpass[es] the DPDK forwarding performance at two cores and above” when packets leave on the interface they arrived on — the buffer never has to be returned to another interface's pool.
  • Send it out of a different card and DPDK is ahead again.
ACM CoNEXT 2018, §5.3. That is the whole of the exception and we are not stretching it further.
WHAT ACTUALLY CONSTRAINS A BNG
Packets per second, not bits per second — which is why the 64-byte benchmark exists at all.
  • The XDP authors verified 100 Gbit/s at line rate on a single core that was half idle with full-sized 1500-byte packets, and said so explicitly: the challenge is processing many packets per second, which is why every figure in their paper uses minimum-sized frames.
  • So treat any bits-per-second headline — ours included — as an incomplete statement until the packet size is next to it.
ACM CoNEXT 2018, §5. Cited to make the point that both camps' 64-byte culture is a deliberate worst case, not a trick.
NOT A RESEARCH PROTOTYPE
XDP carries hyperscale production traffic today.
  • Meta's Katran Layer 4 load balancer is an XDP program running in production in front of Facebook and Instagram.
  • Cloudflare has run XDP-based volumetric attack filtering in production for years, and publishes its own XDP capture tooling.
Cited as qualitative evidence only. No throughput figure is claimed for either — the per-core numbers widely attributed to Katran are not in its documentation, so we do not repeat them.

The full ledger

Eighteen dimensions with the winner named on each, our wins first and theirs immediately after, so that nothing is buried. We win nine, DPDK or VPP wins eight, and one belongs to nobody because nobody has published a fair measurement of it. On a wide screen the table scrolls sideways with the dimension name pinned in place; on a narrow one it becomes one card per dimension, so no value is ever separated from the heading that gives it meaning.

Dimension Kernel bypass — DPDK / VPP In-kernel XDP — BNGSOFT Advantage Basis
Processor cost at low load Pegged at 100% regardless of offered load, by design Scales with offered load BNGSOFT ACM CoNEXT 2018 §5.2; DPDK Poll Mode Driver guide
Cores reserved before any packet moves At least one for control in the reference measurement — one sixth of a six-core box None dedicated BNGSOFT ACM CoNEXT 2018, figure caption
Memory contract Hugepages required; 1 GB pages often must be reserved at boot; a lifecycle tool exists to manage them Ordinary kernel memory; nothing to size, clear or re-mount BNGSOFT DPDK System Requirements; DPDK hugepages tool
Port visible to ip / ethtool Not under vfio-pci or uio binding. Retained under bifurcated drivers Always retained — the port never leaves the kernel BNGSOFT — scoped DPDK Linux Drivers guide, both directions
Routing integration VPP mirrors kernel netlink into a second table; FRRouting documents VPP as an alternate forwarding plane Reads the kernel table directly — no second copy to diverge BNGSOFT — different failure modes, not different capability FRRouting wiki; VPP Linux control-plane write-ups; CoNEXT 2018 §6.1
Dataplane code update The dataplane is a userspace process, so a code change is a process restart. Vendors mitigate with control/user-plane separation and spare user planes Atomic in-place program swap where link-based attach is available; data-structure changes still require a brief restart BNGSOFT — with both caveats libxdp protocol documentation; first-party
Packet capture The DPDK capture tool runs as a secondary process and requires the primary application to have the capture framework already compiled in; it exits when that process exits The XDP capture tool attaches to an already-loaded program on a live interface, including packets the program drops. Plain tcpdump is blind to XDP verdicts — both models need a purpose-built tool BNGSOFT — scoped DPDK pdump docs; xdp-tools; Red Hat
Fault containment A userspace dataplane with direct device access; a crash takes the forwarding path with it. uio binding lacks IOMMU protection — vfio-pci with a working IOMMU does restrict DMA The kernel refuses to load a program it cannot prove safe for the kernel — ours cannot corrupt kernel memory BNGSOFT — kernel safety only, not program correctness DPDK Linux Drivers guide; CoNEXT 2018 §3.4
Forwarding out the same NIC Baseline Surpasses DPDK from two cores upward BNGSOFT ACM CoNEXT 2018, §5.3
Throughput with subscriber state VPP's own published CGNAT44: 6.9 Mpps/core, 359 cycles/pkt, at 1,000 users No comparable public benchmark exists for XDP. Our production figure is in section 4 Neither — nobody has published a fair one FD.io whitepaper, July 2017
Packet queueing & hierarchical shaping Five-level scheduler, token bucket per pipe, strict priority across classes, weighted round-robin within, typically 64K leaf queues None. Returns a verdict; holds nothing. Meters, marks and drops instead DPDK — decisively DPDK QoS Framework (23.11); kernel queueing RFC, 2022, unmerged
Latency at 100 pps 2 µs average · 99.5% under 10 µs 82 µs average · 0% under 10 µs DPDK — decisively ACM CoNEXT 2018
Latency at 1 Mpps 3 µs average · 99.0% under 10 µs 7 µs average · 98.1% under 10 µs DPDK — narrowly ACM CoNEXT 2018
Single-core drop rate, 64B 43.5 Mpps · 22.9 ns/pkt 24.0 Mpps · 41.6 ns/pkt DPDK ACM CoNEXT 2018
Forwarding out a different NIC Higher Below DPDK — the buffer must return to the owning interface DPDK ACM CoNEXT 2018
Program complexity ceiling Unbounded C — no verifier, no instruction limit, no table count, arbitrary data structures Hard instruction and table ceilings. Our main program sits near 90% of an instruction limit that itself needed a kernel patch; one program is at 64 of 64 tables DPDK Kernel BPF design notes; first-party
Micro-architecture Vector processing — up to 256 packets scraped at once, amortising instruction-cache cost Scalar, one packet at a time, on a large program VPP FD.io VPP documentation
Host operating system Installs on unmodified enterprise Linux distributions Requires our own kernel with raised ceilings — an appliance, not a package DPDK / VPP First-party

Two further dimensions are not in the table because they are one-sided and already conceded in section 3: NIC driver breadth and hardware crypto acceleration (DPDK / VPP), and published third-party benchmarks (DPDK / VPP). Counting those, the ledger is nine to ten, with one row belonging to nobody. We are content with that: the nine we hold are the ones an operations team lives inside every day.

3 · The objections, named and answered

An evaluator who knows DPDK arrives with a list, and it is a good list. Here it is, strongest first, each with our answer beside it. Nothing here is new to us and nothing here is buried — every one of these also appears in the ledger above.

THE STRONGEST ONE
“XDP cannot hold a packet, so you have no rte_sched.”
  • Correct: XDP returns a verdict and holds no packet, so we have no equivalent of DPDK's rte_sched — no five-level tree, no token bucket per pipe, no weighted round-robin between queues — and we meter, mark and drop where a shaper would delay and reorder.
  • The answer is the measurement in section 4: on a production node carrying more than 4,400 subscribers with per-subscriber rate enforcement, low-latency queueing, interactive-traffic protection and the full protection suite all enforcing, zero packets were discarded across the window.
  • The structural advantage is real; at real subscriber scale it has not yet produced a practical deficit we can measure.
What we do instead of scheduling: a committed rate per subscriber per direction with a group aggregate above it, a queueing-delay estimate driving the drop decision rather than byte count alone, acknowledgements and connection setup rescued from tail-drop, and congestion marking instead of loss where the flow supports it.
THE SURPRISING ONE
“Your latency at low offered load is far worse.”
  • Also correct, and by a large margin: at 100 packets per second the 2018 head-to-head puts DPDK at 2 µs average with 99.5% of packets under 10 µs, against 82 µs and 0% for XDP — constant polling wins when there is nothing to poll.
  • At a million packets per second the same measurement closes to 98.1% against 99.0%.
  • A subscriber edge is an aggregation device: it is bought for the evening peak and judged on the evening peak, the hour in which the polling advantage has already evaporated.
ACM CoNEXT 2018, end-to-end over 50 seconds, two ports on one NIC, 64-byte packets. DPDK 18.x era, kernel ~4.18 — the direction still holds, the exact ratio should not be treated as current.
RAW DROP RATE
43.5 against 24 Mpps, single core, 64 bytes.
  • Theirs, and the XDP authors say why: “DPDK has incorporated more performance optimisations at the lowest level of the code.” We quote it rather than wait to be shown it.
  • It is also a stateless drop test. Section 1 is the argument that this is the wrong number to buy a BNG on.
ACM CoNEXT 2018. Both stacks scale linearly to a shared ~115 Mpps PCI-bus ceiling.
CEILINGS AND THE HOST OS
We run against limits that C does not have.
  • Our main forwarding program verifies near 90% of its instruction ceiling — a ceiling four times what stock Linux allows, which is why we ship our own kernel. One program sits at 64 of 64 data tables. This has already cost a feature and forced a rollback.
  • So we are an appliance, not a package. Against a standardised host-OS mandate that is a genuine procurement obstacle.
Named deliberately. An experienced reader assumes such limits exist and trusts the rest of the page more once one is on the table.
TOOLING, VECTORS, BENCHMARKS
Three more, without excuses.
  • VPP's vector model gathers up to 256 packets at once and amortises instruction-cache cost across them. Ours is scalar and our program is large. The mechanism is uncontroversial and we do not pick a fight here.
  • Userspace C has thirty years of debuggers, profilers and sanitisers. We compensate with a regression suite and staged rollout; it is compensation, not parity.
  • We have no third-party benchmark. Every BNGSOFT figure here is self-reported and an evaluator is right to treat it as such.
The benchmark gap is the most common reason a technically strong small vendor loses a bake-off. Pretending otherwise helps nobody.

And three things we will not claim, however useful they would be

WE WILL NOT CLAIM
“The verifier proves our datapath is correct.”
  • It does not, and the paper that introduced XDP says so: the verifier's purpose is to protect the kernel, “not to ensure that the programs perform their designated function”, and a program “can corrupt network packets if written incorrectly”.
  • What is true: the kernel will refuse to load a program it cannot prove safe for the kernel. What is not: anything about our logic being right. We have shipped logic bugs that verified perfectly.
WE WILL NOT CLAIM
“You can just tcpdump our interfaces.”
  • Not for the packets that matter most. When an XDP program drops or redirects a packet the kernel never sees it, so neither does tcpdump. We have been caught by this in our own engineering more than once.
  • What is true: the interface stays a normal kernel interface, and a purpose-built XDP capture tool attaches to the running program and shows you the drops.
WE WILL NOT CLAIM
“VPP cannot integrate with a routing daemon.”
  • It can, and this would be rebutted in the first meeting. VPP's Linux control-plane plugin mirrors kernel interfaces and netlink route events into VPP, and FRRouting documents VPP as an alternate forwarding plane.
  • The accurate statement is the narrower one in section 2.4: a synchronised copy can diverge; a table read directly cannot.

4 · The number nobody publishes — so here is ours

The whole argument of this page is that you should demand the figure measured with the subscriber table loaded. That obliges us to publish ours first. And it is worth saying what is missing from the public record: there is no published XDP benchmark carrying heavy per-subscriber state. Not one. There are excellent XDP measurements of packet drop, of load balancing at hyperscale, of volumetric attack filtering — but nothing with tens of thousands of translation entries and per-subscriber rate enforcement. The closest available proxy is the other camp's CGNAT row from 2017 that opens this page.

So we measured our own, on a live network, and published it with the conditions attached — because the conditions matter more than the numbers.

WHAT IT WAS CARRYING

A real evening peak, on old hardware

~16 Gbit/s
of subscriber traffic, with peaks near 18
  • More than 4,400 concurrent subscriber sessions.
  • Close to two million packets per second — the measure that actually stresses a forwarding system.
  • Six cores of a processor generation launched in 2016. Ordinary server hardware, no specialist forwarding silicon.
WHAT IT COST

Less than the arithmetic suggests

72%
the highest any single core reached
  • No core crossed 80% in any one-second sample across the window.
  • Zero packets discarded — no overrun, nothing dropped anywhere.
  • Every protection and quality-of-service feature was enforcing, not waiting to be enabled.

One minute on a production node, sampled every second

Not a lab rig, not a traffic generator, and not a maximum. Read the conditions below before quoting anything here.
Busiest coreworst single one-second sample in the window
72%
headroom
Processor, server-wide averageacross all six cores
~61%
headroom
Network port capacity in usethe resource that actually fills first
~85–88% full
near limit
Packets discardedinterfaces and forwarding system combined
0 — nothing discarded
none
Conditions: a single continuous one-minute observation of one production broadband gateway node, sampled once per second, taken during the evening peak, on a node carrying paying subscribers. Processor utilisation is reported as the busiest individual core in each one-second sample. Per-subscriber rate enforcement, low-latency queueing, interactive traffic protection, DDoS protection, source-address validation and outbound abuse containment were all enabled and enforcing throughout. The node is a broadband gateway; it does not perform carrier-grade address translation, which runs on other nodes in the same product family. These figures describe this node under this load and are not a maximum — the node serves paying customers and was not driven to its limit. Full conditions, including the port-capacity analysis, are published separately in 4,400 Subscribers. One Server From 2016.
Per-subscriber speed controlEvery subscriber metered to their own plan rate
Low-latency queueingKeeps video calls and gaming responsive under load
Interactive traffic protectionProtects latency-sensitive flows when a line is congested
DDoS protectionAttack detection running inline on all traffic
Source-address validationBlocks spoofed traffic leaving your network
Outbound abuse containmentLimits compromised subscribers before they harm your reputation
We are not going to divide that by six. It is the obvious next move, and it would be dishonest, in a way worth spelling out because the same objection applies to every vendor figure on this page. The node was not driven to its limit — it was serving real customers at whatever rate they happened to want. A per-core figure computed from it would be a floor presented as a ceiling: it tells you what six cores were doing at 72% while comfortable, not what they can do. If we published it as a per-core rate it would be smaller than the truth, and it would still be quoted as a maximum. So we publish the load, the utilisation, the hardware and the conditions, and let you do the sizing with us in a design review.
What this measurement is for. Not to beat 6.9 Mpps per core — the two are not comparable, being different functions on different hardware in different decades under different criteria. It is to be the artefact that does not otherwise exist: an XDP forwarding measurement with thousands of subscribers of state loaded and every enforcement feature on, with its conditions attached. That is also the test we would put to any vendor you are comparing us against. Ask for the number with the subscriber table loaded — then ask how big the table was, what was enforcing, and whether the box was driven to its limit.

A note on who this describes

Architecture-level statements about DPDK and VPP are fair game — both are open, documented projects, and this page argues against the architectures, not against companies. Product-level statements about a named vendor are not fair unless they appear in that vendor's own material or a partner's. So, briefly and only where sourced:

SOURCED
DPDK is genuinely what this market is built on.
  • An Intel solution brief states that netElastic “has developed an architecture that leverages CUPS, SDN, DPDK, and Intel processors and Ethernet controllers”, and refers to the product's DPDK mode.
  • A Benu Networks press release (April 2021) describes its broadband network gateway as coupled with Intel Xeon processors, DPDK and Red Hat OpenShift.
  • VPP itself is an FD.io project under the Linux Foundation, and its performance characteristics are published by the project — which is what made section 1 possible.
These are the vendors' and partners' own words, cited so the architectural comparison on this page has a real subject.
DELIBERATELY ABSENT
We do not attribute a dataplane we have not read about.
  • Several BNG vendors are widely assumed to use one framework or another. We have left them out rather than guess, and we have left VPP unattributed to any commercial BNG in this document because we could not source it.
  • Where a vendor's own page names neither framework, that is what we say — we do not fill the gap from a secondary write-up.
  • If you are running a competitive evaluation, ask each vendor directly. The answer changes what every number in their deck means.
The same restraint that makes the concessions in section 3 credible requires this one.

The position, in one paragraph

Our datapath runs as verified code inside the kernel rather than as a kernel-bypass process, and we think that is the right trade for a broadband gateway. It costs us raw single-core drop rate — the reference measurement is 24 Mpps for XDP against 43.5 Mpps for DPDK on 64-byte packets in 2018, and we do not dispute it — and it costs us the ability to hold a packet, which is why we police and mark rather than shape and reorder. What it buys is everything around the packet: no core dedicated to polling, no hugepages reserved at boot, a port that is still a port, one forwarding table instead of two, and a dataplane that is replaced without being restarted. Those are the properties an operations team lives inside every day of a deployment's life.

And it changes which benchmark matters. The forty-plus-Mpps figure everyone quotes is 64-byte Layer 2 forwarding with no subscriber state. The kernel-bypass camp's own published carrier-grade NAT number is 6.9 Mpps per core at a thousand users — a 6.3× fall from the headline, and 3.1× the cycles of the stateless row in its own table. Ask every vendor, including us, for the number with the subscriber table loaded — and ask how big the table was.

Sources

All web sources accessed 4 August 2026. Every figure quoted on this page carries its measurement conditions in the chart or table that contains it. Dates are given because both stacks have moved since the head-to-head measurements were made, and a reader should be able to check.

Trademarks and non-affiliation. BNGSOFT is not affiliated with, endorsed by, or sponsored by any organisation named on this page. DPDK, FD.io, VPP and Linux Foundation are trademarks or projects of the Linux Foundation and its affiliates. Linux is a registered trademark of Linus Torvalds. Intel and Xeon are trademarks of Intel Corporation. NVIDIA and Mellanox are trademarks of NVIDIA Corporation. Red Hat and OpenShift are trademarks of Red Hat, Inc. Ubuntu is a trademark of Canonical Ltd. netElastic is a trademark of netElastic Systems, Inc. Benu Networks is a trademark of Benu Networks, Inc. Meta, Facebook, Instagram and Katran are trademarks or projects of Meta Platforms, Inc. Cloudflare is a trademark of Cloudflare, Inc. Cisco is a trademark of Cisco Systems, Inc. FRRouting, ACM and LWN.net are the marks or publications of their respective owners. All other trademarks are the property of their respective owners.

About the comparative statements. Architectural statements about DPDK and VPP describe those projects' published documentation and published measurements at the access date given in the Sources section, and are not statements about any particular vendor's product unless that vendor is named and cited. Vendor dataplane attributions are made only where the vendor or a named partner states them; where we could not source an attribution, none is given, and the absence of a vendor from this page carries no implication about its architecture. Both DPDK and XDP have changed substantially since the 2018 head-to-head measurement and the 2017 VPP whitepaper reproduced here; those figures are presented with their dates and should be re-verified against current releases before being relied on. Software feature sets change between releases — re-verify before each re-issue.

About the BNGSOFT figures. All BNGSOFT performance figures on this page come from a single continuous one-minute observation of one production broadband gateway node, sampled once per second, taken during the evening peak period. The node was carrying more than 4,400 live subscriber sessions on a six-core server built on a 2016-generation processor, averaging approximately 16 Gbit/s of subscriber traffic and close to two million packets per second, with short-term peaks approaching 18 Gbit/s. The highest utilisation observed on any individual core in any one-second sample was 72%, and no sample on any core reached 80%. No packets were discarded by the network interfaces or the forwarding system during the window. Per-subscriber rate enforcement, low-latency queueing, interactive traffic protection, DDoS protection, source-address validation and outbound abuse containment were all enabled and enforcing throughout. The node is a broadband gateway and does not perform carrier-grade address translation, which is a separate function running on other nodes in the same product family. These figures describe this node under this load and are not a maximum: the node serves paying customers and was not driven to its limit. They should not be converted into a per-core rate or treated as a capacity ceiling. Capacity on any given deployment depends on installed port capacity, traffic profile, per-subscriber rates and feature configuration, and should be confirmed during design review. No subscriber-identifying information was accessed or is reproduced anywhere in this document.