Operations · running state
Operations & NOC

Switched on. Running. Healthy.
And permitting every packet it was bought to stop.

There are three states a capability can be in, not two — and the difference between the second and the third is the entire thing you paid for. Most consoles render all three as one green tick, because they are reading the configuration file and calling it the truth.

A setting written in a file is a statement of intent. Whether the running process ever loaded it is a separate question — and whether it is enforcing or merely watching is a third one that almost nothing asks.

Written
the config file says the feature is on
Running
the process actually loaded it
Enforcing
it is stopping something

The three states

State one

Written

The value is in the configuration file. Nothing has read it yet. A file edited without a reload can sit like this for months, and a console that reads files will call it on.

State two

Running — in observe

The process loaded it. It is watching the traffic, scoring it, counting it. It is permitting all of it. Everything reports healthy because everything is healthy.

State three

Enforcing

The capability is acting on what it sees. This is the only state that changes what reaches your subscribers, and the only one worth paying for.

Observe mode is not a bug. It is how a sane operator introduces anything that drops traffic: turn it on, watch what it would have done, and promote it once the numbers look right. The failure is not that observe exists — it is that nothing reminds you the promotion never happened, and a capability parked in observe is indistinguishable from one doing its job unless you go and ask the daemon directly.

We found one in our own fleet while building this. A production gateway with interactive flow protection loaded, healthy, and running in observe. Its configuration said enabled. Its process said enabled. It was protecting nothing, and had been for as long as anyone could tell.

Where a configuration-derived view loses the truth Three stages: the configuration file, the running daemon, and enforcement. A console that reads only the file reports a green tick after the first stage, while the capability may be loaded in observe mode and permitting all traffic. STATE ONE — WRITTEN enabled: true the file says so STATE TWO — RUNNING mode: observe loaded, watching, permitting STATE THREE — ENFORCING mode: enforce actually stopping traffic reload promote A CONSOLE THAT READS THE FILE reports one green tick here — and cannot tell these two apart ASKING THE GATEWAY returns the mode, not a boolean The gap between state two and state three is the entire thing you paid for. Nothing is broken in state two. Everything reports healthy, because everything is healthy. It is simply not protecting anyone.

Why reading the config file is not enough

Three separate ways a file-derived view goes wrong, all of them seen on real gateways:

What the file saysWhat is actually trueWhy the file cannot tell you
Feature enabled Running in observe, blocking nothing Enablement and mode are separate settings; reading one and not the other reports a watching feature as an enforcing one.
Setting absent Feature running on its built-in default An unwritten key is not false. Treating absence as “off” invents faults that do not exist — see below.
"1" (quoted) Accepted, but flagged by the daemon as the wrong type Provisioning writes a string where a boolean belongs. Today's build tolerates it. A stricter one need not.
The Gateway Features screen: a grid of gateways, each with the same fixed row of capability chips, lit where a capability is on and faded where it is off, above a line stating how many gateways could be read.
The same capabilities, in the same order, on every gateway — so the column can be read down and an operator can see at a glance which boxes are missing something their neighbours have. The line above the grid is the part that matters most: it states how many gateways could actually be read. A gateway nobody could ask is not a gateway with nothing wrong.

The fault our own audit invented

While building the feature audit described here, we had it check whether queue management was enabled by reading a configuration key — one that the current generation of the daemon never writes. The key was absent everywhere, the audit read absence as “off”, and it duly reported that a dependent capability was broken on gateway after gateway.

Sixteen faults. None of them real. The gateways were running the feature perfectly well, under a differently-named setting. We only caught it because two gateways were new enough to be asked directly, and the daemon's answer flatly contradicted our own console.

We are including this because it is the strongest possible argument for the approach: the config-derived view was wrong, and only the daemon's own answer could show that it was wrong. A vendor who has never had this happen has not looked.

So we ask the daemon

The gateway reports what it is actually running, per capability, in its own words — not enabled or disabled, but the real state: enforcing, observing, or off, along with the section it came from and any complaint the daemon has about how the value was stored.

Reading the configuration

Tells you what somebody intended, at the moment they last saved the file.

Cannot distinguish enforcing from watching, or a deliberate default from an oversight.

Silently wrong when a setting is renamed between versions — and confidently so.

Asking the gateway

Tells you what the running process is doing right now, in the mode it is doing it.

Surfaces the daemon's own complaints about malformed values instead of hiding them.

Disagreement between the file and the daemon becomes a visible finding rather than an invisible assumption.

Enforcing

Doing the job

Shown as such, per gateway and per capability, because it is the only state that means the feature is earning its place.

Observing

Watching only

Called out explicitly — deliberately not in the fault colour, because nothing is broken. It simply is not protecting anyone yet.

Unknown

Not asked

Where a gateway runs a build too old to answer, the console says so rather than guessing. An unread gateway is not a clean one.

What this does not claim

Not every gateway can answer. The capability to report running state per feature is recent, and on a mixed-vintage fleet most boxes still fall back to the configuration-derived view. We show the count of gateways that could actually be asked, next to the results, so that “no faults found” is never confused with “everything verified”. That distinction is the whole point of the document.

Ask what your gateway is enforcing, not what its file says. If your current console cannot tell the difference between a capability that is blocking and one that is only watching, it cannot tell you whether you are protected — and it will show you a green tick either way.

About the data. Observations are taken from production gateways across eight independent operators. No operator name, hostname, site, region or subscriber identifier appears in this document, and no per-operator figure is published. The sixteen invented faults, the observe-mode gateway and the string-typed configuration values are all real findings from that fleet, reproduced here without identifying which deployment they came from. About the screenshots. Every screen shown is a real production console. Gateway hostnames are replaced with placeholders before the image is captured, and no operator name, site, region, subscriber identifier or address appears in any image. CPU models are left intact: they describe hardware, not a customer.