What your fleet is, beside what it says it is

Git stays the authority and the app cannot edit it: the declared record is derived by cloning each machine's repo, and it is a cache rather than a second source of truth. Seeing a divergence produces a commit, not a form submission.

A screen that merely re-rendered the declared record would add a second place to read the same stale claim. What is worth showing is where declared and observed disagree.

A label is not a description of what is installed — it is what a host will ACCEPT work for, which is why a true-but-unwanted label is dangerous: it quietly admits builds. Toolchain probes check the promise, and each probe is compared under the label that DECLARED it, because grouping by host instead reports drift on machines that never made the claim.

Labels resolve three ways: verified, hint (declared unassertable, with the reason), and unverified — nobody has ruled either way. Three states rather than two, because an alarm that fires on every macOS host promising a Mac is an alarm nobody reads.

There is a fourth answer to an unverified label and it is often the best one: retire it. One label here promised 31 GB of memory, could not be asked about, and no target had ever required it.

Attached handsets are reported in three states, not two: attached, unplugged (declared, nothing reporting), and undeclared (reported, nobody declared it). A device row is only as durable as its USB cable, so without a declaration an unplugged phone and one nobody ever owned are the same empty set.

Building and installing can be separate targets. One compiles on any host with the toolchain; the other installs a PREVIOUS build's artifact on whichever runner holds the handset — world install <project> <number> --target <install-target> --device <serial> — and proves the install from the device's own record of when the app was updated, rather than trusting the installer's "Success". The install target's build step refuses to run, so nothing quietly compiles where it should only install.

A build can be pinned to one handset with --device, and a pin to a host that cannot run the target is refused with the missing labels named rather than queued forever. Without a pin, a declared priority decides which of a runner's handsets is preferred; it orders, it never reserves.

Form factor is a separate axis from platform, not a finer value of it — an iPad reports its platform as ios, so a target that needs the tablet has to be able to ask for a tablet. And where the fact is observed it is reported as observed: Apple hardware reports its own device type, Android does not, so a declared form factor is marked with a ? rather than being printed as if something had measured it.

A phone can be plugged in, visible, and unusable — unauthorised, or stuck in some state that is not "ready" — and the effect is a lane that looks healthy while every build needing that handset waits forever. One of ours sat that way for three days with the runner online and green.

The alertable condition is stated once, in one place: attached AND not healthy. Unplugged is not broken — a handset nobody has plugged in closes an open incident rather than holding it open, because otherwise every lunch break is a standing alarm. "Attached" is judged on the same freshness the scheduler uses, so the thing that alerts and the thing that refuses to schedule cannot disagree about whether the hardware is there.

Getting this wrong is expensive in the direction nobody checks. Two of the conditions this watch first called faults were a device that was merely ABSENT, and an iPhone whose on-demand network tunnel was simply idle — both were fixed in what usable means rather than by making the alarm wait longer, because a watch that cries wolf is turned off and then catches nothing.

Where a device IS gets overwritten every time it changes, which makes "how long do these faults usually last?" unanswerable — the column that looks like a history is a clock. So every state transition is written down as its own row and kept for ninety days.

Two details are what make it honest rather than decorative. An unchanged heartbeat writes NOTHING, or the table fills with thousands of duplicate rows, looks healthy, and answers nothing. And a handset being unplugged is recorded as an event in its own right — otherwise a spell that ends by somebody pulling a cable has no end, and reads forever after as still running.