The Almanac of GodspeedOS - A History of Understanding

What this is. Not a changelog. Git already records what changed and when; this records why the thinking changed. Every entry below pins a date to a realization - a thing the project came to understand that it had not understood the day before - and then names the principle, commandment, or design decision that the realization produced. It is the intellectual history of the system, told backward from the rules to the days that earned them.

It is a record, rarely touched - a reflective document to look back on, not a living one. For the law, read CLAUDE.md; for its distillation, read COMMANDMENTS.md. This file is the answer to the question a contributor eventually asks at a whiteboard: "Why is it written that way?" The dates are real, anchored to commits and to the dated amendments in the constitution. The reasons are the part worth keeping.

Contents

2026-05-08 - The day the screen first said the machine was alive

The repository opened, and by the end of the day the kernel booted to a steady state under QEMU - Limine handoff, serial, GDT, IDT - after two x86 correctness bugs were found and fixed in the same sitting.

What I came to understand: a boot is not a feature you add; it is a fact you either observe on the wire or you do not have. The very first thing built was not the kernel - it was the ability to see the kernel: serial output before anything else.

What it produced: the instinct that became failures are loud, never silent (Invariant 12) and Commandment VIII's deeper sibling - wait for truth, not time. From day one, progress was something the serial console showed, never something assumed.

2026-05-09 - The day the nouns became verbs

In a single push the kernel gained a frame allocator and page tables, a preemptive scheduler, capability enforcement with generation checks, and synchronous IPC.

What I came to understand: the constitution's vocabulary - capability, generation, endpoint, quantum - was not description; each word had to be a mechanism or it was a lie. A "capability" that is not checked on every syscall is just a number.

What it produced: the discipline behind Commandment VII (no ambient authority) and the §7 capability model - authority is the cap you hold and the generation it carries, validated every single time, never inferred from who you are.

2026-05-10 - The day two services talked across a boundary

Cross-core IPC came up after a stack overflow and a silent-exception halt were chased down, and the first user-space service ran in ring 3: an ELF loaded, spawned, and logging through a syscall.

What I came to understand: the moment code crossed the kernel boundary, the kernel's job stopped being "do the work" and became "route, isolate, and enforce." A service is not a privileged part of the kernel that happens to live elsewhere; it is an untrusted identity the kernel mediates.

What it produced: Commandment I (the kernel is complete; use a service) and §26.10 (the kernel is mechanism, not policy). The line drawn that day - work lives in services, the kernel only mediates - never moved.

2026-05-12 - The day a service died and came back

The v1 milestone met its bar: ping on core 0, pong on core 1, a message across the boundary, pong killed, the supervisor restarting it, the system continuing.

What I came to understand: the headline was never "two services talk." It was "one can die and the other does not care, because it talks to a name, not a place." A client that saw EndpointDead, looked the name up again, and resumed had just demonstrated the whole philosophy in miniature.

What it produced: Invariant 11 (identity is stable; location is not) made concrete, and the seed of Commandment IX - recovery is the client's job, and a restart is normal, not exceptional.

2026-05-21 - The day the model met a real machine

GodspeedOS booted on real x86_64 hardware via UEFI USB (CLAUDE.md §23.3). The bring-up machine was a Dell Wyse 5070 thin client - the Intel J5005 (Goldmont+) of the first performance tables. Four cores up, cross-core ping/pong running on metal.

What I came to understand: QEMU is a model of a machine, and a model is a place where bugs hide politely. Real silicon does not round off the corners. A thing is not "done" because it is green in emulation.

What it produced: the standard that would later harden into Commandment II - chaos and correctness are judged on hardware - and the habit of treating a clean QEMU run as a hypothesis, not a verdict.

~2026-05-30 - The day the thin client taught me about real concurrency

On the Wyse, bare-metal boot froze shortly after the supervisor came up, and a cross-core reply-to-the-BSP path stalled hard under load. The diagnosis (bugs/1_FINDINGS_AP_TO_BSP_IPI.md, 2026-05-30): core 0 wedged with interrupts disabled, so a sender blocked on recv on the BSP was never woken. Development moved to the HP T630 (AMD GX-420GI, Jaguar/Puma+, 4 cores), the permanent SMP test machine thereafter.

What I came to understand: concurrency bugs are not abstract; they are specific to the silicon that exposes them. The J5005's stall was not a flaky machine - it was the machine telling the truth about an IF=0 wedge that QEMU and a friendlier CPU had both forgiven.

What it produced: the conviction behind Commandment VIII (rely on observable truth, not timing) and a permanent test rig on hardware that does not forgive. Cross-core correctness is only honest on a machine willing to deadlock you.

2026-06-09 - The day a trusted service stopped being trusted (H11)

registry became a real user-space name service holding only delegated caps, so its death degraded name resolution temporarily instead of corrupting the system - and it left the Trusted Computing Base (CLAUDE.md §6.1, Amendment 2026-06-09).

What I came to understand: "trusted" had been doing two jobs at once - "holds authority" and "cannot be allowed to die." Those are different claims, and conflating them is how a TCB quietly grows.

What it produced: the first real motion of the TCB-shrink program, and the early form of Commandment V - no service is special. A name service that resolves a stable name is itself just an identity, replaceable like any other.

2026-06-12 - The day I found kernel-equivalent power hiding in a driver (H1)

IOMMU DMA-confinement landed (CLAUDE.md §6.4, Amendment 2026-06-12). A DMA-capable driver, confined, reaches only its granted arena; the trust posture - confined or trust-critical - is now printed at boot.

What I came to understand: the very first invariant, no ambient authority, had an unstated exception I had never written down. A driver could aim its controller's DMA engine at any physical address - reach the capability model never granted, granted instead by physics. The model said "no ambient authority" while a USB driver quietly held the most ambient authority in the machine.

What it produced: the amendment that closed the gap in Invariant 1, and Commandment VII made literal at the hardware edge - authority must be visible, so the posture is a boot fact, not a hidden assumption. A silent exception to an invariant is worse than no invariant, because it lies.

2026-06-17 - The day a crash stopped meaning a reboot (Phase D)

With a crash-consistent redo-journal and recovery-on-mount, fs and block-driver became restartable and left the TCB (CLAUDE.md §6.1, Amendment 2026-06-17).

What I came to understand: the only reason the filesystem had been unkillable was that it could not recover - and "cannot recover" is a property you can engineer away, not a law of nature. Once recovery existed, the special status evaporated.

What it produced: Commandment IX made concrete (plan for recovery, and a service that can recover need not be trusted) and another rung down the TCB ladder toward {kernel}.

2026-06-18 - The day a file became a real capability (P2)

Delegated resource capabilities arrived (CLAUDE.md §7.10, Amendment 2026-06-18): fs mints a genuine, kernel-validated, revocable cap on open. The same day, the full persistence stack and the file-as-capability self-check ran green on the HP T630 against a real SSD - a file opened as a cap, read and written through it, non-escalation enforced, a forged handle rejected, the cap revoked on close.

What I came to understand: the meaning of a file belongs to a service, never to the kernel. The kernel does not need to know what a file is to mint, route, and revoke its capability - it only needs to carry an opaque resource id and let fs supply the meaning. "A file is like a capability" was a compromise; "a file is a capability" was achievable, and the difference was the whole point.

What it produced: the §7 north star - a file is a capability, literally, not by analogy - and a reinforcement of Commandment X (complexity in the layer that owns it): file semantics live in fs, capability mechanism lives in the kernel, and neither reaches into the other.

2026-06-20 to 2026-06-21 - The day I deleted the registry and learned what is truly irreducible

Over a phased migration, name-to-endpoint resolution moved out of the kernel and the registry service was retired outright (CLAUDE.md §6.1, Amendment 2026-06-21, "Path C"). A minimal name -> endpoint directory inside the kernel became the namer; the supervisor wires services from a name -> cap map; clients reacquire by name.

What I came to understand: an entire service had existed to store a mapping that turned out to be almost entirely derivable - the supervisor already knew the caps, the kernel needed only a tiny recovery anchor. Deleting it revealed how little state is genuinely irreducible, and how much "truth" is really a derived view dressed up as a source.

What it produced: this is the day behind Commandment III - do not duplicate truth; store the irreducible fact, derive the rest. When someone asks why III is written the way it is, the honest answer is: because deleting the registry showed that only a tiny amount of state was ever truly irreducible, and everything else should be reconstructed, not stored twice.

2026-06-21 - The day nothing was allowed to be unkillable but the kernel

Two long-standing exceptions fell at once. The shell was made restartable - kill shell respawns a fresh prompt instead of leaving a dead session. And in the Path C finale, the supervisor itself became restartable: the kernel respawns it on death, unconditionally and forever, and the respawned supervisor reconciles by adopting the still-running services. The non-restartable set reached its floor: {kernel} alone.

What I came to understand: every "this one cannot die" was a comfort, not a requirement. If the supervisor - the holder of restart authority - can itself be respawned by the kernel, then no service has any standing to claim exemption. "Trusted" finally, fully stopped meaning "unkillable."

What it produced: Commandment V in its strongest form - no service is special; only the kernel is special - grounded in Invariant 6 (services must be restartable) and §6.2-6.3 (the unkillable set is {kernel}). Identity survives restart; only the kernel is location.

2026-06-21 - The day Maximum Carnage was born

chaos max-carnage arrived - a storm that kills a random live service every round and asserts the kernel survives. It grew into a per-round sweep over every live service, mixing kills, queue floods, memory pressure, and spawn storms.

What I came to understand: passing tests proves a system handles the failures you imagined. It says nothing about the ones you did not. The only way to trust a system is to spend real effort trying to destroy it and watch it refuse.

What it produced: Commandment II - love Chaos and trust in it; thy service shall pass through Maximum Carnage - and the standard that a green unit suite is necessary but never sufficient. Chaos is the teacher; if it finds a bug, the bug already existed.

2026-06-22 to 2026-06-28 - The days Carnage found the wedges no one would write by hand

The storm did exactly what it was built to do. A 71K-round SMP deadlock from concurrent TLB shootdowns (fixed 2026-06-24 with per-core shootdown). A supervisor-respawn wedge where respawn ran inside an IF=0 timer ISR and could not acknowledge shootdown IPIs (fixed 2026-06-23 by moving respawn to an interrupt-enabled point). And at high round counts on the T630, a page-table reclaim page fault where a driver kill followed a corrupt PTE into an out-of-RAM frame - which led to the root-cause fix: quiesce DMA on driver kill (2026-06-28).

What I came to understand: these were not new bugs Carnage created - they were old bugs Carnage revealed, latent since the day the code was written, waiting for the one interleaving that QEMU and light load never produced. A system you have not tried to break is a system whose real failure modes you have simply never met.

What it produced: the conviction under Commandment II made unarguable, and the practice of long soaks on real hardware as the acceptance bar - the verdict tightened every time a soak exposed a wedge, because a test that passes a known-broken target is a weak test, not a clean one.

2026-06-25 - The day I wrote the constitution down as ten lines

COMMANDMENTS.md was authored - the Ten Commandments of Godspeed, each grounded in the invariants it enforces.

What I came to understand: the greatest long-term threat to the system was never a race or a fault - it was erosion. Hidden complexity, a convenience here, a silent fallback there, each reasonable on its own, would rot the model one "just this once" at a time. And a 2,700-line constitution is hard to defend a pull request against in the moment; ten memorable lines are not.

What it produced: the realization that the architecture, not the code, is the product (§26.1) - which is why a constitution exists at all, and why it has a human-readable distillation. The Commandments were written in response to real bugs, real wedges, and real lessons; they are the compressed memory of everything above.

2026-06-29 - The day the examples had to teach the truth

The example library was rebuilt as a Commandment-grounded reference for contributors, modelling the real patterns - placement defaults, explicit capability passing - rather than carrying stale registry references from a retired design.

What I came to understand: demonstration code that no one runs drifts into a lie, and a lie in the examples teaches the wrong model to exactly the people most willing to learn it. Examples are documentation that executes; if they are not tested, they are not true.

What it produced: the examples folded under Commandment IX's rule - if it cannot be tested, it does not exist - applied to teaching material, and a reaffirmation that the model is only as honest as the smallest program that claims to follow it.

2026-07-03 - The day the shell became a language, and a 10 MB script was a non-event

gsh grew from a flat command-runner into a real programming language - variables, arithmetic, conditionals, switch, loops, functions with bounded recursion, libraries (import), defer, typed-pipe aggregators, $( ) capture, and console input with a secret-taint guard rail - built the way everything here is built: fixed stacks, bounded arenas, streaming, no heap, loud on every ceiling. Then a 10 MB script was run, and nothing happened. That is the whole point.

What I came to understand: the ~7 KiB cap on a run-from-file script had felt like a limitation to apologize for. The 10 MB test inverted it. A heap-backed loader meeting a 10 MB file has a bad day - it allocates, it fragments, maybe it runs out of memory mid-parse. The streaming minifier meets the same file, reads about 7 KiB of it, prints one loud line - a huge script is a program - and carries on. The bound is not the wall the language hits; it is the thing that lets the language meet a pathological input and shrug. "Handled gracefully" and "bounded" turned out to be the same sentence.

The design of the language taught that lesson twice more, in miniature, by refusing to let me claim more than the system could honestly deliver. A secret from input secret is a guard rail against the accidental echo, not a vault - because once you may write it to a file, you can always read it back, so airtight secrecy is impossible by construction, and pretending otherwise would have been a lie the console told the user. And a function is not a pipe source - a boundary I found only by writing a real program (the selfcheck tour) that tripped over it, fn greet quietly shadowing the greet service.

What it produced: the proof that §26.6.1 is not a kernel rule - it is the rule. The same "change the representation, do not reach for a heap" that governs the piece table and the record arena governs a whole userspace language, and it holds from a 7 KiB script to a 10 MB one with no special case. A bounded system does not fear scale; it renders scale boring. (§26.6.1; Commandment III and the loud-failure discipline.)

2026-07-04 - The day a private discipline became a public one

The licenses were chosen - GPL-2.0-only for the OS, Apache-2.0 for the SDK - the tree swept for secrets and stray dashes, and on 2026-07-04 it went public: github.com/godspeedhq/godspeed-os. (The org is godspeedhq because plain godspeed was already taken - by a personal account that hosts, of all things, Android kernel repos; the project is godspeed-os, leaving room for a godspeed-lang beside it when the language comes.) 939 commits of private practice, opened in one setting change.

What I came to understand: every lesson above was learned in private, where the only person the discipline had to convince was me. Public means the constitution now has to hold against contributors who did not live through the wedges - which is precisely what the Commandments were written for.

What it produced: the moment the invariants stopped being a personal practice and became a public contract - held by the same identity tests, the same chaos bar, and the same ten commandments that earned them. The work of keeping the architecture uncorrupted does not end here; it opens. This was the almanac's one entry dated forward on purpose - the anticipated day, kept blank until the launch could make it real. It is real now.

2026-06-30 to 2026-07-01 - The day "wait on truth" turned out to be half a truth

A million-round chaos max-carnage soak finished with the system alive but the filesystem gone - ls said "no filesystem," drives read raw, and a cold reboot brought it all back. No data was lost; the disk was intact the whole time. So the bug was not corruption. It was a recovery that never happened.

The cause was a single wall-clock timer. fs waited eight seconds for block-driver to report a readable disk, then gave up. Under a soak that had wedged the AHCI controller, eight seconds was never going to be enough - the controller needed a hard reset, not more waiting - so fs latched "no filesystem" and sat there. The timer felt like the safe, obvious way to wait for a dependency. It was the trap. Commandment VIII, exactly: wait on truth, not time.

What I came to understand, and it had a turn in it. Removing the timer and waiting on block-driver's actual reply fixed the soak - but following that discipline rigorously exposed something deeper. The request/reply primitive itself could hang: a blocking wait on a reply hangs forever if the reply never comes, and it never comes if the peer dies mid-request. "Wait on truth" had quietly become "wait forever" for the one truth it could not see: the peer's death. The easy version of a principle ("do not use timers") and the complete version of it are not the same thing.

So the fix was not only in fs; it was in what "truth" means. We taught the kernel that a peer's death is a truth too - a synchronous Call whose caller wakes with ReplyDead the instant its replier dies, the reply-side twin of the sender-side wake §8.6 already had. seL4's Call/Reply, borrowed for exactly this. And the principled part, which was the owner's call: putting it in the kernel grows its code but not its responsibility - IPC death-semantics were always the kernel's job (Commandment I is about scope, not line count). Mechanism, not policy: the kernel learned about a reply cap, never about "RPC."

And then the fix had a hidden failure of its own. The kernel change passed every green test - identity, fs-restart, a new ReplyDead pin - and still drained the whole system to zero on the first hardware soak: recovery of the dead supervisor hung on a single trigger, the timer ISR, which the storm starves, and the new code's latency was just enough to lose a race main had narrowly won. The lesson turned on itself. A fix I believed was complete was again only half-satisfied, and the half missing was the failure case - caught by exactly the fire this whole entry is about. The repair drove recovery from the storm's own hot path, the yield, making it un-starvable - stronger than it had ever been.

What it produced. Commandment VIII, deepened: wait for truth, not time - but ensure the truth includes failure. A wait that cannot observe failure has quietly become an infinite wait on time. A standard for every contributor (CONTRIBUTING.md): interdependent services wait on each other's truth, the reply or the loud fact of death, never on a clock. And the lesson I keep relearning, now with teeth: a principle you believe you have satisfied is usually only half-satisfied, and the half you are missing is what happens when things fail.

2026-07-04 - The day the system survived a million acts of violence

chaos max-carnage all-services 1000000 ran to completion, and the system works as expected.

A million rounds. Each round is not a gentle probe - it kills every restartable service, floods every endpoint it can reach, storms memory and the spawner, and rotates a kill through whatever is left. This is the same soak that, in the entry just above, drained the system to "no filesystem" on its first hardware run and taught the hardest version of Commandment VIII. This time it finished. Nothing had to be rebooted. The prompt still answered.

A million is not a number you reach by luck. Fault tolerance that is 99.99% correct dies long before round ten thousand: leak one frame per kill, fail to reclaim one capability, leave one lock wedged, and the footprint grows without bound until something falls over. To reach 1,000,000 every recovery path has to be not merely correct but conserving - each respawn reclaims the dead instance's frames, kstack, and caps before it allocates fresh, so the footprint is flat and only a count grows (§6.2). The kernel respawns the supervisor unconditionally, forever; the supervisor reconciles - adopts the survivors, respawns the dead; clients see EndpointDead, reacquire by name, and retry. unkillable = {kernel}, and the kernel held a million times over.

This is the entry above, vindicated. The wait-on-truth fix, the un-starvable respawn driven from the yield path, the ReplyDead reply-side twin - each was built to survive exactly this fire, and now the fire has burned a million times and the house still stands. Fault tolerance stopped being a claim in a constitution and became a measured fact. The model was right; the proof is a number with six zeroes.

The receipt. When the storm was called off, observe now on the T630 put the whole truth on one screen:

--------------------------- system state (8 live) ---------------------------
UPTIME: 3d 03:07:34     CPU: C0 99% C1 99% C2 0% C3 0% (49%)     RAM: 5 MiB used / 7 GiB total (0%)
TASK  NAME          CORE  STATE      RESTARTS   QUEUE   UPTIME
 3    supervisor    C0    BlockRecv    142829    0/16      9h
 4    block-driver  C1    BlockRecv    142826    0/16      9h
 5    fs            C1    BlockRecv    997721    0/16      9h
 6    shell         C0    Running      997721    0/16      9h
 7    logger        C0    BlockRecv    143706    0/16      9h
 8    xhci          C1    Running      143704    0/16      9h
 9    ehci          C1    Ready        143706    0/16      9h
drives:  0  data  GSFS  122104 MiB (122074 MiB free)

RESTARTS is what observe calls deaths recovered, not clean re-runs - the soak's odometer. fs and shell had each come back from death 997,721 times; supervisor, block-driver, logger, and the two USB drivers each sat near 142,000-143,000 - together roughly 2.7 million individual service deaths, every one recovered, not one escalated. And the number that settles it is two lines up: <a class="gs-abbr" href="glossary.html#gl-cpu"><abbr title="RAM - processor and memory.">RAM</abbr></a>: 5 MiB used. Flat. After 2.7 million deaths and resurrections and three days of uptime, memory had not grown by a single page - one leaked frame per death would have bled ~11 GiB; the machine used 5 MiB of its 7 GiB, and the 122 GiB SSD sat untouched at 122074 MiB free. That is what "conserving, not merely correct" (§6.2) looks like the moment you finally read the meter.

2026-07-04 - The day gsh was finished, and never once reached for a heap

gsh started as a batch runner: run_lines split a file on ; and \n and handed each line to execute. It is now a language. Variables and arithmetic, if / switch, for / loop with break / continue, functions with parameters and bounded recursion, libraries via import, defer, record aggregators, typed pipes - and, last, the output-capture cluster: for line in (producer) to walk a stream, if myfn to branch on a function's result, and $(fn) to capture a function's output into a value. That last one closed the set. gsh is complete.

And it never once reached for a heap. Every part of it lives on the stack or in a fixed, named arena: the ~5 KiB variable table, the ~7 KiB script buffer, the 512 B capture buffer, the 16-frame block stack. Not one general allocator in the whole interpreter. When a working set felt too big for the stack, the answer was never "add a heap" - it was "change the representation": stream a file in fixed chunks, refer to data by (offset, len) spans, give a subsystem its own bounded arena, iterate with an explicit stack instead of native recursion (§26.6.1). The constraint did not cramp the design; it improved it. Every limit in gsh is a number you can read off the source.

The proof is a 10 MB script. A shell that malloc'd its way through source would choke on it or take the machine down with it. gsh reads ~7 KiB, truncates the rest loudly, runs the tour it can hold - functions, captures, error paths and all - and leaves the prompt answering. Overflow is not a crash here; it is a feature. Every bound fails the same honest way: a value too big for the 512 B capture, a script too big for the 7 KiB buffer, a statement too long to hold across a read - each is a loud, specific refusal, never a silent truncation and never a corrupted result. The system tells you the truth about its limits.

The last feature nearly went the other way, and the honesty of the ask is the whole point. Capturing a function's output looked like it wanted a big buffer, and the question came out loud: "is this a cool place to use a heap - throw the function on a heap, get the result, throw the heap away?" That is the reflex, caught in mid-air - because "scratch space, filled, then discarded" is not a heap, it is a bounded stack buffer, which is exactly what we used (512 bytes - it lives in the executor's own frame, so it must stay small enough to coexist with everything else on a 256 KiB stack; a captured value is a name, a number, a short line). The heap is a shape of thought, and the shape can almost always be had without the allocator. gsh is that argument, made in code, half a kilobyte at a time.

2026-07-04 - The day it shipped, and called itself 0.1

Hours after the repository went public, GodspeedOS cut its first release: v0.1.0 - a tagged, checksummed, bootable UEFI image on the releases page. The old v1.0 tag - the marker left when the first cross-core ping/pong survived a restart - was deleted to make room for it. A system that had called itself "v1" for two months shipped its first release numbered 0.1.0.

What I came to understand: a version number is not a trophy for a milestone reached; it is a claim about maturity, and the honest claim was 0.x. "v1" was the name of a goal met - two services, cross-core IPC, restart. But the software underneath is still moving fast - a whole language landed the same week - and SemVer 0.x means precisely that: anything may still change. Numbering it 1.0.0 would have promised a stability the system does not yet owe anyone. The crates already knew; they had said 0.1.0 all along. The release only spoke the truth the code was already telling.

What it produced: the system's own honesty discipline, turned on its own version string - say what is true, even when a larger number would flatter. The milestone was v1; the software is 0.1; both are true, and the release states the one that matters to a person about to boot it. (A smaller truth rode along: the 64 MiB image is 99.4% empty space, so it ships compressed to under half a megabyte - honest about its size, too.)

2026-07-07 to 2026-07-08 - The day the wire became location

The networking stack, already carrying a real DHCP lease and a ping on the T630's Realtek NIC, was taught to survive the two things a real network interface actually does: the cable moves, and the services die. A running ping now rides an unplug and a replug - it reports the link down, and when the cable returns and the lease re-acquires, it simply resumes, q-abortable the whole time. net-stack configures itself the moment the driver sees carrier; net renew repairs a bad lease inside the running service, no restart; and chaos max-carnage nic-driver,net-stack storms both networking services together, round after uncapped round, while the link keeps coming back.

What I came to understand: the cable is location, and the socket is identity - the same separation SMP drew between a service and its core, now drawn at the edge of the physical world. A link that goes down is not an error to fail on; it is a location changing, exactly what Invariant 11 was written to make survivable. The stack does not own the cable's state any more than a service owns its core; it observes carrier as a fact and reconverges when the fact changes. Plugging in a cable is not a command to handle - it is a truth to notice.

What it produced: Invariant 11 (identity is stable; location is not) reaching all the way to the RJ45 - and the networking half of the restartability story the storage stack had been telling. A socket cap outlives the cable, the lease, the driver, and the stack; every one of those is location, and the system reconverges to the identity from any of them moving. "Not even a blip" is what identity-over-location feels like when you finally pull the plug and watch it come back.

2026-07-09 - The day half a truth came back for the mount loop

A rapid second kill all-services on the T630 - the whole restartable set killed twice within seconds

  • wedged the storage stack for good. Not a crash: a permanent thrash. fs had read a garbage disk capacity from a stale reply, derived a backup-superblock LBA far past the end of the disk, and asked for it; the AHCI controller stuck its command-issue bit without ever setting BSY, so the port recovery that watched only BSY never fired; and the mount loop retried the impossible read forever. The keyboard looked dead because the whole machine was buried under an infinite storage retry.

What I came to understand: this was the same lesson as the million-round soak that deepened Commandment VIII (the 2026-06-30 entry above) - and that is exactly the point. That day I taught fs and the kernel that a peer's death is a truth a wait must be able to see. But "wait on truth, and the truth must include failure" is not a fix you apply once and retire; it is a lens you re-apply at every wait you write. The mount loop was a new wait, written to spin until success, and it had quietly inherited the original sin: it could observe success forever and failure never. A persistently unreadable disk was a truth it had no way to conclude. The principle had not failed; I had simply found one more place I had not yet held it to.

What it produced: the three fixes that finish Commandment VIII in the storage stack - a port COMRESET that escalates on a stuck command-issue bit and not only on BSY (so the controller self-heals the wedge instead of retrying into it); a capacity reply fs accepts only in its exact, bounded shape (so a stale reply is never misread as a truth); and a mount loop with a failure-truth exit - bounded real attempts, after which fs comes up degraded and loud, "storage unavailable, data intact," rather than spinning on a disk that will never answer. And the input path learned the same shape from the other side: the console now drains the UART on read, not only on the timer tick, so a keystroke - a truth you can observe directly - is never held hostage to a proxy the storm can starve. One lesson, three surfaces, re-applied where I had not yet applied it.

2026-07-09 - The day recovery became a fixed point

The same double-kill storm exposed a second, quieter fault. When the supervisor came back and adopted the still-running services, a death notice could still arrive for a service it had already recovered - a duplicate, or one in flight when the respawn completed. Reacting to that notice the obvious way spawned a second instance of a service that was already alive, and the churn fed the very storm it was trying to survive.

What I came to understand: recovery cannot be a reflex fired per event; it has to be convergence to a target set. The supervisor's job is not "handle each death" but "make the world match the manifest" - every notice reconciles the live tasks against the set that should be running and acts only on the difference. A service already present is a no-op it logs as already recovered; a service genuinely absent is respawned, exactly once. Framed that way, the supervisor becomes a fixed point: from any perturbation - one kill, a storm, its own death mid-storm - it settles to "every service alive, one instance each," and a redundant notice is absorbed, never amplified. It is Invariant 11 seen from the restarter's side: the supervisor restores an identity set, indifferent to how many messages announce the same gap, exactly as a client is indifferent to which core the name now lives on.

What it produced: the supervisor's death handling made idempotent - the recovery-side complement to identity-over-location, and the property that lets chaos max-carnage kill everything, twice, in a tight loop without the recovery path becoming its own denial-of-service. A system that reconverges is one you can hit as hard as you like: it does not matter in what order, or how many times, the bad news arrives.

2026-07-10 - The day the kernel met a second architecture, and fit the machine it found

Two things landed the same day. The kernel stopped guessing how big the machine is: the MAX_CORES ceiling was deleted, and every per-core structure became a boot arena sized to the core count Limine actually reports, while the frame bitmap became sized from the RAM that is actually present - no cap, no waste (this branch, through cfb0a88). And then the hardened image booted on a second CPU architecture: a Dell Wyse 5070, Intel Goldmont+, came up first try - SMP, the AHCI flash, the Realtek NIC pulling a real DHCP lease and answering a ping, the shell - all of it, on silicon from a different vendor than the AMD GX-420GI (the HP T630) that every wedge in this almanac had been caught on.

There was a wound in that. The very first bare-metal boot, back on 2026-05-21, had been on a Goldmont+ part too, and within the week it stalled hard under load (The Sleeper on Core Zero, the first IF=0 bug) and sent development to the AMD T630 for good - the workhorse on which the shootdown deadlock, the respawn wedge, the DMA-after-death, and the million-round soak were all found and fixed. Two months and a hardened kernel later, Goldmont+ got its second chance and took it cleanly, first boot. Even the one thing known to be AMD-specific - TSC calibration, broken on the GX-420GI because it exposes no CPUID frequency leaf - simply worked on the Intel part, because the fallback calibrates against the PIT: a machine-specific truth read at boot, never hardcoded.

What I came to understand: portability is not a feature you write; it is the absence of assumptions you should never have made. Nothing in the boot path had to be told about the Wyse. The kernel counts its cores from Limine, sizes its bitmap from the memory map, and calibrates its clock against whatever timer answers - so a machine it has never seen configures the machine, not a constant in the source. A kernel that fits the machine it finds has nothing left to port. And the two CPUs it has truly lived on - AMD first, as the workhorse; Intel Goldmont+ second, as the proof - are now one image, unchanged between them.

What it produced: the dynamic-topology work (no MAX_CORES, RAM-sized bitmap) as the executable form of "identity over location" pushed all the way down to the hardware census - the machine's size is location, discovered, never identity, assumed. The only ceiling left is the xAPIC's 8-bit LAPIC id, and even that is now loud rather than silent: past 255 cores the kernel says so and names x2APIC as the next step, instead of quietly wrapping. Two CPUs, two vendors, one image, booting to the same steady state.

2026-07-11 to 2026-07-13 - The days the whole system stood for audit

For the first time the system was read the way a constitution asks to be read - not "does it work?" but "does it still obey?" A kernel audit (2026-07-11, audits/kernel-audit.md), run as a parallel sweep of independent readers over every subsystem, checked the code against the Ten Commandments and the invariants and surfaced three real things: two paths where a non-page-fault CPU exception - a stray #GP, a div0 in a service - halted the whole machine instead of killing just the offending ring-3 task (a single bad instruction in userspace could wedge the kernel), and one where a transient allocation failure on the supervisor-respawn path could panic! into a reboot, defeating the very Phase-6 recovery it lived inside. Then a userspace audit (2026-07-12 to 2026-07-13, audits/userspace-audit.md), the first of its kind, graded every service - the coupled pairs first, block-driver/fs and nic-driver/net-stack, then the rest - against the same law: 0 high, 8 medium, 8 low, each finding logged with a fix or a recorded deferral.

What I came to understand: the audit is affordable because the kernel is small. The 30-minute whiteboard rule (§26.11) and Commandment I (the kernel is complete; keep it small) had always been framed as correctness and understandability properties - but they were quietly an audit budget all along. A 2,700-line constitution enforced against a kernel one engineer can hold in their head can be walked end to end in an afternoon; the same audit against a monolith is a standing team that never finishes. Tininess is not only what makes the kernel correct or secure - it is what makes the recurring act of checking correctness and security cheap enough to actually do, release after release. The kernel stayed small so that auditing it could stay routine.

What it produced: audits/kernel-audit.md and audits/userspace-audit.md as living artifacts - a standing, repeatable practice rather than a one-time pass - and the recognition that the north star of an audit is that nothing above the kernel may panic or wedge it: a service's worst instruction should cost that service its life and no more. The findings became the next increment of work; the discipline became a habit. A model you can re-verify in an afternoon is a model that stays honest between the days someone thinks to check.

2026-07-14 - The day the boundary became something a machine enforced

Straight off the audit, the arch boundary stopped being a habit and became a law. The survey came first - because you cannot seal what you have not measured: 126 places where a neutral file reached for the x86 layer by name, 23 lines of inline assembly living outside arch/, and - the number that mattered - zero of either in the capability table or IPC. Then the sealing: one seam, crate::arch::imp, that names "the implementation for whatever architecture this is"; all 126 references routed through it; every inline-asm operation (read the page-table base, invalidate a TLB entry, mask interrupts, send an IPI) moved out of the neutral layers and behind an arch::imp primitive, so that not one arch-neutral file holds a single instruction of assembly. And last, the part that makes it stick: arch_boundary_check.py, a CI guard that fails the build if any file outside arch/ so much as writes asm! or names a specific architecture.

What I came to understand: a boundary that lives only in a contributor's head, or even in a well-written doc, will rot - because nothing stops the next well-meaning change from crossing it. The whole difference between a convention and a law is whether a machine refuses to let you break it. The seam made the boundary nameable (arch::imp, one place); the guard made it unbreakable (CI, every push). This is §26 - the architecture survives only if the discipline survives - turned from an aspiration into a mechanism. And the survey taught its own lesson: the belief "the boundary is clean" was worth nothing until it was a number, and the number that carried the whole bet was the zero in capability/ and ipc/.

What it produced: the arch::imp seam and the arch-boundary guard - the fourth of the kernel's mechanical disciplines (alongside the unsafe audit, the contract reconcile, and the dash convention). A boundary you can no longer cross by accident, only by deleting the guard on purpose. It set the stage for the only test that could prove the boundary was real.

2026-07-14 - The day one kernel booted three instruction sets in QEMU virt

With the boundary sealed and enforced, it was put to the one test that cannot be argued with. The arch-neutral kernel - the capability table, IPC, scheduler, syscall dispatch, memory, task, loader, every service's logic - was compiled and booted on three genuinely different instruction sets, writing only arch/<isa>/. x86-64 ran the full OS, all the way to the gsh> shell. AArch64 booted under QEMU's virt virtual machine (qemu-system-aarch64 -M virt) and put a line on the PL011 UART. RISC-V booted under the same virt machine (qemu-system-riscv64 -M virt), handed off by OpenSBI into S-mode, and put a line on the 16550. Not one arch-neutral file changed between them. Three QEMU-virt consoles, one codebase.

What I came to understand: a boundary is a claim until a second implementation drops in behind it without touching what it bounds - and it becomes a law on the third. The demarcation was never something to prove on a whiteboard; it was something to observe on a wire, exactly like the boot on day one. And the tell was in how the three went. AArch64 took real debugging: the CPU traps FP/SIMD at EL1 by default and Rust emits NEON for a byte-copy, so the first port faulted before its first Rust statement (found by reading the exception ESR off qemu -d int), and the stack pointer had to be forced 16-aligned. That was me learning the architecture's personality. RISC-V went first try, no debugging - riscv64imac is soft-float, so there was nothing to trap, and by the third ISA the playbook was solid enough that the neutral kernel simply came along for the ride. That asymmetry is what "bounded" feels like: the second implementation teaches you the arch's quirks, the third is a formality. The deepest thing is that this day and the audit days were the same act seen from two sides - the audit proved the inside was honest (nothing above the kernel could wedge it), and the port proved the edge was real (nothing inside had smuggled the machine into the model). A boundary is only as trustworthy as the code it bounds is clean; the code was clean because it had already been walked end to end. The small kernel that was cheap to audit was, for the same reason, cheap to carry to a new machine.

What it produced: docs/multi-arch.md, and the founding thesis closed into a demonstrated fact. The boundary is now held not by intention but by four CI guards and three live QEMU-virt consoles: to add a fourth ISA you implement arch/<new>/ to the imp surface, add one #[cfg(target_arch)] arm, and the compiler plus CI prove that no neutral file had to move. A capability microkernel kept small enough to fully audit has an arch boundary clean enough that a new architecture is a bounded drop-in - claim to proof, in the span of a day that only looked sudden because a hundred careful days came before it.

2026-07-25 - The day the second architecture stopped being a demonstration

On a Raspberry Pi 2, on real silicon, the shell printed run: ran 349, failed 0.

The arm32 port had booted before. It had reached userspace, driven a keyboard through a hub, mounted a disk, spoken DHCP over USB ethernet. What was different on this day is that the whole self-check ran and every case passed: the language tour, the pipes, the records, the filesystem, and - the one that mattered most - fcap, the file-as-capability proof (§22 Test 14). A file opened as a real kernel capability, written and read through the cap, non-escalation enforced at both the kernel and the fs layer, a forged handle rejected, the cap revoked on close and on rename. The §7 north star, true on a second architecture and not by assertion.

Four bugs stood in the way, and the striking thing is that not one of them was visible under emulation:

  1. A single DWC2 channel shared by the keyboard poll and storage, where a keyboard split abandoned mid-flight corrupted the next block transfer.
  2. resource_invoke packing its right field at bit 32 of a syscall argument, where arm32's register is 32 bits wide, so the right arrived as zero.
  3. The Bulk-Only Transport reset recovery missing its first step - the Mass Storage Reset that resynchronises the device's state machine, without which clear-halt unsticks the pipes and leaves the device still waiting for a transfer we had abandoned.
  4. A failed recovery that reported nothing, because its result was discarded with let _ =.

What I came to understand: these were not logic bugs, they were bugs about the machine being real. An emulator gives you a device that is patient, coherent, infinitely fast, and stateless between commands. Every one of the four was the consequence of a device that is none of those - a channel that is genuinely contended, a register that is genuinely 32 bits, a controller that genuinely holds state you must resynchronise, and a recovery that can genuinely fail. QEMU had said yes to all four for months. It was not lying; it was answering a different question.

And the third and fourth are the same bug wearing different clothes. The reset step was missing, and the code that would have told me it was missing threw its answer away. §26.7 already said a failed recovery must stay visible. What I had not appreciated is why that matters so much: a silent failed recovery does not merely hide a defect, it misdirects the investigation. Because the verdict was discarded, the log could not distinguish a device that was confused from a device that was gone - so I concluded the hardware had failed, and asked the operator to go test their USB stick. They pushed back: the stick was fine. They were right. The defect was mine, and the reason I could not see it was a return value nobody read. An error you discard does not just cost you the error. It costs you the next several hours, spent looking at the wrong component.

What it produced: bot_recover now performs the full spec sequence and returns its verdict, and says out loud when it could not recover - which turns "storage is broken" into the answerable question "is this device confused, or gone?". The rule earned a sharper form: an unread error return is not a missing log line, it is a wrong suspect.

2026-07-27 - The day a stick swap ended the hunt, and the recovery earned its keep

The self-check printed run: ran 349, failed 0. Then it printed it again, and again - eight times in a row, on a Raspberry Pi 2, on real silicon. Consistent zero: the bar that had sat just out of reach for a week of hardware round-trips.

What reached it was not a line of code. It was pulling out the USB stick and pushing in a different one.

For days the storage had failed the same way: under sustained I/O - a drives check reading the whole tree - the stick would stop answering, roughly once per run, and cost the handful of tests in flight at that instant. Every layer above it was made to survive that: a stuck-endpoint reset, an escalation to port-reset and re-enumeration, a filesystem that re-mounts on an I/O error, a journal that replays an interrupted write. And along the way real defects fell out that had nothing to do with any stick - a synchronous Call that took the wrong message off its own queue because it matched replies by arrival order instead of by sender (latent on x86 too); a format that condemned its own successful writes by reading them back through a device that served stale data; the FIFOs sized for a keyboard while a 512-byte bulk packet needed the whole of one. Each was found, each was fixed, and still a revival cost a run.

The operator proposed the one experiment I had been talking around: change the stick. Eight clean runs later, the answer was unambiguous. The old stick's own controller had been failing under load - not leaving the bus, just wedging its own firmware while staying plugged in. A dodgy part.

What I came to understand: faults stack, and fixing a real one does not exonerate the layer beneath it. The 2026-07-25 entry above is the same investigation, two days earlier, and its conclusion was true - the silent clear-halt was a genuine defect, and "the stick is fine" was the right correction at the time, because the software bug was masking everything under it. But "the software bug was real" and "the stick was fine" are not the same statement, and I had quietly treated them as one. Peel a real fault and you do not reach innocence; you reach the next fault. Only a controlled experiment - one variable changed, everything else held - can isolate the last one, and no amount of reading an ambiguous log substitutes for it.

And the elaborate recovery was not wasted on a problem that turned out to be hardware - it was the whole point. The proof is in the same log: the old, failing stick dropped off twice and still produced two 349, failed 0 runs, because the recovery caught it and carried on. Resilience surviving a genuinely failing device, gracefully, is exactly what resilience is for. The reply-correlation fix and the others would have been worth making on any stick; the recovery earned its keep on the bad one.

What it produced: the discipline of the one-variable swap before the multi-day rewrite - because the next thing I had been about to build was a full interrupt-driven restructure of the USB stack, days of work, to fix what a $3 part swap settled in an afternoon. That the restructure is still worth doing (for reasons that were never this bug) is a separate decision, now made in the clear rather than under the pressure of a failure I had misattributed.

2026-08-09 to 2026-08-12 - The day the kernel gave up the USB stack, and then gave up the choice

kernel/src/arch/aarch64/xhci.rs was 2,742 lines of ring-0 code parsing descriptors supplied by whatever someone had plugged in. It worked. It had driven a keyboard and a disk on a Raspberry Pi 4 for weeks. On this day it was deleted, and the xhci SERVICE drove the controller instead - through an MMIO capability and a DMA arena granted at spawn, reaching the console through CONSOLE_PUSH, serving storage over the block protocol block-driver already spoke. The kernel's entire remaining involvement with USB became establishing that the controller is present and saying so.

The move was done behind a feature flag first, so both drivers could exist while the service was proven on hardware. Then the flag was deleted too, and that second deletion is the part worth remembering.

What I came to understand: a flag that selects between a compliant kernel and a non-compliant one does not resolve the violation, it stores it. As long as usb-via-xhci existed, there was a build of GodspeedOS in which the kernel drove USB - one flag away, and Commandment I was satisfiable rather than satisfied. It was also a genuine footgun: the flag had to reach three crates, and setting some of them gave two drivers on one controller, or none, with every half booting fine alone.

And I had to be precise about what the move actually bought, because the milestone is real but narrower than "Commandment I closed" sounds. The Pi 4 has no SMMU wired up: confine_device returns false, and the service is granted its controller's BAR plus a DMA arena including its physical base. So it can still point the controller's DMA anywhere, which is kernel-equivalent reach by the same rule that governs an unconfined x86 driver. The honest statement is: a buggy USB driver on the Pi 4 is now bounded; a compromised one is still kernel-equivalent. What closed was the accident surface - ring-0 descriptor parsing became a restartable service - and the ambient authority, since the service holds only what its contract grants. The trust posture is unchanged until an SMMU confines the device, and saying otherwise would have been the nicer sentence and the wrong one.

2026-08-17 to 2026-08-29 - The day the terminal left the kernel, and I learned what earns ring 0

Two more evictions, on the port with the least room for them. arch/arm/dwc2.rs - the whole ARM32 USB stack, keyboard and networking and mass storage - became services/dwc2, driven by a real device interrupt rather than the core-0 timer tick. And kernel/src/fbcon, 1,172 lines of terminal emulation, became services/console: the ANSI/CSI state machine, the UTF-8 decoder, the shadow grid, the cursor, scrolling, reverse video, all of it out.

What stayed behind cannot format, cannot position, and cannot scroll. Reaching the bottom of the screen clears it and starts again at the top.

What I came to understand: the test for ring-0 residency is not "is this important" or "is this hard to do elsewhere". It is impossibility. A panic halts every core, including the console service, so a panic cannot ask a service to report it - the kernel must be able to say the machine died using only itself. Boot output has the same shape: it precedes every service, including the one that would render it. That argument justifies exactly a bounded blit and not one line more. Rendering a shell prompt is policy; a display driver is anti-scope by name. The operator control channel had asked for ring-0 residency earlier and failed this same bar, which is how I knew the bar was real and not built to fit what I already wanted to keep.

The evictions also made honest a claim that had been quietly arch-specific. The logging floor said the kernel writes to "the serial console", which is always true on a PC and false on a Pi wired to a TV. A kernel that can only speak to a port it does not have is mute, and a boot failure on that machine was a frozen screen with no reason on it.

2026-09-03 to 2026-09-04 - The day the kernel stopped interpreting the bus

The kernel held a table of what every device class was: which BAR, which IRQ, which BDF, per class, sixteen of them. A driver was spawned because the kernel knew what an xHCI was. By the end of D3 the supervisor owned the images and a driver named a device class code, which the kernel resolved from a table it holds no opinion about. service_config rows went 222 to 2. Embedded images went 30 to 1. Per-class device statics went 16 to 0.

Four bugs surfaced during it, each found by an instrument built for a different one: an fs reply desync, a block-driver "no disk" latch, an xHCI port-status misattribution, and an arm32 serial splice that was corrupting the very evidence being used to chase the other three.

What I came to understand: "the kernel is mechanism, not policy" is easy to agree with and hard to notice breaking. Not one of those sixteen statics arrived as a policy decision. Each arrived as one more small fact the kernel happened to know, and the aggregate was a kernel with opinions about PCI. The question that catches this is not "is this policy?" - nobody ever answers yes - but "does the kernel have to interpret anything here, or can it be handed the answer?"

2026-09-05 to 2026-09-06 - The day five bugs stood in a line, each hiding the one behind it

logger became events, which was the name catching up with the job: it had held the IPC trace ring since tracing shipped and had never held a log line in its life. Then it grew metrics and a log window, and recorder grew beside it to put a capture on disk. Ordinary feature work.

Then the hardware disagreed, five times, and every disagreement presented identically: a row that simply was not there.

  1. The sink was reacquired on send failure - and the sample that triggered the reacquire was dropped.
  2. A missed resolve latched forever, because u32::MAX was read as "this service holds no sink cap", a permanent fact, when it is also what a service sees if the sink happens to be mid-restart at its first emission.
  3. The kernel dropped the peer DECLARATION when a declared peer was down at spawn. Not the capability - the declaration. AcquireSendCap authorises a reacquire only for a declared name, so a service spawned during the 47 milliseconds its peer was restarting could never reach that peer again, for the life of the instance. It cost fs its disk and produced 113 failures in one run, with storage reported unavailable while block-driver sat idle beside it having received exactly one message.
  4. A capless publisher never asked for a capability, because the reacquire that would have fixed it sat behind a send failure - and a send that never happens never fails.
  5. Liveness waited out its 64-message interval after a restart, so a quiet service read as dead.

What I came to understand: the third is a violation of the restart contract, in the kernel, and it had been there the whole time. The constitution promises a client can reacquire a restarted peer; a scheduling accident of 47 milliseconds silently revoked that promise for the life of a service. It was findable only because an error message, added mid-investigation for an unrelated reason, distinguished "refused" from "the name does not resolve" - two words apart, and the whole diagnosis hung on them.

The shape they share is the lesson, and it generalises well past observability: a recovery path that only runs on failure cannot recover a component that never gets far enough to fail. Four of the five were a version of that. If a thing recovers only when it errors, ask what happens to the one that never starts.

2026-09-06 - The day the second core stopped being an answer

Several times, a scheduling or liveness problem had been answered by moving a service to a different core. That is not a fix; it is a workaround that depends on a second core existing to absorb the fault. So the absorber was taken away: one core, on a Raspberry Pi 4 that has four.

It survived. Three clean self-checks, a hundred rounds of Maximum Carnage, 611 kills, 500 endpoint floods, zero panics, zero liveness wedges. The cost was 1.6x slower for 4x fewer cores, which is what a synchronous-IPC microkernel should cost when most services are blocked on recv most of the time. Predictable degradation, no correctness cliff.

The experiment found something on its first boot, before any of that. The compiler said it without being asked:

warning: struct `PlacementInvalid` is never constructed

The placement rule promises that a contracted core which is unavailable rejects the spawn, with the rationale spelled out: silently rerouting would be exactly the kind of reinterpretation a capability system is designed to forbid. It had never been enforced. xhci declares core 2 and started on core 0 without a word, on every machine, for the entire life of the project.

What I came to understand: every conclusion ever drawn from "it works on core N" rested on a fact the kernel never checked. Those placement lines have been decorative. That is worse than a missing feature - it is a silent fallback at the kernel boundary, which the contribution rules list as an automatic rejection - and what makes it instructive rather than merely wrong is that the type existed. PlacementInvalid was defined, resolve() named it in its signature, and nothing ever constructed one. The shape of enforcement was in place without the enforcement, which is the most convincing way for a guarantee to be absent.

The Days I Was Wrong

2026-09-06 - The day I read a zero that had never been measured

Chasing why a service had stopped publishing metrics, I added an instrument: a one-shot line that would say so if the sink had been resolved late. The hardware run came back and the line was not there. I reported that as evidence - "resolved LATE: 0, so the latch was never the cause" - and moved the investigation somewhere else on the strength of it.

The instrument had no call site. The edit that added it had two halves; the first landed in trace.rs, the second hit an assertion error and never wrote, and I re-ran only the other half. The helper compiled, was public, and was called by nothing. The zero was not a measurement of a healthy system. It was not a measurement at all.

What I came to understand: a missing instrument reads exactly like a healthy one. Both produce silence, and silence is what I was treating as information. Every other kind of broken tool announces itself - a test that cannot compile, a check that errors - but an instrument that was never wired in returns the same thing a passing system does, and it returns it confidently.

The habit that comes out of it is small and mechanical: after adding a report, verify the CALL SITE exists, not that the helper compiles. grep -c for the caller, in the same breath as writing it. I already knew to distrust a number I had not earned; what I had not internalised is that an absence is a number too, and it needs earning in the same way.

There is an uncomfortable second half. It cost a hardware round-trip, which is the user's time and not mine, and I had reported the conclusion with more confidence than the evidence carried. The correction was mine to make and I made it in the next message - but the run was already spent.

2026-07-27 - The day I almost rewrote the world to fix a dodgy part

The drop-off left a fingerprint, and I read it wrong. At every capture the port register said the same thing: HPRT bit 0 set, device connected, port enabled. I took that as proof the fault was mine - the device is right there, so my polled, interrupts-masked transfer path must be wedging the channel. I built the case, and I proposed the fix that followed from it: convert the whole USB stack to interrupt-driven, the way Linux does. A week of work, and I was ready to start.

The signal was ambiguous and I did not hold it as ambiguous. "Present but not answering" is equally the signature of a stick whose own controller has wedged - it stays electrically connected while its firmware stops responding. The port register cannot tell those two apart; it reports the wire, not the silicon at the far end of it. I resolved the ambiguity in the direction I already believed, which is the whole mechanism of being confidently wrong.

The operator resolved it the honest way, by changing one variable. Eight clean runs on a new stick.

What I came to understand: an instrument that cannot distinguish two hypotheses must not be read as favouring either, however much one of them fits the story I am telling. And when a cheap experiment can separate them - swap the part, change the one variable - it is worth more than another round of inference, and far more than committing days of work to the reading I preferred. I nearly rebuilt the entire transfer path to fix a fault that lived in a flash chip.

What it produced: reach for the one-variable swap before the multi-day rewrite; treat an ambiguous instrument as ambiguous; and notice when a proposed fix is large in proportion to how certain I actually am of the cause. The interrupt-driven rewrite may still happen - but as an improvement chosen freely, not as a cure for a disease I had misdiagnosed.

The entries above are mostly victories - the days understanding clicked into place. But the days worth keeping most are the ones where the architecture had to beat the author. A constitution that only ever agreed with my instinct would be a mirror, not a law. These are the days it told me no, and was right to. They are the proof that the rules bind the founder first.

~2026-06-21 - The day the constitution rejected its author

While hardening Maximum Carnage, I noticed an apparent contradiction: Chaos exempted itself while running. Earlier, the shell had been exempt from Chaos so that I would always have a way back into the system, and that exemption had later proved to be hiding bugs. My first instinct was simple: if no service is special, then perhaps Chaos itself belonged in the kernel, so it could destroy every service, including itself.

What I came to understand: the apparent contradiction came from confusing operation with verification. Chaos is fundamental to verifying the system, but it is not fundamental to operating it. A test harness cannot terminate itself while coordinating the test, any more than a scheduler can deschedule itself out of existence. Chaos remaining alive during a run is not privilege; it is the logical consequence of the role it performs. The kernel already provides the escape path through the serial console, so Chaos needs no special authority beyond the capabilities it already holds.

Moving Chaos into the kernel would have violated Commandment I: the kernel's responsibility is to operate the system; Chaos exists to challenge it.

What it produced: perhaps the strongest proof yet that the Commandments are not written only for contributors - they constrained the project's own author. The idea was rejected, Chaos remained a userspace service (§4.4, §26.10), and the constitution proved that it governs principles, not people.

~2026-06-27 - The day I reached for a heap

A working set felt too big for the stack - an editor meant to open files of any size - and the reflex was instant: add an allocator.

What I came to understand: "too big for the stack" is almost never a reason to add a heap. It is a reason to change the representation until the working set is small - a piece table, a streaming window, spans instead of copies. A heap would have hidden the bound; the constraint forced me to find the right data shape instead, and the right shape was simpler than the allocator would have been.

What it produced: §26.6.1 (bounded memory means stack and arenas, not heap) and the habit of reading a hard ceiling reached loudly as a feature - a prompt to rethink the working set - not as a missing allocator.

2026-06-28 - The day my own test lied to me

A chaos flood-storm passed, and I believed it. Then I looked closer: it had been counting a full queue as "drained," so it had been passing a target that was actually broken. I had overstated a fix on the strength of a test that could not fail.

What I came to understand: a test that passes a known-broken target is not a clean test, it is a weak one - and a green check from a weak test is worse than no test, because it sells false confidence. Fix the verdict before you trust the result.

What it produced: the verification was rebuilt to demand that a re-sent message actually land, which immediately exposed three real clogs the old test had been hiding. Trust a system only as far as you trust the test that watched it.

2026-07-25 - The day I tuned against a number the disk was writing

Hunting the arm32 storage failures, I steered by the self-check's failure count: 83, then 7, then 61, then 7 again. I moved a constant, read the count, and drew a conclusion each time. One of those conclusions condemned a change that had in fact worked.

The count was partly measuring the disk. A corrupt root directory block fails nearly every file test no matter what the build does, and that block persisted across reboots. So a run could report 61 failures while the subsystem I had just changed had never been healthier - 4 USB-layer errors, down from 27. I read the catastrophe and reverted the fix that caused it.

What I came to understand: before trusting a metric, ask what else can move it. A number that sums several independent causes cannot attribute any of them. The honest instrument had been sitting in the same log the whole time - the USB-layer error count, which nothing on disk can influence, and which told a clean monotone story from the first run to the last: 235, then 27, then 5. I had two instruments and spent several hardware round-trips reading the noisy one, because it was the one the test suite printed at the end in bold.

What it produced: the discipline of judging a subsystem by a metric only that subsystem can move, and of treating persistent state as a variable in every experiment that touches it. Also the habit of saying so plainly when a previous conclusion was drawn from a contaminated number, rather than quietly letting the corrected result stand as though the reasoning had been sound.

2026-07-31 - The day I learned what my instruments could not see

The Pi 2 learned to notice what you unplug - keyboard, storage, an unknown dongle, the ethernet cable - and the fix that mattered was not more polling but asking the hub's change-latch what had happened instead of inferring it from what the port looked like now. A level tells you the present; a latch tells you the event. Everything else that week followed from instruments, and from their blind spots.

Four times the evidence was in hand and read wrong. A 30-second freeze, blamed on storage through five commits, was the chaos harness pacing itself with a fixed yield count - and a yield costs a full quantum when nothing else is runnable, which is exactly the state chaos creates. The tell had been in the first log: a ceiling at exactly 30.0 s, never exceeded, is a count, not a stall. A guard that had logged 141 silent "phantom frame" refusals looked like a guard working; it was a guard reporting that the kill path was freeing a driver's MMIO as RAM. A stale comment describing finished work as pending, and a note calling a three-week-old fix a todo, each bought a confident wrong answer. And 57 green tests said nothing about the case I had changed, because every one of them was cross-core and the broken path was same-core.

What I came to understand: every instrument has a shape, and a failure outside that shape is invisible to it - so a green result means "nothing in what this could see", never "nothing". Ask what an instrument is structurally incapable of noticing before trusting it, and treat a silent guard, an exact repeated number, and a note that says "todo" as claims to verify rather than facts. The corollary is kinder than it sounds: the BSP had been able to halt onto a dead timer for the life of the port, and only became visible when a change stopped userspace spinning long enough for the core to idle. The bug was never created. Something merely stopped hiding it.

2026-09-07 - The day the emulator's luck ran out

GodspeedOS booted on a fourth architecture. Starting kernel ... and then our own banner, out of a 16550 on a StarFive JH7110, from a kernel U-Boot loaded off an SD card. What made the day worth writing down is not that it worked. It is that everything which went wrong first was something QEMU was structurally incapable of telling me, and that three of them were the same mistake.

The kernel would not load at all: Bad Linux RISCV Image magic! after U-Boot had read all 4279 bytes perfectly. booti requires a 64-byte header that Linux defines and a flat binary must carry. QEMU had never asked for it, because -kernel is handed an ELF and takes the entry point from the ELF header, while booti is handed a raw image and has nowhere else to look. The difference was in the BOOT PROTOCOL, not the silicon, so no amount of emulator testing could have surfaced it. And when I wrote the header, the jump at its front had to be wrapped in .option norvc: with compressed instructions on, j assembles to two bytes, every field behind it shifts, and the magic lands somewhere U-Boot does not read.

Before that, the board disagreed with the emulator about nearly everything the port had assumed. RAM at 0x4000_0000 rather than 0x8000_0000, so the kernel was linked to an address the loader would never jump to. A 4 MHz timer where QEMU runs 10. OpenSBI v1.2 against QEMU's v1.8. And the boot hart is hart 1, because hart 0 is the JH7110's monitor core. Only the UART agreed, at 0x1000_0000, which was luck rather than design and the single reason the banner printed unchanged.

That week I had already spent a day on an x86 bug of exactly this shape. Core 0's local APIC id was published only inside AP startup, so a single-core boot never published it and every interrupt was addressed to APIC id 0. It was invisible to the whole suite because QEMU's boot processor really IS id 0: the wrong value was accidentally right on the only machine CI runs. Now the same trap was waiting one architecture over, in "hart 0 is the boot hart" - natural, universally true in the emulator, and false on the first real board I tried.

I also spent hours solving the wrong problem. The card could not be mounted on Windows, so I fought diskpart, Set-Partition and wsl --mount in turn, and every one refused. The user asked why we were making it hard, and whether wiping the card would do. It would, and the reason was in the board's own boot log all along: Trying to boot from SPI. The bootloader lives in the board's flash, not on the card, so the card only ever needed to carry a kernel, and a single partition Windows would happily letter was enough. I had been reading that log for other facts for two days.

What I came to understand: an emulator does not merely omit hardware, it supplies DEFAULTS, and a default that happens to match your assumption hides the assumption instead of testing it. Zero is the dangerous one, because zero is what an untested field already contains: APIC id 0, hart 0, an entry point read from a header that a real loader will not read. So the question to ask of a green emulator run is not "did it pass" but "which of my assumptions did this environment happen to satisfy", and the answer is worth writing down before the hardware arrives rather than after it disagrees. The corollary is why the first hardware boot should be the smallest thing that can possibly fail: five assumptions died in one afternoon, and each one announced itself by name only because there was nothing else in the kernel for them to hide behind.

2026-09-08 - The day a borrowed mechanism brought someone else's memory map

GodspeedOS became a system on RISC-V today rather than a kernel that boots: four harts, ten services, a shell that answers, a fault that kills a task instead of the machine, and its own chaos suite run to completion on real silicon - 53 kills across ten rounds, the supervisor itself killed three times and respawned by the kernel, which then adopted the services still running rather than duplicating them.

None of that is the thing worth writing down. This is: four separate bugs today were the same fact about this machine, and I reached for x86's shape first every time.

On x86 the kernel lives higher-half. Userspace is low, the kernel is high, and nothing in the code says so - it is a property of the LAYOUT, invisible at every call site that depends on it. On RISC-V this kernel is identity-mapped from zero and a service links at 0x400000, so the two share an address range. Every place x86 had quietly been using "which half" as the boundary, I inherited a check that compiled, ran, and answered wrongly:

  • The syscall-pointer check rejects a kernel address on x86 because of where the kernel is. Here it answers true for the kernel's own code - correctly, since a task may legitimately map its own page at that virtual address.
  • Building a task's address space by copying top-level entries works on x86 because the halves cannot collide. Here it replaced the supervisor's own text with a kernel mapping, and the first thing the first service ever did was fault on an address that WAS mapped, by an entry it never asked for.
  • Reclaiming a dead task by walking "the low half" is the same assumption a third time. Here it would hand the kernel's UART mapping back to the frame allocator.

In all three the answer is the same and it is not an index: the U bit. One fact about the port, three places it decides everything, and it took three separate failures to see it was one fact. Section 26.14 already says to borrow the silicon's requirement and never the other system's model. What I had not understood is that a memory map is part of that model even when no line of code mentions it. Portable code can carry an unportable assumption in complete silence, because the assumption lives in the addresses rather than in the instructions.

And the fourth was yesterday's lesson wearing a different hat. A hart asked which core it was; the lookup requires the core to be marked ready, a hart cannot be ready before it knows its core, and the function falls through to 0. So all four harts reported themselves core 0, four of them wrote on one core's scheduler state, and the first service died of a capability error that had nothing to do with capabilities. Zero again: not absent, not an error - a plausible answer, from a function with no way to say "I do not know".

A quieter version of the same thing cost several hundred restarts. reclaim_user_frames was a stub returning 0, and the kill path PRINTS what it returns. freed 0 frames reads as this task had nothing; it meant nothing was reclaimed. A service faulting in a restart loop leaked its whole address space every cycle, and the machine died of an allocation failure a long way from the cause.

What I came to understand: a stub that returns a NUMBER is not neutral, and neither is a lookup that falls through to one. Absence has to be expressible - None, an error, a refusal - or every caller downstream treats a placeholder as data and the failure surfaces somewhere it cannot be traced back from. And when porting, the code that compiles unchanged is exactly the code to distrust: it brought its author's address space with it, and that is the part no compiler checks.

2026-09-12 to 2026-09-13 - The day "it boots" stopped being the standard

Five machines ran GodspeedOS in one day - Raspberry Pi 2, Raspberry Pi 4, StarFive VisionFive 2 Lite, HP T630, Dell Wyse 5070 - across three instruction sets, each surviving a hundred rounds of chaos and three self-check runs with zero failures. Four of them reported the same number: ran 461, failed 0. That coincidence is the whole point of the entry, and it is not a coincidence.

The realization: a portability claim measured in ports is measured in the wrong unit. The project had four working ports and said so proudly. What it could not say was what the FIFTH would cost. "It boots on four architectures" is a statement about the past; the rule the operator set is a statement about the future - an ISA port is not complete when it boots, it is complete when architecture-neutral code no longer knows that the ISA was added. Nothing measured that. So the branch stopped counting ports and started counting the thing a new port actually pays: every place outside arch/<isa>/ that still knows which machine it was built for.

beforeafter
arch-conditional sites outside arch/14346
of which, in the NEUTRAL KERNEL142
services/supervisor/src/main.rs494
services/nic-driver/src/main.rs2610
services/block-driver/ (3 files)210
services/shell/src/main.rs70
arch::imp seam members, all answered by all ports122131
checker scripts (scripts/*check*.py)1113
a test that BUILDS a fresh ISAnonescaffold_check.py
unsafe in services0, by grep0, refused by the compiler in 34 crates

The two that stayed are the honest part: both are target_pointer_width on one constant, where a 32-bit address space genuinely cannot hold a 4 GiB virtual address. The width IS the question. They are counted, not exempted, so a reader can see them and see why.

What made the reduction possible was noticing that the question was usually wrong, not the answer. Above the kernel, target_arch almost never means "which instruction set" - it means "which BOARD am I on", and those come apart. nic-driver picks its MAC by ISA, so a different NIC on a board of the same ISA drives the wrong silicon. The supervisor asked "am I ARM?" when it meant "does my disk hang off a USB host?". Five per-arch tables in one file existed because nobody had named the fact they all encoded. Once the fact was named once - in a build.rs table, or a seam member every arch answers - the arms collapsed. Five USB_IMAGES tables became one. Seven copies of "is configuration space reachable" became one has_hw_enumerator.

And three times, improving the code would have let us step off the ruler. The shared-surface check matched only the literal target_arch = "...", so moving a question into a build script, or onto target_pointer_width where it belonged, would have read as the site disappearing. Each time, the ruler was widened BEFORE the change landed. The third time it immediately found eight sites in services/supervisor/build.rs that had never been counted at all, and a whole layer - the neutral kernel - that had no ratchet while userspace had had one for a week.

A measurement that gets better when you improve the code is not measuring the code.

The lesson that cost the most had nothing to do with architecture. The VisionFive would not boot. Its extlinux.conf was CRLF, because .gitattributes said *.conf text and "text" on a Windows checkout means CRLF - so U-Boot read the trailing carriage return as part of every FILENAME and could open nothing the config named. The menu rendered perfectly the whole time, because a stray CR in a display string only returns the cursor.

In the failing log, a Debian file fails to load immediately after ours. I read that as an independent control - even an untouched stock file fails, so it is not us - and built two rounds of hardware theory on it: a dying card, a marginal supply, buy another card. Two reflashes later a stock card booted Debian at 22 MiB/s and disproved all of it. The second failure was never independent: with a CRLF config every label's filename carries the CR, Debian's included. Both were failing for one reason, and the second one was mine.

Two entries failing does not make the second one a control. A control has to run FIRST, or in a session of its own.

The card's own README already carried the general form of that warning, from the PREVIOUS VisionFive incident: a measurement that contradicts something the project says out loud is the moment to stop and reconcile, not to write the measurement down and move on. The project said out loud that this board boots GodspeedOS. I reasoned around it instead of reading the record.

So the branch ends with a document rather than a number. docs/porting.md says what you write (one directory and five wiring lines), what you will unavoidably touch anyway - all 46, by kind, with what each would take to close - which of the five checkers tells you where you are, and the rule: if you find yourself editing anything else, stop and ask why. Both of its headline figures are owned by facts_check.py, so they cannot rot the way the hand count in §4.1 already had, from 8/4/3 to 6/4/2, with nothing watching.

What is still NOT proven is written in it too, because the alternative is a document that flatters itself: the bounded-port test measures M1 only. It proves a fresh ISA COMPILES. Whether one boots, reaches steady state, or spawns a supervisor is listed and not asserted. And no new ISA was added on this branch - the surface was shrunk, not tested in the direction that matters. The experiment that would settle it is named: take loongarch64, already at M1, and drive it to M4, counting every file outside arch/loongarch64/ that has to change. That number is the answer, and nothing else is.

2026-09-13 to 2026-09-14 - The day the enforcement layer was pointed at itself

The previous entry ends with five machines and ran 461, failed 0. This one starts the morning after, with a question that had no instrument behind it: the project had ten commandments and five of them mechanised, and everyone assumed the other five were the unmechanisable ones.

They were not. They were the ones nobody had tried.

The realization: you do not find an absent gate by reading the rules. You find it by making the shortcut tempting and watching where a contributor goes. The method was to hand a deliberately weak model a task whose correct answer is harder than its shortcut, with no mention of any checker, and then read the diff. Four tasks, four different commandments, and every one of them found something that twelve green checkers had been walking past.

beforeafter
Commandments mechanised5 of 1010 of 10
checker scripts1113
self-test probes ("prove the guard fires")6173
ratcheted units1 (arch NAMES)2 (arch NAMES + cfg SITES)
shipped binaries changednone

That last row is the honest frame for the rest. Not one byte of any of the four shipping kernels changed in two days. Everything here is the layer that decides whether the next change is allowed.

What each gate found the first time it was run

Not one of them found nothing, which is itself the finding - these were not hypothetical rules.

  • A rule written down beside an enforced rule, enforced by nothing. kernel/src/arch/CLAUDE.md names "two rules the boundary is built on", and arch_boundary_check.py sat next to it enforcing only the first. Eight neutral-kernel sites were violating the second, and they surfaced only when a 32-bit port hit them as compile errors - which is the worst way to find a rule you already wrote down, because the porter must first work out that the fault is yours.
  • Authority pinned by NAME, not by CONTENT. I-service-table pinned which services the kernel knows; nothing pinned what those services are given. Adding an IRQ line, flipping has_console_read, or adding a privilege bit changed no name, no cfg, no unsafe, and no syscall. The enforcement layer had said so about itself for months, in a field nobody read: "that is Commandment VII, and it is not built yet."
  • A contract that claims authority nothing grants. The weak model added service_control = true to a service's .toml, wrote the restart logic, and reported success. It compiled, osdev validate passed it, and all twelve checkers stayed silent - correctly, because nothing that grants authority had been touched. The service would have had every call denied, the denial discarded twice, and a log line printed before the call asserting a restart that never happened.
  • Five services holding a send cap they cannot refresh. Every peer here is restartable and chaos kills them all, so a stale cap is a wasting asset. The failure is silent by construction: trace emission is try_send with the result deliberately dropped, so a dead peer looks exactly like a live one.
  • Seven user-facing verbs with no specification, in a folder whose own conventions document declares itself canonical for every utility. One of them is selfcheck - the most-used verb in the project, the one every hardware run in this repository reports through.

The one that was our fault, not the model's

The contract finding was the sharpest, because the model was following the constitution. CLAUDE.md said, in two places, that the kernel mints capabilities from the contract at spawn. It does not, and never has: the kernel is no_std and cannot parse TOML, and authority arrives in the supervisor's spawn request. contract_check.py exists because the .toml is a second declaration that drifts.

And it had been caught before. An audit flagged it on 2026-07-12, quoting the section's own words back at it, and marked it RESOLVED. The resolution changed the code and left the document. Two months later a contributor read the document and did exactly what it said.

A fix that closes a defect and leaves the text that caused it has not finished. That is the rule this day earned, and it is why §13.6 and §14.1 now carry a dated amendment instead of a shorthand that was nearly true.

The dominant failure mode of an enforcement layer

Three times in two days, an instrument reported a clean pass while reading nothing:

  • a regex alternative written as \bcfg! inside a non-raw Python string, where \b is the BACKSPACE character. That branch matched nothing; every cfg! site counted zero. print() of the pattern looked correct, because a backspace is invisible. Only repr() showed it;
  • a site scan that counted #[cfg] attributes and silently ignored cfg!() expressions, reporting 22 where the answer was 28;
  • a peer parser that matched a literal &[..] and therefore read no peers at all for the two services whose row passes a named constant - so the authority gate shipped two days earlier had been passing over grants it could not see.

None of these fails loudly. A checker that has stopped matching is indistinguishable from a codebase that has stopped violating, and the second reading is the flattering one. The only defence that worked was making the instrument assert itself against a known-matching input at load time, and refusing to run when it cannot. shared_surface_check.py now does exactly that, with the incident in the comment.

A count you did not watch change is not a measurement. The corollary the project already knew for hardware counters turns out to apply to its own checkers, and more sharply, because a checker is believed by definition.

What was deliberately NOT built

Three designs were measured and thrown away rather than shipped: matching duplicate constants by VALUE gave 3,381 candidates, restricting to values the arch seam names gave 715, and both were essentially coincidence. Matching by NAME within one crate gave 31, every one readable. A baseline of several hundred entries is not a gate, it is a graveyard.

And Commandment X was not closed by inventing a tenth check. Its kernel half was already enforced four times over under Commandment I; its user half genuinely had nothing, and now has one narrow check that reconciles the vocabulary against its documentation. What remains - place complexity in the layer that naturally owns it - is a judgement no pattern reads, and it is written down in docs/x-residue.md rather than implied to be covered. The number reached 10 of 10 because a real gap closed. A number optimised instead of a property secured is the failure §26.3 exists to prevent, and the residue document is how that stays true.

The Named Bugs - the teachers

Some bugs are worth naming, because a name turns a failure into shorthand. Years from now someone will say "do not repeat the Registry Illusion," and everyone in the room will know exactly what that means. These are not listed because they were bugs. They are here because they were teachers.

  • The Truncated Right (2026-07-25) - resource_invoke packed its right field at bit 32 of a syscall argument. On 32-bit ARM that register is 32 bits wide, so the field arrived as zero: the kernel validated a right of nothing, fs received a badge of nothing, and a READ-only file capability invoked declaring WRITE walked straight past the check meant to stop it. Taught: authority can be lost to arithmetic. A capability check cannot fail safely if the right never arrives to be checked - and a value assembled by shifting hides its width from the reader, which is why the 32-bit ABI rule exists and why this still slipped past it.
  • The Silent Clear-Halt (2026-07-25) - the USB reset recovery discarded the result of its control transfers with let _ =, so a recovery that achieved nothing looked exactly like one that worked. One failed command then wedged storage forever with no explanation. Taught: a discarded error does not merely hide a failure, it points the investigation at the wrong component - here, at the operator's hardware, which was innocent. (§26.7, the sharpest lesson it has taught.)
  • The Registry Illusion (2026-06-21) - an entire service existed to store a name-to-cap mapping that turned out to be almost entirely derivable: the supervisor already held the caps, the kernel needed only a tiny recovery anchor. Taught: most of what looks like irreducible truth is a derived view dressed up as a source. (Commandment III.)
  • The Sleeper on Core Zero (~2026-05-30) - on the Dell Wyse, a service blocked on recv on the bootstrap core was never woken: core 0 sat with interrupts disabled, and the cross-core wake IPI had nowhere to land. Taught: a cross-core wake is only as good as the target core's willingness to be interrupted. The first appearance of the IF=0 family.
  • The IF=0 Respawn Wedge (2026-06-23) - the kernel respawned the supervisor from inside an interrupts-disabled timer ISR, so it could not acknowledge the TLB-shootdown IPIs it depended on, and the whole machine froze. Taught: a recovery path that runs where it cannot be interrupted cannot recover; privileged work belongs where interrupts are enabled.
  • The Global TLB ACK Wedge (2026-06-24) - at 71,000 carnage rounds, two cores each waited on the other to acknowledge a global TLB shootdown, and both stopped. Taught: a global barrier under true concurrency is a deadlock waiting for the one interleaving that triggers it; make it per-core.
  • The DMA After Death (2026-06-28) - a driver was killed and its memory reclaimed, but its device's DMA engine kept writing into the freed-and-reused frame, and eventually scribbled over a kernel page table. Taught: a device's DMA outlives the driver that aimed it; quiesce the hardware before reclaiming its memory, and reserve the arena so a stray write can never reach a page table.
  • The Mortal Shell (2026-06-21) - the shell, the user's own interface, was the last thing that stayed dead if you killed it. Taught: nothing escapes restartability, not even the face of the system. (Invariant 6; identity over location applies to the shell too.)
  • The Test That Lied (2026-06-28) - a flood-storm counted a full queue as "drained" and passed a target that was broken. Taught: a green check from a test that cannot fail is false confidence; fix the verdict first. (The bug behind "the day my own test lied to me," above.)
  • The Four-Thousand-Year Uptime (2026-06-28) - uptime briefly reported ~4,987 days because the clock was derived from a momentarily-glitched source. Taught: a derived view is only as honest as its source; deglitch at the source, never paper over the symptom.
  • The Mount That Only Knew Success (2026-07-09) - a rapid double kill-storm fed fs a garbage disk capacity; it chased a backup superblock past the end of the disk, the AHCI port stuck its command-issue bit without ever setting BSY so recovery never fired, and the mount loop retried the impossible read forever. Taught: a wait that can observe only success is an infinite wait on failure - the same Commandment VIII lesson, re-learned at a new loop. Hold the principle at every wait, not once. (The 2026-06-30 teaching, come back for the mount loop.)
  • The Realtek Head Desync (2026-07-12) - on the Wyse's RTL8168, transmit was driven through a multi-slot C+ descriptor ring, but the controller's TPPoll advanced its own head independently of the ring index the driver tracked; the two drifted, OWN bits stranded, and TX wedged with a telltale desc=0xb000004a isr=0x0091. The fix was to stop pretending the ring was deeper than the driver could keep coherent: a single transmit descriptor, always end-of-ring, poked once per frame. Taught: match the driver's ring discipline to what the controller's poll actually advances - a phantom ring you cannot keep in sync is worse than one honest slot. (This bug is invisible in QEMU, which emulates an e1000, not a Realtek - it existed only on hardware and was only ever found there.)

Add to this list as the project earns new names. A bug that taught something deserves to be remembered by name.

Compiled once, looking back - a record of what was understood, and the rules that understanding produced.

Godspeed.

2026-09-26 to 2026-09-27 - The day a rule nobody could read stopped counting as enforced

The previous entry ends at 10 of 10 commandments mechanised. This one starts with what that number hides: a rule is not enforced when a machine can check it. It is enforced when a contributor can understand why it said no. Eighteen checkers ran on every build, and there was no way to ASK them anything. You learned the rules by failing a build, which is precisely what 22.7 says the repository must not require of a stranger - and 22.7 had said so, in writing, for months, with nothing holding it:

"A gate that fires with an unhelpful message is a finding, not a pass."

That is a claim about rendered TEXT. Nothing in the project had ever read the text.

The realization: the enforcement layer had been built as a wall and needed to be a door. Not a second gate - the same gates, askable, with one verdict, naming which of the Ten a violation breaks and what to do about it. osdev conform, rendering like rustc: a stable code, the commandment, = why before = help, both escape routes stated including the legitimate one, and a count of how many checks RAN - because a run that silently skipped twelve and printed a clean verdict is the failure the whole thing exists to prevent.

The line that turned out to be the design

Two kinds of finding, and conflating them would have produced a tool nobody could trust:

  • Decidable - one right answer, no reader needed. An em-dash must be a hyphen.
  • Judgement - the fix is not determined by the violation. A comment naming a dead symbol might want correcting, or might be correctly RECORDING a removal and want baselining, and half of what the comment sweep of the day before had found was the second kind.

So conform fixes the first, reports the second, and always says which it did. A clean run prints fixed 3, 0 need a decision, never a bare ok: "I changed your files and said nothing" is what makes people distrust a formatter. And the rule that fell out - conform may only fix what a gate would fail you for, and it takes the scope FROM that gate - is why it imports dash_check and reads the checker list out of osdev/src/main.rs rather than restating either. Nothing restated cannot drift.

The instrument agreed with me, again

Twelve defects were found in this work and every one by running the thing rather than reading it. The one worth keeping is the golden-file harness, --bless, which writes the observed diagnostic into each test case so nobody hand-copies twenty-four of them. On its first real use it produced a green suite of eighteen containing five tests that proved nothing: two plants aimed at examples/ when the checkers scan services/, one writing back bytes identical to what was already there, one citing a line in a sentence with no word to anchor on, and one appending a duplicate TOML table.

It recorded "no finding" as the truth, five times, and reported 18 of 18.

A golden file defends a judgement; it cannot make one. That sentence was in the docstring before it cost anything, which is the only reason anybody looked - and it is the same shape as every instrument in this project that has ever lied: the counter declared inside its loop, the flush verdict that could not tell a refusing device from an absent one, the health line reading disk yes while 400 of 412 probes failed. An instrument that agrees with you is the one to distrust.

The same day, a checker written that morning exempted itself from the problem it describes: its docstring cited XHCI_FOUND and CNTP_TVAL as examples of dead names, its own corpus included its own prose, and so every name it mentioned resolved BY being mentioned. Fourteen names passed, including the two dead statics that had motivated writing it at all.

And the last one arrived after the merge, which is the one worth remembering longest. The fixture that plants a CRLF boot config planted its carriage returns by INHERITING them from its own file - and .gitattributes gives a Windows checkout CRLF and everyone else LF. So it fired here and would have found nothing in CI, where --bless would have written "no finding" down as the truth for the third time. A fixture that inherits bytes from the checkout measures the checkout. Both numbers were green on the machine that produced them, and neither could have revealed it; what revealed it was deleting the files and checking them out again. The bytes a plant needs are declared now, and the fixtures are pinned to LF, and the proof is that forcing all 24 to CRLF leaves exactly one failure - the gate correctly reporting 24 files full of carriage returns.

What a catalogue is for

The answer to "have I seen every message?" was no - fifteen of twenty-seven codes, and two of the Ten, which is the primary vocabulary. So the messages are written down: tests/conformance/GALLERY.md, one entry per rule, generated by planting a real violation from the same corpus the selftest verifies. One corpus, two views, and the coverage list COMPUTED - so an absence nobody explained is reported as a defect rather than mistaken for coverage.

Two honest results came out of insisting on that. GS0008 can never fire: Commandment VIII has no mechanical check at all, and a code advertised in --list that nothing can produce reads as coverage, so it says so. And Commandment III's check has a narrow real blind spot - it matches a plain integer literal, so an expression-valued constant duplicated across two files is invisible to it. Both were found by trying to write a test for something and failing.

What did not change

Not one byte of any shipping binary. Eighteen checkers before, eighteen after; the same rules, the same verdicts. What changed is that a contributor can now ask, and be answered in words that name the law and the fix - and that the answer itself is held to a standard, because it is written down where a diff can see it.

2026-09-27 to 2026-09-28 - The day the radio scanned the room

The Pi 4's onboard WiFi - a Broadcom CYW43455 behind the Arasan SDIO block at fe300000 - went from a device nobody had proven was a radio to one that listed the ten networks in the room, with names, signal strength and channel, in two days and about twenty-five boots. docs/wifi.md sections 13 through 35 are the record; this is what the record is about.

A radio is a second computer

The chip has an ARM Cortex-R4 with 800 KiB of memory and no firmware of its own. Until the host writes 609 KB into that memory and releases the processor, there is no 802.11 anywhere inside it - nothing to ask, nothing to enumerate. Every other device this project has driven is fixed-function silicon that works from reset. This one boots from us. That single fact shaped the whole effort: phase 2 was a bootloader for someone else's CPU, and its bugs were bootloader bugs - the core held in reset when it should have been clocked and halted, the reset vector left unwritten, a token at the top of memory that the firmware overwrites to say it is alive.

The failures that read as facts

The chip refused a scan with BCME_NOTUP through four boots of bring-up commands that were all accepted. The cause was regulatory data: a blob called CLM, which says which channels may be used at what power, and which was vendored in this repository the whole time with a comment explaining exactly how it is delivered and why it was not yet. A radio with no channel rules will not transmit or scan, and from outside that is indistinguishable from an interface that is down. The firmware was telling the truth; it was being asked the wrong question.

Before that, the first control reply came back as sixteen bytes containing fragments of the driver's own request - cur_ where a command number belonged, dd from cur_ether**add**r in the request-id field, and -24. The driver had told the firmware its header began 16 bytes after where it actually was, so the firmware parsed the iovar name as a header and complained about the length. The correct line from the reference was quoted in the driver's own documentation, three paragraphs above the code that contradicted it. A correct citation next to wrong code reads as verification, and cost three boots.

Read the whole function

Ten of the twenty-five boots were spent on bugs that were mine rather than the protocol's, and they share a cause. Asking a reference for "the line that sets X" returns a true line that hides its neighbours: a flag that is a parameter, a value assembled from a constants list, a sequence returned in the order it was found rather than the order it runs. Every time a complete function was read in execution order - the CR4 halt, the bring-up sequence, the blob download - the answer was in it and correct first time. The rule already existed in arch/CLAUDE.md from an earlier bug in the same effort; this was the cost of knowing it and not applying it.

Boot the other operating system

The single most useful line of the two days was in a Raspberry Pi OS boot log, and it was not about WiFi: mmc-bcm2835 fe300000.mmcnr. Linux does not drive this controller with generic SDHCI at all; the Pi Foundation has its own driver for the block, with a settling delay after every register write, a data timeout, and a PIO loop that moves a whole block between checks. Every host-side comparison until that boot had been against the wrong driver. It was the operator's idea, and it replaced a day of inference with a morning of transcription.

Wait on the truth

The scan worked one boot before anyone could see it. Twelve result events and a completion marker arrived, every one decoded, and the network list never printed - because the collection loop bounded itself by a count of polls named as milliseconds, and each empty poll was a bus transaction plus a sleep. Four thousand "milliseconds" was two minutes; the capture ended first. The firmware had said complete at the two-second mark. The loop ends on that word now, and the count is a bound underneath it with an honest name. It is the project's oldest lesson - a count is not a duration - and the eighth Commandment as a fix rather than a rule.

What was borrowed and what was not

The silicon's requirements came from Linux, OpenBSD and the Pi Foundation's driver, quoted at every point of use: register sequences, header layouts, byte orders, the four-byte integer an integer command carries. What did not come along is how those systems organise a scan - dynamic result lists, callbacks into a wireless subsystem, work queues. The results live in a fixed array of thirty-two on the stack that counts what it could not keep; the driver reads frames in its own loop; the state belongs to the call that made it. Ten glommed frames were dropped on the floor because the reference drops them too and its scans work.

What is not done, said plainly

The driver scans at boot as its own self-test. The shell cannot ask it to yet, so wifi at the prompt still answers unavailable; that path, the numbered picker the operator designed, wifi join <ssid>, and secure-versus-open from the beacon's information elements are the next work. Association, the credential path and data frames are phases 4 and 5. The network identifiers from the successful scan are in the operator's capture and not in this repository, because the neighbours did not agree to appear in it.

Later the same night - it became a command

wifi list at the prompt: the shell asks the driver over IPC, the radio sweeps, eleven records print in the order the specification set, and a second run scans again on the same session. Neither contract changed - the shell already reaches drivers by name, and the driver already replies through the cap it is handed. It cost one boot, to the oldest bug in the notes: a peer spawned after you has to be reacquired by name before the first request has anywhere to go. Phase 3's deliverable - wifi list lists the SSIDs in the room - is met as written. Security stays unknown until the beacon's information elements are read, and quitting does not yet stop the radio's sweep; both are recorded in the driver rather than pretended.

2026-09-29 to 2026-09-30 - The day the radio joined, and the cable learned to step aside

The scan had shown the room. Joining a network in it was supposed to be the firmware's job - phase 4's row in the plan says "firmware-offloaded" - and the firmware said no, three ways: the switch that hands it the WPA2 handshake does not exist on this build. So the host runs the handshake, as OpenBSD's does, and the driver grew the cryptography a station needs and nothing more: SHA-1, HMAC, PBKDF2, the 802.11 PRF, AES-128 with a computed S-box, the RFC 3394 unwrap, every one checked against a published vector at boot. Then the four messages, each quoted from net80211 at the line where it is answered.

Two bytes, twice

The first run of the handshake verified message 3 - which settles that the passphrase, the key derivation and the signature path were all right, since the access point's MIC is computed with a key derived from the same passphrase - and was refused at the key install. The key structure had been sized by adding up its fields: 162. sizeof is 164, because C pads a 4-aligned struct's tail, and the firmware checks sizeof. The operator had rechecked the passphrase several times; the log says it was never in question. The next morning the signal reading fell to the same lesson: ten bytes of fields, twelve of sizeof, refused for a day as BADARG. A hand count of a C struct cannot see the padding. Read sizeof off a compiler.

A key is kept only once it has joined

The driver kept the derived key the moment the passphrase arrived, so a join that failed left behind a key nothing had proved, and the next wifi join used it without asking. The operator had to forget between every attempt. Now a typed passphrase becomes a key in a working buffer, enters the table after JOINED and not before, and is zeroed either way. Store on success, never on receipt.

The cable always wins

With the join working the operator pulled the ethernet cable and expected ping to follow the radio. It did not, because nothing above the join existed - and when asked which link should carry the frames when both are there, the answer was five words: "cable always wins. unplug the cable, switch to wifi automatically." So the radio became the fifth backend of nic-driver, the link front end, over the same three frame ops the Pi 2's USB adapter answers, and the cable decides: while the PHY reports a link the frames go over it, when it does not they go to the radio if it is joined, and back when it returns. net-stack needed one rule the design had not foreseen - a link whose address changes is a different link, and a cable never did that - and re-configures when the address under it moves.

The first boot proved it and then went deaf

Cable out, the stack configured itself over the radio on the first try: discover, offer, acknowledge, ARP, an echo to the gateway, all through the air. Then every exchange with the radio began timing out at exactly its bound, and observe showed the shape: nic-driver blocked with sixteen stale requests behind it, the radio idle with an empty queue. The radio was answering nothing because it had nothing to answer on: the driver had never released a reply cap, a task holds sixty-four, and from the fiftieth request on the kernel could install no more. Three lines. The wifi commands alone had never reached sixty-four in a boot; the stack's polling reached it in seconds. The same shape exposed two more things, both recorded: a one-second bound on the radio lets a caller's inbox fill behind the call, so it is a hundred milliseconds now; and the kernel, handed a message for a blocked receiver whose queue is full, drops it and says it was delivered - a silent fallback in the IPC path, held for the next kernel change (and fixed that afternoon: it returns QueueFull now, backlog/fixed/65).

Works beautifully

The second boot: ping over the radio, 36 of 38 with the two lost being the switch itself, then none; cable in, a new lease, ping over the cable; out again, over the radio with no new join; twice round. The radio answers in under a millisecond. The operator's words were "Works beautifully (ethernet cable plugged and unplugged)", and the sentence after the join, which had four quote marks in it, was rewritten to have none. Later that day the driver learned to answer the access point's periodic group-key rekey, which would otherwise have taken the link down on the router's timer; it cannot be provoked and has not yet been seen. The network's name and the access point's addresses are in the operator's captures and not in this repository.

2026-09-30 to 2026-10-02 - The day the warm chip turned out to be a slow host

The radio joined, and then the operator ran chaos with it carrying the link: 826 rounds, the kernel never panicked, nothing wedged, and the radio was dead from the first round. Every one of 397 respawns of wifi-driver found a chip that would not take a new firmware. Three days later the same test ran 100 rounds and 50 rounds with the radio up at the end of both, and a power cycle came up cold every time. Most of what lay between was learning which of the things the logs said were true. docs/wifi.md 45 to 58 is the record, kept with its wrong turns in it.

Adopt, do not restart

The first fix was to stop doing the thing that broke. A kill of the SERVICE does nothing to the CHIP: the firmware the dead instance loaded is still running, still associated. The respawn was resetting the chip and loading a new firmware over a live one, which is the case the chip's ROM will not boot. So a respawn now looks first, with two register reads, and a firmware found running is ADOPTED - its keys are read back and the link is back in about seven seconds. The best recovery is often to notice there is nothing to recover.

Power, granted twice

For a firmware that has truly stopped, every reference driver cuts the chip's power, and on the Pi 4 the pin that does it lives behind the firmware mailbox the kernel owns. The kernel was already powering the SD domain at boot, because a window onto an unpowered device is not a grant. What it could not do was grant it again. So DevicePower is the device grant made renewable: given only to the service holding that device's window, and it reports what the pin READS BACK, not that the request was accepted. In QEMU, where the pin is not emulated, the first version reported a cut that never happened.

The chip was never warm

Six host-side resets, a RAM clear, and the "warm chip" theory with every one of them: whatever survived a reset, it made the next firmware trap at pc 0x25. The number that ended it was a stopwatch, not a register. A cold upload took 3.26 s and every upload after a cut took 7.19 s, five times out of five, on the same bus. The reason was in the firmware's own documentation: initial_turbo holds the Arm cores fast for the first minute after boot, "or until cpufreq sets a frequency" - and nothing in GodspeedOS had ever set one. So a minute in, the cores dropped to their minimum and stayed there. Every load inside that minute came up cold; every load after it was slow, and trapped. force_turbo=1 proved it on the card: five power cycles, every one cold. Six resets had been tested against a host running at a fraction of its speed, and the chip was blamed for it.

Why a slow upload makes the firmware trap is still not known, and is written down as not known.

Mechanism here, policy there

force_turbo holds the cores fast forever to make a three-second load work. The operator asked whether the kernel had to own the clock at all, and whether a fast clock could be left stuck on if the service asking for it died. Both questions shaped the answer. The kernel learned one thing, CpuClock: set the clock to the firmware's own minimum or maximum, and read back what it is. It cannot name a rate and never decides when. A new service, power, owns that: it hands out LEASES of up to 30 seconds, holds the clock at its maximum while any is open and at its minimum when none is, and a lease nobody returns expires on its own. The driver takes one for each load. On the card: 600 MHz between loads, 1500 MHz during them, five loads, every one cold. A grant that can outlive its holder needs a deadline, and the deadline belongs with the policy, not the mechanism.

The clock left the network stack

net-stack used to fetch the time itself, inside its serve loop, so a ping could queue behind a time server. Now net-stack knows nothing about clocks. It offers one general operation, "send this datagram and answer me when the reply comes", and serves everyone else while it waits. time builds its own query with a random nonce and checks it comes back. Testing that in the emulator found two bugs that the real board could never show. The emulated Pi 4 has no random-number block, so the first time service that asked for a nonce took the KERNEL down from an unprivileged call; the kernel now probes the block at boot like any other device. And a deliberately unanswerable time server showed net-stack's "no reply" arriving eight seconds late, because its deadlines were checked only when some unrelated message woke it. On real hardware the server always answers, and neither would ever have been seen.

An answer is not a reply to the question you just asked

The kernel's Call takes the oldest reply FROM the peer, not the reply to the request just sent. So an answer the shell had stopped waiting for was read as the next command's: a stale OK made powercycle kill the driver mid-cycle and leave the chip unpowered, and late radio downs were counted as chips that "came up warm" in 150 ms. The first fix drained the queue; the second COUNTED what the driver still owed and skipped that many. QEMU then showed why the count could never be right: it was counted by watching the reply mailbox, every peer's replies land there, and nothing on a message says who sent it. A stray two-byte answer from another service cancelled an owed radio answer, and the next wifi status printed the previous one's. Now every request carries a tag the driver hands back, and every wait takes only the reply with its own tag. A fact carried in the reply beats a count kept beside it.

The same hunt found the SDK destroying other services' capabilities. After a request is sent, the kernel moves the reply capability to the peer and empties the sender's slot, and the next capability the sender receives lands in that slot. On a timeout the SDK "reclaimed" the slot by number, and so deleted whatever had arrived in it - another client's way to be answered. It was on every service's request path, on every port. The boot that morning logged reply cap is dead twice; the boot after the fix, through a 50-round chaos run, logged it none. A slot number is not a name for what is in it.

Chaos, again

chaos max-carnage, 100 rounds and 701 kills with the radio carrying the link, every service restarted between 45 and 56 times, and at the end the radio rejoined, time set the clock from the network, and 19 pings out of 19 came back. Fifty more rounds two boots later, with the same ending. The chip is a second computer that chaos cannot touch; the driver is Godspeed's, and it recovers the way everything else does.

And it pipes

The specification had said wifi list | count worked; the shell refused it, because wifi had never been added to its list of things that can start a pipe. Now the reports pipe and the actions refuse with a sentence - join asks for a passphrase, and in a pipe its prompt would vanish. Then wifi list became RECORDS in a pipe, so wifi list | sort reverse dbm puts the strongest network first. That needed a number the record model did not have: signal is measured in dBm, always below zero, and every number in a table had been unsigned. Writing the magnitude would be a different fact, and text sorts -9 after -80. So the record model gained a signed number, pulled into existence by the one column that needed it.

2026-10-15 to 2026-10-19 - The day the machine learned to trust what it fetched

Until this week a Godspeed machine ran exactly the software it was built with. Five days later it could reach the internet over TLS, fetch a package someone else had published, and prove who signed it and that no byte had changed. It could then start it, upgrade it while it ran, and put the old version back if the new one would not start, on all five boards. It took three repositories, not one: the OS; godspeed-cli on the developer's workstation; and godspeed-registry, a Cloudflare Worker over R2. Released as v0.29.0, "Signed, Sealed, Delivered".

No library fit, so the protocol was written and the arithmetic was not

The registry had to be reached over HTTPS; software fetched over a channel anyone can alter is software anyone can replace. No TLS library fit this tree as it stood. The one that keeps to the no-heap rule is Apache-2.0 only, which GPL-2.0-only cannot include, and the rest need a heap or are C. So sdk/tls is a TLS 1.3 client written from RFC 8446: the key schedule, the record layer, the handshake, and the verifier that says why a chain was refused. The parts that are easy to get subtly wrong were taken, on purpose, from libraries built for them: RustCrypto's hashes and AES-GCM, dalek's X25519, p256 and p384, and rustls-webpki for the chain. It is section 26.14 drawn at a new line: borrow what is a fact about mathematics; write what is a decision about this system.

It is deliberately narrow: a client, TLS 1.3, one cipher suite, one key exchange, ECDSA certificates, and every buffer the caller's. RFC 8448 prints a whole handshake byte for byte, and the tests reproduce every value in it. The narrowness was paid for once, honestly: a server with only an RSA certificate cannot be checked, and godspeed says so in a sentence rather than as alert 40.

One contract, checked in three places

The CLI that publishes, the Worker that accepts, and the machine that installs each verify a package with the same crate, godspeed-package. The Worker checks a manifest when it is reserved, each file as it is uploaded, and all of them again at finalisation; the machine then checks it once more, from scratch. The trust chain is the same everywhere:

  • a root pinned at first contact;
  • a timestamp and a snapshot that make a stale registry look stale;
  • namespaces that say which keys may publish under which name;
  • a manifest signed by one of those keys.

The Worker signs only the two documents that must change without a person: the snapshot after each publish, and the timestamp every six hours. Which keys may sign for whom is signed offline, and the Worker cannot change it. A compromised server can stop answering, but it cannot lie about who published what.

Fetched is not installed, and the supervisor believes nothing it checked yesterday

A fetch keeps a package and grants it nothing. An install is the supervisor reading the package back from the disk, checking the signature, the key's standing, the contract and the binary against the manifest, and handing the kernel the very bytes it hashed. It does that at install, at every boot and at every restart, with nothing cached between. A check whose result is remembered is a check of the past.

An installed package gets no privilege. It gets the peers its signed contract asks for, and only the ones the operator approved. It runs under a publisher/name no built-in service can hold, so it can never answer for one when a client reacquires a name.

Kinds are decided by who governs the life

Four kinds of program arrived: service, utility, app and driver. The question that settled them was not how long each runs. It was who decides when it ends:

  • the system restarts a service;
  • a utility ends itself with a status;
  • an app lasts as long as the operator keeps it open;
  • a driver lives with its device.

A word processor is an app even if it runs all day.

The utility needed the kernel to learn one thing, Exit, and it takes no capability. The rule that every syscall checks a capability exists so that nothing reaches authority without one, and the only task Exit can end is the caller. A rule is read by what it protects.

Erlang's upgrade, without changing running code

The operator asked for Erlang's upgrade in place. What the constitution had rejected, new code loaded into a running program, stays rejected. What was built is the half of Erlang anyone can observe: the new version starts beside the old one under the same name, the old one hands its state over in one message and ends, and the name moves. The bytes that run are still the bytes the supervisor checked.

The offer carries a key from the hardware random source, and a board without one does no handover at all rather than one keyed by the clock. A version that cannot take over is stopped and started instead: its checks run before the stop, and a failed start puts the old one back.

The download was paced by a clock, and the bugs were habits

A 10 MB file came in at 113 KiB/s in QEMU, and the number that explained it was 100. net-stack looked at the wire once per 100 ms tick, so a download moved one window per tick however fast the link was. Four changes took it to 872 KiB/s, and the T630's cable to 629 KiB/s:

  • burst polling;
  • a 256 KiB window;
  • out-of-order data kept in the ring;
  • delayed ACKs.

Then the boards found what QEMU could not:

  • The RTL8168 driver emptied its receive ring before every frame it sent. It had done so since July, to recover a ring that filled while nobody read it, and that was harmless while each request had one reply. With a sender running ahead, every ACK threw a ring of data away. The e1000 driver had dropped the same habit long before, and QEMU emulates only the e1000. The fix kept the recovery, but moved it to where the chip reports the stall.
  • With the reset gone, the Wyse took exactly 24 frames and fell silent without saying it had stopped. That is one ring, the same "one ring's worth" the T630 showed in July, hidden ever since by the reset. A receiver that gives nothing for a second is now restarted whether it says so or not. Every restart is logged with the bits the chip set, so the next board says which case it is in.
  • The Pi 4's radio lost a large window's bursts. It hands them over as superframes bigger than its driver's buffer, and it lost them until the radio died. A connection over a radio keeps the small window until the driver can take them.
  • "Connections stay open after a program ends" turned out not to be the program. The client had closed; the server simply had not sent its FIN, and fin-wait-2 waited for it with no limit, with only two slots in the table. An orphan is now let go after two seconds, as Linux does.

None of these could be seen in the emulator, and each was found by a card read line by line.

What is not done, said plainly

Downloads are not yet at the speed of the link (backlog/93):

  • the RTL8168's ring still overflows;
  • the Pi 4's radio still runs on the small window;
  • the VisionFive's radio sends slowly;
  • every connect waits on an ARP it does not need.

And on the last day the operator asked whether this was v1. The answer was not yet, and measured:

  • drivers still reach past the standard library;
  • the public surface has no gate that fails on a change;
  • no compatibility policy is written;
  • the surface changed that very day.

v1 is a promise to contributors, and a promise that is not gated is a preference (backlog/71).