# LESSON_LEARNED — mgf-standard

> **Distilled, portable version:** [`docs/AGENT_PRACTICES.md`](docs/AGENT_PRACTICES.md) —
> the practices these incidents produced, written for other teams and agents to adopt.
>
> Append-only, dated log of non-obvious lessons learned working in this repo.
> Newest first. One entry per lesson. Supersede a wrong past entry with a new
> one — never rewrite history. See docs/recipes/lesson-learned-md-template.md.

---

## 2026-07-30 — A fixture that starts empty tests only the branch that never runs in production

- **Context:** Closing the gap "`sync` cannot detect a stale Woodpecker secret value",
  I found that **`rotate` never rotated a deployed secret's value at all.** On the
  UPDATE branch — reached whenever the secret already exists, i.e. *every real
  rotation* — it PATCHed events and images and omitted the value. The new value went
  to the store, six stages reported `ok`, `cutover` announced *"surface serves new"*,
  the scope audit compared metadata and found nothing, and `retire-old` then revoked
  the old credential at the provider. The end state is CI authenticating with a value
  that was just revoked.
- **What I learned:** It survived a full per-kind drill suite — the "AC-is-executed
  proof", one representative of *every* kind, end-to-end — because every drill starts
  from `FakeWoodpecker(org_secrets={})`. An **empty** surface. So all of them took the
  CREATE branch, where the value *is* passed. The drills exercised a state that never
  occurs in production: you rotate a credential precisely *because* it is deployed.
  The suite was thorough along the dimension it enumerated (kinds) and blind along the
  one it never varied (**pre-existing state**).
- **Why it matters:** This is the most likely origin of the stale `codeberg_repo_read`
  that cost a session tracing an opaque `ProtocolError` in mgf-hrb. A rotation that
  reports six green stages and revokes the old credential, while the surface still
  serves it, is lockout-shaped: the failure lands later, on a different repo, looking
  like something else entirely.
- **How to apply:** When a fake models a remote surface, ask **what state does this
  fixture start in, and is that the state production is actually in?** Empty is the
  easiest fixture to write and usually the least realistic. For any create-or-update
  branch, the drill must run **twice** — once against absence, once against presence.
  A parametrized suite proves coverage of the axis it varies and nothing about the
  axis it holds constant; name that axis out loud before trusting the green.

---

## 2026-07-30 — A warning that only ever reaches stderr is a warning nobody receives

- **Context:** `mgf-uitest-desktop` declared `[tool.mgf-fed] qt = true` — a knob that
  has never existed, written by analogy with the real `browser = true`. The renderer
  handled it "gracefully": it parsed the table, printed `note: [tool.mgf-fed] key(s)
  not read by the CI renderer: qt` to stderr, and rendered a pipeline. `scaffold`
  reported success and exited **0**. The pipeline had no Qt libraries and went red
  three times before anyone connected it to the note.
- **What I learned:** The note was correct, accurate, and useless. It was emitted in
  the middle of a 46-repo sweep, on the stream nobody greps, by a command that then
  said it succeeded. **Exit code is what CI and humans both actually read** — and it
  said the repo was fine. Worse, this is the one CI-config defect the byte-render
  currency check *structurally cannot* catch: a key the renderer does not read shapes
  nothing, so the render is byte-identical to itself and `ci-generated` stays silent
  while the author believes a knob is on.
- **Why it matters:** "We already warn about that" reads like coverage in a review and
  is not. The difference between a note and a finding is the difference between
  something a gate enforces and something a human might notice.
- **How to apply:** Route a diagnostic to the plane that **acts** on it. If a condition
  should change someone's behaviour, it belongs in a gate's findings and its exit code,
  not in a print statement — and the message should name the **consequence** ("the
  rendered pipeline is NOT shaped by them"), not just the observation ("key not read").
  Where the mistake is a wrong *concept* rather than a typo, a `difflib` near-match will
  never find it: keep an explicit alias table (`qt` → `apt_packages`) and add the entry
  the **first** time a knob is invented, not the second.

---

## 2026-07-29 — A check's success line must state its own scope, or a narrow check reads as a broad one

- **Context:** A tools-inventory round produced seven findings. Six were the same
  defect wearing different clothes. `fed-tools doctor` swept only web consoles and
  printed *"the registry covers every discoverable web console"*. `update --check`
  looped only over web consoles and printed *"all current"*. `mgf-fed status`'s footer
  named the authoritative conformance scorer as a **file path**. The tools front-door
  called itself *"the Magogi Foundation"* while covering one of two families.
- **What I learned:** Every one of those sentences was **true**. Each was also read as
  something broader — *the registry is complete*, *the tools are current*, *the scorer
  is reachable*, *this is the Foundation's tooling*. The bug is never in the fact; it
  is in the gap between the scope checked and the scope implied. Widening the query
  fixes one instance. Making the **success line name what it actually covered** fixes
  the class — and it is the only version that survives the next person adding a
  category nobody thought of.
- **Why it matters:** The cost was not hypothetical. `mgf-secrets` — the credential
  plane that `CLAUDE.md` names as the *first* thing to ask before declaring a
  capability missing — was absent from the front door and not installed, and the
  doctor could not see it because CLIs were never in the doctor's query. That is F-35
  reproduced by tooling. Separately, `update --check` reported three stale tools when
  five were stale; the two invisible ones were CLIs.
- **How to apply:** When you write a checker, write the *clean* verdict last and make
  it enumerate its own scope (`"all 3 web console(s) and 9 command-line tool(s) …"`).
  If a mode skips a check, the clean line must say so out loud — `fed-tools doctor
  --offline` now prints *"currency NOT checked"* rather than a bare green. And treat
  **"I could not look"** as its own outcome, never folded into "clean": unreadable
  pyprojects and an unreachable devpi are now reported as *unverifiable*, not passed
  over. Related: F-36 (a monitor must page on *I could not run*).

## 2026-07-29 — The lesson may already exist in the OTHER family; look before inventing

- **Context:** `scripts/fleet_conformance_audit.py` audits **working trees**. Its
  committed tracker had asserted *"39 repos · 39 clean · 0 real gaps"* since 07-04; a
  fresh run found 90 warns — and that run's own input was 26 repos behind
  `origin/main` and 29 off `main`. The audit had no way to say so.
- **What I learned:** sooth-standard's `verify_family_conformance.py` had already
  solved this: it reads each member's **remote default branch**, and its working-tree
  flag is documented as `(lies more)`. The Sooth Family named this exact failure mode
  and defaulted to the safe side. MGF's audit never compared the two refs, so it could
  not even warn that it might be lying.
- **Why it matters:** We are one Foundation with two families. A lesson learned in one
  and not carried to the other is a lesson the Foundation paid for twice. Sooth had
  also independently reached the principle behind this whole round —
  `verify_handoff.py` is described as *"the CRT-15 intake gate as a command, not a
  sentence"* — which is precisely the fix `mgf-fed conformance` applies.
- **How to apply:** Before designing a check, grep the sibling family's `tools/` for
  the same question. When the two families implement the same check differently, the
  difference is a finding: one of them has learned something the other has not. Carry
  it across by reference, not by copy (sooth `SEC-06`/`PKG-01` is the shape).

## 2026-07-29 — A breaking refactor's ripple hides in managed files, and prose is not a state

- **Context:** Two unrelated-looking symptoms turned out to be one incident each.
  (a) 43 of 44 repos ship an `AGENTS.md` telling agents that `mgf-fed agents` is
  *"always installed"* because `mgf-common` is a dependency — false since
  **mgf-common 0.47.0** extracted the CLI to `mgf-fed`. 39 repos also still carry the
  pre-0.47 `mgf-common catch-up` block marker. The same refactor left a stale
  `mgf-common` binary answering on the operator's PATH for a CLI that no longer exists.
  (b) `mgf-secrets doctor` had 11 pageable findings; ten were five credentials on a
  deliberately **held** deploy path declared `state = "active"`, with the hold written
  only in `purpose` and `recovery` prose.
- **What I learned:** A `refactor(cli)!` changes an installable's contract, and that
  contract is quoted inside *managed* files across the whole fleet. Nothing forced the
  ripple, so it sat for months in the one file whose entire job is telling agents how
  to get started. And a state that exists in the model (`held`) but is expressed in
  prose instead will be contradicted by the field next to it — the doctor then pages
  correctly and constantly on an intentional condition, which is how the **one** real
  page (F-40) ends up 11th in a list of 11.
- **Why it matters:** Both are the founder's *"small bits are harder to track than big
  bites."* Neither was a big change. Both defeated every automated check we had,
  because the checks that would have caught them were themselves unreachable or unrun.
- **How to apply:** A breaking change to *how a tool is installed or invoked* must
  carry a fleet re-render in the same wave — treat managed files (`AGENTS.md`,
  `docs/claude/CLAUDE.md`) as part of the API surface of the CLI they describe. And
  when a declaration's prose contradicts its enum field, the enum is what runs: fix the
  field. Related: F-37 (a state the model *cannot* express gets worked around — this is
  its mirror, a state it *can* express going unused).

## 2026-06-07 — A doc describing ANOTHER repo's file silently drifted from reality

- **Context:** Relaying `pending-feedback/mgf-common.md` — one paper asked to
  re-sync `mgf-common/.claude/commands/sync.md` to the rewritten prompt, on the
  strength of THIS recipe's Note claiming it held an "inline copy" of the prompt.
- **What I learned:** I checked the actual file: it is a thin *delegator*
  (`Run mgf-common sync-prompt …`), not an inline copy — so it can't drift, and
  the paper was a false alarm. The bug was in THIS repo's
  `federation-sync-prompt.md` Note, which described an inline-copy model that
  hadn't been true for some time. A doc that narrates a *different* repo's file
  is drift waiting to happen — nothing tests it.
- **Why it matters:** I almost filed (and acted on) cross-repo work based on a
  stale self-authored Note. Cross-repo descriptions are the highest-drift,
  lowest-verification kind of doc.
- **How to apply:** Before acting on any claim about another repo's file, OPEN
  that file. Treat docs-about-other-repos as hypotheses to verify, never facts —
  especially when they drive cross-repo action.
- **Generalizes?:** partly — the stale Note is fixed here; the habit "verify the
  artifact, not the doc about it" pairs with the earlier "audit the entry point"
  lesson.

## 2026-06-06 — On a PRIVATE registry, an unauth'd 404/401 reads as "absent" — a guesswork trap

- **Context:** Review pass on the sync-prompt hardening — I had written STEP-0
  instructions telling agents to `curl` Codeberg raw-file + package-index URLs
  with NO auth header.
- **What I learned:** magogi-admin is a private account: the unauthenticated
  raw-file URL returns **404** and the package simple-index returns **401** — and
  an agent naturally misreads those as "this file/version doesn't exist" and
  proceeds on a false negative (e.g. "already on latest"). With the PKG-03 token
  header both return **200**. The danger isn't just the missing auth — it's that
  the failure code actively lies in the direction of "nothing to do."
- **Why it matters:** A misleading-but-plausible error is worse than a loud one;
  it produces a confident wrong conclusion, exactly the guesswork the prompt
  exists to remove.
- **How to apply:** Every Codeberg remote-read instruction must (1) carry
  `Authorization: token $CB` (PKG-03) or reuse the already-configured index
  creds, and (2) explicitly state that a 401/404 means AUTHENTICATE, never
  "absent." Test the literal command unauthenticated before shipping it as
  guidance — assume nothing is public.
- **Generalizes?:** yes → the fix is in the sync prompt (STEP-0 ASSERT-LATEST +
  sibling-read blocks); the broader rule "verify the failure code can't be
  misread as success" is a review habit worth keeping.

## 2026-06-06 — Red-team the bootstrap prompt, not just the recipe it loads

- **Context:** Hardening the federation-sync mechanism by simulating a
  sibling-repo agent handed only "run `mgf-common sync-prompt` and follow it"
  (after `pip install -U mgf-common`).
- **What I learned:** The recipe (`federation-sync-prompt.md`) was already
  correct (it installs `'mgf-common[standards]'`), but the *bootstrap one-liner
  humans hand to agents* still said bare `mgf-common` — which loud-degrades
  before the agent ever sees the prompt. The gap was in the handoff message, a
  layer above the artifact we kept reviewing. Reviewing only the recipe would
  have missed it.
- **Why it matters:** The entry-point instruction is part of the mechanism. A
  perfect prompt is useless if the sentence that launches it is wrong.
- **How to apply:** When auditing any "do X then follow Y" flow, test the
  literal first instruction a fresh agent receives — not just Y. Trace from the
  human's exact words inward.
- **Generalizes?:** yes → the canonical bootstrap string fix is folded into the
  sync prompt's GUARD; broader "audit the entry point" is a review habit.

## 2026-06-06 — "Synced" must mean "to the latest released," not "to my pin"

- **Context:** Same red-team — checking whether an agent following the prompt
  actually lands on current standards.
- **What I learned:** The resolve-don't-copy seams read the corpus from the
  version THIS project's pin RESOLVES TO. If the pin ceiling lags the latest
  release, the agent syncs to OLD standards and still reports success — a
  green, confident, WRONG sync. "Resolve from the pinned version" (a deliberate,
  good design for reproducibility) quietly became "stale is fine" without an
  explicit latest-check.
- **Why it matters:** It silently breaks the core "always get the latest"
  guarantee — the worst kind of bug, because every signal says pass.
- **How to apply:** Any "read config/contract from a dependency" design needs an
  explicit installed-vs-latest assertion at the boundary (here: STEP-0
  ASSERT-LATEST against the Codeberg registry), or it will eventually bind to
  stale-as-current. Don't conflate "deterministic" with "current."
- **Generalizes?:** yes → filed to pending-feedback/mgf-common.md (wire a live
  latest-check into `catch-up`) and FEEDBACK PAPER-05.

## 2026-06-06 — Read-only-on-the-federation needs a writable outbox, or value leaks

- **Context:** Same red-team — tracing what happens when the agent finds a bug
  in a dependency or wants to challenge a standard.
- **What I learned:** "Route it to the target's FEEDBACK.md" + "never reach into
  another repo" with no writable destination = the finding dies in the chat
  transcript. Two reasonable rules combined into a dead end. The fix was a
  `pending-feedback/<target>.md` outbox in the agent's OWN repo (the one place
  it may write), which the supervisor relays — reconciling read-only with
  never-lose-value.
- **Why it matters:** The whole "never waste or duplicate effort, everything is
  up for evolution" value depends on findings surviving the session.
- **How to apply:** Whenever a rule forbids writing where the value belongs,
  give it a writable staging area + a relay step — don't leave "surface it
  somehow" as the instruction.
- **Generalizes?:** yes → the outbox mechanism is now in the sync prompt; a
  supervisor-side relay recipe is the open follow-up (FEEDBACK PAPER-04).
