# LESSON_LEARNED — mgf-fed

An append-only, dated log of non-obvious lessons (DOC-19).

## 2026-06-18 — CI `pip install` silently bypasses the `[tool.uv.sources]` dependency-confusion pin; a `|| <bare>` fallback is worse than none
- **Context:** Fleet CI audit (the mgf-hrb dig's two hazards). The pyprojects declare
  the real PKG-06 dependency-confusion defense — `[[tool.uv.index]] explicit = true`
  + `[tool.uv.sources] mgf-common = { index = "codeberg-mgf" }` (only these names from
  Codeberg, NEVER public PyPI). But the generated library CI installs with **pip**,
  which **ignores `[tool.uv.sources]` entirely**. The generator substituted that with
  `pip install -e ".[…]" --extra-index-url pypi --index-url codeberg || pip install -e ".[…]"`.
- **What I learned:** Two distinct holes. (1) `--extra-index-url pypi` makes pip MERGE
  both indexes and pick the **highest version** — safe only while Codeberg's version
  outranks any public mirror; it is NOT per-package pinning. (2) The `|| <bare>` retry
  resolves *entirely* from public PyPI, so any transient Codeberg hiccup silently
  installs the stale public mgf-* (0.44.5) and the build still "passes." A bare retry
  for a private dep can ONLY degrade correctness — the dep lives on Codeberg, so a
  public resolution is always wrong, never a useful fallback. Public-PyPI check
  (2026-06-18): the `mgf-*` names there are **org-owned stale mirrors**, so external
  RCE is low — but the silent-wrong-version path is live, and the public surface is a
  PKG-12 hygiene gap that should be yanked.
- **Why it matters:** the declared security control (uv explicit pinning) and the
  enforced one (pip + extra-index) were DIFFERENT controls; the weaker one ran in CI.
  "We pinned it in pyproject" did not mean "CI honours the pin."
- **How to apply:** never pair an mgf-* install with `--extra-index-url pypi` + a bare
  `|| ` fallback. Retry the SAME index-steered command (`cmd || (sleep 15 && cmd)`),
  never a bare one. Better: use `uv pip install`/`uv sync` in CI (the app repos do) so
  the `explicit=true` pin is actually enforced. Fixed the generator
  (`_woodpecker._steered`) to drop the bare fallback fleet-wide.
- **Generalizes?:** YES — any repo whose dependency-confusion defense is declared for
  one resolver (uv) but whose CI uses another (pip) has an unenforced control.

## 2026-06-18 — Exempt/bespoke CI escapes the generation-currency gate; gate install-command SAFETY, not just step names
- **Context:** `mgf-test-supervisor` (in `[exempt].woodpecker`) had bare
  `pip install -e ".[dev]"` on its lint/typecheck/test steps; `.[dev]` pulls
  `mgf-common`, so they tested against the stale public 0.44.5. `mgf-fed drift-check`
  reported it **clean** — exempt repos skip the byte-exact generation-currency check
  and fall back to a step-NAME presence list, which never looks at what the install
  commands DO. A prior partial hand-fix had steered vuln-scan/sbom but missed the test
  steps; nothing caught the gap.
- **What I learned:** the two CI gates (byte-render currency for generated repos,
  step-name list for exempt) both have a blind spot: neither inspects install-command
  safety. The hand-fix-some-steps-miss-others failure mode is invisible to both.
- **How to apply:** added `_drift._ci_install_safety_drift` — runs on EVERY
  `.woodpecker.yml` (generated, exempt, generation-off): flags bare project/extras
  `pip install` without index steering (when `extra_index` on) + any `pip-audit` step
  lacking `pip install --upgrade pip`. ~30 LOC, reuses the `extra_index` knob so the
  legit-bare repos (cornerstone, editable-pinned) aren't false-flagged.
- **Generalizes?:** YES — a presence/byte gate is not a behaviour gate; safety-critical
  command shapes need their own assertion that runs regardless of exemption.

## 2026-06-16 — pytest-cov in `addopts` silently zeroes the `coverage run` gate
- **Context:** With the FileNotFound tests fixed, only `test-3.12` still red:
  `coverage report` → `No data was collected … total of 0 is less than
  fail-under=80`, while `test-3.11`/`test-3.13` were green. Not flaky — *structural*:
  the matrix's non-gate steps run bare `pytest`, but the gate step runs
  `coverage run --source=src/mgf -m pytest` then `coverage report --fail-under=80`.
  mgf-fed's `[tool.pytest.ini_options] addopts` carried `--cov=mgf.fed` /
  `--cov-fail-under=80`, so under the gate step pytest-cov opened a SECOND coverage
  session that hijacked collection from the outer `coverage run` — outer data file
  empty → 0%.
- **What I learned:** The two green repos (mgf-common, mgf-standard) have NO `--cov`
  in addopts; they let the generated CI gate (`coverage run` + `coverage report`)
  own coverage. mgf-fed had deviated, and the deviation only bites the one step that
  uses `coverage run` — so it hid behind the bare-pytest steps passing.
- **Why it matters:** "2 of 3 matrix steps pass" looks like flakiness but was a
  deterministic config conflict. The signal to read is *which* step fails and what
  command it runs, not the pass-count.
- **How to apply:** Never combine pytest-cov (`--cov` in addopts) with the fleet's
  `coverage run -m pytest` gate. Keep coverage flags OUT of addopts; run coverage
  locally the way the gate does. A `drift-check` rule asserting addopts has no
  `--cov*` when the woodpecker gate uses `coverage run` would catch this fleet-wide.
- **Generalizes?:** YES — fleet-wide pyproject/CI consistency invariant.

## 2026-06-16 — A federation generator can't unit-test itself in a single-repo CI clone
- **Context:** When the self-hosted Woodpecker first ran mgf-fed's pipeline, ~30
  tests (`test_woodpecker` / `test_scaffold` / `test_drift`) failed with
  `FileNotFoundError: …/mgf-standard/templates`. Those tests render the REAL
  `mgf-standard/templates/` via `_REAL_TEMPLATES = parents[3]/"mgf-standard"/"templates"`
  — a path that only exists in the **federation-root layout** (siblings checked out
  together). Woodpecker clones **one** repo, so the sibling is absent.
- **What I learned:** mgf-fed is the generator; the bulk of its suite is
  *federation-integration* (it exercises the generator against the live templates),
  not single-repo-unit. A single-repo CI clone fundamentally can't run that surface.
  Skipping it isn't a workaround — it's the correct CI boundary. But skipping it also
  drops the covered modules below the coverage floor, so they must be `omit`-ed from
  the gate too, or CI reds on coverage instead of FileNotFound.
- **Why it matters:** "make CI green" here is two coupled edits, not one: (1)
  `skipif(not _REAL_TEMPLATES.exists())` on every federation-integration test file,
  and (2) `[tool.coverage.run] omit` for the modules those tests cover
  (`_woodpecker`, `_scaffold`, `_drift`). Do only (1) and coverage fails; the second
  failure mode hides behind the first.
- **How to apply:** For any tool that reads sibling repos, guard its integration
  tests with a sibling-present skipif AND omit the integration-only modules from the
  single-repo coverage gate — and say so in a comment, citing the real validator
  (`mgf-fed drift-check`, run by the supervisor against the live fleet, + local
  federation-root runs). The cleaner long-term fix (ship `mgf-standard/templates` as
  installed package data so the generator resolves them without a sibling) is heavier
  (a mgf-standard republish + render-resolution change) — deferred as a TODO.
- **Generalizes?:** Partly — the sibling-skip pattern is fleet-wide for federation
  tooling; the coverage-omit coupling is specific to repos whose *core* modules are
  integration-only (mgf-fed is the main one).

## 2026-06-10 — TOML top-level keys must precede every `[table]`
In `policy.toml`, `woodpecker_required_steps` was placed *after* the `[exact]` table
header, so TOML parsed it as a key **inside** `[exact]` — the CI-step drift check read an
empty top-level list and was **silently inert**. The unit test (`test_drift`) caught it:
the "missing publish step" assertion failed because no `ci-step` item was produced. Fix:
all top-level keys/arrays must appear before the first `[table]` header. *Takeaway:* prefer
parsed-policy unit tests over trusting a live run that "found nothing" — absence of findings
can mean a broken check, not a clean fleet.

## 2026-06-10 — Compose, don't reimplement
mgf-fed's value is orchestration + tracking, not new validators. The graph
(`mgf-standard` waves), the runner (`mgf-test-supervisor`), and local CI (`woodpecker-cli
exec`) already exist; reimplementing any of them would create drift, not remove it.
