# Federation Automation & Release System — Planning Document

> **Status:** DRAFT v0.1 (2026-06-10) — for review, refinement, and filtering. **Not yet approved; do not implement until signalled.**
> **Owner decisions locked:** Q1 `mgf-fed` thin tool (composes mgf-standard + mgf-test-supervisor) · Q2 file-based local index · **Q3 same version number across local & remote; ordering rule = release locally + validate first, then optionally promote the *same* version to remote (build-once)** · Q4 JSON status + table + run logs · Q5 template + drift-checker · Q6 first slice = local-release gate + gate-runner + status tracker · Q7 codify both gates as standard rules + recipes.
> **Bucket:** in-progress tracker (DOC-06 shape — resumable). Supersedes ad-hoc per-operation bash.

---

## 0. Why this exists

We repeatedly run the *same cross-cutting work* on ~22 Python libraries (plus Zig siblings, a template, and downstream apps): quality/standard checks, build, publish, local tests, CI, registry regen, PR flow. Today each is **manual or ad-hoc bash, error-prone at fleet scale**, and the server CI is blocked (`org_access_denied`). This plan turns that recurring work into a **consistent, controllable, trackable** system — one that knows *how* and *why* each step runs — and codifies a **two-mode release** (local-first, then optional remote promotion).

**Design stance:** *compose, don't reinvent.* The graph, the runner, and the local CI already exist (see §2). The new build is orchestration + tracking + a local index + codified gates.

---

## 1. Scope & shapes

The federation registry (`release_waves()`) currently spans **28 members across 4 waves**:

| Shape | Members | In scope for which patterns |
|---|---|---|
| **Python conformance lib** (22) | mgf-common, mgf-standard, mgf-fastapi, …, mgf-vm-*, mgf-tele-*, mgf-hrb, mgf-test-supervisor | **all** patterns + both release modes |
| **Zig lib** (5) | mgf-common-zig, mgf-common-identity, mgf-hrb-{model,kabi,cap,rt} | gates/release via `zig build`; **separate gate set** (not Python 218-MUST) |
| **Template** (1) | mgf-website-template (cookiecutter) | scaffold/lint only; no wheel release |
| **Downstream apps** (consumers) | hrb-app, TelePlasma, iosRecovery, inthewords, vm-vmanager-* | consume releases; catch-up automation only |

**Phase-1 target = the 22 Python libs.** Zig/template/app support is a later phase with adapted gates.

---

## 2. Part I — The repeated patterns (deep+wide inventory, with evidence)

Every row is something we run **N times across the fleet**. "Reuse" = the existing asset the orchestrator wraps.

| # | Pattern | What it does | Reuse (exists today) | Cadence | State |
|---|---|---|---|---|---|
| P1 | **Quality gates** | ruff check · ruff format --check · lint-imports · mypy --strict · pytest+cov · pip-audit · gitleaks · twine check · py.typed | per-repo `.woodpecker.yml` (canonical 13-step, **byte-identical** across libs) | every change/release | manual-in-venv; **CI server dormant** |
| P2 | **Test orchestration** | tiered runs, selection, quarantine, perf, structured records, supervisor wrapper | **`mgf-test-supervisor` (`mtest`)** — records/quarantine/selection/perf/self_check/supervisor | every test run | shipped, under-used outside itself |
| P3 | **Local CI** | run the *actual* `.woodpecker` pipeline in containers | **`woodpecker-cli exec` 3.15 + Docker 29.5** (proven this session) | pre-release | works (matrix-axis + private-index caveats) |
| P4 | **Conformance audit (§Z)** | evidence-based per-MUST verdicts + adversarial refute | multi-agent **Workflow** template (this session) | per-library, occasional | done once; not yet a one-touch tool |
| P5 | **Conformance gates** | `doctor --standards` (DOC-15/19) · §Z ledger present · SCOPE/DEPS/buckets · PUBLIC_API surface+pins · AG-16 `[standards]` · import-linter contracts | **mgf-common CLI** + per-repo artifacts | per release | manual |
| P6 | **Registry** | roster + dependency **waves** + consistency/currency gate | **`mgf-standard` `gen_mgf_libraries.py --check`** + `release_waves()` | per release | manual regen |
| P7 | **Build & publish** | bump version+`__version__` · CHANGELOG · regen PUBLIC_API.json · build · **annotated** tag · upload devpi · verify +simple listing | `uv`/`twine` + `scripts/verify_codeberg.py` | per release | **fully manual** (Woodpecker dormant) |
| P8 | **CI scaffolding** | the standard pipeline + verify script | canonical `.woodpecker.yml` + `verify_codeberg.py` | per new/repaired repo | hand-copied (drift risk) |
| P9 | **PR/SCM flow** | branch · commit (AG-03 trailers, hooks-off) · push (SSH) · `tea` create/merge · **confirm-before-delete** · paced/distinct-title | `tea` + git | constantly | ad-hoc bash; **hits Forgejo 11-PR/30-min cap** |
| P10 | **Fleet maintenance** | format-normalize · pin/cohort-walk · doc/reference sweeps (e.g. the rename) · consumer catch-up | ad-hoc sweeps | episodic | one-off scripts |

**Recurring failure modes (must become guardrails — see §9):** Forgejo 11-PR/30-min + similar-title throttle; branch-cleanup races; stale/wrong-path venvs; root-owned `.ruff_cache`/`.gates/.lock`; ruff-version format drift; TOML corruption from regex edits; write-from-same-file-read truncation; mid-run dir renames breaking paths.

---

## 3. Part II — The `mgf-fed` orchestrator (architecture)

A thin CLI that **composes** the existing assets; it owns orchestration, ordering, control, and tracking — not new validators.

```
mgf-fed  ──reads──>  mgf-standard.release_waves()      # the ORDER (graph)
         ──runs────>  mgf-test-supervisor (mtest)       # gates + tests + records
         ──runs────>  woodpecker-cli exec               # local CI (real pipeline)
         ──drives──>  tea + git                          # PR/release flow (guard-railed)
         ──writes──>  ~/.mgf/{local-index, runs, MGF_FED_STATUS.json}
```

### Subcommands (each = one Part-I pattern, run consistently)
| Command | Does | Composes |
|---|---|---|
| `mgf-fed status` | render the tracking matrix (read-only) | MGF_FED_STATUS.json |
| `mgf-fed gates <sel> [--fix]` | run P1 quality gates across a selection | mtest + ruff/mypy/… |
| `mgf-fed local-ci <sel>` | run P3 (woodpecker-cli exec) | woodpecker-cli |
| `mgf-fed release-local <sel>` | **Local-release gate (§5 Table C)** → local index, **wave-ordered** | all of the above |
| `mgf-fed release-remote <sel>` | **Remote-release gate (§5 Table D)** → devpi, **promote same version**, wave-ordered | workflow + tea/git |
| `mgf-fed scaffold <repo>` | apply/refresh the standard template (P8) | template (§6) |
| `mgf-fed drift-check [--all]` | diff each repo vs the template; report divergence | template (§6) |
| `mgf-fed registry [--check]` | P6 regen + consistency/currency | gen_mgf_libraries |
| `mgf-fed pr …` | the guard-railed PR primitive (used by all writes) | tea/git + §9 |
| `mgf-fed audit <repo>` / `close` | P4/P5 (later phase) | Workflow |

### Control surface (consistency + safety)
`--repos a,b` · `--wave N` · `--from-wave N` · `--all` · `--shape python|zig` · `--dry-run` (default for destructive) · `--stop-on-fail` (default) · `--pace <sec>` · `--source local|remote` (dep index) · `--explain` (prints *why* each gate runs, citing the standard rule).

### Tracking (Q4)
`~/.mgf/MGF_FED_STATUS.json` — per repo: `{conformance_pct, last_audit, gates:{ruff,mypy,…}, local_version, remote_version, scaffold_drift, last_run}`. `mgf-fed status` renders it as a table; every run appends a structured log to `~/.mgf/runs/<ts>-<cmd>.jsonl` (mirrors mgf-test-supervisor's record format).

---

## 4. Part III — The local index (dual-source dependency resolution)

- **Local index** = `~/.mgf/local-index/` — a directory of built wheels exposed to `uv`/`pip` as a `--find-links` / simple index.
- **Resolution precedence:** `local-index → devpi → public PyPI`. Injected via a uv config / env (not per-repo pyproject edits), so the same repo resolves local-first for dev, remote-pinned for reproducible builds.
- **Always-available invariant:** because `release-local` publishes the wheel to the local index immediately, a dependent in a later wave can always resolve the just-released version **without network/cap**.
- **Build-once (Q3):** the wheel built + validated during local-release is the **same artifact** later promoted to devpi — no rebuild, no `.dev` marker, identical version.

| Need an mgf dep | Local | Remote |
|---|---|---|
| Availability | always (immediately after local-release) | after promotion only |
| Freshness | latest *locally-released* (may lead in time) | latest *published* (authoritative once promoted) |
| Use | dev/QA, validating an unreleased chain | reproducible builds, downstream apps |

---

## 5. Part IV — The two release modes + gates (codified)

**The ordering rule (Q3):** `release-local vX` (gate+validate) **first** → then *optionally* `release-remote vX` (promote the same artifact). A remote release of `vX` is only permitted once every mgf dep is **remote-published** (wave order on the remote side).

### Table C — Local-release gate (`mgf-fed release-local`, per lib, deps resolved from local index)
| # | Stage | Validation / test | Pass condition |
|---|---|---|---|
| 0 | Resolve | fresh env, deps from **local index** | all resolve |
| 1 | Static | ruff check · ruff format --check (pinned ruff) · lint-imports · placeholder grep | clean |
| 2 | Types | mypy --strict | 0 errors |
| 3 | Tests | `mtest` full suite (matrix where local) + coverage floor | pass + ≥floor |
| 4 | Security | pip-audit --strict · gitleaks | none |
| 5 | Conformance | doctor --standards (DOC-15/19) · §Z ledger present · registry --check | pass |
| 6 | Build | python -m build · twine check · py.typed-in-wheel | valid |
| 7 | Coherence | tag==pyproject==`__version__` · PUBLIC_API.json pin · CHANGELOG entry | in sync |
| 8 | Smoke | install built wheel in clean venv · import · CLI smoke | runs |
| 9 | Local CI | `woodpecker-cli exec` (the real pipeline) | green |
| 10 | **Local-publish** | copy the validated wheel → local index | dependents resolve it |

### Table D — Remote-release gate (`mgf-fed release-remote`, workflow-driven)
| # | Stage | Validation / test | Pass condition |
|---|---|---|---|
| 0 | Pre-req | the *same version* was **local-released + validated**; all mgf deps **remote-published** | yes |
| 1–9 | Tables C 1–9 re-run **inside `.woodpecker`** (deps from **devpi**), via `woodpecker-cli exec` now / server later | workflow green |
| 10 | Tag | **annotated** `v<x>` on the build commit (REL-01) | annotated |
| 11 | Publish | tag→build→tag-coherence→smoke→**twine upload (devpi)**→verify-devpi | version listed |
| 12 | Registry | regen + consistency/currency (REL-13) | GREEN+CURRENT |

**Difference between modes = dep source (local vs devpi) + runner (direct vs workflow) + target (local index vs devpi+tag).** Stages 1–9 are otherwise identical, which is the point: *what passes locally is what the workflow re-checks remotely.*

---

## 6. Part V — Template + drift-checker (anti-drift backbone, Q5)

- A **canonical template** in `mgf-standard/templates/` for every shared artifact: the 13-step `.woodpecker.yml` (+ documented per-shape extensions: Zig build, supervisor tiers), `.importlinter`/contracts, `scripts/verify_codeberg.py`, `tools/generate_public_api_json.py`, the pyproject `[tool.*]` gate config, **pinned ruff**, the `[standards]` extra, the doc-bucket charters.
- `mgf-fed scaffold <repo>` applies it; `mgf-fed drift-check` diffs each repo against it and reports divergence (so the byte-identical pipelines *stay* identical and "wired-dormant" stays uniform).
- **Highest-leverage anti-drift move** — it converts "hand-copied, drifts" into "generated, verified."

---

## 7. Part VI — Codify both gates as standard rules + recipes (Q7)

- New standard content in `mgf-standard/docs/standards/`: a **RELEASE_GATES** section (or extend RELEASING) defining the **Local-release gate (LR-01..N)** and reaffirming the **Remote-release gate** (today's WF/REL/PKG) — as MUSTs, so they're *enforced*, not just scripted.
- Recipes: `docs/recipes/local-release.md` + `docs/recipes/remote-release.md` (the runbooks `mgf-fed` automates).
- `doctor --standards` (and `mgf-fed drift-check`) verify a repo carries the gate config — closing the loop.

---

## 8. Part VII — Phased rollout

| Phase | Deliverable | Reuses | Unblocks |
|---|---|---|---|
| **0 Foundations** | pin ruff fleet-wide · template + `drift-check` · local index · `mgf-fed status` (read-only) | mgf-standard, uv | everything below; kills format drift |
| **1 Gate-runner** | `mgf-fed gates` + `mgf-fed local-ci` | mtest + woodpecker exec | consistent local validation |
| **2 Local release** | `mgf-fed release-local` (wave-ordered → local index) | Phase 1 + local index | end-to-end local validation of a change across the chain |
| **3 Remote release** | `mgf-fed release-remote` (promote, workflow-gated) + guard-railed `mgf-fed pr` | Phase 2 + §9 | reliable devpi releases without manual repetition |
| **4 Codify + enforce** | the gate standards + recipes + drift-check in CI | Phase 0/3 | "part of the core standard" |
| **5 (optional) Audit/close** | `mgf-fed audit`/`close` wrapping the Workflow | Workflow | one-touch conformance |

---

## 9. Part VIII — Guardrails baked in (the hard-won lessons)

`mgf-fed` makes these *defaults*, not things to remember:
- **PR pacing:** ≤ ~9 creates / 30 min, with cool-down+retry; **distinct titles** (dodge the similar-name throttle).
- **Confirm-before-delete:** never delete a branch until `origin/<default>` is confirmed advanced.
- **No-regex-on-TOML:** edits via `tomllib`-validated rewrites or exact-line `sed`.
- **Never write a file from a same-file read** (the truncation bug).
- **Pinned-ruff formatting** + scope gates to what CI gates (`src tests`).
- **Fresh, correctly-pathed venvs;** clean root-owned caches (`--no-cache`, `rm .gates/.lock`).
- **Idempotent + `--dry-run`-first** for any destructive sweep.

---

## 10. Part IX — Risks & open items to refine/filter

1. **`mgf-fed` home** — standalone repo vs a subpackage of mgf-test-supervisor. *(Lean: standalone thin tool; revisit.)*
2. **uv index injection** — how to set `local→devpi→PyPI` precedence without polluting each pyproject (uv `--index`/config/env). Needs a concrete, reproducible mechanism.
3. **Local index location & GC** — `~/.mgf/local-index/`; retention/prune policy (old local wheels).
4. **Woodpecker server activation** is still **org-OAuth-blocked** — local CI (`exec`) is the bridge; remote-release "workflow" runs via `exec` until the org grants the app. Container installs of *private* mgf deps need a token strategy (don't bake tokens into pipeline URLs).
5. **Zig + app shapes** — separate gate sets; phase later.
6. **Local-release ≠ a tag** — decide whether local-release records anything in git (suggest: no tag locally; tag only on remote promotion, keeping REL-01 annotated-tag semantics remote-only).
7. **Conformance audit cost** — keep P4 (multi-agent) as an *occasional* `mgf-fed audit`, **not** part of the per-release gate (too heavy/expensive for every release).
8. **Source of truth for status** — `MGF_FED_STATUS.json` location (in mgf-standard repo vs `~/.mgf/`); committing it makes it shareable but adds churn.

---

## 11. Appendix

**Release waves (current `release_waves()`):**
- wave 0: mgf-common, mgf-standard, mgf-common-zig, mgf-hrb-model, mgf-website-template
- wave 1: mgf-apiprobe, mgf-auth, mgf-django, mgf-hrb, mgf-hrb-kabi, mgf-hrb-rt, mgf-http, mgf-livepush, mgf-logship, mgf-sqlalchemy, mgf-tele-core, mgf-test-supervisor, mgf-vm-core
- wave 2: mgf-alembic, mgf-common-identity, mgf-fastapi, mgf-hrb-cap, mgf-tele-teltonika, mgf-vm-firecracker, mgf-vm-forensics, mgf-vm-interchange, mgf-vm-libvirt
- wave 3: mgf-vm-sdk

**Canonical Python pipeline (13 steps):** lint · typecheck · test-3.11/3.12/3.13 · vuln-scan · secret-scan · build · sbom · tag-coherence* · smoke* · publish* · verify-devpi*  (*tag-gated). Per-shape extensions seen: `zig-build-and-test`, `supervisor-self-check`, `supervisor-tier-0-2/0-3`, `nightly-tier-0-3` (mgf-hrb).

**Tooling baseline:** uv 0.10.8 · woodpecker-cli 3.15.0 · Docker 29.5.3 · tea (Forgejo) · ruff (pin TBD) · mgf-test-supervisor `mtest`.
