# Recipe: the cross-family CI contract (MGF ↔ Sooth)

> **Audience:** anyone (human or agent) about to change a piece of the CI
> infrastructure the **Sooth** family runs on — a Woodpecker org secret, the
> server env, the devpi index, an `mgf-fedres` release, or a shared template
> pattern.
> **What this is:** the written contract naming the six MGF-owned surfaces
> Sooth consumes, and the rule that a change to any of them is not "done" until
> the **sooth canary** (`scripts/ping_sooth_ci.py`) has run green.

## Why this exists (CIR-22)

The Sooth family runs on MGF-owned CI infrastructure with **no code
dependency** and six **operational** dependencies. Nothing in the MGF
verification toolchain tested any of those surfaces *from Sooth's side*
when MGF changed them — `ping_sibling_ci.py` reads `MGF_LIBRARIES.json` only,
`ci-ungoverned` globs `mgf-*` only, drift-check governs the MGF fleet only.
Sooth was structurally invisible to every MGF forcing function.

The class already bit twice, in production:

- **The 2026-07 secret event-scope re-scope (CIR-02)** killed sooth-brand's
  manual pipeline (#40, 07-14) and mgf-cost-watch's cron (dead 07-13 **and**
  07-14 — two passes to fix one change). An error-status pipeline runs zero
  steps, so nothing gated and nothing paged.
- **The `$DEVPI_URL` rename (PAPER-113)** needed by-hand coordination across
  both families — Sooth's conversion had to ship first, live-green, before the
  MGF side could cut over.
- Live Sooth-side drift compounded it: **soothglass** missed the PAPER-109 SSH
  clone fix entirely (CIR-23) and nothing either family owned flagged it.

This page is the durable contract that makes those changes not-done until the
canary confirms Sooth still builds — exactly as **WF-13** makes a template
change not-done until the fleet re-render.

This is **how-to + contract**, not a binding standards rule. The rule that
every change lands through green CI is
[`../standards/WORKFLOWS.md`](../standards/WORKFLOWS.md) **WF-01**; this recipe
names the *cross-family* surfaces and the extra gate they carry.

## The six surfaces

Sooth consumes exactly these six MGF-owned surfaces. A change to the "change
events" column MUST run the sooth canary before it is declared done.

| # | Surface | What Sooth consumes | Change events that MUST run the canary |
|---|---|---|---|
| 1 | **Woodpecker server + org-level secrets** | `codeberg_ssh_clone_key` / `codeberg_ssh_host_key` (clone), `sibling_clone_token` (sibling-clone path), the server env (`WOODPECKER_ENVIRONMENT`) | secret add / rotate / **re-scope (the events!)**, server upgrade, agent-capacity change |
| 2 | **devpi + `$DEVPI_URL`** | every `fedres-check` step installs `mgf-fedres` through it | index config change, devpi host/IP move, the `$DEVPI_URL` variable rename or value change |
| 3 | **mgf-fedres** | pinned install in every sooth pipeline; adapters wired through the registration seam | any `mgf-fedres` release; any change to the projector / registration API |
| 4 | **Template patterns** (forked copies) | `plugin-git` / `gitleaks` pins, netrc idioms, the clone shape | a proven-defect fix in a shared pattern (PAPER-109-class) → **notify sooth-standard so the forks follow** |
| 5 | **`mgf-cloud` / `mgf-cost-watch`** | cost visibility, VM ops | cost-watch cron changes, `mgf-cloud` breaking releases |
| 6 | **`mgf-secrets`** (the credential plane) | the shared `SECRETS.toml` registry + store + doctor Sooth's CI actors read on the shared host; Sooth owns its own entries but inherits the mechanism (sooth-standard SEC-06) | any `mgf-secrets` library release; a `sync` that changes a **shared** surface (org secret / deploy key / host file both families touch); a registry schema change | 

Because surfaces 1–3 are the high-frequency ones, they are where the two
production incidents came from. Surface 4 is a *notification* obligation (the
forks are hand-maintained on Sooth's side — CIR-24), not something the canary
can prove on its own; run the canary anyway so a pattern change that also
touches shared infra is caught. Surface 6 (`mgf-secrets`) is the *governance*
surface: it is canary-gated on a release / shared-surface `sync`, and it also
carries the steward-precedent obligation below — the mechanism that keeps any new
cross-family credential from becoming ungoverned sprawl.

## The steward-precedent policy (any actor who adds a shared-infra credential registers it)

> **Any actor — either family — who adds a credential to shared infrastructure
> (a Woodpecker org/repo secret, a forge deploy key, a host credential file) MUST
> declare it in the `mgf-secrets` registry (`SECRETS.toml`, the SC-19 mandate) in
> the same change.** The obligation is not optional and not deferred: the
> `mgf-secrets` doctor's unregistered-actual check (C3) *enforces* it — a live
> credential absent from the registry is a finding that pages within the cron
> interval, exactly as today's steward key would have.

This makes the registry self-enforcing in both directions: declared credentials
are verified live (declared == deployed), and live credentials are verified
declared (no ungoverned actor). The precedent is named for the soothrede steward
actor whose keys/env/minter appeared on shared infra undeclared — legitimate, but
invisible until it was registered. Registering it (sooth-standard `SECRETS.toml`)
is the pattern every future actor follows.

## The rule

> **A change to any surface above is not "done" until the sooth canary has run
> green.**

The event-scope of a secret is part of the change. Re-scoping
`codeberg_ssh_clone_key` from "all events" to "push only" is a surface-1
change even though the secret's *value* never moved — that is precisely the
CIR-02 shape. Two enforcement mechanisms back the rule:

- **The sooth canary** (`scripts/ping_sooth_ci.py`) — triggers a designated set
  of sooth pipelines and polls them green. Run it before declaring done any
  surface-1/2 change, and from the mgf-fedres release flow (tracker 1.3) for
  surface 3.
- **The secret/event-scope audit** (`scripts/audit_secret_scopes.py`) — the
  standing invariant that catches a surface-1 mismatch *before* it reaches a
  pipeline: it cross-checks every activated repo's referenced secrets against
  the events each secret is actually scoped for, and fails on any mismatch. Add
  "run the audit + fire one pipeline per event class" to the secret-change
  playbook in [`../../MGF_INFRA.md`](../../MGF_INFRA.md) §2.

## The canary

`scripts/ping_sooth_ci.py` (pattern: `scripts/ping_sibling_ci.py`) reads
`sooth-standard/SOOTH_LIBRARIES.json` and triggers the **designated canary
set**, chosen to exercise each distinct dependency path once:

- **soothfast** — the kernel; a leaf build that exercises the clone + devpi +
  fedres path with the least surface area.
- **sooth-standard** — the dogfood repo (it runs its own `verify_ci.py`).
- **soothsmith** — the sibling-clone secret path (`sibling_clone_token`).

It triggers each repo's pipeline on branch `main` via the Woodpecker API,
**polls Woodpecker** (never Codeberg) until each reaches a terminal state, and
**exits non-zero if any is red or unreachable**. `--dry-run` lists the set and
calls nothing.

```sh
# List the canary set, call nothing:
python scripts/ping_sooth_ci.py --dry-run

# Trigger the three canary pipelines and poll them green (before declaring a
# surface change done):
python scripts/ping_sooth_ci.py

# Override the set (e.g. add the repo whose pattern you touched):
python scripts/ping_sooth_ci.py --repos soothfast sooth-brand
```

Token handling matches `ping_sibling_ci.py`: `$WOODPECKER_TOKEN` → the
`my_stuff/woodpecker_token.txt` fallback; the token is a JWT that expires (→
401) and is re-minted host-side per [`../../MGF_INFRA.md`](../../MGF_INFRA.md)
§2 — a 401 is never "blocked."

## The audit

`scripts/audit_secret_scopes.py` pages through every activated repo on
Woodpecker (`GET /api/repos?all=true`, paginated), reads each repo's committed
`.woodpecker.yml` from the local sibling checkout, extracts every referenced
secret and the `when:` event classes it is referenced under, and compares that
against the secret's allowed events (read live from the Woodpecker API —
`GET /api/orgs/{id}/secrets` and `GET /api/repos/{id}/secrets`, both of which
expose the `events` scope in v3.8.0). It reports repo × secret × events, flags
every mismatch, and exits non-zero on any.

```sh
python scripts/audit_secret_scopes.py            # audit the live fleet
python scripts/audit_secret_scopes.py --json     # machine-readable
```

The historical failure it reproduces: a secret referenced on a `cron` / `manual`
event it was not scoped for (mgf-cost-watch, 07-13/14). Run it after any
surface-1 change and wire it into the estate watcher so a re-scope that breaks a
pipeline is caught before the next scheduled run, not two days later.

## Ownership

`mgf-standard` owns this page (the infra owner). `sooth-standard` carries only a
**pointer** beside `SOOTH_INFRA.md` (the "inherit authority upstream, own your
pipeline" creed — that pointer is a separate, Sooth-side step). This file is
named from [`../../MGF_INFRA.md`](../../MGF_INFRA.md) §2.

Keep it current: **a surface not listed here is a surface nobody will canary.**
Update the table when a cross-family surface is added or retired.

## Related

- [`../../MGF_INFRA.md`](../../MGF_INFRA.md) — §2 names this contract and holds
  the secret-change playbook + the Woodpecker token mint path.
- [`run-ci.md`](run-ci.md) — how the pipelines this canary triggers are run and
  activated.
- [`../standards/WORKFLOWS.md`](../standards/WORKFLOWS.md) — **WF-01** (every
  change lands via green CI), **WF-13** (generated CI is not-done until the
  re-render) — the same shape this contract applies across the family boundary.
- `scripts/ping_sibling_ci.py` — the in-family fan-out this canary mirrors.
</content>
