# Test Execution Runbook

> **What this is.** Every command to run tests against `mgf-hrb`,
> from "I just cloned the repo" to "we're cutting a release."
> Concrete. Copy-pastable.
>
> Read after [`TAXONOMY.md`](TAXONOMY.md) (the categories) and
> [`INFRASTRUCTURE.md`](INFRASTRUCTURE.md) (the fixtures).

## Prerequisites

```bash
# 1. The Python venv with mgf-hrb + dev deps installed editable.
#    On a fresh clone:
cd mgf-hrb
uv sync --extra dev     # provisions .venv/, installs everything
. .venv/bin/activate    # convenience; or use absolute paths below

# 2. The Zig toolchain.
#    No system Zig is required — the `ziglang` Python package ships
#    Zig 0.16 in the venv:
ZIG=.venv/lib/python3.13/site-packages/ziglang/zig

# 3. The broker binary (for e2e + use_case suites).
$ZIG build --build-file hrb-code/build.zig --cache-dir hrb-code/.zig-cache
```

`mgf-common` is a declared dependency; if you don't have it in a
sibling directory, install the local wheel:

```bash
uv pip install -e ../mgf_common  # if you have the source
# OR
uv pip install ../mgf_common/dist/mgf_common-0.39.0-py3-none-any.whl
```

## Picking what to run — `mgf-test-supervisor`

mgf-hrb consumes the federation test supervisor as a dev dep
(`mgf-test-supervisor` v0.1.1+; see `pyproject.toml`). The
supervisor runs pytest from the OUTSIDE so it can detect hangs,
crashes, and freezes pytest itself cannot. It writes structured
per-test records to `.gates/last-run/run.jsonl` and a one-file
failure bundle per failed test to `.gates/last-run/failures/<id>.txt`.

The full spec lives in
[`mgf-standard/docs/standards/TESTING.md`](https://codeberg.org/magogi-admin/mgf-standard/src/branch/main/docs/standards/TESTING.md)
§13.11 (13-marker taxonomy) and §19 (supervisor implementation
guide). The reference implementation lives in
[`mgf-test-supervisor`](https://codeberg.org/magogi-admin/mgf-test-supervisor).

```bash
# Anchor the supervisor + wire the daemon-name probe (env vars
# expected per MGF-TS-23 + the v0.1.1 daemon-name override).
export MGF_TEST_SUPERVISOR_REPO_ROOT="$PWD"
export MGF_TEST_SUPERVISOR_DAEMON_PROCESS_NAME="mgf-hrb-server"

uv run mgf-test-supervisor                          # all tiers (0-3), no budget
uv run mgf-test-supervisor --tier 0                 # < 1s — am I building?
uv run mgf-test-supervisor --tier 0-1               # < 30s — pre-push default
uv run mgf-test-supervisor --tier 0-2               # < 2m — pre-release
uv run mgf-test-supervisor --tier 0-3               # < 30m — nightly
uv run mgf-test-supervisor --budget 30s             # best fit into 30 seconds
uv run mgf-test-supervisor --report                 # show last run's summary.md
uv run mgf-test-supervisor --quarantine             # show quarantine state
uv run mgf-test-supervisor --resume                 # continue interrupted run
uv run mgf-test-supervisor --self-check             # verify the supervisor works
```

(The `mtest` alias is also available — same binary.)

Tier semantics (matching the federation standard, MGF-TS-22):

| Tier | What it includes | Wall-clock target | Used by |
|---|---|---|---|
| 0 | smoke marker | < 1s total | CI fail-fast; build-broken signal |
| 1 | unit + contract + property + regression + fuzz | < 30s | every push (pre-push hook + PR CI) |
| 2 | adds conformance + use_case + integration + concurrency | < 2 min | pre-release; main-branch CI |
| 3 | adds stress + perf + e2e | < 30 min | nightly + release gate |

Selection is **greedy under budget**: tier-0 is always included,
remaining selected-tier tests sorted by `value / cost` (value =
recently-failed tests rank higher; cost = estimated duration).

## The four gates (every PR)

These four gates are the contract. **Every PR must pass all four
green.** Any gate red blocks merge.

`scripts/gates` is the **canonical 4-gate runner** (lint, types,
pytest+coverage, zig tests). `mgf-test-supervisor` is the **smart
test runner** (tier-aware, budget-aware, supervisor-driven). The
two serve complementary purposes:

* CI's PR job runs `uv run mgf-test-supervisor --tier 0-2` for
  fast feedback.
* CI's main job runs `scripts/gates` + `uv run
  mgf-test-supervisor --tier 0-3`.
* Pre-push hook runs lint + types + `uv run mgf-test-supervisor
  --tier 0-1`.
* Pre-release / nightly: `uv run mgf-test-supervisor --tier 0-3
  --quarantine-attempt`.

The expanded gates if you prefer to run them by hand:

```bash
./scripts/gates              # all four gates, stop on first failure
./scripts/gates --fast       # skip the Zig gate (slow path)
./scripts/gates --keep-going # run every gate even if an earlier one failed
```

The expanded gates if you prefer to run them by hand:

```bash
# Gate 1 — lint
.venv/bin/python -m ruff check src tests

# Gate 2 — types
.venv/bin/python -m mypy --strict src

# Gate 3 — Python tests + coverage (TS-06: ≥ 80%).
# `--cov` (no source arg) honours `[tool.coverage.run].source = ["mgf.hrb"]`
# in pyproject.toml — see "Coverage gotcha" below for why this matters.
.venv/bin/python -m pytest -q --cov --cov-fail-under=80

# Gate 4 — Zig tests
$ZIG build --build-file hrb-code/build.zig --cache-dir hrb-code/.zig-cache test --summary all
```

### Pre-push hook (opt-in, one-line install)

```bash
ln -s ../../hooks/pre-push .git/hooks/pre-push
```

After install, every `git push` runs `scripts/gates --fast` (lint,
types, default pytest + coverage; ~3 s on a clean tree). CI catches
Zig regressions on the push itself. Bypass in a true emergency
with `git push --no-verify`.

### CI (Woodpecker on Codeberg)

`.woodpecker.yml` runs the federation's three-job split per
WF-01..WF-12 (see [`mgf-standard/docs/standards/WORKFLOWS.md`](https://codeberg.org/magogi-admin/mgf-standard/src/branch/main/docs/standards/WORKFLOWS.md)).
Caches the uv-cache + Zig artifact
dir to keep round-trips short. Validates exactly what `scripts/gates`
runs locally — if it's green here it's green there.

Expected output on a clean checkout (numbers will drift):

```
All checks passed!                       ← ruff
Success: no issues found in 13 source files   ← mypy
132 passed in 1.3s                       ← pytest (default suite excludes opt-in markers)
Build Summary: 33/33 steps succeeded; 252/252 tests passed   ← zig
```

## Common traps + fixes

These are the single-line failure modes that have cost the most
debugging time historically. Every one is now caught by an explicit
mechanism — listed here as both the failure shape and the fix.

### Stale broker binary

**Symptom:** opt-in suites (stress / perf / use_case / conformance)
all `FAIL` in setup with the message
`broker binary hrb-code/zig-out/bin/mgf-hrb-server is older than
hrb-code/src/<file>.zig by Ns`. Or — without the freshness check —
tests silently deadlock because the binary is from before a `*.zig`
change.

**Fix:** `$ZIG build --build-file hrb-code/build.zig --cache-dir hrb-code/.zig-cache`

The check lives in `tests/_broker.py::broker_is_stale` and is
mirrored as a dedicated smoke test (`tests/smoke/test_broker_freshness.py`)
so the trap fires early.

### Editable install pointing at a worktree

**Symptom:** tests pass against unexpected code (or fail mysteriously
against missing features). `mgf.hrb.__file__` resolves under
`.claude/worktrees/<name>/src/...` instead of `<repo>/src/`.

**Fix:** `uv pip install -e . --python .venv/bin/python` from the repo root.

The check is a dedicated smoke test
(`tests/smoke/test_install_points_to_main.py`); install drift
fails the smoke gate immediately.

### Hung test

**Symptom:** pytest sits at a test name with no output for 30+ seconds.

**Fix:** `pytest-timeout` (default 30 s) auto-fails with a thread
stack dump. If you suspect output buffering, see
[`DIAGNOSING_HANGS.md`](DIAGNOSING_HANGS.md).

### Coverage gotcha

**Symptom:** `pytest --cov` reports very low or 0% coverage; the
gate "passes" because everything is counted.

**Fix:** the coverage source is now pinned in `pyproject.toml`'s
`[tool.coverage.run]`, so `pytest --cov` (no source arg) measures
`mgf.hrb` correctly. If you see drift, check that block isn't
missing.

### Stale broker zombie from a debug session

**Symptom:** new broker spawns fail to bind, or tests connect to a
stale socket.

**Fix:** `pgrep -af mgf-hrb-server | xargs -r kill -9; rm -f /tmp/mgf-hrb*.sock`

## E2E suite (opt-in)

The e2e test spawns a live broker. It requires the broker binary AND
a writable `/tmp/mgf-hrb.sock` path. Marker-gated so contributors
who skip the Zig build aren't blocked.

```bash
# Build the broker first.
$ZIG build --build-file hrb-code/build.zig --cache-dir hrb-code/.zig-cache

# Run the e2e suite (one test, ~36 internal checks).
.venv/bin/python -m pytest -m e2e -q
```

Expected: `1 passed, 96 deselected`.

## Use-case suite (opt-in)

Reproduces consumer-scenarios. Planned to grow as new use-cases are
documented in `recipes/` and filed in `FEEDBACK.md`.

```bash
.venv/bin/python -m pytest -m use_case -q
```

## Stress + perf (nightly)

Long-running; not part of PR gates because hardware variance would
make them flaky on the four-gate signal.

```bash
.venv/bin/python -m pytest -m stress -q
.venv/bin/python -m pytest -m perf -q
```

For Zig-side stress and perf:

```bash
# Stress is bundled into the standard `zig build test`;
# the `stub_usb_test.zig` suite + the concurrency tests in
# `core_test.zig` run on every default Zig invocation.

# The ring benchmark runs as part of `ring_test.zig` — see its
# trailing test that prints throughput numbers.
$ZIG build --build-file hrb-code/build.zig --cache-dir hrb-code/.zig-cache test --summary all 2>&1 | grep "items/sec"
```

## Conformance (release)

S1–S5 survivability + G1–G4 guarantees. Today the suite lives in
`hrb-code/src/conformance.zig` (Zig side) and is bundled into the
standard `zig build test`. A Python-side conformance suite is on
the roadmap (see [`COVERAGE_MAP.md`](COVERAGE_MAP.md)).

```bash
# Today — runs as part of gate 4.
$ZIG build --build-file hrb-code/build.zig --cache-dir hrb-code/.zig-cache test --summary all

# Future — when tests/conformance/ exists:
.venv/bin/python -m pytest -m conformance -q
```

## Individual test selection

```bash
# One file
.venv/bin/python -m pytest tests/unit/mgf/hrb/test_client.py -v

# One test
.venv/bin/python -m pytest tests/unit/mgf/hrb/test_client.py::test_claim_returns_lease_and_release_round_trip -v

# Pattern match
.venv/bin/python -m pytest -k "doctor" -v

# Run skipped + warnings + slow
.venv/bin/python -m pytest -v --tb=short --no-header
```

Zig:

```bash
# All tests in one file
$ZIG test hrb-code/src/core_test.zig --cache-dir hrb-code/.zig-cache

# Filter by test name (use Zig's --test-filter)
$ZIG build --build-file hrb-code/build.zig --cache-dir hrb-code/.zig-cache test \
  -- --test-filter "concurrency"
```

## Coverage report

```bash
.venv/bin/python -m pytest --cov=mgf.hrb --cov-report=term-missing -q
```

Or HTML:

```bash
.venv/bin/python -m pytest --cov=mgf.hrb --cov-report=html -q
xdg-open htmlcov/index.html
```

The `--cov-fail-under=80` flag is on by default (per
`[tool.coverage.report]` in `pyproject.toml`). A regression that
drops coverage below the line fails gate 3.

## Cleaning state between runs

E2E tests own `/tmp/mgf-hrb.sock`. A test crash can leave it behind.
Clean by hand:

```bash
pkill -f mgf-hrb-server 2>/dev/null
rm -f /tmp/mgf-hrb.sock
```

Stale broker processes from a debug session are the most common
cause of "the e2e test hangs" — always check `pgrep -af
mgf-hrb-server` first.

## Pre-commit (recommended)

Run the four gates before pushing. Add to `.git/hooks/pre-push`:

```bash
#!/bin/sh
set -e
.venv/bin/python -m ruff check src tests
.venv/bin/python -m mypy --strict src
.venv/bin/python -m pytest -q --cov=mgf.hrb --cov-fail-under=80
ZIG=.venv/lib/python3.13/site-packages/ziglang/zig
$ZIG build --build-file hrb-code/build.zig --cache-dir hrb-code/.zig-cache test --summary all
```

```bash
chmod +x .git/hooks/pre-push
```

(`pre-push` not `pre-commit` — gate 4 is too slow to run on every
commit; pre-push is the right boundary.)

## CI matrix (target shape)

Not yet wired (no CI config in-repo today; gates run by hand and on
the maintainer's machine). When CI lands, this is the shape:

| Job | Trigger | Command | Time budget |
|---|---|---|---|
| `lint-types` | every push | gates 1 + 2 | 30s |
| `tests-default` | every push | gates 3 + 4 | 2min |
| `tests-e2e` | every push | `pytest -m e2e` (broker built) | 1min |
| `tests-stress-nightly` | cron 02:00 UTC | `pytest -m stress` | 30min |
| `tests-perf-nightly` | cron 02:00 UTC | `pytest -m perf` | 30min |
| `tests-conformance-release` | release tag | gate 4 + (future) `pytest -m conformance` | 10min |

## When tests fail

The triage flow:

1. **Lint failure** → run with `--fix` for autofix-eligible rules;
   read the rule docs for the rest (`ruff rule <CODE>`).
2. **Mypy failure** → the error is structural. Either the code is
   wrong or the annotation is. Don't reach for `# type: ignore` —
   reach for the right type.
3. **Pytest failure** → run with `-v` and read the specific
   assertion. Most failures fall into:
   * Wire-format change → update the test + the contract doc in the
     same commit.
   * Logic regression → fix the code, not the test.
   * Flake → file it as such; don't paper over.
4. **Zig test failure** → re-run with `--summary all` to see which
   step failed. Zig tests have rich diagnostic output; read the
   stack trace bottom-up.
5. **E2E failure** → first kill stale brokers (see Cleaning State
   above). Then re-run with `pytest -m e2e -v -s` to see the
   broker stdout interleaved.

## Reproducing CI locally

CI runs exactly the four-gate one-liner above. If it passes locally
and fails in CI:

* Check Python version (`.python-version` says 3.13.x).
* Check Zig version — `$ZIG version` should be 0.16.x.
* Check `uv.lock` is in sync — `uv lock --check`.
* Wipe `hrb-code/.zig-cache/` to force a clean Zig rebuild.

## Performance regression triage

When a `perf` test fails:

* Run with `-v` to see the actual number vs. the budget.
* Re-run on a quiet machine (no other CPU-heavy processes).
* Compare against `main` baseline — if main also fails, the
  budget is stale, not the code.
* Don't loosen the budget without a paper trail (file a paper in
  `FEEDBACK.md` documenting why).
