# For QA / test engineers

> **Who you are.** You're running, analyzing, or extending
> the test suite. You want to know what's tested, how to run
> a specific layer, and where a new test belongs.

## In one paragraph

`mgf-common` follows a **four-layer test pyramid** (rules
TS-19, TS-20, TS-21 — see `mgf-fed standards TESTING`). Total
1445 tests across **smoke / unit / integration / e2e** layers
running in ~16 seconds. Each layer catches a different
failure class; markers are registered in `pyproject.toml` so
they can be invoked in isolation.

## The four layers

| Layer | Directory | Marker | Count | Runtime | What it catches |
|---|---|---|--:|---|---|
| smoke | `tests/smoke/` | `@pytest.mark.smoke` | 7 | < 100 ms total | Packaging breaks (broken wheel install, missing `__init__.py`, typo in `__all__`) |
| unit | `tests/unit/` (mirrors `src/`) | (default — no marker) | 1312 | ~ 12 s | Per-component logic regressions; isolated, no I/O |
| public-surface | `tests/public_surface/` | (default sub-tier) | 113 | ~ 2 s | Public API contract pinning |
| integration | `tests/integration/` | `@pytest.mark.integration` | 5 | < 1 s | Cross-module wiring; in-process only |
| e2e | `tests/e2e/` | `@pytest.mark.e2e` | 8 | < 1 s | Full bootstrap-to-output chains |

## Read first

1. `mgf-fed standards TESTING` — the rule corpus (TS-01 through
   TS-21). Skim the Rules box at the top for the bar; §13 for the
   operational layer-marker pattern.
2. [`docs/inprogress/coverage.md`](../inprogress/coverage.md) — per-module
   coverage map showing which subsystems have which layers.
   Also lists coverage gaps as a roadmap for future polish
   rounds.
3. `mgf-fed standards TESTING` §13.9 — decision tree for **which
   layer a new test belongs to**.

## Running tests

```bash
# Default: all 1445 (~16 s)
pytest

# By layer (TS-19 markers — registered in pyproject.toml)
pytest -m smoke              # 7, < 100 ms — post-install sanity
pytest -m e2e                # 8, < 1 s — bootstrap chains
pytest -m integration        # 5, < 1 s — cross-module

# By directory
pytest tests/unit            # 1312
pytest tests/public_surface  # 113

# Just one module's unit tests
pytest tests/unit/process    # 84 process-module tests

# Slow tests excluded by default (TS-18); run them with:
pytest -m slow

# Coverage gate (TS-06): ≥80%
pytest --cov=src/mgf --cov-report=term-missing

# Property tests (TS-13) live in unit/, run with:
pytest tests/unit -k property
```

## Adding a new test — decision tree

From `mgf-fed standards TESTING` §13.9:

```
Single function/class/module?                    → unit/
Two-or-more modules composed (no real bootstrap)? → integration/
Requires real mgf.common.bootstrap()?             → e2e/
"Wheel imports + happy path"?                     → smoke/
> 1 second runtime?                               → add @pytest.mark.slow
Hits real external system (Sentry, Slack, …)?    → §12 pattern (separate marker)
```

Default to the **lower** layer. A test that's faster + more
isolated is a better contract pin than the same logic in a
slower test.

## The fixture hierarchy (TS-17)

Discoverable via the conftest hierarchy:

```
tests/
├── conftest.py              # repo-wide fixtures
├── unit/conftest.py         # unit-only fixtures
├── integration/conftest.py  # cross-module fixtures
├── e2e/conftest.py          # bootstrap fixtures (e.g., bootstrapped_test_app)
└── smoke/                   # rare; smoke needs no fixtures by design
```

Each fixture lives in the lowest conftest that can reach all
its consumers. Cross-package fixture imports are forbidden
(per TS-17).

## Performance benchmarks

Performance budgets (rule LG-12 etc.) are pinned by
benchmarks in `tests/unit/test_logging_perf.py` and similar.
Each benchmark runs in < 5 s and asserts the budget directly.

Detailed runtime budgets per release: [`docs/ops/benchmarks.md`](../ops/benchmarks.md).

## Coverage gate

`mgf-fed standards TESTING` TS-06:

- ≥ 80% on every CI job.
- ≥ 90% on changed lines in a PR.
- Lazy-imported sink code (Sentry, OTLP, journald, syslog, HTTP webhook)
  excluded via `pragma: no cover` only when genuinely
  unreachable without the real backend.

Per-module coverage floors: see `mgf-fed standards TESTING` §9.4.

## Real-backend smoke (TS-15)

Distinct from the `smoke` marker (which is post-install
sanity, no external deps). The "real-backend smoke" pattern
is opt-in, marker-gated, read-only against live external
systems (Slack workspace, Sentry org, etc.). See
`mgf-fed standards TESTING` §12.

## CI — how the layers map

| Layer | CI shape |
|---|---|
| smoke | Pre-flight job; aborts if it fails |
| unit + public_surface | Default suite; runs on every PR |
| integration | Default suite; runs on every PR |
| e2e | Default suite; runs on every PR |
| slow | Separate job; runs nightly OR on demand |
| real-backend | Separate job; runs nightly + against a scratch environment |

Workflow templates: `mgf-fed standards WORKFLOWS_PATTERNS`.

## When a test fails — diagnosis

1. **Failure message must be self-explanatory** (TS-16). If
   the bare `assert x == y` doesn't tell you why, add
   context: `assert x == y, f"context: {ctx}"`.
2. **`--showlocals`** is on by default (`pyproject.toml::addopts`).
   Locals are printed in the failure traceback.
3. **`filterwarnings = ["error"]`** (TS-04) means warnings
   are failures. A new warning surfaces immediately.

## Filing a test gap

If a test gap shows up in production (a bug shipped that the
tests didn't catch), file via [`FEEDBACK.md`](../../FEEDBACK.md)
§1.4 with severity 🟡 (high). The triage process leads to
either (a) a regression test in the same release as the fix
(canonical), or (b) a roadmap entry to add the missing layer
coverage in a future polish round.
