# Performance & Capacity

> **Status:** v0.1 — first pass, finalized 2026-05-15
> (review-pass verified all file:line citations against the
> live source). Last updated 2026-05-15.
> A federation-sibling-scoped performance audit: walked the
> five source files (`_apps.py`, `_middleware.py`, `_logging.py`,
> `_settings.py`, `apps.py`) and identified the per-request
> hot paths, the startup-time costs, and the memory footprint
> the adapter adds on top of bare Django.
>
> **Scope:** mgf-django's OWN performance footprint — what
> every Django request pays for `MgfRequestIdMiddleware` +
> `MgfContextMiddleware`, what every Django startup pays for
> `MgfCommonConfig.ready()`, what every formatter instance
> costs. NOT a re-audit of mgf-common's PC items (per-record
> log emission, redactor cost, bootstrap-once memory). NOT an
> audit of Django itself.
>
> **Audience:** the maintainer + future AI agents doing
> performance work on this sibling.
> **Cross-references:**
> [`RELIABILITY_AND_ACCURACY.md`](RELIABILITY_AND_ACCURACY.md),
> [`SECURITY_HARDENING.md`](SECURITY_HARDENING.md),
> [`mgf-common/docs/inprogress/PERFORMANCE_AND_CAPACITY.md`](https://codeberg.org/magogi-admin/mgf-common/src/branch/main/docs/inprogress/PERFORMANCE_AND_CAPACITY.md)
> (cornerstone tracker — PC-01 cold-import, PC-02 sustained
> throughput, PC-05 memory baseline all apply to consumers
> who pull mgf-common transitively).

The sibling sits on the request path of every Django consumer.
The audit's main concerns are **per-request overhead** (two
middleware classes each touching the request twice) and
**startup latency** (one bootstrap at `manage.py` invocation).

---

## §1 You are here

mgf-django ships five public names; the two middleware classes
touch every request (read header, set contextvar, set
response header, reset contextvar — ~4 syscalls + dict ops per
request). The bootstrap is one-shot per process.

What this tracker exposes is the **per-request cost shape**
— in microseconds — and the startup cost shape (in
milliseconds). For Django apps serving 100+ req/s per worker,
even a 50 µs middleware overhead becomes a 0.5% throughput
tax. For shorter-lived processes (CLI management commands,
test fixtures), startup is what matters.

**Counts (2026-05-17):**

| Priority | Open | Closed | Total |
|---|---|---|---|
| P0 — pre-v1.0 blockers | 0 | 0 | 0 |
| P1 — should land before/with v1.0 | 0 | 3 | 3 |
| P2 — post-v1.0 hardening | 0 | 2 | 2 |
| **Total** | **0** | **5** | **5** |

**Do first:** ALL items closed. PC-01/02/03 (P1, 2026-05-16).
PC-04 (JsonFormatter construction pin) + PC-05 (contextvar
set/reset pin) closed 2026-05-17.

---

## §2 Priority table

| ID | Theme | One-liner | Effort | Status |
|---|---|---|---|---|
| [**PC-01**](#pc-01-per-request-middleware-overhead-pin) | Latency | `MgfRequestIdMiddleware` + `MgfContextMiddleware` per-request overhead is unmeasured. For high-throughput Django apps, a 50 µs overhead per request becomes a 0.5% throughput hit per middleware. Quantify + pin a budget. | S | ✅ closed |
| [**PC-02**](#pc-02-appconfig-ready-bootstrap-time-pin) | Startup | `MgfCommonConfig.ready()` runs once per `manage.py` invocation. The bootstrap calls into `mgf.common._lifecycle.bootstrap()` which transitively loads pydantic-settings + OTel + redactor patterns. mgf-common PC-01 closed the cold-import to ~100 ms; this adds the Django-app-registry overhead. Pin the total. | S | ✅ closed |
| [**PC-03**](#pc-03-djangosettingsbridge-from-django-allocation) | Allocation profile | `DjangoSettingsBridge.from_django()` builds a fresh `BaseAppSettings` instance per call. Each call iterates Django settings + allocates tuples for `allowed_hosts` / `installed_apps`. If consumers call it per-request (rare but possible: `mgf-common explain` dumping config inside a view), the allocation churn matters. | XS | ✅ closed |
| [**PC-04**](#pc-04-jsonformatter-redactor-construction-cost) | Memory + startup | Every `JsonFormatter` instance constructs a fresh `Redactor.default()`. Django's `LOGGING` config typically wires one formatter; the cost is one-shot. But test runs that re-init Django repeatedly (pytest-django with multiple `override_settings(LOGGING=...)` blocks) build N redactors. Measure the construction cost + the memory per instance. | S | ✅ closed |
| [**PC-05**](#pc-05-contextvar-set-reset-overhead-pin) | Latency | `_REQUEST_ID_VAR.set(...)` + `.reset(token)` are the contextvar ops in `MgfRequestIdMiddleware`. PEP 567 contextvars are fast (sub-microsecond per op) but not free; high-throughput async-ASGI apps could see the cost. Pin it. | XS | ✅ closed |

Effort scale: XS (≤ 15 min); S (≤ 1 h); M (≤ 1 session).

---

## §3 Invariants we hold

The list of "closed elsewhere; do not re-open":

- **PC-INV-1** — Bootstrap is **single-shot per process**
  ([`_apps.py:33-34`](../../src/mgf/django/_apps.py#L33)).
  Subsequent `ready()` calls return early after the lock
  check — no work, no cost. Cost amortises across the
  process lifetime; for long-lived Django workers, the
  per-request cost is zero.
- **PC-INV-2** — Middleware operations are **synchronous +
  branch-free** in the happy path. No I/O, no locking,
  just dict reads + contextvar ops + header set
  ([`_middleware.py:51-68`](../../src/mgf/django/_middleware.py#L51)).
- **PC-INV-3** — `DjangoSettingsBridge` **snapshots at
  construction**, NOT on every attribute read. The fields
  are pydantic-validated frozen-after-construction; reading
  `bridge.debug` is just a getattr.
- **PC-INV-4** — `Redactor` construction in `JsonFormatter.__init__`
  builds the **default pattern set** which is module-level
  cached in mgf-common. The cost is the pattern-set look-up
  + the lightweight wrapper allocation.
- **PC-INV-5** — `atexit.register(_cleanup_bootstrap)` adds
  one entry to atexit's list; the cost is one function-pointer
  registration + invocation at shutdown.

---

## §4 Work units

### PC-01 Per-request middleware overhead pin

**Why:** Both middleware classes touch every request. The
ops per-request:

`MgfRequestIdMiddleware.__call__`
([`_middleware.py:51-68`](../../src/mgf/django/_middleware.py#L51)):

1. Read `request.META.get("HTTP_X_REQUEST_ID")` — dict get.
2. Call `_generate_request_id()` if absent — UUID4 allocation.
3. Set `request.mgf_request_id = request_id` — attr set.
4. `_REQUEST_ID_VAR.set(request_id)` — contextvar set.
5. Call `self.get_response(request)` — the view.
6. `_REQUEST_ID_VAR.reset(token)` — contextvar reset.
7. Set `response[REQUEST_ID_HEADER] = request_id` — dict set.

`MgfContextMiddleware.__call__`:

1. Import `mgf.common._lifecycle.current_context` — cached.
2. Call `current_context()` — contextvar read + dict lookup.
3. Set `request.mgf_context = ctx` — attr set.
4. Call `self.get_response(request)`.

For a Django app at 100 req/s per worker, ~10 ms/sec of
worker time is spent in middleware-our-side. At 1000 req/s
(rare for Django without async + multiple workers), it's 1%
of wall-clock per middleware.

No benchmark today; no budget pinned.

**Concrete output:**

- Add `tests/perf/test_middleware_overhead.py` with
  `pytest.mark.perf`:
  - Mock `get_response = lambda req: HttpResponse()` (zero-cost
    view).
  - Run 10 000 requests through `MgfRequestIdMiddleware`;
    measure per-request time.
  - Run 10 000 requests through `MgfContextMiddleware`; same.
  - Assert each ≤ 50 µs per request (generous initial budget;
    measure current actual + multiply by 2-3).
- Document the measured numbers in `docs/ops/benchmarks.md`
  (sibling-local file).
- Tag with `pytest.mark.perf` so it doesn't run per-PR; CI
  runs the perf suite nightly.

**Locked in by:** the pinned test; a 2× regression breaks
the nightly perf-sweep build.

**Cross-references:** mgf-common PC-02 (sustained
throughput — the same discipline applied to logging records;
this is the middleware analog); LG-12.

**Status:** ✅ closed (2026-05-16). See §5.

---

### PC-02 AppConfig.ready bootstrap time pin

**Why:** `MgfCommonConfig.ready()`
([`_apps.py:69-95`](../../src/mgf/django/_apps.py#L69)) runs
once per process at Django app-registry build:

1. Import `mgf.common._lifecycle.bootstrap` (likely
   already-loaded).
2. Acquire `_BOOTSTRAP_LOCK`.
3. Read Django settings (two `getattr` calls).
4. Build `ExitStack`.
5. Call `bootstrap(app_name=..., app_version=...)` — this
   transitively touches: redactor patterns, JSON formatter,
   stderr handler, rotating-file handler, OTel setup,
   excepthook installers.
6. Register `atexit` hook.

mgf-common PC-01 closed `import mgf.common` to ~100 ms; the
bootstrap-side cost is on top of that. For:

- `manage.py runserver` — one-shot at start. Long-lived
  workers; cost amortises.
- `manage.py migrate` — one-shot per command. Each migrate
  pays the bootstrap.
- `pytest-django --reuse-db` — pays per worker on first
  test that triggers app registry.
- `pytest-django` with `override_settings(INSTALLED_APPS=...)` —
  may trigger `ready()` again, but the guard makes it a no-op
  (PC-INV-1).

No measurement today.

**Concrete output:**

- Add `tests/perf/test_bootstrap_time.py`:
  - Fork a fresh Python interpreter (`subprocess`).
  - Invoke `python -c 'import django; from mgf.django.apps import MgfCommonConfig; c = MgfCommonConfig(...); c.ready()'`.
  - Measure wall-clock time.
  - Assert ≤ 500 ms (generous; measure first then pin
    accordingly).
- Document in benchmarks.md.

**Locked in by:** the test pins the cost; a regression in
mgf-common's bootstrap path that wasn't caught upstream
fails here.

**Cross-references:** mgf-common PC-01 (cold-import closed);
mgf-common PC-05 (memory baseline — the bootstrap memory
adds to it).

**Status:** ✅ closed (2026-05-16). See §5.

---

### PC-03 DjangoSettingsBridge.from_django allocation profile

**Why:** `from_django()`
([`_settings.py:71-90`](../../src/mgf/django/_settings.py#L71))
allocates per-call:

- Tuple from `ALLOWED_HOSTS` iterable.
- Tuple from `INSTALLED_APPS` iterable.
- Five getattr reads against `django.conf.settings`.
- Pydantic validation of the resulting fields.
- The bridge instance itself.

For a single-call use case (`mgf-common explain` once at
startup), the cost is negligible. For repeated calls (a
diagnostic endpoint that returns the bridge per request),
the allocation churn matters.

No measurement; not currently a hot path but worth quantifying
before it becomes one.

**Concrete output:**

- Add `tests/perf/test_settings_bridge_construction.py`:
  - `from_django()` × 1000.
  - Measure per-call time.
  - Assert ≤ 1 ms (generous).
- Document baseline in benchmarks.md.

**Locked in by:** the pinned test.

**Cross-references:** mgf-common's `MgfSettings` PC story
(parent's allocation profile).

**Status:** ✅ closed (2026-05-16). See §5.

---

### PC-04 JsonFormatter Redactor construction cost

**Why:** `JsonFormatter.__init__`
([`_logging.py:58-59`](../../src/mgf/django/_logging.py#L58))
calls `Redactor.default()`. Per mgf-common's implementation
this is mostly cached pattern lookup — but each formatter
instance still pays the wrapper-allocation cost.

Django's `LOGGING` dictConfig typically constructs **one**
formatter at startup; cost is one-shot. But:

- pytest-django with `override_settings(LOGGING=...)`
  constructs a new formatter per override.
- A consumer with multiple handlers each using their own
  formatter (admin email handler + console + file)
  multiplies the cost.

Memory per instance: the formatter holds a reference to
the redactor + an empty per-record dict (the `extra`
buffer).

**Concrete output:**

- Add `tests/perf/test_formatter_construction.py`:
  - Build 100 formatters; measure total time + memory delta
    (via `tracemalloc.snapshot`).
  - Assert ≤ 10 µs per instance + ≤ 4 KB per instance.
- Document.

**Locked in by:** pinned test catches regression.

**Cross-references:** mgf-common's Redactor pattern-set
cache; LG-04.

---

### PC-05 Contextvar set/reset overhead pin

**Why:** `_REQUEST_ID_VAR.set(...)` + `.reset(token)` are
the per-request contextvar ops. PEP 567 says sub-microsecond
on CPython 3.11+. But:

- Async ASGI Django apps may have many contextvar transitions
  per request (across `await` boundaries).
- High-throughput apps + the cost compounds.

Worth measuring + pinning a small budget.

**Concrete output:**

- Add `tests/perf/test_contextvar_ops.py`:
  - Loop 100 000 `set/reset` pairs.
  - Assert ≤ 1 µs per pair (generous; measure first).
- Document.

**Locked in by:** pinned test.

**Cross-references:** PEP 567; mgf-fastapi's parallel
test (sister-sibling has the same contextvar).

---

## §5 Closed items

- **PC-01** — Per-request middleware overhead pinned at ≤ 50 µs
  for both `MgfRequestIdMiddleware` and `MgfContextMiddleware`
  (measured ~6 µs and ~3 µs/request respectively, net of
  RequestFactory cost; ~8-15x cushion). Added
  `tests/perf/test_middleware_overhead.py` with two perf-marker
  tests — each warms 200 requests, measures 5000, asserts per-
  request cost < 50 µs against a stub view returning a bare
  `HttpResponse`. Error messages point at the specific middleware
  class. (2026-05-16, commit pending.)
- **PC-02** — `MgfCommonConfig.ready()` bootstrap wall-clock
  pinned at ≤ 500 ms (measured ~174 ms median locally over 5
  fresh-subprocess runs; ~2.9x cushion). Added
  `tests/perf/test_bootstrap_time.py` which spawns 5 fresh
  Python subprocesses, runs `settings.configure` +
  `django.setup()` (which triggers `MgfCommonConfig.ready()`
  via `INSTALLED_APPS`), and asserts the median wall-clock
  under budget. Catches regressions in either mgf.django or
  upstream mgf-common cold-import (see mgf-common PC-01 for
  the cornerstone baseline). (2026-05-16, commit pending.)
- **PC-03** — `DjangoSettingsBridge.from_django()` per-call cost
  pinned at ≤ 1 ms (measured ~175 µs locally; ~5.7x cushion).
  Added `tests/perf/test_settings_bridge_construction.py` with
  one perf-marker test that warms 100 calls, measures 1000, and
  asserts per-call cost < 1 ms. Catches regressions in either
  Django's settings access or pydantic's validation overhead.
  (2026-05-16, commit pending.)

**Infrastructure changes:**

- Added `perf` marker registration + addopts deselect in
  `pyproject.toml` (matches mgf-common's convention — perf pins
  run via `pytest -m perf` for the nightly sweep, not per-PR).
- Added `tests/perf/conftest.py` with a `pytest_configure` hook
  that mirrors `tests/unit/conftest.py`'s Django settings byte-
  identically. Required because both conftests load in the same
  pytest session; whichever fires `pytest_configure` first wins,
  and the second short-circuits on the `configured` guard.
  Drift detection: any settings change to either conftest should
  mirror the other.

---

## §6 Notes on methodology

This first pass walked the five source files (~440 LOC)
against:

- mgf-common's PERFORMANCE_AND_CAPACITY.md methodology +
  measurement discipline.
- `DP-06` (perf as a first-class concern).
- `LG-12` (per-record logging budget).
- `RA-INV-2` (middleware exception-safe — affects measured
  shape only on the happy path).

The audit method:

1. Read each source file end-to-end identifying per-call
   work, per-process work, and allocation patterns.
2. For every concern surfaced, write a falsifiable budget +
   the test shape that pins it.
3. Generous budgets (current × 2-3) — catch order-of-
   magnitude drift, not noise.
4. Cross-reference mgf-common's PC tracker — invariants
   inherited; the sibling-side perf tax is what THIS
   tracker measures.

Five items — all about quantifying current performance with
falsifiable budgets. No items are closed; no perf data yet
exists. The next pass (after the budget tests land) will
have specific numbers to track regression against.

Future passes should look for:

- **Cold-start optimisation.** PC-01 / PC-02 might surface
  improvement opportunities (lazy-loading the redactor,
  deferring atexit registration until first warning).
- **Async-ASGI Django.** When the v0.3+ async middleware
  lands (per RA-05 deferred follow-up), the perf shape
  shifts — `async def __call__` has different overhead
  than the sync wrapper.
- **Multi-worker amortisation.** Bootstrap is one-shot
  per worker; gunicorn / uvicorn pre-forking changes the
  effective cost. Document the operator-side view.

When an item closes, move it to §5 with the closing-commit
SHA + the measured baseline.
