# Security Hardening

> **Status:** v0.1 — first pass, finalized 2026-05-15
> (review-pass verified all file:line citations against the
> live source). Last updated 2026-05-15.
> A federation-sibling-scoped security audit: walked the three
> source files (`_client.py`, `_retry.py`, `_errors.py`) against
> the **SC-*** rules in `mgf-standard/docs/standards/SECURITY_STANDARD.md`
> plus the outbound-HTTP security concerns (TLS, redaction,
> header-value sanitization, certificate handling).
>
> **Scope:** mgf-http's OWN security posture — the outbound
> HTTP client wrapper. NOT a re-audit of mgf-common's redactor
> / vault / crash reporter. NOT an audit of httpx / httpcore —
> they have their own upstream security processes.
>
> **Audience:** the maintainer + future AI agents.
> **Cross-references:**
> [`RELIABILITY_AND_ACCURACY.md`](RELIABILITY_AND_ACCURACY.md)
> (sibling tracker),
> [`mgf-standard/docs/standards/SECURITY_STANDARD.md`](https://codeberg.org/magogi-admin/mgf-standard/src/branch/main/docs/standards/SECURITY_STANDARD.md),
> repo-root [`SECURITY.md`](../../SECURITY.md).

The sibling is the outbound HTTP client every consumer uses
to talk to third-party APIs. Security here covers two things:
**outbound secret handling** (Authorization, API-key, cookie
headers) and **upstream-trust** (TLS verification, cert
expiry, redirect chasing into different domains).

---

## §1 You are here

mgf-http ships three public names; TLS verification is
default-on with a `UserWarning` on opt-out; secret-shaped
headers are redacted in log records via a configurable set;
exponential-with-full-jitter backoff per EH-11; idempotent-
methods-only retry per EH-12. The security primitives are
shipped correctly out of the box.

What this tracker exposes is the **edge of safe defaults**
— what happens at the boundaries: when `verify_tls=False` is
set, when cookies contain control characters, when the redirect
chain crosses domains, when a future exception class leaks
sensitive data through `str(exc)`.

**Counts (2026-05-17):**

| Priority | Open | Closed | Total |
|---|---|---|---|
| P0 — pre-v1.0 blockers | 0 | 0 | 0 |
| P1 — should land before/with v1.0 | 0 | 3 | 3 |
| P2 — post-v1.0 hardening | 0 | 1 | 1 |
| P2 — deferred to v1.1+ | 2 | 0 | 2 |
| **Total** | **2** | **4** | **6** |

**Do first:** P2 sweep complete (2026-05-17). SH-04 closed
(twin of RA-04); SH-05 + SH-06 deferred to v1.1+ — see entries
below for rationale. All v1.0-blocking security items now
closed.

---

## §2 Priority table

| ID | Concern | One-line | Effort | Status |
|---|---|---|---|---|
| [**SH-01**](#sh-01-cookie-value-sanitization) | Header injection | Per-call `cookies=` values interpolated into the `Cookie:` header without `\r\n` validation. httpx ≥ 0.27 rejects at transport; older versions might not. Defence-in-depth at the boundary. Twin of [RA-03](RELIABILITY_AND_ACCURACY.md#ra-03-cookie-value-sanitization). | XS | ✅ closed |
| [**SH-02**](#sh-02-cross-domain-redirect-chase) | Auth-leak via redirect | `follow_redirects=True` default with `Authorization` / `Cookie` headers attached can leak credentials to an attacker-controlled host via a 302 to a different domain. httpx by default DOES strip Auth on cross-domain redirects, but the behaviour isn't audited by us. Verify + document. | S | ✅ closed |
| [**SH-03**](#sh-03-tls-disable-ci-lint) | Mis-configuration class | `verify_tls=False` emits `UserWarning` once per process. A consumer's CI that suppresses UserWarnings (some default pytest configs do) loses the diagnostic. Surface the disabled-TLS state via a structured log line + lintable substring. | S | ✅ closed |
| [**SH-04**](#sh-04-cause-summary-secret-leak) | Info disclosure | `_categorise_exc` interpolates `str(exc)` from httpx errors into `cause_summary`. The string ends up in `HttpTransportError`'s message + consumer logs. A URL with a query-string-embedded credential (`?api_key=...`) could surface. Twin of [RA-04](RELIABILITY_AND_ACCURACY.md#ra-04-cause-summary-leaks-httpx-exception-text). | S | ✅ closed |
| [**SH-05**](#sh-05-certificate-expiry-pinning) | Cert-handling robustness | mgf-http honours httpx's certificate verification; no extra cert-pinning. For a high-stakes consumer (auth provider, payment processor), public-key pinning would harden against CA compromise. Document the consumer-side recipe. | S | ⏳ v1.1+ — needs real consumer trigger |
| [**SH-06**](#sh-06-sibling-sbom-and-vuln-scan-in-ci) | Supply chain | Same gap as the other siblings. SBOM + `pip-audit` CI gate. | S | ⏳ v1.1+ — federation-wide SBOM workflow tracked separately |

Effort scale: XS (≤ 15 min); S (≤ 1 h); M (≤ 1 session).

---

## §3 Invariants we hold

The list of "closed elsewhere; do not re-open":

- **SH-INV-1** — TLS verification is **default-on**. Opt-out
  emits a `UserWarning` once per process
  ([`_client.py:94-114`](../../src/mgf/http/_client.py#L94)).
  Per SC-10 + the "TLS-default-on" pattern.
- **SH-INV-2** — Secret-shaped request headers are
  **redacted** in log lines. The redaction set covers
  `Authorization` / `Cookie` / `Set-Cookie` / `X-Api-Key` /
  `X-Auth-Token` / `X-Secret` / `Proxy-Authorization`
  ([`_client.py:50-61`](../../src/mgf/http/_client.py#L50)).
  Per LG-04 + SC-09.
- **SH-INV-3** — POST is **excluded** from default
  `retryable_methods`. Per EH-12 idempotency rule. Prevents
  retried POST from creating duplicate side-effects (also
  a security concern: duplicate-charge, duplicate-send via
  retry).
- **SH-INV-4** — `HttpTransportError` preserves the original
  httpx exception as `__cause__`
  ([`_client.py:341 + :369`](../../src/mgf/http/_client.py#L341)).
  The traceback chain stays intact for the redactor /
  crash-reporter pipeline to scrub.
- **SH-INV-5** — User-Agent identifies the calling app +
  library version
  ([`_client.py:381`](../../src/mgf/http/_client.py#L381)).
  Upstream provider logs can attribute traffic; CVE-style
  rapid-response notifications can target affected versions.
- **SH-INV-6** — Backoff sleep uses `random.random()` for
  jitter, NOT cryptographically random. Documented +
  acceptable (jitter is a load-shedding hint, not a security
  primitive). The noqa on `S311` at
  [`_retry.py:113`](../../src/mgf/http/_retry.py#L113)
  records the choice.

---

## §4 Work units

### SH-01 Cookie value sanitization

**Why:** `_send_with_retries` builds the Cookie header
([`_client.py:282-292`](../../src/mgf/http/_client.py#L282)):

```python
if cookies:
    cookie_header = "; ".join(f"{k}={v}" for k, v in cookies.items())
    request_headers["Cookie"] = cookie_header
```

A consumer passing `cookies={"sess": "abc\r\nX-Inject: yes"}`
gets that value into a header. httpx ≥ 0.27 with h11 ≥ 0.14
rejects via `LocalProtocolError`; older versions might not.

Header injection on OUTBOUND requests is less critical than
on inbound (we're attacking ourselves, not exposing a server
to injection). But the injected fields could:

- Leak the actual auth header (a follow-up `Authorization:
  ...` from the injection) to a man-in-the-middle.
- Confuse the upstream's request parser (some servers
  accept multiple headers; ambiguity).

**Concrete output:** Same fix as RA-03. Validate at the
boundary; reject `\r\n\0` in cookie values.

**Locked in by:** the unit test in `tests/unit/test_client_cookies.py`.

**Cross-references:** RA-03 (twin); SC-09 (sink-side
redaction principle generalises).

---

### SH-02 Cross-domain redirect chase

**Why:** `AsyncHttpClient.__init__`
([`_client.py:131`](../../src/mgf/http/_client.py#L131)) sets
`follow_redirects=True`. With `Authorization` header
attached to the request, what happens on a 302 to a
different domain?

httpx's documented behaviour: **strips `Authorization` (and
`Proxy-Authorization`) headers on cross-origin redirects**
since httpx 0.20. Verification needed:

- Confirm the strip behaviour holds in the httpx version
  pinned by mgf-http.
- Confirm `Cookie` headers behave correctly (httpx defers
  to cookie domain attributes; if the cookie was set
  without a domain, it shouldn't follow to a different
  host).
- Custom-named auth headers (`X-Api-Key`, etc.) are NOT
  stripped — httpx doesn't know they're secret.

For a consumer using a custom auth header, the cross-domain
redirect is a real leak risk.

**Concrete output:**

- A unit test that:
  - Builds an `AsyncHttpClient`.
  - Mocks a 302 redirect from `example.com` to `attacker.com`.
  - Sends with `Authorization: Bearer secret` AND
    `X-Api-Key: secret`.
  - Asserts:
    - `Authorization` is stripped at the redirect (httpx's
      contract).
    - `X-Api-Key` is leaked to attacker.com — **OR** we
      tighten our wrapper to strip ALL secret-shaped headers
      on cross-origin redirects.
- Document the behaviour in the constructor docstring:
  > **Cross-origin redirects:** httpx strips `Authorization`
  > / `Proxy-Authorization` on redirects to a different
  > origin. Custom auth headers (`X-Api-Key`, etc.) are NOT
  > stripped. For high-stakes endpoints, set
  > `follow_redirects=False` (planned per RA-05) or limit
  > the redirect chain to known origins.
- Pair with [RA-05](RELIABILITY_AND_ACCURACY.md#ra-05-follow-redirects-true-default-open-redirect-class)'s
  fix (`follow_redirects` kwarg).

**Locked in by:** the unit test pins the strip behaviour;
the docstring sets consumer expectations.

**Cross-references:** RA-05 (the broader redirect-policy
opening); SC-09; httpx's cross-origin redirect handling
docs.

---

### SH-03 TLS-disable CI lint

**Why:** `_client.py:101-114` emits a `UserWarning` once per
process when `verify_tls=False`. The warning is the audit
trail. But:

- Many pytest setups suppress UserWarnings.
- A consumer's CI that uses
  `pyproject.toml::tool.pytest.ini_options.filterwarnings`
  with `"ignore::UserWarning"` loses the signal.
- The warning fires ONCE per process — by the time someone
  reviews logs, it may have rolled off.

A more durable shape:

- A structured INFO log line at construction (always, not
  one-shot) that operators can grep for.
- An audit event with `audit=true` extra-field per SC-17.

**Concrete output:**

- Add a structured log line every time `verify_tls=False`
  is set:
  ```python
  if not verify_tls:
      _LOG.warning(
          "AsyncHttpClient: verify_tls=False",
          extra={
              "audit": True,
              "event": "mgf_http.tls_disabled",
              "trace_id": current_request_id() or "",
          },
      )
  ```
- Keep the `UserWarning` for backward compatibility (some
  consumers' CI flags it as a test failure — wanted).
- Update the docstring to mention both surfaces (warning +
  structured log).

**Locked in by:** a unit test asserting both surfaces fire
on construction with `verify_tls=False`.

**Cross-references:** SC-10 (vuln + cert posture), SC-17
(audit log records).

---

### SH-04 cause_summary secret leak

**Why:** Same defect as RA-04. `_categorise_exc`
([`_client.py:443-451`](../../src/mgf/http/_client.py#L443))
interpolates `str(exc)` from httpx errors:

```python
def _categorise_exc(exc):
    if isinstance(exc, httpx.TimeoutException):
        return f"timeout: {exc}"
    ...
    return f"{type(exc).__name__}: {exc}"
```

A URL like `https://api.example.com/v1/users?api_key=secret123`
in a `httpx.ConnectError`'s repr could surface in the
`cause_summary`. The string ends up in
`HttpTransportError.message`.

The RA framing is "information completeness for logs"; the
SH framing is "secret material leak via exception path."

**Concrete output:** Same fix as RA-04. Category-name-only
summary + optional detail field. Pair the close.

**Locked in by:** the unit tests cover both framings.

**Cross-references:** RA-04 (twin); SC-11 (traceback
scrubbing — same principle on outbound exception path).

---

### SH-05 Certificate expiry pinning

**Why:** mgf-http relies on httpx's TLS-verification via
the system trust store. This catches:

- Expired certs (the OS refuses them).
- Self-signed certs (unless `verify_tls=False`).
- Cert chain mismatches.

It does NOT catch:

- A compromised CA issuing a fraudulent cert for the target
  host (CA-compromise attack).
- A consumer-side trust-store mutation (an attacker
  installing their root CA into the OS).

For high-stakes consumers (auth providers, payment
processors, customer-data endpoints), public-key pinning
adds a defence layer:

- Pin the expected public-key hash at construction.
- httpx's `httpx.Client(cert=...)` + a custom transport can
  enforce.

This is overkill for casual consumers; useful for the
security-sensitive subset.

**Concrete output:**

- Add a recipe in `docs/recipes/`:
  > **How to pin a public key for high-stakes endpoints.**
  > Steps to extract the SPKI hash + configure httpx's
  > transport to enforce.
- Document the consumer-side CA-rotation procedure (every
  N years; matches Let's Encrypt + cloud-managed cert
  rotation cadence).
- Optionally: add an opt-in kwarg `expected_spki_hash=`
  to `AsyncHttpClient.__init__` that wraps httpx's verify
  with the pin check. Defer to v0.3+.

**Locked in by:** the recipe + optional kwarg. Consumer-side
adoption is the real verification.

**Cross-references:** SC-10 (cert posture), DEP-09 (TLS as
managed infra — the consumer's deployment side).

---

### SH-06 Sibling SBOM and vuln scan in CI

**Why:** Same gap as mgf-django SH-04 + mgf-fastapi SH-07.

**Concrete output:** Same shape — CycloneDX SBOM at release;
`pip-audit --strict` in CI.

**Locked in by:** the CI step.

**Cross-references:** SC-10, SC-18; mgf-django SH-04 +
mgf-fastapi SH-07 (twins).

---

## §5 Closed items

- **SH-01** — Cookie value CR/LF/NUL guard. Validate each cookie
  value before interpolating into the `Cookie:` header (in
  `_send_with_retries`). Reject with `HttpTransportError` —
  same shape every other transport failure uses, so the suite's
  error path handles it uniformly. Defence-in-depth against
  outbound header-injection. httpx ≥ 0.27 catches some of this
  at the transport layer; failing here makes the error
  attributable to OUR code path. Twin of RA-03 — same bytes,
  two trackers. 4 new unit tests pin the rejection (CRLF, LF,
  NUL) + the clean-value passthrough. (2026-05-16, commit pending.)
- **SH-02** — Cross-domain redirect documentation. Expanded
  `AsyncHttpClient`'s class docstring with a new "Security —
  cross-origin redirects (SH-02)" section that captures httpx's
  documented strip behaviour (`Authorization` / `Proxy-Authorization`
  stripped on cross-origin redirects since httpx 0.20) AND the
  gap (custom auth headers like `X-Api-Key` are NOT stripped).
  Lists three mitigations: don't follow on high-stakes calls,
  restrict redirect origins, prefer standard `Authorization`.
  Cross-references RA-05 for the upcoming `follow_redirects=`
  kwarg. No behaviour change — the docs are the deliverable
  because the strip behaviour is correct; consumers needed to
  see the limit. (2026-05-16, commit pending.)
- **SH-03** — TLS-disable structured WARNING log. Added a
  `_LOG.warning(..., extra={"audit": True, "event":
  "mgf_http.tls_disabled"})` emission alongside the existing
  one-shot `UserWarning` in `AsyncHttpClient.__init__`. The
  UserWarning is fragile (pytest configs that suppress
  UserWarning lose the signal); the structured log fires on
  EVERY construction (not once-per-process) so the audit
  trail is durable, and SIEMs filter via `audit=True`. Safe
  default (`verify_tls=True`) is silent — operators only see
  the warning when they take the risky shape, so they don't
  learn to ignore it. 2 new tests pin the emission on True
  + silence on default. (2026-05-16, commit pending.)
- **SH-04** — `cause_summary` is now category-only (twin of
  RA-04). `_categorise_exc` no longer interpolates `str(exc)`
  from httpx exceptions; returns a fixed category string per
  known class (`"timeout"`, `"connect-error"`, `"network-error"`)
  and `type(exc).__name__` for the catch-all. The non-retryable
  `httpx.HTTPError` branch in `_send_with_retries` got the same
  treatment. The risk closed: an httpx exception whose `__str__`
  carries the request URL (which can carry a query-string-
  embedded credential like `?api_key=secret`) would otherwise
  land in the user-facing `HttpTransportError.message` →
  consumer logs / CLI output / response bodies. The original
  exception is still preserved as `__cause__` so the
  debugger / crash-reporter chain (which has its own Redactor
  pass) stays intact. 4 new tests use a deliberately leaky
  string as the httpx exception message and assert it does NOT
  appear in `cause_summary` or `str(exc_info.value)`. See RA-04
  in `RELIABILITY_AND_ACCURACY.md` for the dual-framed defect.
  (2026-05-17, commit pending.)

### Deferred to v1.1+

- **SH-05** — Certificate pinning recipe + optional kwarg.
  Deferred to v1.1+: the existing TLS posture (default-on
  verification, opt-out with WARNING + structured audit log)
  is sufficient for the typical consumer; SPKI pinning is a
  high-stakes feature that needs a real consumer trigger
  (auth-provider integration, payment-processor integration)
  to justify the kwarg shape + recipe-doc surface area.
  Federation has no such consumer yet. Re-evaluate when the
  first high-stakes consumer surfaces. (2026-05-17.)
- **SH-06** — Sibling SBOM + `pip-audit` CI gate. Deferred to
  v1.1+: federation-wide SBOM workflow is tracked separately —
  this gap is blocked by a federation-wide CI strategy, not
  by anything specific to mgf-http. Twin items in mgf-django
  / mgf-fastapi / mgf-common are similarly deferred. The
  federation-wide rollout will close all four in lockstep.
  (2026-05-17.)

---

## §6 Notes on methodology

This first pass walked the three source files (~430 LOC)
end-to-end against:

- **SC** rules from
  `mgf-standard/docs/standards/SECURITY_STANDARD.md`
  — especially SC-09 (sink redaction), SC-10 (vuln scan +
  cert posture), SC-11 (traceback scrubbing), SC-17 (audit
  log), SC-18 (SBOM).
- **EH-12** idempotency boundary — the retryable-methods
  rule has a security dimension (retried POST = duplicate
  side-effects, potentially security-relevant for
  payment / send operations).
- mgf-common's own SECURITY_HARDENING.md as the methodology
  template.

The audit method:

1. Read each source file end-to-end with the SC rule set in
   mind.
2. For every concern surfaced, verify `file:line` against the
   live source.
3. Cross-reference against the sibling's RA tracker —
   reframe shared items in the SH lens.
4. Cross-reference against the cornerstone's SH tracker —
   confirm inherited invariants.
5. Cross-reference against sister-sibling trackers —
   identify cross-sibling patterns.

Six items + six invariants. The library is focused; the
security shape reflects that. Future passes should look for:

- **httpx security advisories.** httpx 1.0 ships with
  stricter semantics + new attack-surface guards. Bump
  cautiously + re-audit.
- **HTTP/3 + QUIC**. As Python's HTTP ecosystem adds these
  transports, the audit shape changes — new TLS handshake
  shape, different connection-reuse semantics.
- **Provider-specific quirks**. Stripe / GitHub / Clerk
  each have rate-limit + retry-header conventions. As the
  consumer set grows, the security audit corpus should
  cover provider-specific edges.
- **Circuit-breaker integration** (EH-13). Currently parked
  on the watch list; if it ships, adds another security
  control (fail-open vs fail-closed under attack).

When an item closes, move it to §5 with the closing-commit
SHA.
