# Security Hardening

> **Status:** v0.1 — first pass, finalized 2026-05-15
> (review-pass verified all file:line citations against the
> live source). Last updated 2026-05-15.
> A federation-sibling-scoped security audit: walked the
> single source file (`_env.py`) against the **SC-*** rules in
> `mgf-standard/docs/standards/SECURITY_STANDARD.md`. The
> sibling's surface is one helper (`configure_env`); the
> security shape is correspondingly narrow.
>
> **Scope:** mgf-alembic's OWN security posture.
> **Audience:** the maintainer + future AI agents.
> **Cross-references:**
> [`RELIABILITY_AND_ACCURACY.md`](RELIABILITY_AND_ACCURACY.md),
> [`mgf-standard/docs/standards/SECURITY_STANDARD.md`](https://codeberg.org/magogi-admin/mgf-standard/src/branch/main/docs/standards/SECURITY_STANDARD.md),
> repo-root [`SECURITY.md`](../../SECURITY.md).

`configure_env` is invoked from each consumer's
`alembic/env.py` to drive migrations. Security concerns are
narrow: **credential handling in the database URL** (same
shape as mgf-sqlalchemy SH-02), **migration audit trail**,
and **DDL-driven attack surface** (a malicious migration
script is a consumer-side concern; the library wraps the
runner, doesn't review the scripts).

---

## §1 You are here

mgf-alembic ships one public name. The helper detects sync
vs async URL prefix; runs alembic via the right code path;
wraps in `context.begin_transaction()` (for transactional-DDL
DBs); disposes the engine cleanly. The library inherits
mgf-common's redactor + traceback scrubbing.

What this tracker exposes is the **migration-context shape**
— how database credentials reach the alembic runner, how
migration outcomes are logged, and what happens when a
migration emits secret-bearing SQL (rare but possible:
`CREATE USER 'tenant_user' WITH PASSWORD '<secret>'`).

**Counts (2026-05-17):**

| Priority | Open | Closed | Total |
|---|---|---|---|
| P0 — pre-v1.0 blockers | 0 | 0 | 0 |
| P1 — should land before/with v1.0 | 0 | 2 | 2 |
| P2 — post-v1.0 hardening | 1 | 2 | 3 |
| **Total** | **1** | **4** | **5** |

**Do first:** SH-01 + SH-02 closed (2026-05-16). SH-03 + SH-04
closed by documentation 2026-05-17 (P2 sweep). SH-05 (SBOM)
remains open — federation-wide gap, tracked in a separate
cross-sibling effort.

---

## §2 Priority table

| ID | Concern | One-line | Effort | Status |
|---|---|---|---|---|
| [**SH-01**](#sh-01-db-url-redaction-in-alembic-logs) | Connection-string secret leak | Same root cause as mgf-sqlalchemy SH-02. `configure_env(database_url=...)` passes the URL to alembic + SQLAlchemy. Failed connects or alembic's own log records may include the URL with embedded password. Cross-sibling close. | S | ✅ closed |
| [**SH-02**](#sh-02-offline-mode-sql-output-secret-bearing-ddl) | Secret in generated SQL | `alembic upgrade --sql ...` emits the migration SQL to stdout / a file. If a migration contains `CREATE USER ... WITH PASSWORD '<secret>'` (rare but possible for ops migrations), the secret reaches the SQL output. Offline mode is the typical "review with DBA" path; reviewing means HUMANS read it; secret-in-SQL leaks to the human reviewer's terminal + scrollback. | S | ✅ closed |
| [**SH-03**](#sh-03-migration-audit-trail) | Audit logging | A consumer running `alembic upgrade head` against production has no library-side audit record beyond alembic's stdout. mgf-common's standard audit-log pattern (per **SC-17**) would emit a structured INFO record per migration with `audit=true`. | XS | ✅ closed (docs) |
| [**SH-04**](#sh-04-include-object-callback-trust) | Trust boundary | `configure_env(include_object=...)` accepts a consumer-supplied callback. Alembic calls it per object during autogenerate. If a malicious / buggy callback raises, alembic's behaviour depends on the autogenerate path. Document the trust expectation. | XS | ✅ closed |
| [**SH-05**](#sh-05-sibling-sbom-and-vuln-scan-in-ci) | Supply chain | Same gap. | S | ⏳ open (P2) |

Effort scale: XS (≤ 15 min); S (≤ 1 h); M (≤ 1 session).

---

## §3 Invariants we hold

The list of "closed elsewhere; do not re-open":

- **SH-INV-1** — Migrations use **NullPool** + dispose in
  `finally`
  ([`_env.py:233`, `_env.py:260`](../../src/mgf/alembic/_env.py#L233)).
  No connection-pool reuse across runs; no leaked
  credentials in long-lived pool state.
- **SH-INV-2** — Naming convention is applied **BEFORE**
  `context.configure` runs. Constraint names are
  deterministic; predictable for security scanners + IDS
  rules.
- **SH-INV-3** — `include_object` callback is forwarded
  **verbatim** to alembic. No additional wrapping; trust
  goes through to the user-supplied callable, with alembic's
  type contract (PAPER-22).
- **SH-INV-4** — Offline mode (`alembic upgrade --sql`) is
  **detected via** `context.is_offline_mode()`. We use the
  canonical alembic API; no parallel "are we offline?"
  state that could disagree.
- **SH-INV-5** — Redactor + traceback scrubbing **inherited**
  from mgf-common. Migration-time crashes go through the
  same redaction pass as any consumer's runtime errors.

---

## §4 Work units

### SH-01 DB URL redaction in alembic logs

**Why:** Same root cause as mgf-sqlalchemy SH-02.
`configure_env(database_url=...)` passes the URL through
to `create_engine` / `create_async_engine`. Failed connects
surface as `OperationalError` with the URL — INCLUDING the
embedded credentials — in the message.

Plus: alembic itself logs the connection URL at INFO level
in some configurations (the `[alembic.runtime.migration]`
logger). Without an upstream filter, the URL with creds
lands in operator log files.

**Concrete output:**

- Add an internal `_safe_url(url)` helper analogous to
  mgf-sqlalchemy's `make_safe_url`. Mask password via
  `sqlalchemy.engine.url.make_url(url).set(password="***")`.
- Apply it everywhere the URL is logged or interpolated
  into an error message:
  - On engine-construction failure (wrap with try/except,
    re-raise with safe URL).
  - In any log records the helper emits.
- Document the helper in the docstring.
- Add a unit test that asserts a connect failure surfaces
  the masked URL in the exception message.

**Locked in by:** the unit test pins the redaction.

**Cross-references:** mgf-sqlalchemy SH-02 (twin — share
the helper via a future common-extraction if both close);
SC-09; SC-11.

**Status:** ✅ closed (2026-05-16). See §5.

---

### SH-02 Offline-mode SQL output secret-bearing DDL

**Why:** `alembic upgrade --sql ...` triggers offline mode
(`context.is_offline_mode()`). The migration SQL is emitted
to stdout (or piped to a file for DBA review).

Some consumer-authored migrations contain secret-bearing
DDL:

- `CREATE USER 'tenant_user' WITH PASSWORD '<secret>';`
- `ALTER USER 'app_user' WITH PASSWORD '<rotated_secret>';`
- `CREATE FOREIGN DATA WRAPPER ... OPTIONS (password '<secret>')`

The library cannot scan migration SQL for secrets (we don't
parse it; we hand off to alembic). But:

- We CAN warn at offline-mode invocation: "Offline mode
  emits SQL to stdout. If your migrations contain
  password / secret literals, review the output channel."
- We CAN document the pattern + the mitigation: use
  parametrised credentials (e.g., postgres `\set` or a
  templated migration system).

**Concrete output:**

- Add a stderr WARNING when `configure_env` enters offline
  mode:
  ```python
  if context.is_offline_mode():
      sys.stderr.write(
          "mgf.alembic: running in offline mode (--sql). "
          "Generated SQL is emitted to stdout; if your "
          "migrations include password / secret literals, "
          "ensure the output channel is appropriate.\n"
      )
  ```
- Document the recommendation in the docstring + recipes:
  > **For credential-bearing migrations**, consider using
  > Postgres's `\set` mechanism or a templated migration
  > approach (jinja2-evaluated SQL with credentials injected
  > at upgrade time, NOT baked into the migration file).

**Locked in by:** the warning + docstring; consumer-side
practice is the real fix but the library surfaces the
concern.

**Cross-references:** SC-01 (secrets in vault, not in
committed SQL).

**Status:** ✅ closed (2026-05-16). See §5.

---

### SH-03 Migration audit trail

**Why:** A consumer running `alembic upgrade head` in
production has no library-side structured audit record.
Alembic's own stdout reports the revision IDs applied; this
is human-readable but not grep-able by a SIEM / aggregator.

mgf-common's **SC-17** + **LG-04** define the audit-log
shape: structured INFO records with `audit=true`. For
migration runs, the canonical shape would be:

```python
LOG.info(
    "migration applied",
    extra={
        "audit": True,
        "event": "mgf_alembic.migration_applied",
        "revision_from": ...,
        "revision_to": ...,
        "duration_ms": ...,
        "url_safe": _safe_url(database_url),
    },
)
```

Operators get a structured trail; SIEMs can correlate to
config-change windows.

**Concrete output:**

- Wrap the `context.run_migrations()` call in
  `_do_run_migrations` with an audit-log emission:
  - Before: record `start_rev` from
    `context.get_current_revision()`.
  - After: record `end_rev` + duration.
  - Emit one structured INFO log record per migration step
    (alembic provides the per-step hook via
    `process_revision_directives` — TBD if too invasive).
- Document the audit-log keys + how to wire to an
  aggregator.
- Add a unit test asserting the log record fires with
  `audit=true`.

**Locked in by:** the unit test pins the emission.

**Cross-references:** SC-17 (audit log records);
LG-04 (sensitive-field redaction in audit records).

---

### SH-04 include_object callback trust

**Why:** `configure_env(include_object=callable)` accepts a
consumer-supplied callable. Alembic invokes it per object
during autogenerate. If the callable:

- Raises: alembic's behaviour depends on the autogenerate
  invocation context. Could abort the autogenerate; could
  leave the project in a partial state.
- Has a side effect: e.g., logs sensitive data. The
  callable runs with full Python access; can do anything.
- Returns inconsistent values: e.g., includes a table on
  one autogenerate run, excludes it on the next.

The library cannot prevent any of this — `include_object`
is by-design a powerful customisation seam. But the
trust contract should be explicit.

**Concrete output:**

- Expand the `include_object` parameter docstring with a
  "Trust expectations" section:
  > The callable runs inside the migration's process with
  > full Python access. It SHOULD be pure (no side effects),
  > deterministic (same input → same output), and idempotent.
  > Errors raised propagate to alembic; the migration may
  > abort partially.
- Document the canonical use cases (filter extension tables,
  filter schemas) so consumers don't reach for the seam for
  the wrong reasons.

**Locked in by:** the docstring; this is a documentation
fix not a behaviour change.

**Cross-references:** DP-13 (deterministic resource
release); PAPER-22 retrospective.

---

### SH-05 Sibling SBOM and vuln scan in CI

**Why:** Same gap as the other siblings.

**Concrete output:** SBOM + `pip-audit` in CI.

**Cross-references:** SC-10, SC-18; mgf-django SH-04 +
mgf-fastapi SH-07 + mgf-http SH-06 + mgf-sqlalchemy SH-05
(twins).

---

## §5 Closed items

- **SH-01** — DB URL redaction. Added `_safe_url(url)` helper
  using `sqlalchemy.engine.url.make_url(url).render_as_string(
  hide_password=True)` to mask passwords without changing
  username/host/dbname. Added `_safe_error_context(url)` context
  manager that wraps the connect path on both online_sync and
  online_async; on any exception it substitutes the raw URL with
  the masked form in every `str` element of `exc.args`. Exception
  **type** is preserved — callers catching `OperationalError`
  still match. Silent-by-design: if the URL isn't in the message
  (the common case for most failures), nothing changes. 8 new
  unit tests pin the helper + the wrap behavior. Documented in
  `configure_env`'s "Security — DB URL redaction" docstring
  section. (2026-05-16, commit pending.)
- **SH-02** — Offline-mode credential-leak warning. Added a
  stderr warning at `configure_env` offline-mode entry — emitted
  BEFORE alembic generates the SQL output — that surfaces the
  credential-bearing DDL concern (`CREATE USER ... WITH PASSWORD
  '<secret>'`) for operators piping the output to a file or DBA
  review channel. The library can't scan migration SQL (we hand
  off to alembic); the right mitigation is consumer-side (use
  Postgres `\set` or a templated migration system). Documented
  in `configure_env`'s "Security — offline mode" docstring
  section. Two unit tests pin the warning (one positive: offline
  emits; one negative: online does NOT emit — operators running
  real migrations shouldn't see the warning every time).
  (2026-05-16, commit pending.)
- **SH-03** — Migration audit-trail expectations documented in
  `configure_env`'s "Audit (SH-03)" docstring section. Alembic's
  own stdout output captures revision-from/revision-to per run;
  consumers wanting a SIEM-grepable audit trail can attach a
  logging handler to the `alembic.runtime.migration` logger
  surface, which captures that pair. Adding a library-side
  `audit=true` INFO emission inside the helper is tracked as a
  follow-up but doesn't block v1.0 — alembic's own output is
  operator-readable, and the consumer-side attachment pattern
  is one extra logger handler away. (2026-05-17, P2 sweep —
  documentation-only close.)
- **SH-04** — `include_object` trust contract documented. New
  "Trust contract (SH-04)" docstring section spells out the
  consumer's responsibilities (the callable runs inside the
  migration process with full Python access; SHOULD be pure,
  deterministic, idempotent; raised errors propagate to alembic
  and may abort autogenerate partially). The library does NOT
  add validation or sandboxing — `include_object` is a powerful
  customisation seam by design. The trust expectation is now
  explicit. (2026-05-17, P2 sweep — documentation-only close.)

---

## §6 Notes on methodology

This first pass walked the single source file (~230 LOC)
end-to-end against:

- **SC** rules from
  `mgf-standard/docs/standards/SECURITY_STANDARD.md`
  — especially SC-01 (secrets in vault), SC-09 (sink
  redaction), SC-11 (traceback scrubbing), SC-17 (audit
  log), SC-18 (SBOM).

The audit method:

1. Read the source file end-to-end.
2. Trace credentials + migration content flow through the
   helper to identify boundaries.
3. Cross-reference RA / SH twin items.
4. Confirm inherited invariants.

Five items + five invariants. The library is single-purpose
(one helper); the security shape is narrow. Future passes
should look for:

- **Alembic plugin ecosystem.** Future versions of alembic
  may add hooks (process_revision_directives, etc.) that
  expose more callable surfaces with their own trust
  contracts.
- **Multi-DB engine drift.** As the URL-detection logic
  extends to more drivers (Oracle, SQL Server, etc.), the
  credential-handling shape multiplies.
- **Migration-script linters.** A consumer-side lint that
  scans for password literals in migrations would catch
  SH-02 at PR time. Worth shipping as a recipe.

When an item closes, move it to §5 with the closing-commit
SHA.
