# `mgf-common` — library overview

The shared ground every component doc in this directory builds
on. Read this once; the per-component docs assume it.

`mgf-common` is the cornerstone library for the MGF project
family. **One typed, opt-in observability + configuration stack**
— typed exceptions, central configuration, structured logging,
OpenTelemetry tracing, credential vault, CLI helpers — so every
project in the family inherits the same shape instead of
re-inventing it.

The library's competitive advantage is the **closed-box
interface**: consumers get full power without bypassing to
underlying layers (`pydantic`, `opentelemetry`, stdlib
`logging`). Every capability is exposed through `mgf.common.*`;
consumers should almost never need `import pydantic` or
`import opentelemetry` for normal use.

---

## Install

```bash
# Core — slim install, no optional sinks (pulls only pydantic +
# pydantic-settings + pyyaml — three deps total).
pip install mgf-common

# Tier C observability — OpenTelemetry SDK + Sentry SDK +
# httpx instrumentation.
pip install 'mgf-common[observability]'

# Vault backends (system keyring + encrypted file).
pip install 'mgf-common[vault]'

# Federation siblings (extracted from mgf-common over the v0.28 -> v1.0 split):
pip install 'mgf-fastapi'                 # lifespan + middleware + Depends helpers (v0.28+)
pip install 'mgf-fastapi[testing]'        # + uvicorn + httpx for run_test_app
pip install 'mgf-fastapi[sqlalchemy]'     # + mgf-sqlalchemy for get_session/setup_db (v0.2+)
pip install 'mgf-http'                    # typed async httpx wrapper (v0.29+)
pip install 'mgf-http[test]'              # + respx for mocking httpx in tests
pip install 'mgf-sqlalchemy'              # async SQLAlchemy engine + sessionmaker + tenant_session (v0.30+)
pip install 'mgf-sqlalchemy[test]'        # + aiosqlite for in-memory tests
pip install 'mgf-alembic'                 # async-aware alembic env.py helper (v0.30+)
pip install 'mgf-alembic[test]'           # + aiosqlite for async-driver round-trip tests
pip install 'mgf-django'                  # AppConfig + middleware + JsonFormatter + settings bridge (v0.31+)

# Combine as needed:
pip install 'mgf-common[observability]' mgf-fastapi mgf-sqlalchemy mgf-alembic mgf-http mgf-django
```

Requires **Python 3.11+**. No system-package dependencies —
installs cleanly into any virtualenv.

Every adapter is **opt-in via extra**. Bare `pip install mgf-common`
adds three transitive deps. The library never reaches past its public
surface — `import pydantic` / `import opentelemetry` / `import
fastapi` from consumer code is the rare exception, not the rule.

---

## Package layout

```
mgf.common
│
├── Core (no extras required)
│ ├── (top-level) bootstrap, AppContext, current_context,
│ │ app_name, app_version, bootstrap_for_test
│ ├── exceptions typed hierarchy: AppError + 7 categories + 7 generics
│ ├── config BaseAppSettings + atomic load/save + path helpers
│ ├── settings MgfSettings: the library's own typed config
│ ├── observability logging + OTel + Redactor + JsonFormatter +
│ │ AsyncJsonHandler + crash reporter + excepthooks
│ ├── typing 4 runtime_checkable Protocols
│ ├── progress CancellationToken + ProgressReporter (+ asyncio sibling)
│ ├── vault EnvVault / KeyringVault / EncryptedFileVault + registry
│ ├── cli exit codes + translator + subcommand registry +
│ │ diagnostic CLI (explain / validate / doctor /
│ │ migrate / bench)
│ ├── providers ProviderABC + @translate_at_seam ()
│ ├── pytest_plugin auto-registered fixtures (mgf_settings, mgf_context,
│ │ redaction_recorder, …)
│ └── registry generic Registry[F] primitive (extension point)
│
└── (no service-shape adapters — they all live in sibling
    packages now; see "Federation siblings" below)

# Federation siblings (extracted from mgf-common over v0.28 -> v0.31):
mgf.fastapi    (mgf-fastapi)            lifespan + middleware + Depends + webhooks (v0.28+)
mgf.fastapi.db (mgf-fastapi[sqlalchemy]) get_session + setup_db (v0.2+)
mgf.http       (mgf-http)               AsyncHttpClient + RetryPolicy + HttpTransportError (v0.29+)
mgf.sqlalchemy (mgf-sqlalchemy)         create_engine + create_sessionmaker + tenant_session (v0.30+)
mgf.alembic    (mgf-alembic)            configure_env (v0.30+)
mgf.django     (mgf-django)             MgfCommonConfig + middleware + JsonFormatter + settings bridge (v0.31+)
```

Every adapter under "Service-shape" raises `ImportError` at
import time when its extra isn't installed — the message names
the missing extra so the fix is obvious. The closed-box rule
applies uniformly: consumers MUST import from
`mgf.common.<adapter>`, never reach into `mgf.common.<adapter>._foo`
(and `mgf.fastapi.X`, never `mgf.fastapi._foo`, in the sibling).

The library is a [PEP 420 namespace
package](https://peps.python.org/pep-0420/): no `__init__.py` at
`src/mgf/`, so federation siblings (e.g., `mgf.apiprobe`) can
contribute their own subpackages under the same `mgf.*` umbrella
without touching this leaf.

---

## Three integration shapes — pick yours

The most common decision when adopting the library: how do I
integrate? Three shapes, distinct postures.

```
Are you building a top-level application
(an executable or a service)?
│
├─ Yes ─► §"I'm an app" — bootstrap()
│
└─ No
   │
   Are you a library that other apps depend on?
   │
   ├─ Yes ─► §"I'm a library"  — get_logger / operation_span; don't bootstrap
   │
   └─ No (you're writing tests)
      │
      └─►  §"I'm a test"       — bootstrap_for_test()
```

### §"I'm an app"

Top-level executable / service. You own the process lifecycle.

```python
import mgf.common

def main() -> int:
    with mgf.common.bootstrap() as ctx:
        # ... your app's logic
        return 0

if __name__ == "__main__":
    raise SystemExit(main())
```

`bootstrap()` with no args auto-derives identity from the
launching package's distribution metadata (works for `python -m
mypackage` and console-script entries). Pass identity explicitly
for ad-hoc scripts:

```python
with mgf.common.bootstrap(app_name="myapp", app_version="1.0") as ctx:
    ...
```

For more shape, see [`01_lifecycle.md`](lifecycle.md).

### §"I'm a library"

Your package is imported INTO an app process; you don't own the
process. **Don't** call `bootstrap()` — the host owns identity /
logging / OTel wiring.

```python
from mgf.common.observability import get_logger, operation_span

LOG = get_logger(__name__)

def fetch_user(user_id: str) -> dict:
    with operation_span("my_library.fetch_user", user_id=user_id):
        LOG.info("fetching user %s", user_id)
        # ... real work ...
```

`operation_span` is no-op-safe when no host has bootstrapped;
your library doesn't crash and doesn't reach past the public
surface. See [`05_observability.md`](observability.md).

### §"I'm a test"

```python
import pytest
from mgf.common import bootstrap_for_test, AppContext

@pytest.fixture
def app_ctx() -> AppContext:
    with bootstrap_for_test() as ctx:
        yield ctx

def test_my_thing(app_ctx: AppContext) -> None:
    # ... ctx.settings is MgfSettings.testing() ...
    ...
```

`bootstrap_for_test()` uses test-friendly defaults (WARNING-level
logging, OTel off, deterministic preset). Pass
`capture_logs=True` / `capture_spans=True` to assert on emitted
records. See [`01_lifecycle.md`](lifecycle.md) §"Test bootstrap".

---

## Configuration sources (precedence)

`MgfSettings` reads from five sources, highest precedence first:

1. **Explicit constructor kwargs** — `MgfSettings(logging=...)`
2. **`MGF_*` environment variables** — both
   `MGF_LOGGING_LEVEL=DEBUG` (single underscore, +) and
   `MGF_LOGGING__LEVEL=DEBUG` (double, legacy) work
3. **`[tool.mgf]` in `pyproject.toml`** — walks up from CWD; kill
   switch `MGF_DISABLE_PYPROJECT=1`
4. **`.env` file** — if explicitly configured
5. **Field defaults** — declared on the sub-models

The `preset` field (`MgfSettings.preset`) layers on top of all
these as a final pass — preset values fill in fields no source
supplied. See [`04_settings.md`](settings.md).

---

## Best practices

- **Bootstrap once per entry point.** App-level code calls
  `bootstrap()` exactly once. Library-level code never bootstraps.
- **Compose, don't subclass `MgfSettings`.** Your app's settings
  carry an `mgf: MgfSettings` field. Pattern in
  [`04_settings.md`](settings.md) §"Composing with your app".
- **Subclass typed exceptions**, don't invent ad-hoc strings.
  Pattern in [`02_exceptions.md`](exceptions.md).
- **Use `operation_span` even when OTel is disabled.** It's
  no-op-safe; the spans get exported when OTel is enabled later
  without code changes.
- **Tests MUST be deterministic.** `bootstrap_for_test()` +
  `MGF_DISABLE_PYPROJECT=1` autouse fixture keeps host-shell env
  out of test runs.
- **Don't reach into `mgf.common._*` modules.** Underscore-prefix
  is private; the closed-box interface is the contract. The
  diagnostic CLI's `mgf-fed app-doctor` prints a deprecation hint
  if any -shape API leaks back in.

---

## Where to look next

- **Component-level how-to:** [`01_lifecycle.md`](lifecycle.md)
  through [`10_full_app.md`](full_app.md) in this directory.
- **Recipes directory** (task-shaped how-to):
  [`recipes/`](../recipes/) — one file per integration shape:
  - [`recipes/cli.md`](../recipes/cli.md) — building CLIs.
  - [`recipes/script.md`](../recipes/script.md) — one-off scripts.
  - [`recipes/testing.md`](../recipes/testing.md) — pytest fixtures.
  - `recipes/http.md` (sibling: `mgf-http/docs/recipes/http.md`) — `AsyncHttpClient` + migration recipe.
  - `recipes/fastapi.md` (sibling: `mgf-fastapi/docs/recipes/fastapi.md`) — FastAPI services.
  - `recipes/sqlalchemy.md` (sibling: `mgf-sqlalchemy/docs/recipes/sqlalchemy.md`) — async SQLAlchemy + RLS.
  - `recipes/alembic.md` (sibling: `mgf-alembic/docs/recipes/alembic.md`) — env.py one-liner.
  - `recipes/django.md` (sibling: `mgf-django/docs/recipes/django.md`) — Django integration.
  - [`recipes/providers.md`](../recipes/providers.md) — provider seams.
  - [`recipes/multi_tenant_settings.md`](../recipes/multi_tenant_settings.md) — per-tenant config (recipe-as-primitive; promotion criteria in [`docs/internal/lift_checklist.md`](../internal/lift_checklist.md)).
- **Public-API contract:** [`PUBLIC_API.md`](../../PUBLIC_API.md)
  — every public name with stability tier.
- **Engineering standards** (cross-project):
  run `mgf-fed standards <TOPIC>` — what every consuming
  project commits to.
- **Performance:**
  [`docs/ops/benchmarks.md`](../ops/benchmarks.md) —
  microbench numbers tracked across releases. Reproduce with
  `mgf-fed bench`.
- **Migrating from ?** +
  `mgf-fed migrate` codemod.
- **Reference consumer:**
  [`example_app/`](../../example_app/) — a real packaged
  consumer demonstrating composition + custom CLI subcommand +
  `bootstrap_for_test`.
- **Layered runnable examples:**
  [`examples/`](../../examples/) — six topics × three layers
  (simple → practical → advanced); the smoke test at
  `tests/unit/test_examples.py` keeps them from drifting.

---

## Appendix — cross-reference graph

The reference graph between docs in this directory plus the
cross-cutting docs they reach out to. Single source of truth;
update this table when adding a link rather than per-doc
appendix tables (which rot).

| Doc | References (out) | Referenced by (in) |
|---|---|---|
| **library_common.md** (this) | every per-component doc; PUBLIC_API.md; standards/; MIGRATION_0.1_TO_0.3.md; example_app/; examples/ | every per-component doc (as the "common ground") |
| **01_lifecycle.md** | library_common; 04_settings; 05_observability; 10_full_app | 02–10 (assumes lifecycle for any code that bootstraps) |
| **02_exceptions.md** | library_common; PUBLIC_API.md; standards/ERROR_HANDLING.md | 01_lifecycle (`BootstrapError`); 06_crash_reporting; 09_cli (exit-code map) |
| **03_config.md** | library_common; 04_settings; standards/CONFIGURATION.md | 04_settings (BaseAppSettings is the base) |
| **04_settings.md** | library_common; 03_config; 01_lifecycle; standards/CONFIGURATION.md; examples/02_config | 01_lifecycle (`bootstrap(settings=...)`); 05_observability (LoggingSettings); 07_vault (VaultSettings); 09_cli (`mgf-fed explain`) |
| **05_observability.md** | library_common; 04_settings; 02_exceptions; standards/LOGGING.md; standards/DESIGN_PRINCIPLES.md (DP-12); examples/03_logging; examples/04_observability | 06_crash_reporting; 10_full_app |
| **06_crash_reporting.md** | library_common; 02_exceptions; 05_observability; 09_cli (registry pattern); examples/05_crash_reporting | 01_lifecycle (excepthooks); 10_full_app |
| **07_vault.md** | library_common; 04_settings; standards/SECURITY_STANDARD.md (SC-01, SC-13, SC-17) | 04_settings (`VaultSettings`) |
| **08_progress.md** | library_common; 02_exceptions (`CancellationError`); standards/DESIGN_PRINCIPLES.md (DP-09, DP-11) | 09_cli (translator handles `CancellationError`) |
| **09_cli.md** | library_common; 02_exceptions; 04_settings (`mgf-fed explain` reads MgfSettings); examples (none yet) | 10_full_app (custom subcommands) |
| **10_full_app.md** | every other doc + example_app/ + examples/06_full_app | — (terminal node; the integration) |

External (not in this dir):
- [`PUBLIC_API.md`](../../PUBLIC_API.md) — API contract; every component doc links to relevant rows here.
- [`mgf-standard/docs/standards/`](https://codeberg.org/magogi-admin/mgf-standard/src/branch/main/docs/standards/) — engineering bar; component docs reference the relevant topic doc.
- [`docs/claude/CLAUDE.md`](../claude/CLAUDE.md) — Claude session rules; not user-facing.
- — version-upgrade recipe.
- [`docs/internal/lift_checklist.md`](../internal/lift_checklist.md) — contributor recipe for adding components.
