Metadata-Version: 2.4
Name: mgf-logship
Version: 0.1.8
Summary: Centralized log shipping for mgf-common consumers — a fail-open logging handler that batches canonical LogRecord JSON + statistics and POSTs them to a collector over a versioned, language-neutral wire protocol, plus a store-now stub collector. Sibling of mgf-common under the mgf.* namespace.
Project-URL: Homepage, https://codeberg.org/magogi-admin/mgf-logship
Project-URL: Issues, https://codeberg.org/magogi-admin/mgf-logship/issues
Project-URL: Changelog, https://codeberg.org/magogi-admin/mgf-logship/src/branch/main/CHANGELOG.md
Author: Bassam Alsanie, mgf-logship contributors
License: MIT
License-File: LICENSE
Keywords: centralized-logs,log-shipping,logging,ndjson,observability
Classifier: Development Status :: 3 - Alpha
Classifier: Intended Audience :: Developers
Classifier: License :: OSI Approved :: MIT License
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.11
Classifier: Programming Language :: Python :: 3.12
Classifier: Programming Language :: Python :: 3.13
Classifier: Topic :: Software Development :: Libraries
Classifier: Topic :: System :: Logging
Classifier: Typing :: Typed
Requires-Python: >=3.11
Requires-Dist: mgf-common<0.50,>=0.42
Provides-Extra: collector
Requires-Dist: fastapi>=0.110; extra == 'collector'
Requires-Dist: httpx>=0.27; extra == 'collector'
Requires-Dist: uvicorn>=0.30; extra == 'collector'
Provides-Extra: dev
Requires-Dist: fastapi>=0.110; extra == 'dev'
Requires-Dist: httpx>=0.27; extra == 'dev'
Requires-Dist: import-linter>=2.0; extra == 'dev'
Requires-Dist: mgf-test-supervisor<0.2,>=0.1.3; extra == 'dev'
Requires-Dist: mypy>=1.10; extra == 'dev'
Requires-Dist: pytest-cov>=5.0; extra == 'dev'
Requires-Dist: pytest-timeout>=2.3; extra == 'dev'
Requires-Dist: pytest>=8.0; extra == 'dev'
Requires-Dist: ruff<0.16,>=0.4; extra == 'dev'
Provides-Extra: standards
Requires-Dist: mgf-common[standards]<0.50,>=0.46; extra == 'standards'
Provides-Extra: test
Requires-Dist: fastapi>=0.110; extra == 'test'
Requires-Dist: httpx>=0.27; extra == 'test'
Description-Content-Type: text/markdown

# mgf-logship

Centralized log shipping for the Magogi federation — the Layer-B half of the
unified-logging program (see `mgf-common/docs/inprogress/UNIFIED_LOGGING_DESIGN.md`,
decision **D4**: *ship + protocol now, stub the server*).

A consumer attaches **`LogShipper`** to its logging tree; it batches the
canonical mgf-common `LogRecord` JSON (the same shape every runtime emits —
OB-10), **redacts** each record, and `POST`s them to a collector over a
versioned, language-neutral wire protocol. A **store-now stub collector**
(FastAPI + sqlite) is included so the protocol works end-to-end today; analysis
and presentation are a later program.

## Install

```bash
pip install mgf-logship                # shipper only (stdlib transport)
pip install mgf-logship[collector]     # + the FastAPI/sqlite stub collector
```

## Ship logs from an app

```python
import logging
from mgf.logship import LogShipper
from mgf.common.observability import current_log_stats  # optional stats shipping

shipper = LogShipper(
    "https://collector.internal",
    token="…",                       # Authorization: Bearer …
    stats_provider=current_log_stats,  # also ship LogStats every 30s
)
logging.getLogger().addHandler(shipper)
# … on shutdown: shipper.close()  (flushes the queue)
```

`LogShipper` is **fail-open** (a slow/down collector never blocks or crashes the
app), **bounded** (the queue is capped; oldest dropped on overflow, counted in
`dropped_count`), and **redacts before ship** (off-box data is scrubbed).

### In production (env-driven, with app context)

The pattern the fleet uses (reference: `mgf.fed._logging.bootstrap_logging`, the
mgf-fed CLI):

```python
import logging, os
from mgf.common import bootstrap   # canonical app bootstrap (mgf-common)
from mgf.logship import LogShipper

def bootstrap_logging() -> None:
    url, token = os.environ.get("MGF_LOGSHIP_URL"), os.environ.get("MGF_LOGSHIP_INGEST_TOKEN")
    if not (url and token):
        return                      # opt-in: unconfigured ⇒ no-op, app unchanged
    bootstrap(app_name="myapp", app_version=__version__)   # ← see note below
    logging.getLogger().addHandler(
        LogShipper(url, token=token, app="myapp", env=os.environ.get("MGF_LOGSHIP_ENV", "prod"))
    )
```

- **Call `mgf.common.bootstrap()` first.** The shipper enriches each record via
  `mgf.common.app_version()` / `current_context()`; without an active app
  context mgf-common warns (`current_context() called before bootstrap()`) and
  records ship without app/trace context. `bootstrap()` also installs the
  standard console+file sinks.
- **`MGF_LOGSHIP_URL` is the collector *base*** (e.g. `https://<host>.sslip.io`)
  — the shipper appends `/ingest` itself.
- **Flush on exit:** the bounded queue flushes via `LogShipper.close()`, which
  Python's `logging.shutdown` (registered at interpreter exit) calls — so even a
  short-lived CLI run drains before exit. No extra wiring needed.
- **Keep it guarded.** Make the `LogShipper`/`mgf.common` imports lazy and
  failure-silent if your package has a dependency-light core; logging setup must
  never crash the app.

## Run the stub collector

```python
from mgf.logship.collector import create_collector_app
app = create_collector_app(db_path="logship.db", token="…")
# uvicorn module:app
```

Endpoints: `POST /ingest` (NDJSON of `LogRecord`s), `POST /stats` (a `LogStats`
JSON), `GET /health`. It persists to sqlite and stops there.

## The wire protocol

`mgf.logship._protocol` is the source of truth: `SHIP_PROTOCOL_VERSION`, the
endpoint paths, the `X-Logship-*` headers, and the NDJSON framing. Deliberately
HTTP + NDJSON so a Zig or TS shipper can talk to the same collector with nothing
but a socket and a JSON encoder.

## License

MIT — see [LICENSE](LICENSE).
