Metadata-Version: 2.4
Name: mgf-cloud-provision
Version: 0.1.0
Summary: Declarative, idempotent provisioning of the Magogi Foundation's self-hosted services (devpi, CI agents, Vault, …) on top of mgf-cloud. plan / apply / destroy over a ServiceSpec. Sibling of mgf-cloud under the mgf.* namespace.
Project-URL: Homepage, https://codeberg.org/magogi-admin/mgf-cloud-provision
Project-URL: Issues, https://codeberg.org/magogi-admin/mgf-cloud-provision/issues
Project-URL: Changelog, https://codeberg.org/magogi-admin/mgf-cloud-provision/src/branch/main/CHANGELOG.md
Author: Bassam Alsanie, mgf-cloud-provision contributors
License: MIT
License-File: LICENSE
Keywords: cloud,iac,infomaniak,infrastructure,openstack,provisioning
Classifier: Development Status :: 2 - Pre-Alpha
Classifier: Intended Audience :: System Administrators
Classifier: License :: OSI Approved :: MIT License
Classifier: Operating System :: POSIX :: Linux
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.11
Classifier: Programming Language :: Python :: 3.12
Classifier: Programming Language :: Python :: 3.13
Classifier: Topic :: System :: Systems Administration
Classifier: Typing :: Typed
Requires-Python: >=3.11
Requires-Dist: mgf-cloud<0.2,>=0.1.1
Requires-Dist: mgf-common<0.50,>=0.47
Requires-Dist: pydantic<3,>=2.6
Provides-Extra: dev
Requires-Dist: import-linter<3.0,>=2.0; extra == 'dev'
Requires-Dist: mgf-standard<0.2,>=0.1.6; extra == 'dev'
Requires-Dist: mgf-test-supervisor<0.2,>=0.1.3; extra == 'dev'
Requires-Dist: mypy>=1.10; extra == 'dev'
Requires-Dist: pytest-cov>=5.0; extra == 'dev'
Requires-Dist: pytest-timeout<3,>=2.3; extra == 'dev'
Requires-Dist: pytest>=8.0; extra == 'dev'
Requires-Dist: ruff<0.16,>=0.4; extra == 'dev'
Provides-Extra: infomaniak
Requires-Dist: mgf-cloud[infomaniak]<0.2,>=0.1.1; extra == 'infomaniak'
Provides-Extra: standards
Requires-Dist: mgf-common[standards]<0.50,>=0.47; extra == 'standards'
Description-Content-Type: text/markdown

# mgf-cloud-provision

> **Shape.** Magogi Foundation infrastructure library. Declarative, idempotent provisioning of the federation's **own self-hosted services** on Infomaniak Public Cloud.
> **Status.** Pre-`0.1.0` skeleton (Phase 1a). The core API + the devpi service spec exist and run end-to-end against `mgf-cloud`'s in-memory `FakeProvider`; the live provider + cloud-init converge land in Phase 1b.

`mgf-cloud-provision` sits **above** [`mgf-cloud`](../mgf-cloud) (the OpenStack/Infomaniak adapter). Where `mgf-cloud` is "talk to OpenStack," this is "**declare + converge** our services." A `ServiceSpec` describes the desired state of one service-bearing VM (devpi, a CI agent, Vault, …) as a VM + cloud-init; `plan` / `apply` / `destroy` converge the cloud to it.

```python
from mgf.cloud.providers import get_provider
from mgf.cloud_provision import devpi_service_spec, plan, apply

spec = devpi_service_spec(key_name="woodpecker-key")
print(plan(spec, get_provider("fake")).action)      # Action.CREATE
apply(spec, get_provider("fake"))                    # creates the VM (fake, in-memory)
```

CLI:

```bash
mgf-cloud-provision plan    examples/devpi.toml --provider fake
mgf-cloud-provision apply   examples/devpi.toml --provider fake --yes
mgf-cloud-provision destroy examples/devpi.toml --provider fake --yes
```

The whole surface runs with **no cloud and no credentials** (the `fake` provider), so it is fully testable in CI. The real provider (`--provider infomaniak`) loads credentials via `mgf.cloud.config.load`.

## First customer — devpi

The reason this library exists now is the **self-hosted package index** rollout (replacing the Codeberg PyPI registry — closes the dependency-confusion findings of the 2026-06-18 CI audit). `devpi_service_spec()` / `examples/devpi.toml` encode the Phase-0-resolved sizing: `a2-ram4-disk50-perf1` on `ext-net1`, reachable at `<ip>.sslip.io`. See the rollout plan in `~/.claude/plans/plan-self-hosted-pypi-sequencing-2026-06-18.md`.

## CI host — Woodpecker

**Full operator runbook: [`docs/woodpecker-ci-runbook.md`](docs/woodpecker-ci-runbook.md)**
(architecture, hosts/access, secrets, self-healing, operations, failure-mode catalogue, rebuild).

The federation's self-hosted Woodpecker CI runs from `examples/woodpecker-compose.yml`
(server + **Postgres** + agent + Caddy auto-TLS) — the sanitized live `/opt/woodpecker`
compose, with every secret a `${VAR}` sourced from `.env` (template:
`examples/woodpecker.env.example`). The server was migrated SQLite → Postgres on
2026-06-21 to remove the single-writer "database is locked" ceiling that surfaced under
fleet-wide CI bursts.

A **second agent on its own VM** (`examples/woodpecker-agent.toml` +
`examples/woodpecker-agent-cloud-init.yml`) removes the single-agent / single-Docker-daemon
ceiling — it connects to the server's gRPC over TLS (a Caddy `h2c` proxy at
`grpc.<server-domain>:443`; `WOODPECKER_GRPC_SECURE=true`). The shared `WOODPECKER_AGENT_SECRET`
and the netrc are injected post-provision over SSH, never committed. Fleet capacity is now
2 agents × `MAX_WORKFLOWS=2` = 4 concurrent workflows.

Two **self-healing crons** keep the agents honest: `self-heal.sh` (every 5 min, reaps leaked
`wp_*` Docker debris from daemon-teardown timeouts) and `examples/agent-reconciler.py` (every
3 min on the server host) — the reconciler recovers the "connected but not scheduling" state
(an agent whose server-side worker slots were orphaned, sitting idle while work pends) by
restarting that agent (local `docker compose`, or agent2 over SSH). It reads its API token +
agent2 SSH key from root-600 files on the host (`/opt/woodpecker/reconciler.token`,
`/opt/woodpecker/agent2_id`), never committed; cron: `*/3 * * * * python3 /opt/woodpecker/agent-reconciler.py`.

## Four green gates

```bash
ruff check src tests
mypy src
pytest --cov --cov-fail-under=80
lint-imports
```

## Roadmap

- ✅ **Phase 1a (this skeleton)** — `ServiceSpec` + `plan`/`apply`/`destroy` over a `CloudProvider`; TOML spec loader; devpi service spec; CLI; full test coverage against the fake provider.
- ⏳ **Phase 1b** — cloud-init converge for devpi (devpi-server + Caddy/LE + index init + backup); stand up `devpi-prod` on Infomaniak; absorb `mgf-fed provision-ci-host`.
- ⏳ **Later** — `woodpecker-agent`, `vault`, `registry-mirror` service specs; `plan`-side drift report + idempotent re-apply across the fleet.

## License

[MIT](LICENSE).
