# `mgf.common` Extraction Plan (round 5)

> **Status:** v2.0 (EXECUTED) — Last updated 2026-04-22.
>
> All five phases complete:
> - **Phase 1** (skeleton bootstrap) ✅
> - **Phase 2** (lift exceptions + observability + config) ✅
> - **Phase 3** (switch vm-vmanager-app to consume `mgf.common`) ✅
> - **Phase 4** (move standards docs) ✅
> - **Phase 5** (move this plan + write the PyPI roadmap) ✅ (this commit)
>
> Quality gates after the lift:
>
> | Repo | Tests | mypy strict | ruff | import-linter |
> |---|---|---|---|---|
> | `mgf_common` | 157 pass | clean (18 source files) | clean | `mgf-common-is-a-leaf` kept |
> | `vm-vmanager-app` | 1905 pass (2 skipped — optional deps) | clean (173 source files) | clean | 4/4 contracts kept |
>
> **Repo:** `/home/bassam/PycharmProjects/mgf-common` (Python package import: `mgf.common`)
> **Companion docs:** [`docs/standards/`](https://codeberg.org/magogi-admin/mgf-standard/src/branch/main/docs/standards/) (the specs this package implements)

This document captures what got done and the rationale, plus the
forward-looking roadmap to PyPI publish (§8) and the open
follow-up work (§12). The phase-by-phase narrative below is preserved
verbatim from the original plan as a worked example for any future
extraction round.

---

## 1. Why and what

### 1.1 Why

vm-vmanager-app was the cornerstone reference implementation of `docs/standards/`. With the standards stable and validated against real infrastructure (Jaeger smoke test, round 4), the next step is to lift the implementations into a project that other apps can consume directly — not by copy-paste.

Concrete benefits, in order:

1. **Single source of truth.** Bug fixes in `setup_logging`, `operation_span`, `report_now`, `Settings`, `Redactor`, etc. propagate to every consumer on the next bump.
2. **Faster bootstrap for new apps.** A new desktop / mobile / CLI project starts with `pip install -e ../mgf_common` and gets the entire `docs/standards/` L1+ surface for free.
3. **Forces clean boundaries.** The lift exposes any leaky coupling (e.g., `ssh.py`'s assumption of a specific `VMCredential` type) that the in-repo version got away with.
4. **Doc clarity.** `docs/standards/` lives with the code that implements it — no version-skew between spec and implementation.

### 1.2 What is being extracted

Three logical groupings + the standards docs:

| Grouping | Source files (in vm-vmanager-app today) |
|---|---|
| **Error handling** | `src/vmanager/exceptions.py` (split: base + categories → `mgf.common`; concretes stay) <br> `src/vmanager/observability/{crash_reporter,excepthook,threading_excepthook}.py` |
| **Logging + tracing** | `src/vmanager/observability/{logging_setup,json_formatter,text_formatter,redaction,otel_setup}.py` |
| **Configuration infrastructure** | `src/vmanager/config.py` (split: `BaseAppSettings` + loader/migrator/`_platform_dir` → `mgf.common`; concrete `AppConfig` fields stay) |
| **Standards docs** | `docs/standards/{README,DESIGN_PRINCIPLES,ERROR_HANDLING,LOGGING,CONFIGURATION,COMMON_COMPONENTS}.md` |

What stays in vm-vmanager-app:

- `vmanager.exceptions` — keeps every concrete domain exception (`DomainNotFoundError`, `BackupError`, `SpecValidationError`, etc.) as a subclass of the lifted categories.
- `vmanager.config.AppConfig` — keeps the vm-vmanager-app field schema (libvirt_uri, image_store, etc.) as a subclass of `mgf.common.config.BaseAppSettings`.
- **`vmanager.common.{shell,ssh,progress,guest_agent}`** — deferred. They're catalogued and lift-ready (with the Protocol-based decoupling sketched in §12 below) but staying in vm-vmanager-app for round 5 keeps the cutover small and the dependency graph clean. After round 5, `vmanager.common.*` simply imports `mgf.common.exceptions` + `mgf.common.observability` instead of `vmanager.exceptions` + `vmanager.observability`. Lift to `mgf.common` in a later round if/when a second consumer needs them.
- `vmanager.providers/`, `vmanager.lifecycle/`, `vmanager.operations/`, `vmanager.monitor/`, `vmanager.creator/`, `vmanager.lab/`, `vmanager.interchange/`, `vmanager.image/`, `vmanager.network/`, `vmanager.storage/`, `vmanager.scheduler/`, `vmanager.hooks/`, `vmanager.configurator/`, `vmanager.installer/`, `vmanager.environment/`, `vmanager.profiles/`, `vmanager.provisioning/`, `vmanager.vault/`, `vmanager.cli/`, `vmanager.gui/` — all vm-vmanager-app-specific.

---

## 2. Architecture

### 2.1 Namespace package layout (PEP 420)

You picked `mgf.common` as the import name. We use a **PEP 420 implicit namespace package** so `mgf.<future_subproject>` siblings can be added without changing this project. That means **no `__init__.py` at `src/mgf/`**:

```
mgf_common/
├── pyproject.toml
├── README.md
├── COMMON.md                                        moves here from vm-vmanager-app once bootstrapped
├── LICENSE                                          MIT, matches vm-vmanager-app
├── .gitignore
├── .importlinter
├── .woodpecker.yml                                  CI (lint + mypy + test + import-linter)
├── uv.lock
├── docs/
│   └── standards/                                   moved verbatim from vm-vmanager-app
│       ├── README.md
│       ├── DESIGN_PRINCIPLES.md
│       ├── ERROR_HANDLING.md
│       ├── LOGGING.md
│       ├── CONFIGURATION.md
│       └── COMMON_COMPONENTS.md
├── src/
│   └── mgf/                                         NO __init__.py (namespace package)
│       └── common/                                  has __init__.py
│           ├── __init__.py                          public re-exports + configure(app_name=...)
│           ├── _identity.py                         module-level app-name / app-version cache
│           ├── exceptions/
│           │   ├── __init__.py
│           │   ├── _base.py                         AppError
│           │   └── _well_known.py                   ConfigError, ProviderError, OperationError,
│           │                                        GuestError, EnvironmentError, VaultError,
│           │                                        SchedulerError + generic concretes
│           │                                        (CommandError, SchemaValidationError,
│           │                                         ResourceNotFoundError,
│           │                                         ResourceAlreadyExistsError)
│           ├── config/
│           │   ├── __init__.py
│           │   ├── _settings.py                     BaseAppSettings (the BaseSettings subclass
│           │   │                                    with YAML source customisation + version
│           │   │                                    validator + path-expansion validator)
│           │   ├── _loader.py                       load_settings, save_settings,
│           │   │                                    _write_yaml_atomic
│           │   └── _paths.py                        default_config_path(app_name),
│           │                                        _platform_dir(kind)
│           └── observability/
│               ├── __init__.py
│               ├── logging_setup.py                 setup_logging(...) reads _identity
│               ├── json_formatter.py
│               ├── text_formatter.py
│               ├── redaction.py
│               ├── crash_reporter.py                report_now(exc, source) reads _identity
│               ├── excepthook.py                    install_excepthook()
│               ├── threading_excepthook.py          install_threading_excepthook()
│               └── otel_setup.py                    setup_otel(enabled=...) reads _identity
└── tests/
    └── unit/
        ├── conftest.py
        ├── test_exceptions_base.py                  was tests/unit/test_exceptions.py (the
        │                                            generic parts; vm-vmanager-app-specific
        │                                            concretes stay in vm-vmanager-app)
        ├── test_config_settings.py                  was tests/unit/test_config.py + the
        │                                            generic platform-dir tests
        └── observability/                           lifted entirely, conftest.py included
            ├── conftest.py
            ├── test_redaction.py
            ├── test_json_formatter.py
            ├── test_text_formatter.py
            ├── test_logging_setup.py
            ├── test_crash_reporter.py
            ├── test_excepthooks.py
            └── test_otel_setup.py
```

### 2.2 The "app identity" abstraction

Today, the observability code has `_APP_NAME = "vmanager"` hard-coded in three places (`logging_setup.py`, `crash_reporter.py`, `otel_setup.py`). After the lift, that identity must be a parameter — the package can't know what app is consuming it.

**Recommended shape:** a small `configure(app_name, app_version)` bootstrap function that caches the identity in a module-level variable read by all the observability functions. Cleaner than passing `app_name=` to every call.

```python
# src/mgf/common/__init__.py
from mgf.common._identity import configure
__all__ = ["configure", ...]

# src/mgf/common/_identity.py
"""App-identity cache. Set once at startup via mgf.common.configure(...)."""
_APP_NAME: str = "app"          # caller MUST override
_APP_VERSION: str = "unknown"   # used in crash reports

def configure(app_name: str, app_version: str = "unknown") -> None:
    global _APP_NAME, _APP_VERSION
    _APP_NAME = app_name
    _APP_VERSION = app_version

def app_name() -> str:
    return _APP_NAME

def app_version() -> str:
    return _APP_VERSION
```

vm-vmanager-app calls `mgf.common.configure(app_name="vmanager", app_version=__version__)` once in `cli_main()` and once in `gui.app.run()`, before `setup_logging` / `setup_otel`. Every subsequent call to `setup_logging`, `setup_otel`, `report_now`, etc. reads the identity from `_identity`.

**Why module-level state and not contextvar:** the identity is process-scoped, not request-scoped. A contextvar would suggest "different apps in the same process" which is not a real use case here.

### 2.3 Subclassing recipes

#### `AppError` → `vm-vmanager-appError`

```python
# src/mgf/common/exceptions/_base.py
class AppError(Exception):
    """Base class for every exception raised by application code.
    Subclasses MUST set machine-readable fields in __init__ before
    super().__init__(message) so callers can read .field without
    parsing the message."""

# src/vmanager/exceptions.py — after lift
from mgf.common.exceptions import (
    AppError,
    ConfigError, ProviderError, OperationError, GuestError,
    EnvironmentError, VaultError, SchedulerError,
    CommandError, SchemaValidationError,
    ResourceNotFoundError, ResourceAlreadyExistsError,
)

vm-vmanager-appError = AppError  # alias for backwards-compat — every existing
                          # `from vmanager.exceptions import vm-vmanager-appError`
                          # keeps working unchanged.

# Concrete vm-vmanager-app exceptions stay here, subclassing the lifted categories.
class SpecValidationError(SchemaValidationError):
    pass

class AppConfigError(ConfigError):
    pass

class DomainNotFoundError(ResourceNotFoundError):
    """Inherits the action hint convention from ResourceNotFoundError;
    overrides the default message to mention `vmanager vm list`."""
    def __init__(self, name: str) -> None:
        self.name = name
        super().__init__(name)
        # Hint stays as it is today.

# ... etc for every concrete exception
```

#### `BaseAppSettings` → `AppConfig`

```python
# src/mgf/common/config/_settings.py
class BaseAppSettings(BaseSettings):
    """Shared infrastructure: YAML source customisation, version
    validator, path expansion, atomic save. Subclass and add your
    project's typed fields."""

    model_config = SettingsConfigDict(
        env_file=".env",
        env_file_encoding="utf-8",
        extra="forbid",
        case_sensitive=False,
        validate_default=True,
        arbitrary_types_allowed=True,
        # NOTE: env_prefix is NOT set here — subclasses set their own
        # (e.g., VMANAGER_, MYAPP_).
    )

    _yaml_file_override: ClassVar[Path | None] = None
    version: int = 1

    @classmethod
    def settings_customise_sources(cls, ...):
        # Same YAML wiring as vm-vmanager-app has today.
        ...

# src/vmanager/config.py — after lift
from mgf.common.config import BaseAppSettings, default_config_path, load_settings, save_settings

class AppConfig(BaseAppSettings):
    model_config = SettingsConfigDict(
        env_prefix="VMANAGER_",
        # other model_config inherited from BaseAppSettings
    )

    # Every existing AppConfig field exactly as it is today:
    libvirt_uri: str = "qemu:///system"
    use_mock: bool = False
    image_store: Path = Path("~/.local/share/vmanager/images")
    # ... etc

# Public API unchanged:
def load_config(path: Path | None = None) -> AppConfig:
    target = path if path is not None else default_config_path("vmanager")
    return load_settings(AppConfig, path=target, app_name="vmanager")

def save_config(cfg: AppConfig, path: Path | None = None) -> None:
    target = path if path is not None else default_config_path("vmanager")
    save_settings(cfg, path=target)

Settings = AppConfig  # alias preserved
```

### 2.4 Coupling — none to solve

With `common/{shell,ssh,progress,guest_agent}.py` deferred, the lift is purely **leaf code with no domain dependencies** — exceptions, observability, and config infrastructure all depend only on stdlib + pydantic + pyyaml + (optional) opentelemetry. There are no upward references to vault, providers, or domain types to disentangle.

Should the four `common/*.py` files be lifted in a later round, the structural-typing Protocols required (`Credential` for `ssh.py`'s `VMCredential` dependency, `GuestAgentProviderProtocol` + `HandleProtocol` for `guest_agent.py`'s provider+handle dependencies) are sketched in §12 below as deferred design notes.

### 2.5 Dependency graph (after lift)

```
                     ┌──────────────────────────────────────────┐
                     │                vm-vmanager-app                   │
                     │  cli/, gui/, providers/, operations/      │
                     │  vault/, configurator/, ...               │
                     │  common/{shell,ssh,progress,guest_agent}  │
                     │                                           │
                     │  exceptions.py    ── subclass ──┐         │
                     │  config.py::AppConfig ── subcls ┼─┐       │
                     └────────────┬─────────────────────┘ │       │
                                  │ depends on            │       │
                                  ▼                       ▼       ▼
                     ┌──────────────────────────────────────────┐
                     │              mgf.common                  │
                     │                                          │
                     │  exceptions/   (AppError + categories)   │
                     │  config/       (BaseAppSettings + loader)│
                     │  observability/                          │
                     │                                          │
                     │  Depends on: pydantic, pydantic-settings,│
                     │  pyyaml (+ optional opentelemetry-*,     │
                     │  sentry-sdk via [observability] extra)   │
                     │  Depends on NO consumer project.         │
                     └──────────────────────────────────────────┘
```

Strict rule, enforced by `mgf_common/.importlinter`:
- `mgf.common` MUST NOT import `vmanager`, `mgf.<anything-else>`, or any other consumer project.

---

## 3. Scope: file-by-file lift list

For each file, the table shows: source path in vm-vmanager-app → destination in mgf_common → action required (rename imports, parameterise app name, replace direct dep with Protocol, etc.).

### 3.1 Error handling (8 files)

| Source (vmanager) | Destination (mgf_common) | Action |
|---|---|---|
| `exceptions.py` (lines 21-119: AppError + categories + generic concretes) | `src/mgf/common/exceptions/_base.py` + `src/mgf/common/exceptions/_well_known.py` | **SPLIT.** Lift `AppError`, `ConfigError`, `ProviderError`, `OperationError`, `GuestError`, `EnvironmentError`, `VaultError`, `SchedulerError`, `SchemaValidationError`, `CommandError`, `ResourceNotFoundError` (new generic of `DomainNotFoundError`), `ResourceAlreadyExistsError`. Concrete vm-vmanager-app exceptions (`SpecValidationError`, `AppConfigError`, `LabSpecError`, `DomainNotFoundError`, `DomainAlreadyExistsError`, `HotplugNotSupportedError`, `CreationError`, `SnapshotError`, `CloneError`, `DiskOperationError`, `BackupError`, `MigrationError`, `CustomizeError`, `ImportExportError`, `GuestUnreachableError`, `GuestCommandError`, `HealthCheckError`, `HookExecutionError`, `ScheduleParseError`, `KeyringUnavailableError`, `VirtualizationNotSupportedError`, `KernelModuleError`) stay in `vmanager.exceptions` as subclasses of the lifted categories. |
| `observability/crash_reporter.py` | `src/mgf/common/observability/crash_reporter.py` | Replace `_APP_NAME = "vmanager"` constant with `from mgf.common._identity import app_name`; `_SINK_ENV_VAR` becomes `f"{app_name().upper()}_CRASH_SINK"` evaluated at call time. Same change for the `_crash_dir()` helper. |
| `observability/excepthook.py` | `src/mgf/common/observability/excepthook.py` | Logger name `"vmanager.unhandled"` becomes `f"{app_name()}.unhandled"`. No other change. |
| `observability/threading_excepthook.py` | `src/mgf/common/observability/threading_excepthook.py` | Same as above. |

### 3.2 Logging + tracing (5 files)

| Source | Destination | Action |
|---|---|---|
| `observability/logging_setup.py` | `src/mgf/common/observability/logging_setup.py` | Replace `_APP_NAME = "vmanager"` constant with reads from `_identity.app_name()`. The default log file path becomes `<state>/<app_name>/logs/<app_name>.log`. The env-var prefixes for syslog / HTTP sinks (`VMANAGER_SYSLOG_ADDR`, `VMANAGER_LOG_HTTP_SINK`) become `f"{app_name().upper()}_SYSLOG_ADDR"` etc. |
| `observability/json_formatter.py` | `src/mgf/common/observability/json_formatter.py` | No app-name dependency. Lift verbatim. |
| `observability/text_formatter.py` | `src/mgf/common/observability/text_formatter.py` | No app-name dependency. Lift verbatim. |
| `observability/redaction.py` | `src/mgf/common/observability/redaction.py` | No app-name dependency. Lift verbatim. |
| `observability/otel_setup.py` | `src/mgf/common/observability/otel_setup.py` | Replace `_APP_NAME = "vmanager"` with `from mgf.common._identity import app_name`. The `OTEL_SERVICE_NAME` default becomes `app_name()` rather than the hard-coded string. |
| `observability/__init__.py` | `src/mgf/common/observability/__init__.py` | Re-exports — same shape, plus add `configure` re-export from parent `mgf.common.__init__`. |

### 3.3 Configuration (1 file, split into 3) 

| Source | Destination | Action |
|---|---|---|
| `config.py` (lines 1-218: `BaseSettings` infrastructure, `_PATH_FIELDS`, `_expand_path`, `_check_version`, `settings_customise_sources`, `to_yaml_dict`) | `src/mgf/common/config/_settings.py` (`BaseAppSettings` class) | Lift the class shell + the YAML source customisation + the version validator + the path-expansion validator. Drop vm-vmanager-app-specific fields (libvirt_uri, image_store, etc.) and the `env_prefix="VMANAGER_"` setting. Subclasses set their own. The `_PATH_FIELDS` tuple becomes a class attribute that subclasses override. |
| `config.py` (lines 248-305: `default_config_path`, `_platform_dir`) | `src/mgf/common/config/_paths.py` | `default_config_path()` takes `app_name: str` parameter (no longer hard-coded "vmanager"). `_platform_dir(kind)` lifts unchanged. |
| `config.py` (lines 308-407: `load_config`, `save_config`, `_write_yaml_atomic`, `_migrate`) | `src/mgf/common/config/_loader.py` | Renamed: `load_config` → `load_settings(settings_cls, path, app_name)`; `save_config` → `save_settings(settings, path)`. `_write_yaml_atomic` lifts unchanged. `_migrate` becomes a no-op stub — projects that need migration override via a callback. Callsite-side: vm-vmanager-app's `vmanager.config.load_config(path)` becomes a thin wrapper around `mgf.common.config.load_settings(AppConfig, path, "vmanager")`. |

### 3.4 Common utilities — DEFERRED

`vmanager/common/{shell,ssh,progress,guest_agent}.py` stay in vm-vmanager-app for round 5. After Phase 3, their imports change as follows (one-line edits inside files that otherwise stay put):

- `vmanager/common/shell.py`:
  - `from vmanager.exceptions import CommandError` → `from mgf.common.exceptions import CommandError`
  - `from vmanager.observability import operation_span` → `from mgf.common.observability import operation_span`
- `vmanager/common/ssh.py`:
  - `from vmanager.exceptions import GuestCommandError, GuestUnreachableError` → `from mgf.common.exceptions import GuestCommandError, GuestUnreachableError`
  - `from vmanager.vault.models import VMCredential` — **unchanged** (vault stays in vm-vmanager-app).
- `vmanager/common/progress.py`:
  - No vmanager imports today. **Unchanged.**
- `vmanager/common/guest_agent.py`:
  - `from vmanager.exceptions import GuestUnreachableError, ProviderError` → `from mgf.common.exceptions import GuestUnreachableError, ProviderError`
  - `TYPE_CHECKING` imports of `vmanager.domain.VMHandle` and `vmanager.providers.base.VMProvider` — **unchanged**.

These edits are part of the broader Phase 3 import-rename sweep described in §5; they don't constitute a separate phase.

### 3.5 Standards docs (6 files)

Move the entire `docs/standards/` tree from vm-vmanager-app to mgf_common verbatim:

| Source | Destination | Action |
|---|---|---|
| `docs/standards/README.md` | `mgf_common/docs/standards/README.md` | Move. Update the "Reference implementation" line: vm-vmanager-app → mgf_common ITSELF is the reference implementation, with vm-vmanager-app as the first-consumer validation case. |
| `docs/standards/DESIGN_PRINCIPLES.md` | `mgf_common/docs/standards/DESIGN_PRINCIPLES.md` | Move verbatim. |
| `docs/standards/ERROR_HANDLING.md` | `mgf_common/docs/standards/ERROR_HANDLING.md` | Move verbatim. |
| `docs/standards/LOGGING.md` | `mgf_common/docs/standards/LOGGING.md` | Move verbatim. |
| `docs/standards/CONFIGURATION.md` | `mgf_common/docs/standards/CONFIGURATION.md` | Move verbatim. |
| `docs/standards/COMMON_COMPONENTS.md` | `mgf_common/docs/standards/COMMON_COMPONENTS.md` | Move + rewrite the "Where it lives in vm-vmanager-app" sections to "Where it lives in mgf.common" (with vm-vmanager-app mentioned as the first consumer). |

vm-vmanager-app's `docs/standards/` becomes a single-file pointer:

```markdown
# Engineering Standards

These standards now live with their canonical implementation at:
  https://codeberg.org/<future-org>/mgf_common/src/branch/main/docs/standards/

(Or, locally during development: `../mgf_common/docs/standards/`.)

vm-vmanager-app pulls the implementation in via `mgf-common` (see pyproject.toml).
```

---

## 4. Tests: what lifts, what stays

### 4.1 Lifted to `mgf_common/tests/unit/`

| Source | Destination in mgf_common | Notes |
|---|---|---|
| `tests/unit/test_exceptions.py` | `tests/unit/test_exceptions_base.py` | Tests for the generic categories + concretes only (`AppError`, `ConfigError`, `CommandError`, `SchemaValidationError`, `ResourceNotFoundError`, etc.). The vm-vmanager-app-specific concretes (`SpecValidationError`, `DomainNotFoundError`, etc.) get a smaller test that stays in vm-vmanager-app. |
| `tests/unit/test_config.py` | `tests/unit/test_config_settings.py` | Tests for `BaseAppSettings` + `load_settings`/`save_settings` + atomic write + migration shape. The vm-vmanager-app-specific field tests (verifying e.g. `libvirt_uri` default) stay in vm-vmanager-app. |
| `tests/unit/test_config_platform_dir.py` | `tests/unit/test_config_platform_dir.py` | Lift verbatim — `_platform_dir` is fully generic. |
| `tests/unit/observability/conftest.py` | `tests/unit/observability/conftest.py` | Lift verbatim (the session-scoped TracerProvider fixture). |
| `tests/unit/observability/test_redaction.py` | same path | Lift verbatim. |
| `tests/unit/observability/test_json_formatter.py` | same path | Lift verbatim. |
| `tests/unit/observability/test_text_formatter.py` | same path | Lift verbatim. |
| `tests/unit/observability/test_logging_setup.py` | same path | Lift; tests that set `XDG_STATE_HOME` keep working. The journald-skipped test still works (env-driven). The default-log-path test now goes against `mgf.common._identity.app_name()` — fixture sets it to `"test"`. |
| `tests/unit/observability/test_crash_reporter.py` | same path | Lift; the env var name `VMANAGER_CRASH_SINK` becomes whatever `f"{app_name().upper()}_CRASH_SINK"` resolves to at fixture setup (set `app_name()` to `"test"` in conftest, env var becomes `TEST_CRASH_SINK`). |
| `tests/unit/observability/test_excepthooks.py` | same path | Lift verbatim. |
| `tests/unit/observability/test_otel_setup.py` | same path | Lift verbatim. |

### 4.2 Stays in vm-vmanager-app

| Test | Why |
|---|---|
| `tests/unit/test_exceptions_actionable_hints.py` | Tests assert on `vmanager` command names ("vmanager vm list", "vmanager vm port-forward remove"). Stays. |
| `tests/unit/observability/test_cli_catchall.py` | Tests the CLI catch-all in `cli/__main__.py`. Stays. |
| `tests/unit/observability/test_disk_ops_spans.py` | Tests `disk.<verb>` spans in `vmanager/operations/disk_ops.py`. Stays. |
| `tests/unit/test_shell.py` | Tests `vmanager.common.shell.run_command` (which stays in vm-vmanager-app). Stays. |
| `tests/unit/test_progress.py` | Tests `vmanager.common.progress`. Stays. |
| `tests/unit/test_common_ssh.py` | Tests `vmanager.common.ssh`. Stays. |
| `tests/unit/test_common_guest_agent.py` | Tests `vmanager.common.guest_agent`. Stays. |
| `tests/unit/test_config.py` (parts) | New small file in vm-vmanager-app that tests `AppConfig`-specific defaults (`libvirt_uri == "qemu:///system"`, etc.). |
| `tests/unit/test_exceptions.py` (parts) | New small file in vm-vmanager-app that tests the vm-vmanager-app-specific concretes (`SpecValidationError`, `DomainNotFoundError`, etc.). |
| Everything else under `tests/unit/` | All non-observability tests stay (lifecycle, operations, providers, lab, hooks, scheduler, monitor, GUI, CLI commands). |

### 4.3 Test-suite count after the lift

vm-vmanager-app today: **1999** unit tests pass. After the lift:
- mgf_common picks up roughly **~110** lifted tests (10 observability test files + base exceptions + base config + cross-platform paths).
- vm-vmanager-app keeps roughly **~1890** + small new tests for the post-lift wrappers.

Pre-cutover, both counts have to add up to ≥1999 (we don't lose coverage).

---

## 5. Cutover sequence

Five discrete phases. Each phase is one git commit on each side (so both projects can roll back independently). I'll execute them sequentially in subsequent rounds — this doc is the plan, not the work.

### Phase 1 — Bootstrap mgf_common (skeleton only, no lift)

In `/home/bassam/PycharmProjects/mgf-common`:

1. `mkdir -p mgf_common/src/mgf/common mgf_common/tests/unit/observability mgf_common/docs/standards`
2. `git init`
3. Add files:
   - `pyproject.toml` (project name `mgf-common`, deps: `pydantic>=2.6`, `pydantic-settings>=2.2`, `pyyaml>=6.0`. Optional `[observability]` extra: `opentelemetry-{api,sdk,exporter-otlp-proto-http,instrumentation-httpx}`, `sentry-sdk`. Optional `[dev]` extra: pytest, ruff, mypy, hypothesis, types-PyYAML, import-linter. PEP 420 namespace package via `[tool.hatch.build.targets.wheel] packages = ["src/mgf"]`.)
   - `README.md` (one paragraph + reference to `docs/standards/`)
   - `LICENSE` (MIT, copy from vm-vmanager-app)
   - `.gitignore` (Python defaults + `.venv/`, `__pycache__/`, `dist/`, `*.egg-info`)
   - `.importlinter` (one contract: `mgf.common` MUST NOT import any other top-level package)
   - `.woodpecker.yml` (lint + mypy + test + import-linter; copy vm-vmanager-app's structure, drop the install-script + libvirt-aware bits)
   - Empty `src/mgf/common/__init__.py` with a placeholder `configure` function
   - Empty test placeholder so pytest finds the dir
4. `uv venv && uv sync`
5. `git commit -m "Bootstrap mgf_common skeleton"`

**Smoke test:** `pytest` reports zero tests, zero failures. `mypy src` and `ruff check src tests` both clean. `lint-imports` reports 1/1 contract kept.

### Phase 2 — Lift error handling + observability + config infrastructure

In `mgf_common`:

1. Copy each file from vm-vmanager-app per §3.1 / §3.2 / §3.3, applying the file-specific actions.
2. Implement `mgf.common._identity` and `mgf.common.configure`.
3. Lift the relevant tests from vm-vmanager-app per §4.1, with import path renames.
4. `pytest` passes.
5. `lint-imports`, `ruff`, `mypy --strict` all clean.
6. `git commit -m "Phase 2: lift exceptions + observability + config"`

vm-vmanager-app is **not yet touched** — these phases are independent commits in independent repos. The next phase wires vm-vmanager-app up to consume mgf_common.

### Phase 3 — Switch vm-vmanager-app to consume mgf.common

In `/home/bassam/PycharmProjects/vm-vmanager-app`:

1. Add the path dependency to `pyproject.toml`:
   ```toml
   dependencies = [
       # ... existing ...
       "mgf-common",
   ]

   [tool.uv.sources]
   mgf-common = { path = "../mgf_common", editable = true }
   ```
2. Mechanical rename across the codebase (one large find/replace, validated by mypy + tests):
   - `from vmanager.observability` → `from mgf.common.observability` (every consumer including `vmanager/common/shell.py`)
   - `from vmanager.exceptions import {generic categories}` → `from mgf.common.exceptions import {same names}` for the lifted categories. The aliases `from vmanager.exceptions import vm-vmanager-appError` keep working unchanged because `vm-vmanager-appError = AppError`.
   - `vmanager/common/{shell,ssh,guest_agent}.py` get the targeted import edits per §3.4.
3. Rewrite `vmanager.exceptions` to import + re-export from `mgf.common.exceptions` and define vm-vmanager-app-specific concretes as subclasses (per §2.3 recipe).
4. Rewrite `vmanager.config` to subclass `mgf.common.config.BaseAppSettings` (per §2.3 recipe). Public functions `load_config`/`save_config` become thin wrappers around `mgf.common.config.load_settings`/`save_settings`.
5. Add `mgf.common.configure(app_name="vmanager", app_version=__version__)` calls in `cli/__main__.py::cli_main` and `gui/app.py::run`, BEFORE `setup_logging`.
6. Delete `src/vmanager/observability/` (the lifted package). `src/vmanager/common/{shell,ssh,progress,guest_agent}.py` STAY — they only get the import-rename edits in step 2.
7. Delete the lifted parts of `src/vmanager/config.py` and `src/vmanager/exceptions.py`; keep the vm-vmanager-app-specific subclasses + concrete fields.
8. Delete the lifted tests (anything in §4.1). Tests for `vmanager/common/{shell,ssh,progress,guest_agent}.py` STAY (they test code that stays in vm-vmanager-app).
9. Add small new tests verifying the consume-mgf path: `vmanager.exceptions.vm-vmanager-appError is mgf.common.exceptions.AppError`, `isinstance(AppConfig(), BaseAppSettings)`, etc.
10. Update `.importlinter`: drop the "observability is lift-ready" contract (no longer applicable — observability is gone). Keep the libvirt-isolation, ui-is-a-leaf, exceptions-is-a-leaf contracts. Add a new contract: "vmanager modules MAY import mgf.common, but mgf.common MUST NOT import vmanager" (this catches accidental upward dependencies in PRs).
11. Run full sweep: `pytest`, `mypy --strict`, `ruff`, `lint-imports`. Expected: green.
12. `git commit -m "Phase 3: switch vm-vmanager-app to consume mgf.common via path dependency"`

**Smoke test:** `vmanager vm create --profile ubuntu-server --name demo --dry-run` still works. The OTel demo from round 4 still produces the same trace shape in Jaeger.

### Phase 4 — Move standards docs

Two commits, one in each repo:

1. **mgf_common:** copy `vm-vmanager-app/docs/standards/` to `mgf_common/docs/standards/`, then rewrite the "Reference implementation" / "Where it lives in vm-vmanager-app" sections to "Where it lives in mgf.common" (per §3.5). Commit: `"Phase 4a: standards docs land at canonical home"`.
2. **vm-vmanager-app:** delete the six markdown files under `docs/standards/`, replace with the one-line pointer file. Update vm-vmanager-app `docs/claude/CLAUDE.md` "Documentation" section to point at the new location. Commit: `"Phase 4b: standards docs moved to mgf_common"`.

### Phase 5 — Move COMMON.md, set up the path-to-PyPI doc

1. **mgf_common:** copy `vm-vmanager-app/docs/COMMON.md` to `mgf_common/COMMON.md`. Add a section "Path to PyPI" (described below). Commit: `"Phase 5a: COMMON plan + PyPI roadmap"`.
2. **vm-vmanager-app:** delete `docs/COMMON.md`, replace with a one-liner pointing at `../mgf_common/COMMON.md`. Commit: `"Phase 5b: COMMON plan moved with the project"`.

---

## 6. Backward compatibility

**Policy: clean cutover.** No deprecation shims, no warnings.

After Phase 3, vm-vmanager-app imports `mgf.common.*` directly. Old code paths like `from vmanager.observability import setup_logging` stop working in the same commit. The `vmanager.exceptions.vm-vmanager-appError = AppError` alias is the ONLY back-compat trick we keep — it's a single line and it preserves a heavily-used import path that would otherwise force a sweep across every test file.

Why no shims:
- vm-vmanager-app is the only consumer today. Shims for an audience of one is over-engineering.
- A clean cutover surfaces every coupling immediately. Hidden coupling caught now is cheaper than hidden coupling caught later.
- mypy + the test suite are the safety net. If the rename misses a site, `mypy --strict` flags it.

---

## 7. Rollback plan

Each phase is a separate commit in its own repo. Rollback is `git revert <commit>` in either or both repos.

The high-risk phase is Phase 3 (the cutover). Before merging:

1. Run the full test suite in vm-vmanager-app: `pytest --tb=short -q` — expect 1999+ pass.
2. Run the full test suite in mgf_common: `pytest --tb=short -q` — expect ~110 pass.
3. Run `lint-imports` in both: 4/4 in vm-vmanager-app (after the contract update), 1+ in mgf_common.
4. Run `mypy --strict src` in both: zero errors.
5. Run the live-Jaeger smoke from round 4 — verify the trace shape is identical to before the lift.
6. Run `vmanager vm list`, `vmanager profile list`, `vmanager image list` — basic smoke.

If any of those fail, revert Phase 3 in vm-vmanager-app (single commit) and diagnose. mgf_common stays usable for any other future consumer.

---

## 8. Path to PyPI (the remaining roadmap)

Round 5 shipped **path-source dependency** — vm-vmanager-app consumes
`mgf-common` via `[tool.uv.sources] mgf-common = {path="../mgf_common", editable=true}`.
Going to PyPI is a follow-up round. Trigger when EITHER:

- A second project starts consuming `mgf.common` (proves the API is
  reusable, not just lifted-from-vm-vmanager-app-shaped). This is the
  preferred trigger.
- OR the API is judged stable enough for an external release on the
  strength of vm-vmanager-app use alone.

### 8.1 Pre-flight checklist (before any tag)

| Item | State today | What's needed before v0.1.0 |
|---|---|---|
| Source on Codeberg | ✅ pushed to `codeberg.org/magogi-admin/mgf-common` | — |
| `.woodpecker.yml` | ✅ shipped (lint + typecheck + test matrix + build) | — |
| Coverage gate | ✅ `--cov-fail-under=80` on the 3.12 test job | — |
| `py.typed` marker | ✅ shipped (`src/mgf/common/py.typed`) | — |
| `pyproject.toml` URLs | ✅ point at the real Codeberg slug | — |
| LICENSE | ✅ MIT | — |
| `__version__` | ✅ `0.1.0` in pyproject + `__init__.py` | — |
| Changelog | ✅ `CHANGELOG.md` seeded with the v0.1.0 entry | — |
| Dual-tag verification | ✅ `python -m build` round-trip ships `py.typed`; `twine check` passes both artefacts | — |

### 8.2 Release workflow (manual upload from a developer machine)

PyPI's **trusted-publisher OIDC does not (yet) support Codeberg /
Forgejo** as an issuer — the supported list is GitHub, GitLab, Google,
ActiveState. So `mgf-common` releases are **manual `twine upload`**
from a developer machine using a PyPI API token. The CI in
`.woodpecker.yml` builds + checks the wheel on every push (so
packaging regressions fail fast) but does NOT publish.

Per-release procedure:

```bash
# 1) Make the change, run local gates.
pytest --tb=short -q --cov --cov-fail-under=80
mypy src
ruff check src tests
lint-imports

# 2) Bump version + add CHANGELOG entry.
$EDITOR pyproject.toml src/mgf/common/__init__.py CHANGELOG.md

# 3) Commit, push, tag, push tag (annotated tag preferred).
git commit -am "vX.Y.Z release"
git push
git tag -a vX.Y.Z -m "vX.Y.Z release"
git push --tags

# 4) Build the artefacts.
rm -rf dist/
python -m build         # produces dist/*.whl and dist/*.tar.gz

# 5) Sanity-check the artefacts before upload.
twine check dist/*

# 6) Upload to PyPI. The token lives at
#    /home/bassam/PycharmProjects/vm-vmanager-app/my_stuff/ssh_key/pypi_token.txt
#    (mode 0600). __token__ is the literal username PyPI expects.
TWINE_USERNAME=__token__ \
  TWINE_PASSWORD="$(cat /home/bassam/PycharmProjects/vm-vmanager-app/my_stuff/ssh_key/pypi_token.txt)" \
  twine upload dist/*
```

After the very first publish PyPI claims the project name. From then
on you SHOULD generate a **project-scoped token** (much narrower than
the account-scoped one used for the initial upload) at
https://pypi.org/manage/account/token/ and replace the file content.
Old account-scoped token can then be revoked.

### 8.3 Switching vm-vmanager-app from path-source to PyPI

Once v0.1.0 is on PyPI, vm-vmanager-app's `pyproject.toml` flips from path
source to a real version pin. Mechanical change, no source edits:

```diff
 dependencies = [
     ...
-    "mgf-common",
+    "mgf-common>=0.1.0,<0.2",
 ]

-[tool.uv.sources]
-mgf-common = { path = "../mgf_common", editable = true }
```

Then `uv sync` pulls from PyPI; `pytest`, `mypy`, `ruff`,
`lint-imports` should all stay green.

### 8.4 Future: when CI publishing becomes possible

Two paths could move publishing into CI later:

- **PyPI adds Codeberg as an OIDC issuer.** Track
  https://github.com/pypa/warehouse for movement. The day it lands,
  add a `publish` step gated on `event: tag` to `.woodpecker.yml`
  using `pypi-publish` or a manual OIDC token exchange.
- **Token-secret CI publish.** If the manual workflow becomes a
  bottleneck, store a project-scoped PyPI token in the Codeberg
  Woodpecker secret store (when CI access is granted) and add a
  publish step that reads `TWINE_PASSWORD` from `${PYPI_TOKEN}`. Less
  secure than OIDC but works on Codeberg today. Apply for CI access
  at https://codeberg.org/Codeberg-CI/request-access first.

### 8.5 Version bump workflow (post-v0.1.0)

Each subsequent release follows §8.2 verbatim. Semver applies:

- **MAJOR** — A MUST change in the standards docs OR a breaking change
  to the public Python surface. Bump the leading 0 (within 0.x) or
  the first segment after v1.0+.
- **MINOR** — A SHOULD change OR a non-breaking addition (new public
  function, new config setting). Consumers on the `>=X.0,<X+1`
  range auto-receive these.
- **PATCH** — Wording, examples, internal refactor. No consumer
  action needed.

The standards docs follow the same versioning rules documented in
[`docs/standards/README.md`](https://codeberg.org/magogi-admin/mgf-standard/src/branch/main/docs/standards/README.md). The package
version and the standards version MAY drift — they are released
together but bumped independently.

### 8.6 Backwards compatibility for consumers

Until `mgf-common` reaches v1.0:

- **MINOR bumps** (0.X.0 → 0.X+1.0) MAY introduce non-source-breaking
  additions (new public functions, new optional config keys). Consumers
  on the `>=0.1.0,<0.2` pin auto-receive them.
- **MAJOR bumps within 0.x** (0.1 → 0.2) MAY break the public surface.
  Consumers MUST update their pin and run their tests.
- **`AppError` and the seven category bases are SOURCE-STABLE** within a
  major series. Removing a category requires bumping to v1.0+.
- **`BaseAppSettings.CURRENT_VERSION` semantics** are DOCUMENTED-STABLE:
  consumers' migration callbacks are the contract. Renaming
  `_yaml_file_override` or changing the migration callback signature
  requires a major bump.

After v1.0 the rules tighten — see the v1.0 release-notes template
when we get there.

---

## 9. mgf_common CI

Shipped at `.woodpecker.yml` — four steps on every push / PR / tag:

| Step | Image | Action |
|---|---|---|
| `lint` | `python:3.12-slim` | `ruff check src tests` + `lint-imports` |
| `typecheck` | `python:3.12-slim` | `mypy --strict src` (uses the `[dev,observability]` install) |
| `test` (matrix 3.11/3.12/3.13) | `python:${PYTHON_VERSION}-slim` | `pytest --tb=short -q`. Coverage gate `--cov-fail-under=80` on the 3.12 job only — see the inline rationale in `.woodpecker.yml`. |
| `build` | `python:3.12-slim` | `python -m build` + `twine check dist/*` + `unzip -l` to confirm `py.typed` ships in the wheel. Catches packaging regressions on every push. |

**No CI publish step.** PyPI's trusted-publisher OIDC does not (yet)
support Codeberg / Forgejo as an issuer. The release flow is the
manual `twine upload` documented in §8.2. The `build` step here
exists to catch packaging regressions, NOT to gate the release.

No real-libvirt smoke (mgf.common doesn't touch libvirt). No
install-script step. No GUI test (mgf.common doesn't ship a GUI
today — `core_worker` and `log_viewer_dialog` stay in vm-vmanager-app
because Qt is GUI-stack-specific; whether to lift them is a future
decision, not part of round 5).

**`ci.codeberg.org` access.** Codeberg's official Woodpecker
instance is allow-list gated. The first time you visit it as
`magogi-admin`, login fails with `org_access_denied`. Apply for
access at https://codeberg.org/Codeberg-CI/request-access (an
issue tracker — file a request in the existing thread or open
your own). Until access is granted, the CI runs nowhere; the
`.woodpecker.yml` lives in the repo as documentation +
copy-paste-ready reference. Once granted, every push / PR / tag
fires the four steps above with no further config change.

---

## 10. Standards-docs ownership change

Today: `docs/standards/` lives in vm-vmanager-app. The README says "Reference implementation: vm-vmanager-app."

After the lift: `docs/standards/` lives in mgf_common. The README says:
> "Reference implementation: **mgf.common itself** (this repo). vm-vmanager-app is the first consumer and validation case for round 1–4."

The single biggest doc edit is in `COMMON_COMPONENTS.md`. Every "Where it lives in vm-vmanager-app" subsection becomes "Where it lives in `mgf.common`," with vm-vmanager-app mentioned as the consuming project. The MUST/SHOULD rules stay unchanged.

---

## 11. What this plan does NOT change

To set expectations for the actual execution rounds:

- **vm-vmanager-app's user-facing CLI / GUI surface** — zero change. Every `vmanager X Y Z` command keeps working.
- **vm-vmanager-app's exception types as observed by callers** — `vmanager.exceptions.DomainNotFoundError`, `BackupError`, etc. all keep their names and import paths.
- **vm-vmanager-app's config file format** — `~/.config/vmanager/config.yaml` is unchanged. The `version: 1` schema is unchanged.
- **vm-vmanager-app's logging output** — same JSON/text format, same default file path, same env vars (`VMANAGER_LOG_HTTP_SINK` etc.).
- **vm-vmanager-app's OTel behaviour** — same `vm.create`, `subprocess.run`, etc. spans against any OTLP collector.
- **vm-vmanager-app's Jaeger smoke from round 4** — produces an identical trace shape after the lift.
- **vm-vmanager-app's test count** — total unit tests across both repos ≥ 1999.

What changes (operator-visible):

- `pip install -e ../mgf_common` becomes a setup prerequisite for vm-vmanager-app dev.
- `from vmanager.observability import setup_logging` no longer works (use `from mgf.common.observability import setup_logging`).
- `from vmanager.common.{shell,ssh,progress,guest_agent} import …` **keeps working** (those modules stay in vm-vmanager-app — only their internal imports of `vmanager.exceptions` / `vmanager.observability` change to `mgf.common.exceptions` / `mgf.common.observability`).

What changes (internal only):

- `docs/standards/` content lives in mgf_common.
- The dependency direction is now: vm-vmanager-app → mgf.common (one-way; enforced by import-linter).

---

## 12. Open questions / deferred items

These don't block round 5 — flagging for future rounds.

### 12.1 Deferred lifts (specifically the four `common/*.py`)

`vmanager/common/{shell,ssh,progress,guest_agent}.py` are catalogued and lift-ready, but stay in vm-vmanager-app for round 5 (per the tightened scope). When a second consumer needs them, the lift-time decoupling work is:

- **`shell.py`** — clean lift; no decoupling needed. Today it depends only on `mgf.common.exceptions.CommandError` (after Phase 3) and `mgf.common.observability.operation_span`.
- **`progress.py`** — clean lift; pure stdlib + `Protocol`.
- **`ssh.py`** — depends on `vmanager.vault.models.VMCredential`. To lift, define a structural-typing `Credential` Protocol in `mgf.common.ssh`:
  ```python
  @runtime_checkable
  class Credential(Protocol):
      @property
      def vm_name(self) -> str: ...
      @property
      def username(self) -> str: ...
      @property
      def auth_type(self) -> str: ...   # "password" | "key"
      @property
      def secret(self) -> str: ...
  ```
  `VMCredential` already has these four attributes — no code change at the call sites. `SSHClient.__init__` and `_credential_digest` change their `credential` parameter annotation from `VMCredential` to `Credential`.
- **`guest_agent.py`** — depends (TYPE_CHECKING-only) on `vmanager.domain.VMHandle` and `vmanager.providers.base.VMProvider`. To lift, define structural Protocols:
  ```python
  class HandleProtocol(Protocol):
      @property
      def name(self) -> str: ...
      @property
      def uuid(self) -> str: ...

  class GuestAgentProviderProtocol(Protocol):
      def guest_agent_command(
          self, handle: HandleProtocol, command: str,
          arguments: dict[str, object] | None = None,
      ) -> dict[str, object]: ...
  ```
  Both VMProvider and VMHandle already satisfy them structurally — no inheritance or call-site change needed.

### 12.2 Other deferred lifts

1. **`mgf.common.gui`** (CoreWorker, ui_loader, log_viewer_dialog, qt_excepthook) — Qt-specific. Lift in a later round once a second GUI consumer exists, OR keep in vm-vmanager-app indefinitely if Qt-specific code never gets a second user.
2. **`mgf.common.vault`** — VMCredential is generic but the storage backends (libsecret, encrypted file) are tied to vm-vmanager-app's deployment surface. Future round if needed.
3. **`mgf.common.providers.base`** — the `VMProvider` ABC is hypervisor-specific so it stays. But the *pattern* (Provider ABC + MockProvider + FailurePlan) is generic. Future round may extract a `mgf.common.provider_pattern` micro-library with the ABC machinery (lock-in tests, FailurePlan helper, base mixin).
4. **Versioning policy for the `BaseAppSettings.version` field** — today vm-vmanager-app treats `1` as the first/only version. When a project bumps to v2, the migration runs in `mgf.common.config._loader._migrate`. The current `_migrate` is a no-op stub. Round 5 keeps it as a no-op; future rounds add a registration mechanism (`@register_migration(from_version=1)`) when a real bump happens.

---

## 13. TL;DR

| Aspect | Answer |
|---|---|
| **What moves** | exceptions (base + categories), config (base), observability (all), all of docs/standards/ |
| **What stays in vm-vmanager-app** | All concrete domain code, AppConfig field schema, vm-vmanager-app-specific exception subclasses, vault, GUI, **common/{shell,ssh,progress,guest_agent}** (deferred — see §12.1) |
| **Import name** | `mgf.common` (PEP 420 namespace package) |
| **vm-vmanager-app consumes via** | `pip install -e ../mgf_common` (uv `[tool.uv.sources]` path dependency) |
| **vm-vmanager-app `vm-vmanager-appError`** | alias for `mgf.common.exceptions.AppError` (one-line back-compat) |
| **vm-vmanager-app `AppConfig`** | subclass of `mgf.common.config.BaseAppSettings` (adds vm-vmanager-app fields) |
| **`vmanager.common.{shell,ssh,...}`** | stay in vm-vmanager-app; only their internal `vmanager.exceptions` / `vmanager.observability` imports rewire to `mgf.common.*` |
| **Cutover style** | Clean — no deprecation shims; mypy + tests catch any miss |
| **Phases** | 1 bootstrap → 2 lift core → 3 switch vm-vmanager-app → 4 move docs → 5 move plan |
| **Test count target** | mgf_common ~110, vm-vmanager-app ~1890, total ≥ 1999 |
| **Path to PyPI** | Documented in §8 — separate future round |

---

*This document landed at `mgf_common/COMMON.md` at the end of Phase 5. vm-vmanager-app's `docs/COMMON.md` is now a one-line forwarding pointer. The standards docs followed the same pattern in Phase 4.*
