# Recipe: claim a device for VFIO passthrough

Hand a PCI function (or, equivalently, one or more USB interfaces) to
a running libvirt VM through `mgf-hrb`, take the device back when the
guest is done, and never leak ownership on a crash.

This is the **vm-vmanager-app pattern**: the host agent claims the device
through the broker (which atomically VFIO-binds + leases it,
ghost-free and capability-scoped), then hands the assignment to libvirt
to plug into the guest. Two systems meet here — `mgf-hrb` for the
**device** half, libvirt for the **VM** half — and the recipe owns the
seam between them.

> **What ships today vs. what's still designed.** §1–§3 below use the
> real `mgf.hrb` SDK API (commit `05da7a1`, `__version__ = "0.1.0"`):
> `Client.connect`, `ClaimMode`, `deadline_ms` on every call, typed
> exception hierarchy. §4 (event-driven release waits) uses
> `Client.monitor*` — those entry points are **reserved in the public
> API but raise `NotImplementedError` today** (Tier-1 hardening pass;
> `examples/README.md` carries the same caveat). The libvirt handoff
> in §1 is sketched against a future `mgf.hrb.libvirt` extra that has
> not yet been published — the broker half works now; the libvirt
> helper module is the next consumer-side wiring pass driven by
> vm-vmanager-app's real-world adoption.

---

## §1. The pattern

Four steps: **find → claim → attach → release-and-detach.**

```python
from mgf.hrb import Client, ClaimMode, ConflictError

SOCKET = "/tmp/mgf-hrb.sock"   # the skeleton broker's default

# 1. Connect to the broker. The capability the broker minted for this
#    client (cap_id) is presented on every call; the broker enforces
#    it server-side. `Client.connect` is a classmethod and the Client
#    is a context manager — the socket closes on scope exit (G3:
#    closing the connection reclaims any leases this client held).
with Client.connect(SOCKET, cap_id=my_cap_for_pci_devices) as client:

    # 2. Find the device by URN. The URN is stable across this
    #    session (topological — `hw:/pci/0000:01:00.0/fn0`).
    #    `client.list()` returns a `list[str]` of matching URNs.
    [target_urn] = [
        urn for urn in client.list("hw:/pci/**")
        if urn.endswith("0000:01:00.0/fn0")
    ]

    # 3. Claim exclusive ownership. The broker atomically VFIO-binds
    #    the device, isolates it in its IOMMU group, and grants you
    #    a lease. `deadline_ms` is the survivability-contract budget
    #    (D-029) — `acquire` must complete, error, or be abandonable
    #    within 5 s; on timeout, atomicity guarantees no lease is
    #    committed and the resource stays Free.
    try:
        with client.claim(target_urn, ClaimMode.EXCLUSIVE, deadline_ms=5_000) as lease:

            # 4. Hand the device to the VM via libvirt. The broker
            #    has handed you a raw_channel (an fd to /dev/vfio/<group>
            #    once VFIO acquire is wired); libvirt consumes it via
            #    the hostdev XML. The `mgf.hrb.libvirt` extra (planned)
            #    will encapsulate the attach/detach dance; until then,
            #    drive libvirt directly.
            raw = lease.raw(deadline_ms=1_000)

            # ... drive libvirt with `raw` here (see §3 below) ...
            # The guest is now running with the device. The broker is
            # NOT in the I/O path — the guest talks directly to hardware
            # via VFIO (G4 — owned access is broker-independent).
            ...

        # Leaving the `with` block releases the lease. The broker
        # safe-resets the device, tears down the IOMMU mapping (D-029
        # / revocation below the fd — the guest's outstanding access
        # faults instantly), and unbinds VFIO. If your process
        # crashes inside the block, the lease still ends (Part 3.2
        # G3 — leases are bound to liveness, ghost-free).
    except ConflictError:
        # Another tenant holds the device exclusively right now.
        ...
```

If the caller's process is killed mid-block, the broker's
`LivenessSource` (Part 5.5) detects the socket close, reclaims the
lease, and safe-resets the device. The next claimant — possibly the
operator manually re-attaching to the host — sees a clean device. **No
recovery code required on the caller side.**

---

## §2. Why claim through the broker, not VFIO directly

A direct VFIO-bind dance from inside vm-vmanager-app has been the working
pattern in many projects, including the original vm-vmanager-app. The
trade-offs that drive the migration to `mgf-hrb`:

| Concern | Direct VFIO | Via `mgf-hrb` |
|---|---|---|
| Multiple tenants competing for the same device | manual coordination across processes (file locks, dconf flags, `nm-ignore` hacks); race-prone | one lease, atomic grant, the broker is the arbiter |
| Device cleanup on crash | requires careful `errdefer` in the caller, often misses cases | broker reclaims on connection death; **structural** guarantee |
| IOMMU-group sibling devices | the caller must understand the group and detach every member | the broker hides it; the caller asks for one function |
| Auditing "who owns this device right now?" | scan `/sys/bus/pci/drivers/vfio-pci/` and correlate against process IDs | the broker's lease table is the answer (operator surface in `mgf-hrb scan` today) |
| Privilege | the caller needs root or `cap_sys_admin` | the caller needs a capability the broker minted from policy |

For a single-tenant host running one vm-vmanager-app, direct VFIO is
adequate. For a host with mixed workloads — multiple VMs, recovery
tools, host-driver users, GPU passthrough — the broker is the only
sustainable answer.

---

## §3. Scoped passthrough — the context manager

`client.claim(urn, ClaimMode.EXCLUSIVE)` returns a `Lease` that doubles
as a context manager. Leaving the block releases. This is the
**recommended shape** for bounded passthrough:

```python
with client.claim(
    "hw:/usb/bus1/port-2.3/dev/if0",
    ClaimMode.EXCLUSIVE,
    deadline_ms=5_000,
) as lease:
    raw = lease.raw(deadline_ms=1_000)
    # ... attach to libvirt, run guest-side workflow ...
# Detached here, even on exception.
```

For unbounded passthrough (the device stays with the VM until the VM
is shut down by a separate code path), the explicit form:

```python
lease = client.claim(urn, ClaimMode.EXCLUSIVE, deadline_ms=5_000)
try:
    raw = lease.raw(deadline_ms=1_000)
    # ... attach to libvirt, return both `lease` and the libvirt plan ...
    return lease, plan
except Exception:
    lease.release()
    raise
```

In both cases, the broker's liveness binding (Part 3.2) is the
ultimate safety net — even an `os._exit()`-shaped catastrophe releases
the lease cleanly.

---

## §4. Waiting for the guest to release the device  *(Tier-1 — planned)*

> **Status.** The three subscribe shapes below mirror Part 8.3 of the
> design. The underlying `Client.monitor()`, `Client.monitor_fd()`,
> and a future `Client.amonitor()` are **reserved in the public API
> but not yet wired** — calling them today raises
> `NotImplementedError`. The shapes are documented here so consumers
> can plan their architecture; live implementations land with the
> broker-side subscription protocol (post-Tier-0 hardening).
>
> **What works today:** poll `client.list(scope)` on a coarse interval,
> or block on `lease.release(...)` returning from inside the holding
> process. The subscribe API is the right answer once it ships.

### §4.1 Synchronous (CLI / worker thread)

```python
# Future shape — Tier-1 stub today.
with client.monitor(scope="hw:/pci/**") as events:
    for event in events:
        if event.kind == "state_changed" and event.urn == plan.device.urn:
            if event.new.state == "free":
                detach(plan)
                break
```

### §4.2 Async (asyncio service)

```python
# Future shape — Tier-1 stub today.
async with client.amonitor(scope="hw:/pci/**") as events:
    async for event in events:
        if event.kind == "state_changed" and event.urn == plan.device.urn:
            if event.new.state == "free":
                detach(plan)
                return
```

### §4.3 Qt (PySide6 main thread)

```python
# Future shape — Tier-1 stub today.
from PySide6.QtCore import QSocketNotifier

fd = client.monitor_fd(scope="hw:/pci/**")
notifier = QSocketNotifier(fd, QSocketNotifier.Read, self)
notifier.activated.connect(self.on_hw_event)

def on_hw_event(self, _fd):
    for event in client.drain_events(fd):
        if event.urn == self.plan.device.urn and event.new.state == "free":
            detach(self.plan)
```

The three layers subscribe to the *same* underlying stream; pick the
one that fits your loop.

---

## §5. Failure modes and what happens

| Failure | What `mgf-hrb` does | What you need to do |
|---|---|---|
| Your process crashes between claim and attach | broker releases the lease + safe-resets the device on connection close | nothing — recover and reclaim on restart |
| Hardware `acquire` takes too long | typed `TimeoutError` raised; atomicity holds (D-029) — no lease, device stays Free | catch `mgf.hrb.TimeoutError`, surface to the operator |
| libvirt rejects the hostdev (device in wrong group, capability missing) | broker call already returned the raw fd — the failure is on the libvirt side | catch the libvirt error, `lease.release()`, report |
| The broker dies | the device stays attached to the VM (G4 — direct access is broker-independent); your *control* operations pause until the broker restarts | your service shouldn't notice anything until it needs to make a new claim |
| The device DMA-misbehaves | the IOMMU contains it; the broker's interrupt-rate watchdog may revoke + quarantine (post-Tier-0) | accept the revocation; report to the operator |
| Another tenant tries to claim the same device | the lease lattice (Part 3.3) returns `ConflictError`; your lease stays valid | nothing |

The pattern that does *not* work — and that the broker prevents — is
*two* tenants quietly succeeding in claiming the same device. The
broker is the single source of ownership truth (G3).

---

## §6. Testing this pattern  *(planned — Tier-1)*

> The `mgf.hrb.testing` module (`FakeClient`, `FakeResource`,
> `install_fake`) is part of the public test surface designed in
> Part 8.6.1 of the design but **not yet published** — the testing
> fakes are the next SDK surface to land after the monitor wiring.
>
> **What works today:** run `examples/02_claim_and_use.py` against
> the skeleton broker (`zig build && ./hrb-code/zig-out/bin/mgf-hrb-server`)
> — the seeded fake PCI devices exercise the same claim/raw/release
> flow this recipe documents.

The intended test shape, for reference:

```python
# Future shape — pseudocode against a not-yet-shipped fakes module.
from mgf.hrb.testing import FakeClient, FakeResource, install_fake

def test_passthrough_releases_on_exception():
    pci_fn = FakeResource.pci_function(urn="hw:/pci/0000:01:00.0/fn0")
    with install_fake([pci_fn]) as fake_client:
        with pytest.raises(RuntimeError):
            with fake_client.claim(pci_fn.urn, ClaimMode.EXCLUSIVE, deadline_ms=5_000):
                raise RuntimeError("guest setup failed")
        # The lease must have been released — the resource is `free`.
        assert fake_client.inspect(pci_fn.urn).state == "free"
```

---

## §7. Related

- [`usb-event-monitor-with-role-interfaces.md`](usb-event-monitor-with-role-interfaces.md) — the canonical shape for the `monitor()` / `amonitor()` / `monitor_fd()` consumer side.
- [`track-a-device-across-a-mode-change.md`](track-a-device-across-a-mode-change.md) — the identity-comparator pattern for when the device re-enumerates mid-flow.
- [`examples/02_claim_and_use.py`](../examples/02_claim_and_use.py) — the runnable, today-API counterpart to this recipe.
- `mgf-hrb-complete-design.md` Part 3.5 (revocation), 3.8 (raw channel), 4.5 (leases + capabilities), 5.4 (fd-passing) for the underlying mechanics.
- `mgf-hrb-usb-worker.md` for the USB-specific shape of this same pattern.
