Skip to content

Guest Agent

Every microVM built with mkGuest includes mvm-guest-agent, a lightweight Rust daemon that runs inside the guest on vsock port 52.

CapabilityDescription
Health checksRuns per-service health commands on a schedule, reports results to the host
Worker statusTracks idle/busy state by sampling /proc/loadavg — used by fleet autoscaling
Snapshot lifecycleCoordinates sleep/wake: flushes data, drops page cache before snapshot, signals restore
Integration managementLoads service definitions from /etc/mvm/integrations.d/*.json
ProbesLoads read-only system checks from /etc/mvm/probes.d/*.json (disk usage, custom metrics)
Filesystem diffWalks the overlay upper dir to report files created, modified, or deleted since boot
Remote commandDev-only: execute commands inside the guest via vsock

The agent communicates using length-prefixed JSON frames over vsock (Firecracker, Apple Container, microvm.nix) or a unix socket (Docker):

  1. Host writes CONNECT 52\n to the socket
  2. Agent responds with OK 52\n
  3. All subsequent communication is request/response pairs

Request types: ping, status, sleep-prep, wake, and more.

Health checks defined in mkGuest’s healthChecks parameter are automatically written to /etc/mvm/integrations.d/ at build time:

{
"name": "my-service",
"health_cmd": "curl -sf http://localhost:8080/health",
"health_interval_secs": 10,
"health_timeout_secs": 5
}

The agent picks them up on boot and begins periodic checks immediately.

Services that take time to initialize (e.g., running database migrations) can specify a grace period. During the grace period, health check failures are suppressed and the service reports Starting status instead of Error:

{
"name": "my-service",
"health_cmd": "curl -sf http://localhost:8080/health",
"health_interval_secs": 10,
"health_timeout_secs": 5,
"startup_grace_secs": 120
}

In a Nix flake, set the grace period via startupGraceSecs:

healthChecks.my-app = {
healthCmd = "curl -sf http://localhost:8080/health";
healthIntervalSecs = 10;
startupGraceSecs = 120; # suppress failures for 2 minutes after boot
};

After the grace period expires, normal health reporting resumes.

Terminal window
# Check guest console output
mvmctl logs my-vm
# Follow logs in real time
mvmctl logs my-vm -f
# List VMs and their status
mvmctl ls

Health check results and probe output are included in the guest console logs.

Probes are read-only system checks loaded from /etc/mvm/probes.d/*.json:

{
"name": "disk-usage",
"command": "df -h /mnt/data | tail -1 | awk '{print $5}'",
"interval_secs": 60
}

Probe results are reported via the vsock protocol and included in guest console logs.

Before creating a snapshot, the host sends a sleep-prep request. The agent:

  1. Runs checkpoint commands for each integration
  2. Syncs filesystem buffers
  3. Drops page cache
  4. Responds with “ready”

On wake (snapshot restore), the host sends a wake request and the agent runs restore commands for each integration.

The agent can report all filesystem changes since boot by walking the overlay upper directory. When the rootfs is mounted read-only with an overlay (readOnlyRoot = true in mkGuest), all writes go to the upper dir. The agent detects:

  • Created files: present in the overlay upper dir
  • Deleted files: overlay whiteout files (.wh.*)
  • Modified files: existing files overwritten in the upper dir

Query the diff from the host:

Terminal window
mvmctl diff my-vm # human-readable output
mvmctl diff my-vm --json # JSON array of {path, kind, size}

This is useful for auditing what an AI agent modified during execution.