Phase 3 — Restore (P3-X1, X2, 01, 02, 03, 09, X3-X6) #6
Reference in New Issue
Block a user
Delete Branch "p3-restore"
Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?
Summary
Phase 3, Restore sub-phase. Operator can pick a snapshot, drill into a tree
browser, choose paths, dispatch a restore, watch live progress, cancel
mid-flight, download the log, and (separately) diff two snapshots — all from
the UI.
Spec:
docs/superpowers/specs/2026-05-04-p3-restore-design.md. Wireframe:_diag/p3-restore-wizard/wireframe.html. Sweep screenshots:_diag/p3-restore-sweep/.Commits (in order)
454a241) —spec doc, P3-04 cross-host moved to Future / unscheduled.
9fa2ef4) —command.cancelenvelope, agenttracks per-job ctx.CancelFunc, SIGTERM (5s grace) → SIGKILL via
cmd.Cancel+cmd.WaitDelay. Wires the existing UI Cancel button.6d295bc) —generic
Hub.SendRPC,restic lswrapper with direct-children filter,30-min in-memory wizard-session cache.
265b4b6) —JobRestore+JobDiff,restic.RunRestore/RunDiff, runner translation,RestorePayload/DiffPayload.page (
6e47efc) —/hosts/{id}/restorefour-step wizard, tree partial,host.restoreaudit, restore-shaped progress widget, migration 0012.1111124) —POST /hosts/{id}/snapshots/diff+ UI panel +last restorechrome line.e22b41d) —.snap-rowCSS, tree expand JS, defaulttarget moved into ReadWritePaths-friendly path.
a2398d0) — backed byjob_logstable; works any time, no pause needed against the live WS.lint (
f0dfa68) —\$HOME/rm-restore/<job-id>/default + agent-sideexpansion,
Env.AtLeastVersion(0,17)-gated--no-ownership, UK locale +73-correction sweep.
8e06bc7) — nativedetails/summarydropdown, reusable.dropdowntokens.(
0225067) —LegacyResticflag +titleattribute on column headers.(
e4031d2) —os.MkdirAllon new-dir restore so operator-typed pathsjust work; tasks.md refreshed with all six P3-X follow-ups.
What's new for operators
/hosts/{id}/restore(or/hosts/{id}/snapshots/{sid}/restoreto deep-link)..txtfor humans,.ndjsonforjq.What's deferred
tasks.md. Disaster recovery is already covered by re-enrolling a replacement
host with the same repo creds; the file-sharing use case isn't confirmed.
Migration
Migration 0012 widens the
jobs.kindCHECK to includerestoreanddiff.Rebuild required (SQLite can't ALTER CHECK in place); follows the safe pattern
from 0005, with a defensive temp-table backup of
job_logsso the cascadetrap that bit migration 0007 wouldn't take the log history with it.
Restage notes
Per CLAUDE.md, this PR touches agent code, the systemd unit, and install.sh —
the full restage block applies on smoke env. install.sh now pre-creates
/root/rm-restore(root-owned 0700) so the default new-dir target works underthe sandbox out of the box; existing installs need a re-run of
install.shtopick up the new dir.
Test plan
\$HOME/rm-restore/<job-id>/, step 4 dispatches./root/rm-restore/<job-id>/...on the agent host.cancelled..txtand.ndjsonwith the right Content-Disposition; both work mid-run and post-run.cursor: help).go test ./...pass;go vetclean.🤖 Generated with Claude Code
Wires the existing job_detail Cancel button (which was a UI stub) into real backend behaviour: - internal/api already declared MsgCommandCancel + CommandCancelPayload; promote those from forward-declarations to a working envelope. Agent side: cmd/agent/main.go drops the TODO-stub and gains a per-job ctx.CancelFunc map. runJob's switch is refactored around a small spawn() helper so each kind's goroutine derives a per-job context, registers the cancel, and removes itself on completion regardless of outcome. command.cancel looks up the func and fires it. - internal/agent/runner.sendFinished now takes ctx and rebadges ctx.Canceled errors as JobCancelled (exit 130) rather than JobFailed. All Run* call sites updated. - internal/restic.resticCmd sets cmd.Cancel to send SIGTERM (via build-tagged sigterm constant; os.Kill on Windows since SIGTERM isn't deliverable there) and cmd.WaitDelay=5s for the SIGKILL fallback. SIGTERM lets restic remove its lock file before exiting. - New POST /api/jobs/{id}/cancel server endpoint validates the job is non-terminal and the host is online, sends command.cancel via the hub, writes a job.cancel audit row, returns 202. The agent's resulting job.finished (status=cancelled) is what actually transitions the row. Tests: - internal/server/http/cancel_test.go covers happy path (envelope shape + audit row), 409 for terminal jobs, 404 for missing jobs, 503 for offline hosts. - internal/agent/runner/cancel_test.go covers cancel mid-run: a fake restic that exec'd into 'sleep 30' is canceled 150ms after start and the resulting job.finished reports JobCancelled with exit 130 in well under the WaitDelay. Foundational for P3 restore (operator needs to be able to cancel a running backup if they need to restore urgently). Independently useful for prune/check/backup that are stuck.Wires JobRestore and JobDiff end-to-end at the agent layer (the wizard backend that drives this lands in the next slice). - internal/api: JobRestore + JobDiff JobKind constants. CommandRunPayload grows nullable Restore + Diff sub-payloads. RestorePayload carries snapshot_id, paths, in_place, target_dir; DiffPayload carries snapshot_a + snapshot_b. - internal/restic.RunRestore wraps 'restic restore <sid> --target ... [--no-ownership] [--include p]...' with --json. New pumpRestoreStdout parses the per-line status / summary objects (drops raw status from log.stream — the throttled job.progress envelope covers it). New RestoreStatus + RestoreSummary types mirror restic's wire shape. - internal/restic.RunDiff wraps 'restic diff --json <a> <b>'. - internal/agent/runner: RunRestore translates RestoreStatus into job.progress (mapping FilesRestored → FilesDone etc) with a small estimateETA helper since restic doesn't provide ETA for restore. RunDiff is a thin streamHandler wrapper. - cmd/agent dispatcher gains JobRestore + JobDiff cases. Both reuse the spawn() helper from P3-X1 so cancel just works. - Drive-by fix: lastProgress was initialised to time.Now() so the very first status event was suppressed by the 1s throttle if the agent reported quickly. Initialise to time.Time{} (zero) so the first event always emits. Affects backup + restore. Tests: - restore_test covers restore happy path (started → progress → finished, kind=restore on the started envelope), in-place argv asserts no --no-ownership, new-dir argv asserts --no-ownership + --target + --include, diff produces the expected log.stream lines. Restage block (CLAUDE.md) is deferred to the end of the restore sub-phase so we restage once with all changes.End-to-end wizard from /hosts/{id}/restore (or per-snapshot deep link /hosts/{id}/snapshots/{sid}/restore) → tree-browse → dispatch → restore-shaped live job page. Backend (internal/server/http/ui_restore.go): - GET handlers render the four-step wizard against the wireframe shape in docs/superpowers/specs/2026-05-04-p3-restore-design.md. - HTMX tree partial endpoint hits fetchTreeWithCache (P3-X2) so each directory expansion is a sub-second cached lookup after the first miss. - POST validates: snapshot_id non-empty, ≥1 absolute path, in-place mode requires confirm_hostname == host name, agent online. On error re-renders the wizard with the operator's input intact. Happy path mints a job_id, computes the new-directory target as /var/restic-restore/<job-id>/ (operator can't escape the prefix — server picks it), creates the job row, ships command.run with kind=restore + RestorePayload, writes a host.restore audit row, returns HX-Redirect (or 303) to the live job page. Templates: - host_restore.html: single-page progressively-enabled wizard matching _diag/p3-restore-wizard wireframe. Form-state-driven JS computes a running tally of selected paths and the step-4 confirm summary client-side; the server re-renders on validation failure with form fields preserved. - partials/tree_node.html: recursive HTMX-served tree fragment. - Top-level Restore button on host_detail right rail + per-snapshot Restore action on snapshot rows replace the previous P3-stub. Restore-shaped job page (job_detail.html): - Progress widget rendered as a panel rather than a bare strip when the job is active. - Current-file display under the bar, updated from log.stream stdout lines that look like absolute paths. Hidden for non-restore kinds. Migration 0012: - Add restore + diff to the jobs.kind CHECK. Rebuild required (SQLite can't ALTER CHECK in place); follows the safe pattern from 0005. Defensive: stash job_logs into a temp table before the rebuild and INSERT OR IGNORE back afterwards so even if SQLite cascades on DROP TABLE jobs the log history survives. Tests: - ui_restore_test covers GET step-1 render, GET pre-selected snapshot summary card, POST missing snapshot, POST missing paths, POST in-place wrong-hostname rejection (no command.run leaks to the agent), POST happy path (HX-Redirect + correct payload + audit row), POST against offline host returns 503. Restage block (CLAUDE.md) deferred to the end of the restore phase.P3-09 — snapshot diff dispatcher. - POST /api/hosts/{id}/snapshots/diff (and the unprefixed HTMX-form variant) takes {snapshot_a, snapshot_b}, validates both belong to the host (long id / short id / prefix match), checks the agent is online, mints a JobDiff, ships command.run with DiffPayload, writes a host.snapshot_diff audit row, returns HX-Redirect to the live job page (or JSON {job_id, job_url} for REST callers). - Two-snapshot guard: POSTing diff(a,a) returns 422. - UI: small panel on the host_detail right rail (visible when the host has 2+ snapshots) with two short-id inputs and a Diff button. Output renders on the standard live job page where the operator reads the per-line diff text directly. P3-X3 — recent-restores line. - hostChromeData grows RestoreStatus / RestoreAt / RestoreJobID populated via store.LatestJobByKind(host_id, 'restore') (already exists, used by the init line). - host_chrome.html renders a small line below the existing init-status one with status-coloured copy + a link to the job log. Hidden when no restore has ever run on this host. Tests: - diff_test covers happy path (correct DiffPayload + HX-Redirect), same-id rejection (422), unknown-id rejection (422). Adds a seedTwoSnapshots helper since ReplaceHostSnapshots is atomic-swap (calling seedSnapshot twice would only leave the second). Restage block (CLAUDE.md) deferred to the end of the restore phase.Bug fixes from the Playwright sweep against the live smoke server: 1. Snapshot-picker layout. The .snap-row class was used in the wireframe but never landed in web/styles/input.css; rows rendered as vertical blocks instead of a 6-column grid. Added the token (mirrors host-row shape with restore-specific column widths). 2. Tree expansion. hx-target='closest .tree-row + .tree-children' isn't a valid HTMX selector — modifiers don't chain. Replaced HTMX-driven expansion with a small window.__rmTreeToggle helper that uses plain fetch + .tree-pair wrapper structure for trivial sibling lookup. Caches loaded state per node. 3. --no-ownership flag dropped. Restic 0.17 introduced --no-ownership; 0.16 rejects it ('unknown flag') before doing any work. Since the agent runs as root in the systemd unit, restored files keep their original uid/gid either way and the parent dir is root-owned, so the 'cp without sudo' rationale doesn't hold. Drop the flag entirely. 4. Default target dir moved to /var/lib/restic-manager/restore. The systemd unit pins ReadWritePaths to /etc/restic-manager + /var/lib/restic-manager (with ProtectSystem=strict making the rest of /var read-only); writes to /var/restic-restore failed with 'read-only file system'. 5. Confirm summary HTML escaping. defaultTarget JS literal evaluates to a string with literal angle brackets; insertion into innerHTML must escape them. Added an inline HTML-escape pass. tasks.md ticked for the Restore sub-phase with a sweep summary covering the live end-to-end test.The diff job's full output streams to the standard live job log page, which can be a lot of text the operator wants to grep through or paste into a ticket. Add a Download button. Source of truth is the persisted job_logs table — works any time (running or finished) and doesn't need to pause the live WS stream. The download is 'everything the server has up to right now'; if the operator wants a fuller snapshot of a still-running job, they hit Download again. - New endpoint GET /api/jobs/{id}/log.{txt,ndjson} (chi {format} matcher constrained to the two known suffixes). Auth via session cookie. 404 on unknown job. - internal/server/http/job_download.go writeLogsText emits a small header + 'HH:MM:SS.mmm TAG payload' rows mirroring what the live page shows. writeLogsNDJSON emits one self-contained {seq,ts,stream, payload} JSON object per line — appending stays valid (each line stands alone), and the whole file pipes cleanly into jq. NDJSON is newline-delimited JSON; not the same as a JSON array. - web/templates/pages/job_detail.html grows two header buttons: 'Download log' (txt) + '.ndjson' ghost variant for tooling. Tests cover the txt format (header + per-row shape), the ndjson format (each line round-trips through json.Unmarshal), unknown job 404, unauthenticated 401.Three small follow-ups from review: 1. Restore target is now operator-editable. Default value is the literal '\$HOME/rm-restore/<job-id>/' (agent expands \$HOME at run time using os.UserHomeDir(); also handles \${HOME} and ~/ prefixes). Operator can replace with any absolute path. - ui_restore.go validates the input is either absolute or starts with one of the recognised prefixes; other env-var refs (\$PATH etc.) are deliberately rejected so operator paths can't pick up arbitrary agent env values. - host_restore.html replaces the read-only mono-text display with a real <input>; help text spells out that \$HOME resolves agent-side and <job-id> is substituted on dispatch. - install.sh + the systemd unit prep /root/rm-restore so the default works under the sandbox: ReadWritePaths gains a soft '-/root/rm-restore' entry (the '-' makes the bind-mount soft-fail if missing, but install.sh pre-creates it root-owned 0700). 2. --no-ownership flag now gated on restic version. The flag was added in restic 0.17 and 0.16 rejects it. Previously dropped it wholesale — that meant new-dir restores silently preserved ownership against design intent on 0.17+. Now the agent threads its detected restic version (sysinfo already collects it) through runner.Config -> restic.Env, and RunRestore appends --no-ownership only when AtLeastVersion(0, 17) returns true. 0.16 hosts still restore with original uid/gid; help text in the wizard explicitly notes this. The previous 'Original ownership is preserved' copy was wrong for new-dir mode and is corrected. 3. golangci-lint misspell locale switched US -> UK and the codebase swept (73 corrections, mostly behaviour/serialise/recognise/honour). Wire-format ErrorCode 'unauthorized' -> 'unauthorised' is a tiny contract change but the agent doesn't parse those codes today and no external API consumers exist yet. Tests passed before + after. Tests: - internal/restic/version_test.go covers Env.AtLeastVersion across edge cases (empty, exact match, patch above, minor below, non- numeric) and expandHome on \$HOME / \${HOME} / ~/, plus pass-through for absolute paths and refusal of other env vars. - ui_restore_test updated: TargetDir now starts '\$HOME/rm-restore/' with the job_id substituted into the placeholder. Live verified on the smoke env: default target restored to /root/rm-restore/<job-id>/ as the agent's expanded \$HOME (2 files, 14 bytes); custom override '/tmp/custom-restore/<job-id>/' restored into the agent's PrivateTmp namespace (1 file, 6 bytes); both jobs 'succeeded', exit 0.1. Agent-side MkdirAll on the new-dir restore target. Restic creates missing leaves but won't traverse multiple missing levels, and under the systemd sandbox writes outside ReadWritePaths fail anyway. Calling os.MkdirAll(target, 0700) before invoking restic means the operator never has to pre-create the per-job subdir, and a path the sandbox rejects surfaces as a clean 'restic restore: prepare target ...: read-only file system' error in the job log instead of a cryptic restic-side stat failure. 2. tasks.md Phase 3 — Restore section refreshed: - P3-X4 added (job log download dropdown — txt + ndjson) - P3-X5 added (UK lint locale switch + 73-correction sweep) - P3-X6 added (SIZE/FILES tooltip when host's restic < 0.17) - P3-03 entry expanded to cover version-gated --no-ownership, editable target, $HOME expansion, agent-side MkdirAll - As-shipped sweep summary mentions custom-target restore + download dropdown + tooltip in addition to the original walk Test: TestRunRestoreNewDirAutoCreatesTarget seeds a multi-level target the operator hasn't created and confirms RunRestore mkdir's the chain before invoking restic.