946b6db1379285cd71b3e244030f3042583e1e5e
16 Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
 steve
|
946b6db137 |
P2-02 (server side): schedule reconciliation push + ack handling
Server is now the source of truth for the agent's cron set.
* Helpers in schedule_push.go:
- loadScheduleSetPayload reads the host's schedules + canonical
version into the wire shape.
- pushScheduleSetOnConn writes directly to a just-handshaken conn
(avoids racing against Hub.Register on a brand-new connection).
- pushScheduleSetAsync is the post-CRUD flavour — no-op when the
host is offline (the next reconnect's on-hello path catches it
up, so a missed push is non-fatal).
- applyScheduleAck records what version the agent has confirmed.
* onAgentHello restructured: was returning early when the host had
no repo credentials, which made the schedule push unreachable for
fresh hosts. Split into pushRepoCredsOnHello (silent no-op on
ErrNotFound) + pushScheduleSetOnConn (always runs). Empty schedule
list is a valid push: tells the agent to drop stale cron entries.
* WS dispatcher gains an OnScheduleAck hook on HandlerDeps; the
http server wires it to applyScheduleAck. MsgScheduleAck moves
out of the "TODO(P2)" group into a real case that decodes the
payload and forwards to the callback.
* Schedule CRUD handlers each fire pushScheduleSetAsync after the
audit-log write so the agent picks up changes within seconds.
Tests cover:
- On-hello push of an already-created schedule, agent acks,
applied_schedule_version flips on the host row.
- Connect-then-CRUD: empty initial push (version 0), then a
follow-on push at version 1 after the operator creates a
schedule via REST.
Agent-side `schedule.set` handler (parse, replace local cron,
emit `schedule.ack`) is the remainder of P2-02 and lands with
P2-03's local scheduler.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
|
||
|
steve
|
4b075840a1 |
P2-01: schedule schema + CRUD API
The `schedules` table was already laid down in migration 0001; this
slice adds the Go-side data model, store CRUD with atomic version
bumps, and REST endpoints.
* `store.Schedule` + `RetentionPolicy` + `ScheduleOptions` typed
views (the wire form on the agent side keeps retention/options
as raw JSON since the agent just forwards them to restic).
* Store CRUD: CreateSchedule / GetSchedule / ListSchedulesByHost /
UpdateSchedule / DeleteSchedule. Each mutation bumps
`host_schedule_version` atomically in the same tx via UPSERT on
`host_schedule_version`. SetHostAppliedScheduleVersion records
what the agent has confirmed via schedule.ack (P2-02 will use it).
* REST endpoints under /api/hosts/{id}/schedules + /{sid}:
GET (list, with the version envelope so callers can detect
drift), POST (create), PUT (update — kind is immutable), DELETE.
* Validation: cron expressions parse via robfig/cron/v3 (same
parser the agent will use, so anything that validates here will
fire there); kind ∈ {backup, forget, prune, check} (init/unlock
are operator-only one-shot kinds, not schedulable); backup
schedules require ≥1 path; hooks rejected on non-backup kinds
(spec §14.3).
* All mutations audit-logged.
* Tests: store-level CRUD + version-bump invariants; REST happy
path (create→list→update→delete with version progression); REST
validation table covers each rejection code.
newTestServerWithHub now sets BootstrapToken so the schedules
handler tests can use the existing login flow without a parallel
test-server constructor.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
|
||
|
steve
|
bd434bd1d0 |
P1-26: live job log viewer + WS browser fan-out hub
Closes the P1-21 remainder.
internal/server/ws/jobhub.go — new JobHub. Per-job_id set of
subscribers; each gets a 64-deep buffered channel with a writer
goroutine. Broadcast is non-blocking: if a subscriber is slow,
its channel fills and messages are dropped for that subscriber
only — the agent's read loop is never blocked by a stuck browser.
The agent dispatchAgentMessage path mirrors job.started /
job.progress / log.stream / job.finished envelopes onto the hub
in addition to its existing persistence work. The wire shape is
the same end-to-end, so client-side JS switches on env.type the
same way Go code does.
GET /api/jobs/{id}/stream is the browser endpoint. Auth via
session cookie (HTTP layer); upgrade; subscribe; pump until
context closes.
GET /jobs/{id} renders the live log page. Three states (queued/
running/succeeded/failed) drive the header pill, the progress
bar block, the failure summary panel, and the action button
(Cancel job while running, Back to host afterwards). Already-
persisted log lines are server-rendered on initial load; new
lines arrive over the WS and append to #log-stream. Auto-scrolls
unless the user scrolls up (a "⇢ Follow" pill re-attaches).
On job.finished the page reloads after 600ms to pick up the
final-state header rendered server-side.
POST /hosts/{id}/run-backup now sets HX-Redirect → /jobs/{job_id}
on success so HTMX lands the operator straight on the live log.
For non-HTMX callers (curl / plain form post) it 303s to the
same target.
store.ListJobLogs returns persisted log lines for initial render
on page load.
Browser-verified end-to-end: enrol → run a real backup against a
sibling restic/rest-server → live progress + 11 log lines stream
in → succeeded pill + final stats land after page reload.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
|
||
|
steve
|
26a2b85e13 |
P1-25: host detail page (snapshots tab default)
GET /hosts/{id} renders the v1 host detail layout:
- persistent header: status dot (pulse if a job is in flight),
monospace name, tags, plus a metadata strip (os/arch, agent
version, restic version, "last seen Xs ago" or "online · last
heartbeat …").
- vitals strip: four tiles for last backup (status + relative
time), repo size, snapshot count, open alerts.
- sub-tabs: Snapshots is active; Jobs / Repo / Settings are
visible but inert until P2.
- snapshot table: short id, time (absolute), paths joined with
" · ", size, file count, restore button (disabled — wires up
in P3).
- right rail: run-now stack (backup live, forget/prune/check/
unlock disabled with the Phase tag), danger-zone remove panel
(also disabled for now).
Empty state: when a host has no snapshots yet, the table replaces
itself with a "no snapshots yet" prompt that includes the run-now
button (provided the agent is online).
Pagination cap of 50 most-recent snapshots; full pagination lands
when fleet sizes demand it.
Template helpers grew: comma() now accepts int / int32 / int64 so
templates don't fight Go's type inference; joinDot() concatenates
a []string with " · "; absTime() formats time.Time as
YYYY-MM-DD HH:MM:SS; the existing relTime() already accepts T or
*T after P1-27.
Browser-verified end-to-end with seeded fixture data.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
|
||
|
steve
|
dad8c7fe99 |
P1-27: Add host flow — form + minted-token result page
GET /hosts/new renders the focused two-column form (hostname,
tags, repo URL/username/password). POST /hosts/new validates,
mints a one-time token via the new mintEnrollmentToken helper —
shared with the existing JSON /api/enrollment-tokens endpoint —
and re-renders the same page in result state showing:
- the install command with RM_SERVER + RM_TOKEN filled in (and
an inline copy-to-clipboard button),
- an "awaiting agent connection" panel with the hostname
pre-filled,
- a troubleshooting list pointing at the most common reasons
the agent doesn't appear,
- back-to-dashboard / add-another-host links.
publicURL() resolves RM_BASE_URL first, falling back to scheme +
Host on the inbound request — useful for local smoke without a
proxy.
Browser-verified end-to-end: form submit → token minted → install
command renders with the right values from the form input.
template fn formatRelTime now accepts time.Time *or* *time.Time
so templates can pass either without fighting Go's lack of an
address-of operator.
Deferred: download-preconfigured-installer (a templated .sh with
the values baked in) — copy-paste covers v1; nice-to-have later.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
|
||
|
steve
|
ee16bc7ce7 |
P1-24: live dashboard — fleet summary tiles + host table
Server-rendered HTML view backed by:
- new store.FleetSummary aggregating host counts + repo bytes +
snapshot total + open alerts + last-24h job rollup in two queries.
- GET /api/hosts (JSON list of hosts in the dashboard projection).
- GET /api/fleet/summary (JSON aggregate, same shape as above).
The HTML page (web/templates/pages/dashboard.html) renders the four
summary tiles + host table directly from store data — no separate
fetch. Per-row state colour comes from .host-row.{degraded,failed,
offline} which paint a 3px left edge so problem hosts are scannable
without reading. HTMX is loaded into the base layout so per-row
"Run now" buttons can hx-post to /hosts/{id}/run-backup, a thin
HTML wrapper that funnels into a new dispatchJob helper shared
with the JSON /api/hosts/{id}/jobs endpoint.
Empty state (zero hosts) collapses to the "no hosts yet" prompt
with the + Add host CTA — matches the v1 mockup.
Template helpers (internal/server/ui/funcs.go) added for byte
formatting (412 GB / 3.7 TB), relative time (3m ago / 2d ago), and
comma grouping (1,847). Pure Go, no template-magic dependency.
Browser-verified end-to-end with seeded fixture data: five hosts
across all four states render with correct dots, accents, last-
backup pills, sizes, snapshot counts, alerts, tags, and the right
action button (Run now / Retry / Run first / View → / offline).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
|
||
|
steve
|
229f89fee2 |
P1-23 / P1-28: base layout, login, session-aware nav + Tailwind build
P1-28: Tailwind standalone CLI wired into the Makefile. `make tailwind` downloads the pinned v3.4.17 binary into bin/tailwindcss (gitignored), builds web/styles/input.css → web/static/css/styles.css. `make build` now runs the CSS pass first; `make tailwind-watch` for dev. Output is embedded in the binary via web.FS — single static binary, no Node. The CSS source carries every component class the v1 mockups defined (status dots, buttons, host row, log viewer, progress bar, fields, chips, snippet panel, empty state) so screens that land later can just reach for them. P1-23: html/template tree at web/templates with two layouts (base with chrome, chromeless for login + bootstrap), one nav partial, and two pages (dashboard placeholder, login). internal/server/ui parses the tree at startup; ui_handlers.go in the http package wires: GET / dashboard (303 → /login when unauthed) GET /login sign-in form POST /login consume form, mint session cookie, 303 → / POST /logout drop cookie, 303 → /login GET /static/* embedded Tailwind bundle The HTML login flow shares store/session logic with /api/auth/login via a new authenticateAndSession helper — same security guarantees, two surface representations (HTML form / JSON). Verified end-to-end: bootstrap → form-login → authed dashboard → sign-out → 303 cycle works in the browser; Tailwind output emits only the component classes referenced in the live templates (9.6kB minified). Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> |
||
|
steve
|
5d1951ad94 |
P1-34: e2e smoke runbook + redacted GET /repo-credentials
Adds docs/e2e-smoke.md — an ~5-minute runbook that walks the full
P1 happy path against a sibling restic/rest-server: bootstrap
admin, mint token with repo creds, enrol an agent, watch the
config.update push land, run a backup, confirm the snapshot, edit
creds and watch the second push fire. Per the design discussion
this is a runbook (not a Go integration test); the Playwright
version lands in P5-06.
GET /api/hosts/{id}/repo-credentials returns the redacted view —
{repo_url, repo_username, has_password} — so the UI can pre-fill
the edit form without ever pulling the password out of the AEAD
blob.
Marks P1-32 / P1-33 / P1-34 done in tasks.md.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
|
||
|
steve
|
e58917106d |
spec/tasks: pull repo-credential plumbing into Phase 1
Adds P1-32/33/34: encrypted repo creds carried on the enrollment token, agent-side AEAD secrets file, end-to-end smoke. spec.md §4.2 and §7.3 rewritten to describe the full flow (server-issued at token time, pushed via config.update on hello, persisted encrypted on the agent) and to make the encrypted-file-now / OS-keyring-Phase-2 split explicit. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> |
||
|
steve
|
6c9558c703 |
tasks: add P2-18 announce-and-approve, expand P1-27 with preconfigured installer
P2-18 captures the keypair + fingerprint-comparison enrollment flow as a Phase 2 alternative to the token model. Includes guards (rate limit, pending cap, hostname-collision flagging) and explicit acceptance criteria. P1-27 grows to mint encrypted repo creds alongside the token and expose a one-click preconfigured-installer download from the "Add host" form (cf. UrBackup Internet-mode push installer). Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> |
||
|
steve
|
3904a78f14 |
P1-22: snapshot listing via restic snapshots --json
Agent calls restic snapshots --json after each successful backup
(60s timeout, separate from the backup ctx) and ships the projection
over the existing snapshots.report WS envelope. Failure here is
logged but doesn't fail the job — the next successful backup catches
the projection up.
Server-side ReplaceHostSnapshots is delete-then-insert plus a
hosts.snapshot_count update in one transaction so the dashboard's
per-host count stays consistent with the projection. New read
endpoint GET /api/hosts/{id}/snapshots returns the cached list with
a refreshed_at marker so the UI can show staleness when an agent
has been offline.
Schema: dropped the unused snapshots.repo_id FK (repos as a
first-class entity is P2 work), added short_id and refreshed_at
columns, switched the time index to DESC for the most-recent-first
list query. api.Snapshot gains short_id; size_bytes/file_count come
from the embedded summary block on restic 0.16+ and stay zero on
older clients.
Tests cover round-trip, authoritative replacement after forget+prune
shrinkage, and empty-after-wipe.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
|
||
|
steve
|
77a305d064 |
tasks.md: mark Phase 1 progress
Captures the state landed in this session: Done (P1-01..03, P1-05, P1-06, P1-08..16, P1-17..20, P1-29): HTTP server, store + schema, crypto, first-run bootstrap, every API type with wire-shape tests, WS transport, enrollment + hello + heartbeat round-trip, agent config + service unit + WS client + sysinfo, restic wrapper, job lifecycle store + run-now endpoint, agent runner. Partial (P1-04, P1-07, P1-21, P1-31): CSRF middleware lives with the UI work; audit middleware sweep lives with rest of API; live job-log fan-out needs the per-job browser hub; signed agent binaries deferred to Phase 5. Open (P1-22..28): Snapshot listing, full UI suite (login, dashboard, host detail, live job log, add-host, Tailwind build). Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> |
||
|
steve
|
c275f4ff4c |
phase 1 foundations: api types, store, crypto, auth
Lands the bottom three layers of Phase 1: P1-08 internal/api: protocol_version + envelope + every WS message shape from spec.md §6.2 (Hello, Heartbeat, Job*, Schedule*, etc). Wire-format tests pin the JSON shape so a rename here breaks tests instead of silently breaking the agent. P1-02 + P1-03 internal/store: SQLite via modernc.org/sqlite, embed.FS + a tiny version table for hand-rolled migrations. 0001_initial.sql covers every table from spec.md §5 plus enrollment_tokens and host_schedule_version. Typed accessors for users / sessions / enrollment / audit. WAL + foreign_keys + busy_timeout on by default. P1-06 internal/crypto: XChaCha20-Poly1305 AEAD wrapper with per-message random nonce. Key file lifecycle (generate + refuse-to-overwrite, load with size validation). Optional additionalData binds ciphertext to the row that owns it. P1-04 internal/auth (partial — passwords + tokens; sessions middleware lands with the HTTP handlers): argon2id following RFC 9106 (64 MiB / t=3 / p=4 / 32B), constant-time verify. HashToken stores SHA-256 of session/agent/enrollment tokens so a stolen DB doesn't hand over credentials. Build floor moves to Go 1.25 (modernc.org/sqlite v1.50+ requires it); CI + Dockerfile + README updated. Markdown lint diagnostics on tasks.md cleared. All packages tested. ~70 new tests pass in <1s. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> |
||
|
steve
|
595546afb9 |
spec/tasks: address pre-Phase-1 design feedback
Doc-only changes captured before any Phase 1 code lands. spec.md: - §4.1 nhooyr.io/websocket → github.com/coder/websocket (the maintained fork; the original is unmaintained) - §4.1 RM_LISTEN documented as source of truth for the bind port; add RM_TRUSTED_PROXY env var for X-Forwarded-* handling behind Caddy/Traefik - §4.2 Phase 1 ships Linux only; Windows binaries continue to build in CI to keep the codebase portable, but service integration + installer move to Phase 2 - §4.2 self-update via apt/choco, not bespoke signed binaries - §5 add Host.protocol_version + Host.applied_schedule_version - §6.2 lock protocol_version handshake semantics (clean error on mismatch, not weird JSON parse failures) - §6.2 schedule reconciliation when server unreachable: agent keeps firing last-known-good indefinitely; server's view canonical on reconnect; UI surfaces drift via applied_schedule_version - §6.2 schedule.set carries schedule_version; new schedule.ack agent→server message - §10.1 cross-reference RM_LISTEN ↔ compose port mapping - §14.3 hooks rejected at validation on non-backup schedule kinds tasks.md: - P1-14 / P1-30 (Windows service + install.ps1) → Phase 2 as P2-16 / P2-17 - P1-29 install.sh detects existing restic timers/cron and prints disable commands, doesn't auto-disable - Phase 1 acceptance: drop Windows from end-to-end criterion, require windows cross-compile in CI - P4-01 rewritten: package-manager-based update delivery - P5-08 removed (duplicate of P4-08 Prometheus /metrics) - Various references updated No Go code changes; build still clean. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> |
||
|
steve
|
c9368de904 |
phase 0: project bootstrap
P0-01 Go module + cmd/server + cmd/agent skeletons + internal/ tree
P0-02 LICENSE (PolyForm NC 1.0.0), README, CONTRIBUTING
P0-03 golangci-lint, pre-commit, .editorconfig, .gitignore
P0-04 Gitea Actions CI: test (race+coverage), lint, cross-platform build matrix
P0-05 Dockerfile.server (multi-stage, distroless/static), docker-compose.yml
P0-06 Makefile with build/test/lint/fmt/run/release targets
build, vet, test, and cross-compile to linux/{amd64,arm64} + windows/amd64
all verified locally.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
|
||
|
steve
|
7612687a14 | initial setup ready |