Adds docs/e2e-smoke.md — an ~5-minute runbook that walks the full
P1 happy path against a sibling restic/rest-server: bootstrap
admin, mint token with repo creds, enrol an agent, watch the
config.update push land, run a backup, confirm the snapshot, edit
creds and watch the second push fire. Per the design discussion
this is a runbook (not a Go integration test); the Playwright
version lands in P5-06.
GET /api/hosts/{id}/repo-credentials returns the redacted view —
{repo_url, repo_username, has_password} — so the UI can pre-fill
the edit form without ever pulling the password out of the AEAD
blob.
Marks P1-32 / P1-33 / P1-34 done in tasks.md.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
New internal/agent/secrets package: AEAD blob at
/var/lib/restic-manager/secrets.enc, atomic write (os.CreateTemp +
Sync + Rename), 0600. Key lives in agent.yaml as base64
(SecretsKey) — same trust boundary as the bearer token, minted on
first start via EnsureSecretsKey.
cmd/agent: dispatcher reads creds fresh from secrets.Load() on
each job rather than from in-memory config. config.update merges
the push with what's on disk and persists, so a daemon restart
keeps the latest values. Legacy plaintext repo_url/repo_password
in agent.yaml are silently migrated into secrets.enc on next start
and stripped from the YAML on the following save.
Tests: round-trip + wrong-key rejection + atomic-write
post-condition for secrets; key idempotence + legacy-field
parse/clear for config.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Operator-minted enrollment tokens now carry the repo URL/username/
password as one AEAD blob bound (via additional-data) to the token
hash. ConsumeEnrollmentToken re-encrypts under host_id and writes a
host_credentials row in the same tx as token-burn, so the binding
moves with the credential.
PUT /api/hosts/{id}/repo-credentials lets an operator edit creds
post-enrollment; merges with the existing blob, audits, and pushes
config.update if the agent is connected.
WS handler grows an OnHello hook that the HTTP layer wires to send
the host's decrypted creds as a config.update immediately after the
hello succeeds — synchronously, so a racing command.run lands after
the agent has its repo password.
Schema: 0002_host_credentials.sql adds enc_repo_creds to
enrollment_tokens and a host_credentials table (PK = host_id, FK
ON DELETE CASCADE).
Tests: round-trip token → consume → host_credentials with AAD swap
detection; no-creds path stays compatible.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Adds P1-32/33/34: encrypted repo creds carried on the enrollment token,
agent-side AEAD secrets file, end-to-end smoke. spec.md §4.2 and §7.3
rewritten to describe the full flow (server-issued at token time,
pushed via config.update on hello, persisted encrypted on the agent)
and to make the encrypted-file-now / OS-keyring-Phase-2 split
explicit.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
P2-18 captures the keypair + fingerprint-comparison enrollment flow
as a Phase 2 alternative to the token model. Includes guards
(rate limit, pending cap, hostname-collision flagging) and explicit
acceptance criteria.
P1-27 grows to mint encrypted repo creds alongside the token and
expose a one-click preconfigured-installer download from the
"Add host" form (cf. UrBackup Internet-mode push installer).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Agent calls restic snapshots --json after each successful backup
(60s timeout, separate from the backup ctx) and ships the projection
over the existing snapshots.report WS envelope. Failure here is
logged but doesn't fail the job — the next successful backup catches
the projection up.
Server-side ReplaceHostSnapshots is delete-then-insert plus a
hosts.snapshot_count update in one transaction so the dashboard's
per-host count stays consistent with the projection. New read
endpoint GET /api/hosts/{id}/snapshots returns the cached list with
a refreshed_at marker so the UI can show staleness when an agent
has been offline.
Schema: dropped the unused snapshots.repo_id FK (repos as a
first-class entity is P2 work), added short_id and refreshed_at
columns, switched the time index to DESC for the most-recent-first
list query. api.Snapshot gains short_id; size_bytes/file_count come
from the embedded summary block on restic 0.16+ and stay zero on
older clients.
Tests cover round-trip, authoritative replacement after forget+prune
shrinkage, and empty-after-wipe.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Self-hosted deployments already terminate TLS at Caddy/Traefik/nginx;
making the server do TLS too means double cert config, dual ACME
plumbing, and an untested code path. Drop RM_TLS_CERT/RM_TLS_KEY,
remove TLSEnabled() and the ListenAndServeTLS branch.
Replace the cookie's "Secure if TLS-in-process" check with a new
RM_COOKIE_SECURE flag (default true). Local HTTP-only testing sets
RM_COOKIE_SECURE=false; production is always behind a TLS proxy and
the cookie stays Secure.
Default port :8443 → :8080. docker-compose binds 127.0.0.1 only and
populates RM_TRUSTED_PROXY. spec.md §4.1/§10.1 rewritten with a
Caddyfile snippet and a hard "do not expose RM_LISTEN publicly"
warning. enrollResponse keeps cert_pin_sha256 in the shape but the
server can't introspect a cert it doesn't terminate — operator
pastes the proxy's hash into -cert-pin at install time.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Captures the state landed in this session:
Done (P1-01..03, P1-05, P1-06, P1-08..16, P1-17..20, P1-29):
HTTP server, store + schema, crypto, first-run bootstrap,
every API type with wire-shape tests, WS transport,
enrollment + hello + heartbeat round-trip, agent config +
service unit + WS client + sysinfo, restic wrapper, job
lifecycle store + run-now endpoint, agent runner.
Partial (P1-04, P1-07, P1-21, P1-31):
CSRF middleware lives with the UI work; audit middleware
sweep lives with rest of API; live job-log fan-out needs
the per-job browser hub; signed agent binaries deferred to
Phase 5.
Open (P1-22..28):
Snapshot listing, full UI suite (login, dashboard, host
detail, live job log, add-host, Tailwind build).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Lands the operator → server → agent → restic → server roundtrip for
on-demand backups. The flow:
POST /api/hosts/{id}/jobs {kind:"backup",args:["/path"]}
→ server creates a queued Job row
→ server emits command.run over WS to the host's agent
→ agent dispatcher spawns runner.RunBackup in a goroutine
→ runner spawns `restic backup --json`, parses each line
→ forwards: job.started, log.stream (every line), job.progress
(throttled to 1/sec), job.finished (with summary stats blob)
→ server WS handler persists those into jobs / job_logs
P1-16 internal/restic: thin Locate + Env wrapper that runs `restic
backup --json`, scans stdout/stderr, parses BackupStatus +
BackupSummary, calls back into a LineHandler so the agent can fan
out to log.stream + job.progress. Treats exit code 3 as
"succeeded with issues" (matches restic's contract).
P1-18 store: jobs accessors (CreateJob, MarkJobStarted,
MarkJobFinished, AppendJobLog, GetJob).
P1-19 server: POST /api/hosts/{id}/jobs creates the Job row,
validates kind, dispatches via Hub.Send, audit-logs the action.
P1-20 agent runner: wraps restic.RunBackup with throttled progress
emission. Sender abstraction was added to wsclient.Handler so
background goroutines can keep replying after dispatch returns.
P1-21 server WS: dispatchAgentMessage now persists job.started,
job.finished, log.stream into the database. Browser fan-out for
live tailing lands with the UI work.
Agent gets repo_url + repo_password from agent.yaml in plaintext
for now (mode 0600, owned by service user); spec.md §7.3's keyring
storage moves there in P2. config.update over WS overrides the
in-memory copy (does not persist).
Build clean; all tests pass. End-to-end with a real restic still
needs a host that has restic installed — wire shape verified by
the existing hello/heartbeat round-trip test.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
P1-14 deploy/install/restic-manager-agent.service: standard systemd
unit with the usual hardening switches (NoNewPrivileges, Protect*,
RestrictRealtime, MemoryDenyWriteExecute). Restart=always with a
5s backoff. Runs as a dedicated unprivileged restic-manager-agent
user; the install script creates it.
P1-29 deploy/install/install.sh: arch detection (amd64/arm64), pulls
the agent binary from /agent/binary, creates the service user
+ dirs (/etc/restic-manager, /var/lib/restic-manager), runs
enrollment via `agent -enroll-server -enroll-token`, lays down
the systemd unit, enables and starts it.
Honours the spec's "detect, don't auto-disable" rule for existing
schedulers: scans systemd timers, /etc/cron.d/*, /etc/cron.daily/*,
root crontab for restic-named entries and prints them with the
exact disable command — operator decides.
P1-31 server endpoints to ship the agent installation payload:
GET /agent/binary?os=linux&arch=amd64 → serves
<DataDir>/agent-binaries/restic-manager-agent-linux-amd64
GET /install/<file> → serves
<DataDir>/install/<file>
Both endpoints reject path traversal and return 404 if the file
isn't published. Operators drop the binaries + service unit into
these directories at release time. Signed-bundle verification is
deferred to Phase 5 OSS readiness.
All tests still pass.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Lands the protocol layer end-to-end: an agent can be enrolled
through the operator UI, store credentials, dial back to the server
over WS, complete the protocol_version handshake, and stay
connected with periodic heartbeats.
Server side:
- P1-09 ws.Hub: one Conn per host_id, last-write-wins eviction,
json envelope writer with a write mutex, reader, error envelopes.
- P1-09 ws.AgentHandler: bearer-auth, accept upgrade, hello-stage
(10s deadline, protocol_version checked against
api.MinAgentProtocolVersion → ErrProtocolTooOld with help URL on
reject), main read loop, defer hub register/unregister.
- P1-10 POST /api/agents/enroll consumes a one-time token, mints a
persistent agent bearer (sha-256 stored), creates a host row.
- P1-10 POST /api/enrollment-tokens (operator, session-auth)
issues a 1h one-time token.
- P1-11 hello upserts agent_version + restic_version +
protocol_version on the host row, flips status to online.
- P1-12 heartbeat touches last_seen_at; background sweeper marks
hosts offline after 90s without one.
- store: hosts table accessors, host_schedule_version,
enrollment_tokens FK on consumed_host dropped (audit-only field;
the token gets burned before the host row exists).
Agent side:
- P1-13 internal/agent/config: yaml at /etc/restic-manager/agent.yaml,
atomic Save (tmp+fsync+rename), Enrolled() helper.
- P1-15 internal/agent/wsclient: dial with bearer + optional
TLS cert pinning (sha-256 of leaf), exponential backoff with
jitter (1s → 60s cap), heartbeat goroutine, fatal handling for
ErrProtocolTooOld.
- P1-15 wsclient.Enroll: HTTP POST /api/agents/enroll with sysinfo.
- P1-17 internal/agent/sysinfo: hostname/OS/arch/restic-version
collection. restic detected by `restic version` parse; absent
restic doesn't block startup.
- cmd/agent: -enroll-server / -enroll-token flags drive first-run
enrollment then exit (so the install script can hand off to
systemd to run the persistent service).
End-to-end smoke verified: bootstrap → login → issue token →
enroll → run agent → server logs `ws agent connected` with the
right host_id and protocol_version 1.
All tests still pass.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
P1-01 chi router, slog request log, graceful shutdown via signal
context. Health endpoint, /api/auth/login, /api/auth/logout,
/api/bootstrap. Background sweeper for expired sessions and
enrollment tokens (15 min cadence).
P1-04 (sessions half) HttpOnly Secure-when-TLS cookie carrying a
base64url token; server stores SHA-256(token) so a stolen DB
doesn't yield credentials. Unknown user and bad password collapse
to the same 401 response code so a probe can't enumerate names.
P1-05 first-run admin bootstrap. On a fresh DB the server mints a
one-time token and prints it to stderr inside a banner. The
/api/bootstrap handler accepts {token, username, password},
creates the first admin, then becomes a 409 forever.
P1-07 (partial) audit hooks fire on auth.login and auth.bootstrap.
Full middleware-driven coverage lands with the rest of the API.
internal/server/config: env > YAML > defaults. RM_LISTEN /
RM_DATA_DIR / RM_BASE_URL / RM_TLS_CERT / RM_TLS_KEY /
RM_SECRET_KEY_FILE / RM_TRUSTED_PROXY (CIDR list, validated).
End-to-end smoke test passes: server boots on a fresh dir,
prints the bootstrap token, POST /api/bootstrap creates the admin,
POST /api/auth/login returns 200 with a session cookie.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Lands the bottom three layers of Phase 1:
P1-08 internal/api: protocol_version + envelope + every WS message
shape from spec.md §6.2 (Hello, Heartbeat, Job*, Schedule*, etc).
Wire-format tests pin the JSON shape so a rename here breaks
tests instead of silently breaking the agent.
P1-02 + P1-03 internal/store: SQLite via modernc.org/sqlite,
embed.FS + a tiny version table for hand-rolled migrations.
0001_initial.sql covers every table from spec.md §5 plus
enrollment_tokens and host_schedule_version. Typed accessors
for users / sessions / enrollment / audit. WAL + foreign_keys
+ busy_timeout on by default.
P1-06 internal/crypto: XChaCha20-Poly1305 AEAD wrapper with
per-message random nonce. Key file lifecycle (generate +
refuse-to-overwrite, load with size validation). Optional
additionalData binds ciphertext to the row that owns it.
P1-04 internal/auth (partial — passwords + tokens; sessions
middleware lands with the HTTP handlers): argon2id following
RFC 9106 (64 MiB / t=3 / p=4 / 32B), constant-time verify.
HashToken stores SHA-256 of session/agent/enrollment tokens
so a stolen DB doesn't hand over credentials.
Build floor moves to Go 1.25 (modernc.org/sqlite v1.50+ requires
it); CI + Dockerfile + README updated. Markdown lint diagnostics
on tasks.md cleared.
All packages tested. ~70 new tests pass in <1s.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Doc-only changes captured before any Phase 1 code lands.
spec.md:
- §4.1 nhooyr.io/websocket → github.com/coder/websocket (the
maintained fork; the original is unmaintained)
- §4.1 RM_LISTEN documented as source of truth for the bind port;
add RM_TRUSTED_PROXY env var for X-Forwarded-* handling behind
Caddy/Traefik
- §4.2 Phase 1 ships Linux only; Windows binaries continue to build
in CI to keep the codebase portable, but service integration +
installer move to Phase 2
- §4.2 self-update via apt/choco, not bespoke signed binaries
- §5 add Host.protocol_version + Host.applied_schedule_version
- §6.2 lock protocol_version handshake semantics (clean error on
mismatch, not weird JSON parse failures)
- §6.2 schedule reconciliation when server unreachable: agent keeps
firing last-known-good indefinitely; server's view canonical on
reconnect; UI surfaces drift via applied_schedule_version
- §6.2 schedule.set carries schedule_version; new schedule.ack
agent→server message
- §10.1 cross-reference RM_LISTEN ↔ compose port mapping
- §14.3 hooks rejected at validation on non-backup schedule kinds
tasks.md:
- P1-14 / P1-30 (Windows service + install.ps1) → Phase 2 as
P2-16 / P2-17
- P1-29 install.sh detects existing restic timers/cron and prints
disable commands, doesn't auto-disable
- Phase 1 acceptance: drop Windows from end-to-end criterion,
require windows cross-compile in CI
- P4-01 rewritten: package-manager-based update delivery
- P5-08 removed (duplicate of P4-08 Prometheus /metrics)
- Various references updated
No Go code changes; build still clean.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
steve