P1-34: e2e smoke runbook + redacted GET /repo-credentials
Adds docs/e2e-smoke.md — an ~5-minute runbook that walks the full
P1 happy path against a sibling restic/rest-server: bootstrap
admin, mint token with repo creds, enrol an agent, watch the
config.update push land, run a backup, confirm the snapshot, edit
creds and watch the second push fire. Per the design discussion
this is a runbook (not a Go integration test); the Playwright
version lands in P5-06.
GET /api/hosts/{id}/repo-credentials returns the redacted view —
{repo_url, repo_username, has_password} — so the UI can pre-fill
the edit form without ever pulling the password out of the AEAD
blob.
Marks P1-32 / P1-33 / P1-34 done in tasks.md.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
This commit is contained in:
@@ -0,0 +1,273 @@
|
|||||||
|
# End-to-end smoke test (P1-34)
|
||||||
|
|
||||||
|
A runbook for verifying the Phase 1 happy path against a real
|
||||||
|
`restic/rest-server`. Run this on any Linux host with Docker; nothing
|
||||||
|
here touches your real Proxmox cluster or Unraid storage.
|
||||||
|
|
||||||
|
The test exercises:
|
||||||
|
|
||||||
|
1. Operator mints an enrollment token **with repo creds** (P1-32).
|
||||||
|
2. Agent enrols, server burns the token, host_credentials row lands.
|
||||||
|
3. Agent connects over WS, server pushes `config.update` containing
|
||||||
|
the decrypted creds **before** the agent sees any command.
|
||||||
|
4. Agent persists creds into `secrets.enc` (P1-33).
|
||||||
|
5. Run-now backup against the live `restic/rest-server`.
|
||||||
|
6. `snapshots.report` updates the per-host projection.
|
||||||
|
7. `GET /api/hosts/{id}/snapshots` returns the new snapshot.
|
||||||
|
|
||||||
|
Total time: ~5 minutes on a warm machine.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Prereqs
|
||||||
|
|
||||||
|
- Docker + Docker Compose
|
||||||
|
- `restic` v0.16+ on the host running the agent (the agent does **not**
|
||||||
|
install it; that's a deliberate design choice — see spec §4.2)
|
||||||
|
- `curl`, `jq`
|
||||||
|
|
||||||
|
## Layout
|
||||||
|
|
||||||
|
Everything lives under `/tmp/rm-smoke/`. Nothing escapes it; remove the
|
||||||
|
directory to clean up.
|
||||||
|
|
||||||
|
```
|
||||||
|
/tmp/rm-smoke/
|
||||||
|
├── compose.yaml # rest-server + control-plane
|
||||||
|
├── data/ # control-plane SQLite + secret key
|
||||||
|
│ └── agent-binaries/ # built agent binaries served by /agent/binary
|
||||||
|
├── rest/ # rest-server data volume
|
||||||
|
│ └── htpasswd
|
||||||
|
└── agent/ # this host plays the part of an endpoint
|
||||||
|
├── etc/ # → bind-mounted as /etc/restic-manager
|
||||||
|
└── var-lib/ # → bind-mounted as /var/lib/restic-manager
|
||||||
|
```
|
||||||
|
|
||||||
|
## 1. Build the binaries
|
||||||
|
|
||||||
|
```sh
|
||||||
|
mkdir -p /tmp/rm-smoke/data/agent-binaries
|
||||||
|
cd ~/src/restic-manager
|
||||||
|
make build
|
||||||
|
cp bin/restic-manager-agent /tmp/rm-smoke/data/agent-binaries/restic-manager-agent-linux-amd64
|
||||||
|
```
|
||||||
|
|
||||||
|
The server's `/agent/binary?os=linux&arch=amd64` resolves to that path.
|
||||||
|
|
||||||
|
## 2. Compose the stack
|
||||||
|
|
||||||
|
`/tmp/rm-smoke/compose.yaml`:
|
||||||
|
|
||||||
|
```yaml
|
||||||
|
services:
|
||||||
|
rest-server:
|
||||||
|
image: restic/rest-server:latest
|
||||||
|
restart: unless-stopped
|
||||||
|
environment:
|
||||||
|
- OPTIONS=--no-auth # smoke-test only; real deploys use --append-only + htpasswd
|
||||||
|
ports:
|
||||||
|
- "127.0.0.1:8000:8000"
|
||||||
|
volumes:
|
||||||
|
- ./rest:/data
|
||||||
|
|
||||||
|
control-plane:
|
||||||
|
image: ghcr.io/dcglab/restic-manager:dev # or build locally; see §1
|
||||||
|
restart: unless-stopped
|
||||||
|
ports:
|
||||||
|
- "127.0.0.1:8080:8080"
|
||||||
|
volumes:
|
||||||
|
- ./data:/data
|
||||||
|
environment:
|
||||||
|
- RM_LISTEN=:8080
|
||||||
|
- RM_DATA_DIR=/data
|
||||||
|
- RM_BASE_URL=http://127.0.0.1:8080
|
||||||
|
- RM_SECRET_KEY_FILE=/data/secret.key
|
||||||
|
- RM_COOKIE_SECURE=false # smoke-test only — we're on plain HTTP
|
||||||
|
```
|
||||||
|
|
||||||
|
For local-only smoke: skip the image and run the server straight from
|
||||||
|
the binary instead, pointing at `/tmp/rm-smoke/data`:
|
||||||
|
|
||||||
|
```sh
|
||||||
|
RM_LISTEN=:8080 RM_DATA_DIR=/tmp/rm-smoke/data \
|
||||||
|
RM_SECRET_KEY_FILE=/tmp/rm-smoke/data/secret.key \
|
||||||
|
RM_COOKIE_SECURE=false \
|
||||||
|
./bin/restic-manager-server
|
||||||
|
```
|
||||||
|
|
||||||
|
Either way, watch stderr for the **bootstrap token** — printed on first
|
||||||
|
run, used in the next step.
|
||||||
|
|
||||||
|
## 3. Bootstrap the admin account
|
||||||
|
|
||||||
|
```sh
|
||||||
|
BOOTSTRAP_TOKEN='<paste from server logs>'
|
||||||
|
curl -s -X POST http://127.0.0.1:8080/api/bootstrap \
|
||||||
|
-H 'content-type: application/json' \
|
||||||
|
-d "{\"token\":\"$BOOTSTRAP_TOKEN\",\"username\":\"admin\",\"password\":\"correct horse battery staple\"}"
|
||||||
|
```
|
||||||
|
|
||||||
|
## 4. Mint an enrollment token (with repo creds)
|
||||||
|
|
||||||
|
```sh
|
||||||
|
curl -s -c /tmp/rm-smoke/cookies -X POST http://127.0.0.1:8080/api/auth/login \
|
||||||
|
-H 'content-type: application/json' \
|
||||||
|
-d '{"username":"admin","password":"correct horse battery staple"}'
|
||||||
|
|
||||||
|
ENROLL=$(curl -s -b /tmp/rm-smoke/cookies -X POST http://127.0.0.1:8080/api/enrollment-tokens \
|
||||||
|
-H 'content-type: application/json' \
|
||||||
|
-d '{
|
||||||
|
"hostname":"smoke-host",
|
||||||
|
"repo_url":"rest:http://127.0.0.1:8000/smoke",
|
||||||
|
"repo_username":"",
|
||||||
|
"repo_password":"smoke-pw"
|
||||||
|
}')
|
||||||
|
TOKEN=$(echo "$ENROLL" | jq -r .token)
|
||||||
|
echo "token: $TOKEN"
|
||||||
|
```
|
||||||
|
|
||||||
|
If the server rejects with `missing_field`, you forgot
|
||||||
|
`repo_url`/`repo_password` — both are required (P1-32).
|
||||||
|
|
||||||
|
## 5. Initialise the rest-server repo
|
||||||
|
|
||||||
|
`restic/rest-server` will lazy-create the path on first write, but
|
||||||
|
restic itself wants the repo initialised:
|
||||||
|
|
||||||
|
```sh
|
||||||
|
RESTIC_PASSWORD=smoke-pw \
|
||||||
|
restic -r rest:http://127.0.0.1:8000/smoke init
|
||||||
|
```
|
||||||
|
|
||||||
|
## 6. Pretend to be a fresh endpoint
|
||||||
|
|
||||||
|
The agent will write `agent.yaml` + `secrets.enc` under
|
||||||
|
`/tmp/rm-smoke/agent/etc` and `/tmp/rm-smoke/agent/var-lib`. We point
|
||||||
|
both at those dirs to keep the smoke run isolated from your real
|
||||||
|
`/etc/restic-manager`.
|
||||||
|
|
||||||
|
```sh
|
||||||
|
mkdir -p /tmp/rm-smoke/agent/etc /tmp/rm-smoke/agent/var-lib
|
||||||
|
CONFIG=/tmp/rm-smoke/agent/etc/agent.yaml
|
||||||
|
|
||||||
|
# Pre-write the secrets path so we don't hit the system default.
|
||||||
|
cat > "$CONFIG" <<EOF
|
||||||
|
secrets_path: /tmp/rm-smoke/agent/var-lib/secrets.enc
|
||||||
|
EOF
|
||||||
|
|
||||||
|
# Enroll. This call talks to the server, returns the persistent
|
||||||
|
# bearer, and writes server_url/host_id/agent_token/secrets_key
|
||||||
|
# back into agent.yaml. secrets.enc is empty until the first
|
||||||
|
# config.update push lands.
|
||||||
|
./bin/restic-manager-agent \
|
||||||
|
-config "$CONFIG" \
|
||||||
|
-enroll-server http://127.0.0.1:8080 \
|
||||||
|
-enroll-token "$TOKEN"
|
||||||
|
|
||||||
|
# Read off the host_id for later steps.
|
||||||
|
HOST_ID=$(grep host_id "$CONFIG" | awk '{print $2}' | tr -d '"')
|
||||||
|
echo "host id: $HOST_ID"
|
||||||
|
```
|
||||||
|
|
||||||
|
After enrolment, `agent.yaml` should contain `secrets_key:` (a
|
||||||
|
base64 32-byte key) and `host_id:` (a ULID). It should **not**
|
||||||
|
contain `repo_url:` or `repo_password:`.
|
||||||
|
|
||||||
|
```sh
|
||||||
|
cat "$CONFIG"
|
||||||
|
```
|
||||||
|
|
||||||
|
## 7. Run the agent
|
||||||
|
|
||||||
|
In a second terminal:
|
||||||
|
|
||||||
|
```sh
|
||||||
|
./bin/restic-manager-agent -config /tmp/rm-smoke/agent/etc/agent.yaml
|
||||||
|
```
|
||||||
|
|
||||||
|
You should see, in order:
|
||||||
|
|
||||||
|
```
|
||||||
|
agent starting host_id=01H… server=http://127.0.0.1:8080 …
|
||||||
|
ws agent connected protocol_version=…
|
||||||
|
ws agent: repo credentials updated via config.update
|
||||||
|
```
|
||||||
|
|
||||||
|
That last line confirms slice 1 + 2 of P1-32/33: the server pushed
|
||||||
|
the encrypted creds, the agent decrypted, persisted to `secrets.enc`,
|
||||||
|
and is now ready to back up. `secrets.enc` should now exist and be
|
||||||
|
0600.
|
||||||
|
|
||||||
|
```sh
|
||||||
|
ls -l /tmp/rm-smoke/agent/var-lib/secrets.enc
|
||||||
|
```
|
||||||
|
|
||||||
|
## 8. Run a backup
|
||||||
|
|
||||||
|
Back in the first terminal:
|
||||||
|
|
||||||
|
```sh
|
||||||
|
JOB=$(curl -s -b /tmp/rm-smoke/cookies -X POST \
|
||||||
|
"http://127.0.0.1:8080/api/hosts/$HOST_ID/jobs" \
|
||||||
|
-H 'content-type: application/json' \
|
||||||
|
-d '{"kind":"backup","args":["/etc/hostname","/etc/os-release"]}')
|
||||||
|
JOB_ID=$(echo "$JOB" | jq -r .job_id)
|
||||||
|
echo "job: $JOB_ID"
|
||||||
|
```
|
||||||
|
|
||||||
|
The agent terminal will show restic chugging through two tiny files;
|
||||||
|
the server terminal will log the lifecycle (`mark job started` /
|
||||||
|
`mark job finished` / `snapshots refreshed count=1`).
|
||||||
|
|
||||||
|
## 9. Confirm the snapshot
|
||||||
|
|
||||||
|
```sh
|
||||||
|
curl -s -b /tmp/rm-smoke/cookies \
|
||||||
|
"http://127.0.0.1:8080/api/hosts/$HOST_ID/snapshots" | jq
|
||||||
|
```
|
||||||
|
|
||||||
|
Expect one snapshot with the two paths and a non-zero `size_bytes`.
|
||||||
|
|
||||||
|
## 10. Verify the redacted credential view (sanity)
|
||||||
|
|
||||||
|
```sh
|
||||||
|
curl -s -b /tmp/rm-smoke/cookies \
|
||||||
|
"http://127.0.0.1:8080/api/hosts/$HOST_ID/repo-credentials" | jq
|
||||||
|
```
|
||||||
|
|
||||||
|
Expect `{"repo_url":"rest:http://127.0.0.1:8000/smoke","has_password":true}`.
|
||||||
|
The password is never returned over this endpoint.
|
||||||
|
|
||||||
|
## 11. Edit creds + verify push-on-update
|
||||||
|
|
||||||
|
```sh
|
||||||
|
curl -s -b /tmp/rm-smoke/cookies -X PUT \
|
||||||
|
"http://127.0.0.1:8080/api/hosts/$HOST_ID/repo-credentials" \
|
||||||
|
-H 'content-type: application/json' \
|
||||||
|
-d '{"repo_password":"new-smoke-pw"}'
|
||||||
|
```
|
||||||
|
|
||||||
|
Agent terminal should log `repo credentials updated via config.update`
|
||||||
|
again. (Backups will then fail until you also update the rest-server
|
||||||
|
auth — but that proves the push path is live.)
|
||||||
|
|
||||||
|
## Cleanup
|
||||||
|
|
||||||
|
```sh
|
||||||
|
docker compose -f /tmp/rm-smoke/compose.yaml down -v
|
||||||
|
rm -rf /tmp/rm-smoke
|
||||||
|
```
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## What this runbook does NOT cover
|
||||||
|
|
||||||
|
These are intentionally out of scope for Phase 1; revisit when the
|
||||||
|
relevant tasks land:
|
||||||
|
|
||||||
|
- TLS termination at a reverse proxy (covered by P5-07 reference deployment)
|
||||||
|
- Append-only restic creds + separate prune credential (P2-06)
|
||||||
|
- Live job log streaming in a browser (P1-21 remainder; needs the UI)
|
||||||
|
- Cancellation (P2)
|
||||||
|
- Schedule-driven backups (P2-01 onwards)
|
||||||
|
- Windows agent (P2-16/17)
|
||||||
@@ -18,6 +18,53 @@ import (
|
|||||||
|
|
||||||
func nowUTC() time.Time { return time.Now().UTC() }
|
func nowUTC() time.Time { return time.Now().UTC() }
|
||||||
|
|
||||||
|
// hostRepoCredsView is the body of GET /api/hosts/{id}/repo-credentials.
|
||||||
|
// Password is always redacted; the UI uses this to pre-fill an edit
|
||||||
|
// form with the URL/username already populated.
|
||||||
|
type hostRepoCredsView struct {
|
||||||
|
RepoURL string `json:"repo_url"`
|
||||||
|
RepoUsername string `json:"repo_username,omitempty"`
|
||||||
|
HasPassword bool `json:"has_password"`
|
||||||
|
}
|
||||||
|
|
||||||
|
// handleGetHostCredentials returns a redacted view of the host's repo
|
||||||
|
// creds for UI display. 404 if no credential has ever been set.
|
||||||
|
func (s *Server) handleGetHostCredentials(w stdhttp.ResponseWriter, r *stdhttp.Request) {
|
||||||
|
if !s.authedUser(r) {
|
||||||
|
writeJSONError(w, stdhttp.StatusUnauthorized, "unauthorized", "")
|
||||||
|
return
|
||||||
|
}
|
||||||
|
hostID := chi.URLParam(r, "id")
|
||||||
|
if hostID == "" {
|
||||||
|
writeJSONError(w, stdhttp.StatusBadRequest, "missing_id", "")
|
||||||
|
return
|
||||||
|
}
|
||||||
|
enc, err := s.deps.Store.GetHostCredentials(r.Context(), hostID)
|
||||||
|
if err != nil {
|
||||||
|
if errors.Is(err, store.ErrNotFound) {
|
||||||
|
writeJSONError(w, stdhttp.StatusNotFound, "not_set", "")
|
||||||
|
return
|
||||||
|
}
|
||||||
|
writeJSONError(w, stdhttp.StatusInternalServerError, "internal", "")
|
||||||
|
return
|
||||||
|
}
|
||||||
|
plain, err := s.deps.AEAD.Decrypt(enc, []byte("host:"+hostID))
|
||||||
|
if err != nil {
|
||||||
|
writeJSONError(w, stdhttp.StatusInternalServerError, "decrypt_failed", "")
|
||||||
|
return
|
||||||
|
}
|
||||||
|
var blob repoCredsBlob
|
||||||
|
if err := json.Unmarshal(plain, &blob); err != nil {
|
||||||
|
writeJSONError(w, stdhttp.StatusInternalServerError, "internal", "")
|
||||||
|
return
|
||||||
|
}
|
||||||
|
writeJSON(w, stdhttp.StatusOK, hostRepoCredsView{
|
||||||
|
RepoURL: blob.RepoURL,
|
||||||
|
RepoUsername: blob.RepoUsername,
|
||||||
|
HasPassword: blob.RepoPassword != "",
|
||||||
|
})
|
||||||
|
}
|
||||||
|
|
||||||
// hostRepoCredsRequest is the body of PUT /api/hosts/{id}/repo-credentials.
|
// hostRepoCredsRequest is the body of PUT /api/hosts/{id}/repo-credentials.
|
||||||
// Operator can edit any subset; missing fields preserve the existing
|
// Operator can edit any subset; missing fields preserve the existing
|
||||||
// value (so changing only the password doesn't require resending the URL).
|
// value (so changing only the password doesn't require resending the URL).
|
||||||
|
|||||||
@@ -92,6 +92,8 @@ func (s *Server) routes(r chi.Router) {
|
|||||||
|
|
||||||
// Repo credentials — operator can edit after enrollment. The
|
// Repo credentials — operator can edit after enrollment. The
|
||||||
// initial set is supplied at token-mint time (see enrollment.go).
|
// initial set is supplied at token-mint time (see enrollment.go).
|
||||||
|
// GET returns a redacted view (URL, username, has_password).
|
||||||
|
r.Get("/hosts/{id}/repo-credentials", s.handleGetHostCredentials)
|
||||||
r.Put("/hosts/{id}/repo-credentials", s.handleSetHostCredentials)
|
r.Put("/hosts/{id}/repo-credentials", s.handleSetHostCredentials)
|
||||||
})
|
})
|
||||||
|
|
||||||
|
|||||||
@@ -69,21 +69,22 @@ Sizes: **S** = under a day, **M** = 1–3 days, **L** = 3–7 days.
|
|||||||
|
|
||||||
### Repo credentials (pulled forward from Phase 2)
|
### Repo credentials (pulled forward from Phase 2)
|
||||||
|
|
||||||
- [ ] **P1-32** (M) Server-side encrypted repo creds carried on the enrollment token:
|
- [x] **P1-32** (M) Server-side encrypted repo creds carried on the enrollment token:
|
||||||
- `POST /api/enrollment-tokens` body grows `repo_url`, `repo_username`, `repo_password` (all required).
|
- `POST /api/enrollment-tokens` body grows `repo_url`, `repo_username`, `repo_password` (all required).
|
||||||
- Token row stores them as one AEAD-encrypted blob (existing `crypto.AEAD`); `ConsumeEnrollmentToken` moves the blob to a new `host_credentials` row keyed by `host_id` in the same tx.
|
- Token row stores them as one AEAD-encrypted blob (existing `crypto.AEAD`); `ConsumeEnrollmentToken` moves the blob to a new `host_credentials` row keyed by `host_id` in the same tx.
|
||||||
- `PUT /api/hosts/{id}/repo-credentials` (admin/operator) re-encrypts and replaces the row, emits an in-memory event to the WS hub.
|
- `PUT /api/hosts/{id}/repo-credentials` (admin/operator) re-encrypts and replaces the row, emits an in-memory event to the WS hub.
|
||||||
|
- `GET /api/hosts/{id}/repo-credentials` returns the redacted view (URL + username + `has_password`) so the UI can pre-fill the edit form. Password never leaves the server outside the WS push.
|
||||||
- On WS `hello`, server pushes a `config.update` with decrypted creds **before** returning the connection to idle. Same path on edit-while-connected.
|
- On WS `hello`, server pushes a `config.update` with decrypted creds **before** returning the connection to idle. Same path on edit-while-connected.
|
||||||
- Audit-logged on create / consume / edit; payload omits the secret material.
|
- Audit-logged on create / consume / edit; payload omits the secret material.
|
||||||
|
|
||||||
- [ ] **P1-33** (M) Agent-side encrypted secrets store:
|
- [x] **P1-33** (M) Agent-side encrypted secrets store:
|
||||||
- New `internal/agent/secrets` package: AEAD blob at `/var/lib/restic-manager/secrets.enc`, atomic write (tmp+fsync+rename, mode 0600).
|
- New `internal/agent/secrets` package: AEAD blob at `/var/lib/restic-manager/secrets.enc`, atomic write (tmp+fsync+rename, mode 0600).
|
||||||
- Per-host 32-byte secrets key minted at enrollment, persisted in `agent.yaml` (already 0600 root-only — same trust boundary as the bearer; explicit comment in the file).
|
- Per-host 32-byte secrets key minted at enrollment, persisted in `agent.yaml` (already 0600 root-only — same trust boundary as the bearer; explicit comment in the file).
|
||||||
- Strip `repo_url` / `repo_password` from `agent.config.Config`. Agent loads creds from `secrets.enc` at startup; `config.update` handler writes through to the file.
|
- Strip `repo_url` / `repo_password` from `agent.config.Config`. Agent loads creds from `secrets.enc` at startup; `config.update` handler writes through to the file.
|
||||||
- Dispatcher reads from the secrets store on every job rather than from in-memory config.
|
- Dispatcher reads from the secrets store on every job rather than from in-memory config.
|
||||||
- Migration path: if `agent.yaml` still contains `repo_url`/`repo_password`, copy them into `secrets.enc` on next start, then strip from the YAML on save.
|
- Migration path: if `agent.yaml` still contains `repo_url`/`repo_password`, copy them into `secrets.enc` on next start, then strip from the YAML on save.
|
||||||
|
|
||||||
- [ ] **P1-34** (S) End-to-end smoke: enrollment with repo creds → agent receives them via push-on-connect → run-now backup completes against a real `restic/rest-server` in a sibling container → host appears with snapshot count.
|
- [x] **P1-34** (S) End-to-end smoke runbook: `docs/e2e-smoke.md` walks through enrollment with repo creds → agent receives them via push-on-connect → run-now backup completes against a real `restic/rest-server` in a sibling container → host appears with snapshot count. Test-driven version (Playwright + compose) deferred to P5-06.
|
||||||
|
|
||||||
### Phase 1 acceptance
|
### Phase 1 acceptance
|
||||||
|
|
||||||
|
|||||||
Reference in New Issue
Block a user