P1 polish: agent-as-root, init-repo flow, rest creds passthrough, UX fixes
Cohesive batch from a smoke-test session against a real rest-server.
Themed bullets:
* Agent runs as root, sandboxed via systemd. CapabilityBoundingSet
drops to CAP_DAC_READ_SEARCH + restore caps; ProtectSystem=strict
with ReadWritePaths confined to /etc + /var/lib/restic-manager;
NoNewPrivileges blocks escalation. Install script no longer
creates a service user. spec.md §4.2 / §14.1 / §14.3 explain the
rationale (matches UrBackup / Veeam / Bareos defaults; trying to
back up "everything" as an unprivileged user creates silent skips
on /home, /root, /var/lib/* with no upside vs the threat model
the agent already implies).
* Init-repo end-to-end. New JobKind="init" wired through agent
runner, restic.Env.RunInit, server dispatcher, and a UI button
(red "Initialise repo" in the run-now panel). hosts.repo_initialised_at
flips on init success, on backup success, or on a non-empty
snapshots.report. The "Run now" / "Init" / "Retry" branching now
drives both the dashboard host row and the host-detail panel.
Migrations 0004 (column), 0005 (jobs.kind CHECK widened — using
the safe create-new-then-rename pattern; first version corrupted
job_logs.job_id FK), 0006 (cleans up job_logs FK on already-
affected DBs).
* rest-server creds embedded at exec time only. restic.Env gains
RepoUsername; mergeRestCreds() builds the user:pass@-prefixed URL
inside envSlice() and never assigns it back to the struct, so
nothing slog-able ever sees the cleartext form. RedactURL helper
for any future surface that needs to log a URL safely. Both
helpers tested.
* Add-host UX. Repo password is now optional — server mints a
24-byte URL-safe random one and surfaces it once, alongside an
htpasswd snippet ("echo PASS | htpasswd -B -i ... USERNAME") so
the operator pastes one command on the rest-server host and one
on the endpoint. Result page also links the install snippet at
/install/install.sh (was /install.sh — 404'd before) and pipes
to bash (not sh — script uses set -o pipefail and other
bashisms; on Debian/Ubuntu sh is dash).
* Late-subscriber race in JobHub. A fast-failing job could finish
(DB write + Broadcast) before the browser's HX-Redirect → page
load → WS-connect path completed, so the JS sat forever waiting
on a job.finished that already passed. JobHub split into
Register + Send + Run; handleJobStream now subscribes first,
re-fetches the job, and sends a synthetic job.finished if the
state is already terminal.
* HTMX error visibility. New toast partial listens to
htmx:responseError and surfaces the response body as a
bottom-right toast — every server-side validation error now
becomes visible without per-handler JS wiring. Also handles
custom rm:toast events for future server-pushed notifications
via the HX-Trigger header. Themed via existing CSS vars.
* Dashboard rows are now whole-row clickable to host detail
(CSS card-link pattern: absolute-positioned anchor + .row-action
z-index restoration so the action button stays clickable).
"View →" on a running job links to /jobs/<id> rather than
/hosts/<id> since the row click already covers the host page.
* "Run first" / "Run first backup" → "Run now" everywhere for
consistency.
* runbook (docs/e2e-smoke.md) updated — live-log streaming step
now reflects P1-26; mentions the browser-driven Run-now flow.
* _diag/dump-creds — moved out of cmd/ so go build doesn't pick
it up; .gitignore now excludes /_diag/ entirely.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
This commit is contained in:
@@ -33,10 +33,19 @@ func Locate(override string) (string, error) {
|
||||
}
|
||||
|
||||
// Env is the per-invocation context for a restic command.
|
||||
//
|
||||
// RepoURL is the bare URL as the operator typed it — no embedded
|
||||
// credentials. RepoUsername (optional) carries the HTTP basic-auth
|
||||
// user for `rest:` repos. The merged URL (with `user:pass@host`
|
||||
// embedded) is built once inside envSlice() at the moment of exec
|
||||
// and fed straight to the subprocess via RESTIC_REPOSITORY; we
|
||||
// never assign it back to Env, never pass it to slog. If anything
|
||||
// in this package ever needs to *log* a URL, use RedactURL.
|
||||
type Env struct {
|
||||
Bin string // path to restic binary
|
||||
RepoURL string // RESTIC_REPOSITORY
|
||||
RepoPassword string // RESTIC_PASSWORD (passed via env, never argv)
|
||||
RepoURL string // RESTIC_REPOSITORY (no embedded creds)
|
||||
RepoUsername string // optional HTTP basic-auth user for rest: URLs
|
||||
RepoPassword string // doubles as RESTIC_PASSWORD and (for rest:) HTTP basic-auth password
|
||||
ExtraEnv map[string]string // any other RESTIC_* / passthrough
|
||||
WorkDir string // CWD; default = current
|
||||
}
|
||||
@@ -140,6 +149,55 @@ func (e Env) RunBackup(ctx context.Context, paths, excludes, tags []string, hand
|
||||
return summary, nil
|
||||
}
|
||||
|
||||
// RunInit executes `restic init` against the configured repo. Returns
|
||||
// nil on success. Restic init's output is small and not JSON-rich;
|
||||
// we tee stdout/stderr verbatim through handle so the operator sees
|
||||
// the same lines they'd see at the CLI ("created restic repository
|
||||
// <id> at <url>" on success, "config file already exists" on a
|
||||
// re-init attempt, etc.).
|
||||
func (e Env) RunInit(ctx context.Context, handle LineHandler) error {
|
||||
cmd := exec.CommandContext(ctx, e.Bin, "init")
|
||||
cmd.Env = e.envSlice()
|
||||
cmd.Dir = e.WorkDir
|
||||
|
||||
stdout, err := cmd.StdoutPipe()
|
||||
if err != nil {
|
||||
return fmt.Errorf("restic init: stdout pipe: %w", err)
|
||||
}
|
||||
stderr, err := cmd.StderrPipe()
|
||||
if err != nil {
|
||||
return fmt.Errorf("restic init: stderr pipe: %w", err)
|
||||
}
|
||||
|
||||
if err := cmd.Start(); err != nil {
|
||||
return fmt.Errorf("restic init: start: %w", err)
|
||||
}
|
||||
|
||||
done := make(chan error, 2)
|
||||
go func() { done <- pumpPlain(stdout, "stdout", handle) }()
|
||||
go func() { done <- pumpPlain(stderr, "stderr", handle) }()
|
||||
for i := 0; i < 2; i++ {
|
||||
if err := <-done; err != nil && handle != nil {
|
||||
handle("event", fmt.Sprintf("pump error: %v", err), nil)
|
||||
}
|
||||
}
|
||||
if werr := cmd.Wait(); werr != nil {
|
||||
return fmt.Errorf("restic init: %w", werr)
|
||||
}
|
||||
return nil
|
||||
}
|
||||
|
||||
func pumpPlain(r io.Reader, stream string, handle LineHandler) error {
|
||||
scanner := bufio.NewScanner(r)
|
||||
scanner.Buffer(make([]byte, 0, 64*1024), 1024*1024)
|
||||
for scanner.Scan() {
|
||||
if handle != nil {
|
||||
handle(stream, scanner.Text(), nil)
|
||||
}
|
||||
}
|
||||
return scanner.Err()
|
||||
}
|
||||
|
||||
// envSlice converts Env's typed fields into the os/exec env shape.
|
||||
//
|
||||
// Deliberately does NOT inherit the parent process's environment:
|
||||
@@ -164,7 +222,7 @@ func (e Env) envSlice() []string {
|
||||
xdg = x
|
||||
}
|
||||
out := []string{
|
||||
"RESTIC_REPOSITORY=" + e.RepoURL,
|
||||
"RESTIC_REPOSITORY=" + mergeRestCreds(e.RepoURL, e.RepoUsername, e.RepoPassword),
|
||||
"RESTIC_PASSWORD=" + e.RepoPassword,
|
||||
// Feed restic via env-only — keeps creds off ps(1).
|
||||
"PATH=/usr/local/bin:/usr/bin:/bin",
|
||||
|
||||
@@ -0,0 +1,85 @@
|
||||
package restic
|
||||
|
||||
import (
|
||||
"net/url"
|
||||
"strings"
|
||||
)
|
||||
|
||||
// mergeRestCreds embeds basic-auth user:pass into a `rest:` URL, only
|
||||
// at the moment we hand it off to the restic subprocess. The result
|
||||
// is intentionally NOT stored on Env or logged — restic's REST
|
||||
// backend reads basic-auth from the URL only, so we have nowhere
|
||||
// else to put them. Callers must treat the return value as
|
||||
// secret-bearing and feed it straight into exec env.
|
||||
//
|
||||
// No-ops when:
|
||||
// - the URL has no `rest:` prefix (other backends — s3, b2, sftp,
|
||||
// etc. — get creds via their own env vars);
|
||||
// - the URL already embeds user:pass (operator typed creds inline);
|
||||
// - username is empty.
|
||||
//
|
||||
// Returns rawURL unchanged if it can't be parsed; restic will then
|
||||
// reject it and the operator gets a clear error rather than a silent
|
||||
// "I quietly stripped your URL" surprise.
|
||||
func mergeRestCreds(rawURL, username, password string) string {
|
||||
if !strings.HasPrefix(rawURL, "rest:") {
|
||||
return rawURL
|
||||
}
|
||||
if username == "" {
|
||||
return rawURL
|
||||
}
|
||||
inner := strings.TrimPrefix(rawURL, "rest:")
|
||||
u, err := url.Parse(inner)
|
||||
if err != nil || u.Host == "" {
|
||||
// Either unparseable or a relative URL we shouldn't touch —
|
||||
// pass through and let restic complain with a clear message.
|
||||
return rawURL
|
||||
}
|
||||
if u.User != nil {
|
||||
// Operator already embedded creds — don't overwrite.
|
||||
return rawURL
|
||||
}
|
||||
u.User = url.UserPassword(username, password)
|
||||
return "rest:" + u.String()
|
||||
}
|
||||
|
||||
// RedactURL returns a logging-safe version of u with any password in
|
||||
// the userinfo replaced by ***. Mirrors restic's own redaction so
|
||||
// our logs match what restic prints. Use this — never the bare URL —
|
||||
// whenever a URL might end up in slog output, audit entries, or any
|
||||
// surface an operator can read.
|
||||
//
|
||||
// Non-restic URLs (s3, b2, sftp, …) pass through unchanged unless
|
||||
// they happen to embed userinfo, in which case we redact the same
|
||||
// way for consistency.
|
||||
func RedactURL(u string) string {
|
||||
prefix := ""
|
||||
rest := u
|
||||
if i := strings.Index(u, ":"); i > 0 && i+3 < len(u) && u[i+1:i+3] == "//" {
|
||||
// scheme://… — keep "scheme:" intact.
|
||||
prefix = u[:i+1]
|
||||
rest = u[i+1:]
|
||||
} else if strings.HasPrefix(u, "rest:") {
|
||||
prefix = "rest:"
|
||||
rest = strings.TrimPrefix(u, "rest:")
|
||||
}
|
||||
parsed, err := url.Parse(rest)
|
||||
if err != nil || parsed.User == nil {
|
||||
return u
|
||||
}
|
||||
if _, hasPass := parsed.User.Password(); !hasPass {
|
||||
return u
|
||||
}
|
||||
// Build the redacted form by hand rather than via url.URL.String(),
|
||||
// which percent-encodes the redaction marker into "%2A%2A%2A".
|
||||
user := parsed.User.Username()
|
||||
parsed.User = nil
|
||||
rebuilt := parsed.String()
|
||||
// rebuilt is "scheme://host/path…"; splice user:***@ in after "//".
|
||||
const sep = "//"
|
||||
idx := strings.Index(rebuilt, sep)
|
||||
if idx < 0 {
|
||||
return u
|
||||
}
|
||||
return prefix + rebuilt[:idx+len(sep)] + user + ":***@" + rebuilt[idx+len(sep):]
|
||||
}
|
||||
@@ -0,0 +1,41 @@
|
||||
package restic
|
||||
|
||||
import "testing"
|
||||
|
||||
func TestMergeRestCreds(t *testing.T) {
|
||||
cases := []struct {
|
||||
name, url, user, pass, want string
|
||||
}{
|
||||
{"rest with creds", "rest:http://h:8000/p/", "u", "p", "rest:http://u:p@h:8000/p/"},
|
||||
{"rest no user — no-op", "rest:http://h:8000/p/", "", "p", "rest:http://h:8000/p/"},
|
||||
{"rest creds already inline — no-op",
|
||||
"rest:http://existing:secret@h:8000/p/", "u", "p",
|
||||
"rest:http://existing:secret@h:8000/p/"},
|
||||
{"non-rest s3 — no-op", "s3:s3.amazonaws.com/bucket", "u", "p", "s3:s3.amazonaws.com/bucket"},
|
||||
{"unparseable — pass through", "rest:not a url", "u", "p", "rest:not a url"},
|
||||
{"https URL kept intact", "rest:https://h/p/", "u", "p", "rest:https://u:p@h/p/"},
|
||||
}
|
||||
for _, c := range cases {
|
||||
t.Run(c.name, func(t *testing.T) {
|
||||
got := mergeRestCreds(c.url, c.user, c.pass)
|
||||
if got != c.want {
|
||||
t.Fatalf("mergeRestCreds(%q,%q,***) = %q; want %q", c.url, c.user, got, c.want)
|
||||
}
|
||||
})
|
||||
}
|
||||
}
|
||||
|
||||
func TestRedactURL(t *testing.T) {
|
||||
cases := []struct{ in, want string }{
|
||||
{"rest:http://u:p@h:8000/p/", "rest:http://u:***@h:8000/p/"},
|
||||
{"rest:http://h:8000/p/", "rest:http://h:8000/p/"},
|
||||
{"https://u:p@example/", "https://u:***@example/"},
|
||||
{"s3:s3.amazonaws.com/bucket", "s3:s3.amazonaws.com/bucket"},
|
||||
}
|
||||
for _, c := range cases {
|
||||
got := RedactURL(c.in)
|
||||
if got != c.want {
|
||||
t.Fatalf("RedactURL(%q) = %q; want %q", c.in, got, c.want)
|
||||
}
|
||||
}
|
||||
}
|
||||
Reference in New Issue
Block a user