steve/restic-manager
1
0
Fork 0
Code Issues Pull Requests Actions Projects Releases 1 Wiki Activity

P2 redesign · phase 2: store rewrite — sources, slim schedules, repo maintenance

Browse Source 
Go-side data model rebuilt against migration 0008. The fat-Schedule
shape (paths/excludes/tags/retention/manual/kind/options/hooks) is
gone; that surface lives on source_groups now.

* store/types.go
  - Schedule slimmed to {id, host_id, cron, enabled, source_group_ids,
    timestamps}. SourceGroupIDs populated by Get/List, accepted on
    Create/Update so callers pass desired junction state in one shape.
  - SourceGroup added: name (= snapshot tag), includes/excludes,
    retention_policy, retry_max + retry_backoff_seconds, cached
    conflict_dimension.
  - HostRepoMaintenance added: forget/prune/check cadences + enabled.
  - PendingRun added: offline-retry queue.
  - Host loses RepoInitialisedAt; gains BandwidthUpKBps + BandwidthDownKBps.
  - RetentionPolicy moves home from "schedule field" to "source group
    field" but the type itself + Summary() method unchanged.

* store/sources.go (new) — CRUD + GetByName + ConflictDimension cache.
  Group writes bump host_schedule_version; conflict cache writes don't
  (server-internal projection, agent doesn't see it).
* store/maintenance.go (new) — CreateDefault is idempotent (INSERT OR
  IGNORE). UpdateRepoMaintenance doesn't bump schedule version because
  these run on the server's own ticker, not the agent's local cron.
* store/pending.go (new) — Enqueue / DueRunsForRetry / Bump / Delete.
* store/schedules.go — rewritten for slim shape + junction CRUD.
  Update wipes the schedule_source_groups junction wholesale and
  re-inserts (simpler than diffing). Adds SchedulesUsingGroup for
  retention-conflict detection + UI labels.
* store/hosts.go — drops repo_initialised_at scan, adds bandwidth scan.
  New SetHostBandwidth helper.

* HTTP layer — temporarily stubbed during this rewrite (501 returns
  with redesign_in_progress error code). Phase 3 fills these in
  against the new shape:
    - schedules.go REST CRUD
    - schedule_push.go agent reconciliation
    - ui_schedules.go HTML form CRUD
  Run-now-per-host + Init-repo handlers in ui_handlers.go also stubbed
  — both go away in the new model (Run-now per source group; auto-init
  at host enrolment).

* enrollment.go — replaces "seed manual schedule from typed paths"
  with "seed default source group + repo-maintenance row." The default
  group gets the typed paths as its includes; operator edits later
  via Sources tab.

* ws/handler.go — drops the MarkHostRepoInitialised projection (column
  is gone; auto-init makes it derivable from latest init job's status).

Tests:
* store: existing schedule test rewritten for slim shape + junction;
  new sources_test.go covers source-group CRUD, name uniqueness,
  conflict cache, repo-maintenance defaults + idempotent seed,
  pending-runs queue lifecycle.
* http: schedules_test.go and schedule_push_test.go deleted — both
  exercised the obsolete fat-schedule API. Phase 3 rewrites them
  against the new endpoints.

go test ./... green. cmd/server + cmd/agent build. The UI is broken
end-to-end (schedules / sources / repo tabs all hit 501 stubs); Phase 3
restores REST + on-the-wire reconciliation; Phase 4 rewires the UI
templates against the new model.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
This commit is contained in:
Steve Cliff
2026-05-02 21:30:41 +01:00
parent e717b6998c
commit e7eea7afac
16 changed files with 1076 additions and 1928 deletions
Download Patch File Download Diff File Expand all files Collapse all files
internal/server/http/ui_schedules.go
+15 -511
View File
@@ -1,534 +1,38 @@
package http
import (
"errors"
"fmt"
"log/slog"
stdhttp "net/http"
"strconv"
"strings"
"github.com/go-chi/chi/v5"
"github.com/oklog/ulid/v2"
"gitea.dcglab.co.uk/steve/restic-manager/internal/api"
"gitea.dcglab.co.uk/steve/restic-manager/internal/server/ui"
"gitea.dcglab.co.uk/steve/restic-manager/internal/store"
)
// schedulesListPage carries everything the Schedules tab needs.
type schedulesListPage struct {
Host store.Host
Schedules []store.Schedule
Version int64
AppliedVersion int64
}
// ui_schedules.go — HTML form-driven schedule CRUD.
//
// Stubbed during the P2 redesign template rewrite. Phase 4 of the
// redesign rebuilds the schedule editor against the new slim shape
// (cron + source-group multi-select + enabled), the source-group
// list/edit pages, and the repo-maintenance tab. Until then these
// routes return 501; the dashboard's host-row "View →" link is the
// only operator entry point that still works.
// scheduleEditPage drives both the Create form (Schedule.ID empty)
// and the Edit form (Schedule populated). Errors come back via Error
// to be rendered as a banner; the rest of the fields hold the just-
// submitted raw values so a failed POST can re-render with the
// operator's typed input still in place.
type scheduleEditPage struct {
Host store.Host
IsNew bool
ScheduleID string
Error string
// Kind is settable on create, immutable on edit. The form's
// kind picker is hidden when !IsNew.
Kind string
// Form values — strings so partial input survives validation
// errors (e.g. operator typed "abc" into keep_last).
CronExpr string
PathsRaw string
ExcludesRaw string
TagsRaw string
KeepLast string
KeepHourly string
KeepDaily string
KeepWeekly string
KeepMonthly string
KeepYearly string
LimitUpKBps string
LimitDownKBps string
Enabled bool
Manual bool
}
// handleUISchedulesList renders the Schedules sub-tab on a host.
func (s *Server) handleUISchedulesList(w stdhttp.ResponseWriter, r *stdhttp.Request) {
u := s.requireUIUser(w, r)
if u == nil {
return
}
hostID := chi.URLParam(r, "id")
host, err := s.deps.Store.GetHost(r.Context(), hostID)
if err != nil {
if errors.Is(err, store.ErrNotFound) {
stdhttp.NotFound(w, r)
return
}
stdhttp.Error(w, "internal", stdhttp.StatusInternalServerError)
return
}
rows, err := s.deps.Store.ListSchedulesByHost(r.Context(), hostID)
if err != nil {
stdhttp.Error(w, "internal", stdhttp.StatusInternalServerError)
return
}
version, _ := s.deps.Store.GetHostScheduleVersion(r.Context(), hostID)
view := s.baseView(u, "dashboard")
view.Title = host.Name + " · schedules · restic-manager"
view.Page = schedulesListPage{
Host: *host,
Schedules: rows,
Version: version,
AppliedVersion: host.AppliedScheduleVersion,
}
if err := s.deps.UI.Render(w, "schedules_list", view); err != nil {
slog.Error("ui: render schedules_list", "err", err)
stdhttp.Error(w, "internal", stdhttp.StatusInternalServerError)
}
stdhttp.Error(w, "schedules UI is being rebuilt — see P2 redesign Phase 4", stdhttp.StatusNotImplemented)
}
// handleUIScheduleNewGet renders the empty Create form.
func (s *Server) handleUIScheduleNewGet(w stdhttp.ResponseWriter, r *stdhttp.Request) {
u := s.requireUIUser(w, r)
if u == nil {
return
}
hostID := chi.URLParam(r, "id")
host, err := s.deps.Store.GetHost(r.Context(), hostID)
if err != nil {
if errors.Is(err, store.ErrNotFound) {
stdhttp.NotFound(w, r)
return
}
stdhttp.Error(w, "internal", stdhttp.StatusInternalServerError)
return
}
view := s.baseView(u, "dashboard")
view.Title = "New schedule · " + host.Name
view.Page = scheduleEditPage{
Host: *host,
IsNew: true,
Kind: string(api.JobBackup),
CronExpr: "0 3 * * *",
Enabled: true,
}
s.renderScheduleEdit(w, view)
stdhttp.Error(w, "schedules UI is being rebuilt — see P2 redesign Phase 4", stdhttp.StatusNotImplemented)
}
// handleUIScheduleEditGet renders the Edit form pre-filled from the
// existing schedule row.
func (s *Server) handleUIScheduleEditGet(w stdhttp.ResponseWriter, r *stdhttp.Request) {
u := s.requireUIUser(w, r)
if u == nil {
return
}
hostID := chi.URLParam(r, "id")
scheduleID := chi.URLParam(r, "sid")
host, err := s.deps.Store.GetHost(r.Context(), hostID)
if err != nil {
if errors.Is(err, store.ErrNotFound) {
stdhttp.NotFound(w, r)
return
}
stdhttp.Error(w, "internal", stdhttp.StatusInternalServerError)
return
}
sched, err := s.deps.Store.GetSchedule(r.Context(), hostID, scheduleID)
if err != nil {
if errors.Is(err, store.ErrNotFound) {
stdhttp.NotFound(w, r)
return
}
stdhttp.Error(w, "internal", stdhttp.StatusInternalServerError)
return
}
page := scheduleEditPage{
Host: *host,
IsNew: false,
ScheduleID: sched.ID,
Kind: sched.Kind,
CronExpr: sched.CronExpr,
PathsRaw: strings.Join(sched.Paths, "\n"),
ExcludesRaw: strings.Join(sched.Excludes, "\n"),
TagsRaw: strings.Join(sched.Tags, ", "),
Enabled: sched.Enabled,
Manual: sched.Manual,
}
page.KeepLast = intStringPtr(sched.RetentionPolicy.KeepLast)
page.KeepHourly = intStringPtr(sched.RetentionPolicy.KeepHourly)
page.KeepDaily = intStringPtr(sched.RetentionPolicy.KeepDaily)
page.KeepWeekly = intStringPtr(sched.RetentionPolicy.KeepWeekly)
page.KeepMonthly = intStringPtr(sched.RetentionPolicy.KeepMonthly)
page.KeepYearly = intStringPtr(sched.RetentionPolicy.KeepYearly)
page.LimitUpKBps = intStringPtr(sched.Options.LimitUploadKBps)
page.LimitDownKBps = intStringPtr(sched.Options.LimitDownloadKBps)
view := s.baseView(u, "dashboard")
view.Title = "Edit schedule · " + host.Name
view.Page = page
s.renderScheduleEdit(w, view)
stdhttp.Error(w, "schedules UI is being rebuilt — see P2 redesign Phase 4", stdhttp.StatusNotImplemented)
}
// handleUIScheduleSave handles POST for both create and update. The
// edit form posts to /hosts/{id}/schedules/new (for create) or
// /hosts/{id}/schedules/{sid}/edit (for update); we branch on whether
// {sid} is present in the route params.
func (s *Server) handleUIScheduleSave(w stdhttp.ResponseWriter, r *stdhttp.Request) {
u := s.requireUIUser(w, r)
if u == nil {
return
}
hostID := chi.URLParam(r, "id")
scheduleID := chi.URLParam(r, "sid")
storeUser, _, err := s.userByID(r, u.ID)
if err != nil || storeUser == nil {
stdhttp.Error(w, "internal", stdhttp.StatusInternalServerError)
return
}
host, err := s.deps.Store.GetHost(r.Context(), hostID)
if err != nil {
if errors.Is(err, store.ErrNotFound) {
stdhttp.NotFound(w, r)
return
}
stdhttp.Error(w, "internal", stdhttp.StatusInternalServerError)
return
}
if err := r.ParseForm(); err != nil {
stdhttp.Error(w, "bad request", stdhttp.StatusBadRequest)
return
}
page := scheduleEditPage{
Host: *host,
IsNew: scheduleID == "",
ScheduleID: scheduleID,
Kind: strings.TrimSpace(r.PostForm.Get("kind")),
CronExpr: strings.TrimSpace(r.PostForm.Get("cron_expr")),
PathsRaw: r.PostForm.Get("paths"),
ExcludesRaw: r.PostForm.Get("excludes"),
TagsRaw: strings.TrimSpace(r.PostForm.Get("tags")),
KeepLast: strings.TrimSpace(r.PostForm.Get("keep_last")),
KeepHourly: strings.TrimSpace(r.PostForm.Get("keep_hourly")),
KeepDaily: strings.TrimSpace(r.PostForm.Get("keep_daily")),
KeepWeekly: strings.TrimSpace(r.PostForm.Get("keep_weekly")),
KeepMonthly: strings.TrimSpace(r.PostForm.Get("keep_monthly")),
KeepYearly: strings.TrimSpace(r.PostForm.Get("keep_yearly")),
LimitUpKBps: strings.TrimSpace(r.PostForm.Get("limit_up_kbps")),
LimitDownKBps: strings.TrimSpace(r.PostForm.Get("limit_down_kbps")),
Enabled: r.PostForm.Get("enabled") == "on",
Manual: r.PostForm.Get("manual") == "on",
}
// Kind is immutable on edit — use the existing schedule's kind
// regardless of what the form submitted.
if !page.IsNew {
if existing, err := s.deps.Store.GetSchedule(r.Context(), hostID, scheduleID); err == nil {
page.Kind = existing.Kind
}
}
if page.Kind == "" {
page.Kind = string(api.JobBackup)
}
// Convert the raw form values into store-shape data, surfacing
// the first parse error as a banner.
paths := splitPaths(page.PathsRaw)
excludes := splitPaths(page.ExcludesRaw)
tags := splitCSV(page.TagsRaw)
retention, err := parseRetention(page)
if err != nil {
page.Error = err.Error()
s.renderEditPage(w, u, page)
return
}
options, err := parseOptions(page)
if err != nil {
page.Error = err.Error()
s.renderEditPage(w, u, page)
return
}
// Validate against the same rules the JSON API uses. Manual
// schedules skip the cron-expr requirement; forget schedules
// require a non-empty retention policy. Other validation
// (kind in allowed set, paths required for backup, hooks
// rejected on non-backup) lives in validateSchedule.
apiShape := scheduleAPI{
Kind: api.JobKind(page.Kind),
CronExpr: page.CronExpr,
Paths: paths,
Manual: page.Manual,
RetentionPolicy: retention,
}
if code, msg := validateSchedule(&apiShape); code != "" {
page.Error = uiErrorMessage(code, msg)
s.renderEditPage(w, u, page)
return
}
if page.IsNew {
row := store.Schedule{
ID: ulid.Make().String(),
HostID: hostID,
Kind: page.Kind,
CronExpr: page.CronExpr,
Paths: paths,
Excludes: excludes,
Tags: tags,
RetentionPolicy: retention,
Options: options,
Enabled: page.Enabled,
Manual: page.Manual,
}
if err := s.deps.Store.CreateSchedule(r.Context(), &row); err != nil {
page.Error = "Couldn't save schedule — see server log."
slog.Error("ui schedule create", "err", err)
s.renderEditPage(w, u, page)
return
}
_ = s.deps.Store.AppendAudit(r.Context(), store.AuditEntry{
ID: ulid.Make().String(),
UserID: &storeUser.ID,
Actor: "user",
Action: "schedule.created",
TargetKind: ptr("schedule"),
TargetID: &row.ID,
TS: nowUTC(),
})
s.pushScheduleSetAsync(hostID)
} else {
existing, err := s.deps.Store.GetSchedule(r.Context(), hostID, scheduleID)
if err != nil {
if errors.Is(err, store.ErrNotFound) {
stdhttp.NotFound(w, r)
return
}
stdhttp.Error(w, "internal", stdhttp.StatusInternalServerError)
return
}
existing.CronExpr = page.CronExpr
existing.Paths = paths
existing.Excludes = excludes
existing.Tags = tags
existing.RetentionPolicy = retention
existing.Options = options
existing.Enabled = page.Enabled
existing.Manual = page.Manual
if err := s.deps.Store.UpdateSchedule(r.Context(), existing); err != nil {
page.Error = "Couldn't save schedule — see server log."
slog.Error("ui schedule update", "err", err)
s.renderEditPage(w, u, page)
return
}
_ = s.deps.Store.AppendAudit(r.Context(), store.AuditEntry{
ID: ulid.Make().String(),
UserID: &storeUser.ID,
Actor: "user",
Action: "schedule.updated",
TargetKind: ptr("schedule"),
TargetID: &scheduleID,
TS: nowUTC(),
})
s.pushScheduleSetAsync(hostID)
}
stdhttp.Redirect(w, r, "/hosts/"+hostID+"/schedules", stdhttp.StatusSeeOther)
stdhttp.Error(w, "schedules UI is being rebuilt — see P2 redesign Phase 4", stdhttp.StatusNotImplemented)
}
// handleUIScheduleRun is the POST target of per-schedule Run-now
// buttons. Reuses dispatchScheduledJob (the same code path used by
// the agent's local cron firing) so manual + automated runs flow
// through identical job lifecycle. Sets HX-Redirect to the live
// log on success.
func (s *Server) handleUIScheduleRun(w stdhttp.ResponseWriter, r *stdhttp.Request) {
u := s.requireUIUser(w, r)
if u == nil {
return
}
hostID := chi.URLParam(r, "id")
scheduleID := chi.URLParam(r, "sid")
host, err := s.deps.Store.GetHost(r.Context(), hostID)
if err != nil {
if errors.Is(err, store.ErrNotFound) {
stdhttp.NotFound(w, r)
return
}
stdhttp.Error(w, "internal", stdhttp.StatusInternalServerError)
return
}
if !s.deps.Hub.Connected(hostID) {
stdhttp.Error(w, "agent is offline", stdhttp.StatusBadRequest)
return
}
_ = host
jobID, err := s.dispatchScheduleNow(r.Context(), hostID, scheduleID, nil)
if err != nil {
stdhttp.Error(w, err.Error(), stdhttp.StatusBadRequest)
return
}
target := "/jobs/" + jobID
if r.Header.Get("HX-Request") == "true" {
w.Header().Set("HX-Redirect", target)
w.WriteHeader(stdhttp.StatusOK)
return
}
stdhttp.Redirect(w, r, target, stdhttp.StatusSeeOther)
}
// handleUIScheduleDelete is the POST target of the Delete buttons on
// the list view. Confirm-then-redirect; no AJAX.
func (s *Server) handleUIScheduleDelete(w stdhttp.ResponseWriter, r *stdhttp.Request) {
u := s.requireUIUser(w, r)
if u == nil {
return
}
hostID := chi.URLParam(r, "id")
scheduleID := chi.URLParam(r, "sid")
storeUser, _, err := s.userByID(r, u.ID)
if err != nil || storeUser == nil {
stdhttp.Error(w, "internal", stdhttp.StatusInternalServerError)
return
}
if err := s.deps.Store.DeleteSchedule(r.Context(), hostID, scheduleID); err != nil {
if errors.Is(err, store.ErrNotFound) {
stdhttp.NotFound(w, r)
return
}
stdhttp.Error(w, "internal", stdhttp.StatusInternalServerError)
return
}
_ = s.deps.Store.AppendAudit(r.Context(), store.AuditEntry{
ID: ulid.Make().String(),
UserID: &storeUser.ID,
Actor: "user",
Action: "schedule.deleted",
TargetKind: ptr("schedule"),
TargetID: &scheduleID,
TS: nowUTC(),
})
s.pushScheduleSetAsync(hostID)
stdhttp.Redirect(w, r, "/hosts/"+hostID+"/schedules", stdhttp.StatusSeeOther)
stdhttp.Error(w, "schedules UI is being rebuilt — see P2 redesign Phase 4", stdhttp.StatusNotImplemented)
}
func (s *Server) renderScheduleEdit(w stdhttp.ResponseWriter, view ui.ViewData) {
if err := s.deps.UI.Render(w, "schedule_edit", view); err != nil {
slog.Error("ui: render schedule_edit", "err", err)
stdhttp.Error(w, "internal", stdhttp.StatusInternalServerError)
}
}
func (s *Server) renderEditPage(w stdhttp.ResponseWriter, u *ui.User, page scheduleEditPage) {
view := s.baseView(u, "dashboard")
if page.IsNew {
view.Title = "New schedule · " + page.Host.Name
} else {
view.Title = "Edit schedule · " + page.Host.Name
}
view.Page = page
w.WriteHeader(stdhttp.StatusUnprocessableEntity)
s.renderScheduleEdit(w, view)
}
// ----- helpers --------------------------------------------------------
// splitCSV parses comma-separated values into a clean []string —
// leading/trailing whitespace trimmed, blanks dropped.
func splitCSV(s string) []string {
out := []string{}
for _, p := range strings.Split(s, ",") {
if t := strings.TrimSpace(p); t != "" {
out = append(out, t)
}
}
return out
}
func parseRetention(p scheduleEditPage) (store.RetentionPolicy, error) {
var r store.RetentionPolicy
for _, f := range []struct {
raw string
dest **int
name string
}{
{p.KeepLast, &r.KeepLast, "keep last"},
{p.KeepHourly, &r.KeepHourly, "keep hourly"},
{p.KeepDaily, &r.KeepDaily, "keep daily"},
{p.KeepWeekly, &r.KeepWeekly, "keep weekly"},
{p.KeepMonthly, &r.KeepMonthly, "keep monthly"},
{p.KeepYearly, &r.KeepYearly, "keep yearly"},
} {
v, err := parsePosInt(f.raw)
if err != nil {
return r, errFmtf("%s: %s", f.name, err)
}
*f.dest = v
}
return r, nil
}
func parseOptions(p scheduleEditPage) (store.ScheduleOptions, error) {
var o store.ScheduleOptions
up, err := parsePosInt(p.LimitUpKBps)
if err != nil {
return o, errFmtf("limit upload: %s", err)
}
o.LimitUploadKBps = up
down, err := parsePosInt(p.LimitDownKBps)
if err != nil {
return o, errFmtf("limit download: %s", err)
}
o.LimitDownloadKBps = down
return o, nil
}
// parsePosInt turns a possibly-empty string into *int. Empty → nil
// (no value). Non-empty must parse as a positive int.
func parsePosInt(raw string) (*int, error) {
if raw == "" {
return nil, nil
}
v, err := strconv.Atoi(raw)
if err != nil {
return nil, errFmtf("must be a whole number")
}
if v < 0 {
return nil, errFmtf("must be non-negative")
}
return &v, nil
}
func intStringPtr(p *int) string {
if p == nil {
return ""
}
return strconv.Itoa(*p)
}
// uiErrorMessage maps the JSON-API validation codes to operator-
// friendly banner text.
func uiErrorMessage(code, msg string) string {
switch code {
case "missing_cron_expr":
return "Cron expression is required."
case "invalid_cron_expr":
return "Cron expression doesn't parse: " + msg
case "missing_paths":
return "At least one backup path is required (one per line)."
case "missing_retention":
return "Forget schedules need at least one Keep-* value, otherwise restic would delete every snapshot."
case "invalid_kind":
return "Unsupported schedule kind."
default:
return msg
}
}
// errFmtf wraps fmt.Errorf so the validators read consistently.
func errFmtf(format string, args ...any) error {
return fmt.Errorf(format, args...)
func (s *Server) handleUIScheduleRun(w stdhttp.ResponseWriter, r *stdhttp.Request) {
stdhttp.Error(w, "schedules UI is being rebuilt — see P2 redesign Phase 4", stdhttp.StatusNotImplemented)
}