P2 redesign · phase 2: store rewrite — sources, slim schedules, repo maintenance

Go-side data model rebuilt against migration 0008. The fat-Schedule
shape (paths/excludes/tags/retention/manual/kind/options/hooks) is
gone; that surface lives on source_groups now.

* store/types.go
  - Schedule slimmed to {id, host_id, cron, enabled, source_group_ids,
    timestamps}. SourceGroupIDs populated by Get/List, accepted on
    Create/Update so callers pass desired junction state in one shape.
  - SourceGroup added: name (= snapshot tag), includes/excludes,
    retention_policy, retry_max + retry_backoff_seconds, cached
    conflict_dimension.
  - HostRepoMaintenance added: forget/prune/check cadences + enabled.
  - PendingRun added: offline-retry queue.
  - Host loses RepoInitialisedAt; gains BandwidthUpKBps + BandwidthDownKBps.
  - RetentionPolicy moves home from "schedule field" to "source group
    field" but the type itself + Summary() method unchanged.

* store/sources.go (new) — CRUD + GetByName + ConflictDimension cache.
  Group writes bump host_schedule_version; conflict cache writes don't
  (server-internal projection, agent doesn't see it).
* store/maintenance.go (new) — CreateDefault is idempotent (INSERT OR
  IGNORE). UpdateRepoMaintenance doesn't bump schedule version because
  these run on the server's own ticker, not the agent's local cron.
* store/pending.go (new) — Enqueue / DueRunsForRetry / Bump / Delete.
* store/schedules.go — rewritten for slim shape + junction CRUD.
  Update wipes the schedule_source_groups junction wholesale and
  re-inserts (simpler than diffing). Adds SchedulesUsingGroup for
  retention-conflict detection + UI labels.
* store/hosts.go — drops repo_initialised_at scan, adds bandwidth scan.
  New SetHostBandwidth helper.

* HTTP layer — temporarily stubbed during this rewrite (501 returns
  with redesign_in_progress error code). Phase 3 fills these in
  against the new shape:
    - schedules.go REST CRUD
    - schedule_push.go agent reconciliation
    - ui_schedules.go HTML form CRUD
  Run-now-per-host + Init-repo handlers in ui_handlers.go also stubbed
  — both go away in the new model (Run-now per source group; auto-init
  at host enrolment).

* enrollment.go — replaces "seed manual schedule from typed paths"
  with "seed default source group + repo-maintenance row." The default
  group gets the typed paths as its includes; operator edits later
  via Sources tab.

* ws/handler.go — drops the MarkHostRepoInitialised projection (column
  is gone; auto-init makes it derivable from latest init job's status).

Tests:
* store: existing schedule test rewritten for slim shape + junction;
  new sources_test.go covers source-group CRUD, name uniqueness,
  conflict cache, repo-maintenance defaults + idempotent seed,
  pending-runs queue lifecycle.
* http: schedules_test.go and schedule_push_test.go deleted — both
  exercised the obsolete fat-schedule API. Phase 3 rewrites them
  against the new endpoints.

go test ./... green. cmd/server + cmd/agent build. The UI is broken
end-to-end (schedules / sources / repo tabs all hit 501 stubs); Phase 3
restores REST + on-the-wire reconciliation; Phase 4 rewires the UI
templates against the new model.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
This commit is contained in:
2026-05-02 21:30:41 +01:00
parent e717b6998c
commit e7eea7afac
16 changed files with 1076 additions and 1928 deletions
+19 -292
View File
@@ -1,310 +1,37 @@
package http
import (
"encoding/json"
"errors"
stdhttp "net/http"
"strings"
"github.com/go-chi/chi/v5"
"github.com/oklog/ulid/v2"
"github.com/robfig/cron/v3"
"gitea.dcglab.co.uk/steve/restic-manager/internal/api"
"gitea.dcglab.co.uk/steve/restic-manager/internal/store"
)
// scheduleAPI is the JSON shape for /api/hosts/{id}/schedules. We
// avoid leaking host_id in the body since it's already in the URL,
// and we render booleans + retention as typed JSON rather than
// strings so the UI can edit fields directly.
type scheduleAPI struct {
ID string `json:"id,omitempty"`
Kind api.JobKind `json:"kind"`
CronExpr string `json:"cron_expr"`
Paths []string `json:"paths"`
Excludes []string `json:"excludes"`
Tags []string `json:"tags"`
RetentionPolicy store.RetentionPolicy `json:"retention_policy"`
Options store.ScheduleOptions `json:"options"`
PreHook string `json:"pre_hook,omitempty"`
PostHook string `json:"post_hook,omitempty"`
Enabled bool `json:"enabled"`
// Manual = no cron, fires only when the operator triggers a
// run-now. Cron expr is ignored when this is true.
Manual bool `json:"manual"`
CreatedAt string `json:"created_at,omitempty"`
UpdatedAt string `json:"updated_at,omitempty"`
}
// listSchedulesResp wraps the array so the response is forward-
// compatible (we may want to add a top-level `version` later).
type listSchedulesResp struct {
Version int64 `json:"version"`
Schedules []scheduleAPI `json:"schedules"`
}
// cron parser used for input validation. Standard 5-field syntax
// with descriptors (@hourly etc.) — same parser the agent will
// run against, so a schedule that validates here will fire there.
var cronParser = cron.NewParser(
cron.Minute | cron.Hour | cron.Dom | cron.Month | cron.Dow | cron.Descriptor,
)
// schedules.go — REST API for /api/hosts/{id}/schedules.
//
// Stubbed during the P2 redesign data-model rewrite (commit chain).
// Phase 2 dropped the fat Schedule shape (paths/excludes/tags/
// retention/manual/kind/options/hooks) — the slim Schedule + source
// groups model lives in store/. Phase 3 of the redesign will fill in
// these handlers against the new shape.
//
// Returning 501 here keeps the routes addressable; UI calls will
// surface the unimplemented state via the toast component until the
// new handlers land.
func (s *Server) handleListSchedules(w stdhttp.ResponseWriter, r *stdhttp.Request) {
if _, ok := s.requireUser(r); !ok {
writeJSONError(w, stdhttp.StatusUnauthorized, "unauthorized", "")
return
}
hostID := chi.URLParam(r, "id")
if hostID == "" {
writeJSONError(w, stdhttp.StatusBadRequest, "missing_host_id", "")
return
}
if _, err := s.deps.Store.GetHost(r.Context(), hostID); err != nil {
if errors.Is(err, store.ErrNotFound) {
writeJSONError(w, stdhttp.StatusNotFound, "host_not_found", "")
return
}
writeJSONError(w, stdhttp.StatusInternalServerError, "internal", "")
return
}
rows, err := s.deps.Store.ListSchedulesByHost(r.Context(), hostID)
if err != nil {
writeJSONError(w, stdhttp.StatusInternalServerError, "internal", "")
return
}
version, err := s.deps.Store.GetHostScheduleVersion(r.Context(), hostID)
if err != nil {
writeJSONError(w, stdhttp.StatusInternalServerError, "internal", "")
return
}
out := listSchedulesResp{Version: version, Schedules: make([]scheduleAPI, len(rows))}
for i, row := range rows {
out.Schedules[i] = toScheduleAPI(row)
}
writeJSON(w, stdhttp.StatusOK, out)
writeJSONError(w, stdhttp.StatusNotImplemented, "redesign_in_progress",
"schedule REST API is being rebuilt — see P2 redesign Phase 3")
}
func (s *Server) handleCreateSchedule(w stdhttp.ResponseWriter, r *stdhttp.Request) {
user, ok := s.requireUser(r)
if !ok {
writeJSONError(w, stdhttp.StatusUnauthorized, "unauthorized", "")
return
}
hostID := chi.URLParam(r, "id")
if hostID == "" {
writeJSONError(w, stdhttp.StatusBadRequest, "missing_host_id", "")
return
}
if _, err := s.deps.Store.GetHost(r.Context(), hostID); err != nil {
if errors.Is(err, store.ErrNotFound) {
writeJSONError(w, stdhttp.StatusNotFound, "host_not_found", "")
return
}
writeJSONError(w, stdhttp.StatusInternalServerError, "internal", "")
return
}
var req scheduleAPI
if err := json.NewDecoder(r.Body).Decode(&req); err != nil {
writeJSONError(w, stdhttp.StatusBadRequest, "invalid_json", err.Error())
return
}
if code, msg := validateSchedule(&req); code != "" {
writeJSONError(w, stdhttp.StatusBadRequest, code, msg)
return
}
row := store.Schedule{
ID: ulid.Make().String(),
HostID: hostID,
Kind: string(req.Kind),
CronExpr: req.CronExpr,
Paths: req.Paths,
Excludes: req.Excludes,
Tags: req.Tags,
RetentionPolicy: req.RetentionPolicy,
Options: req.Options,
PreHook: req.PreHook,
PostHook: req.PostHook,
Enabled: req.Enabled,
}
if err := s.deps.Store.CreateSchedule(r.Context(), &row); err != nil {
writeJSONError(w, stdhttp.StatusInternalServerError, "internal", err.Error())
return
}
_ = s.deps.Store.AppendAudit(r.Context(), store.AuditEntry{
ID: ulid.Make().String(),
UserID: &user.ID,
Actor: "user",
Action: "schedule.created",
TargetKind: ptr("schedule"),
TargetID: &row.ID,
TS: nowUTC(),
})
s.pushScheduleSetAsync(hostID)
writeJSON(w, stdhttp.StatusCreated, toScheduleAPI(row))
writeJSONError(w, stdhttp.StatusNotImplemented, "redesign_in_progress",
"schedule REST API is being rebuilt — see P2 redesign Phase 3")
}
func (s *Server) handleUpdateSchedule(w stdhttp.ResponseWriter, r *stdhttp.Request) {
user, ok := s.requireUser(r)
if !ok {
writeJSONError(w, stdhttp.StatusUnauthorized, "unauthorized", "")
return
}
hostID := chi.URLParam(r, "id")
scheduleID := chi.URLParam(r, "sid")
if hostID == "" || scheduleID == "" {
writeJSONError(w, stdhttp.StatusBadRequest, "missing_id", "")
return
}
existing, err := s.deps.Store.GetSchedule(r.Context(), hostID, scheduleID)
if err != nil {
if errors.Is(err, store.ErrNotFound) {
writeJSONError(w, stdhttp.StatusNotFound, "schedule_not_found", "")
return
}
writeJSONError(w, stdhttp.StatusInternalServerError, "internal", "")
return
}
var req scheduleAPI
if err := json.NewDecoder(r.Body).Decode(&req); err != nil {
writeJSONError(w, stdhttp.StatusBadRequest, "invalid_json", err.Error())
return
}
// Kind is immutable; ignore whatever was sent and fix up before
// validation so validateSchedule sees the existing kind.
req.Kind = api.JobKind(existing.Kind)
if code, msg := validateSchedule(&req); code != "" {
writeJSONError(w, stdhttp.StatusBadRequest, code, msg)
return
}
existing.CronExpr = req.CronExpr
existing.Paths = req.Paths
existing.Excludes = req.Excludes
existing.Tags = req.Tags
existing.RetentionPolicy = req.RetentionPolicy
existing.Options = req.Options
existing.PreHook = req.PreHook
existing.PostHook = req.PostHook
existing.Enabled = req.Enabled
if err := s.deps.Store.UpdateSchedule(r.Context(), existing); err != nil {
if errors.Is(err, store.ErrNotFound) {
writeJSONError(w, stdhttp.StatusNotFound, "schedule_not_found", "")
return
}
writeJSONError(w, stdhttp.StatusInternalServerError, "internal", err.Error())
return
}
_ = s.deps.Store.AppendAudit(r.Context(), store.AuditEntry{
ID: ulid.Make().String(),
UserID: &user.ID,
Actor: "user",
Action: "schedule.updated",
TargetKind: ptr("schedule"),
TargetID: &existing.ID,
TS: nowUTC(),
})
s.pushScheduleSetAsync(hostID)
writeJSON(w, stdhttp.StatusOK, toScheduleAPI(*existing))
writeJSONError(w, stdhttp.StatusNotImplemented, "redesign_in_progress",
"schedule REST API is being rebuilt — see P2 redesign Phase 3")
}
func (s *Server) handleDeleteSchedule(w stdhttp.ResponseWriter, r *stdhttp.Request) {
user, ok := s.requireUser(r)
if !ok {
writeJSONError(w, stdhttp.StatusUnauthorized, "unauthorized", "")
return
}
hostID := chi.URLParam(r, "id")
scheduleID := chi.URLParam(r, "sid")
if hostID == "" || scheduleID == "" {
writeJSONError(w, stdhttp.StatusBadRequest, "missing_id", "")
return
}
if err := s.deps.Store.DeleteSchedule(r.Context(), hostID, scheduleID); err != nil {
if errors.Is(err, store.ErrNotFound) {
writeJSONError(w, stdhttp.StatusNotFound, "schedule_not_found", "")
return
}
writeJSONError(w, stdhttp.StatusInternalServerError, "internal", err.Error())
return
}
_ = s.deps.Store.AppendAudit(r.Context(), store.AuditEntry{
ID: ulid.Make().String(),
UserID: &user.ID,
Actor: "user",
Action: "schedule.deleted",
TargetKind: ptr("schedule"),
TargetID: &scheduleID,
TS: nowUTC(),
})
s.pushScheduleSetAsync(hostID)
w.WriteHeader(stdhttp.StatusNoContent)
}
// validateSchedule rejects malformed inputs with a stable error code.
// Returns ("", "") on success.
func validateSchedule(s *scheduleAPI) (code, msg string) {
switch s.Kind {
case api.JobBackup, api.JobForget, api.JobPrune, api.JobCheck:
// ok — valid schedule kinds (init/unlock are operator-driven only).
default:
return "invalid_kind", "kind must be one of backup|forget|prune|check"
}
if !s.Manual {
if strings.TrimSpace(s.CronExpr) == "" {
return "missing_cron_expr", "cron_expr is required (or set manual=true)"
}
if _, err := cronParser.Parse(s.CronExpr); err != nil {
return "invalid_cron_expr", err.Error()
}
}
if s.Kind == api.JobBackup && len(s.Paths) == 0 {
return "missing_paths", "backup schedules require at least one path"
}
// forget needs at least one keep-* dimension; otherwise restic
// would happily delete every snapshot.
if s.Kind == api.JobForget && (s.RetentionPolicy == store.RetentionPolicy{}) {
return "missing_retention", "forget schedules require at least one Keep-* value"
}
// Hooks are only meaningful on backup schedules (spec §14.3).
if s.Kind != api.JobBackup && (s.PreHook != "" || s.PostHook != "") {
return "hooks_not_allowed", "pre_hook / post_hook only apply to backup schedules"
}
return "", ""
}
func toScheduleAPI(s store.Schedule) scheduleAPI {
out := scheduleAPI{
ID: s.ID,
Kind: api.JobKind(s.Kind),
CronExpr: s.CronExpr,
Paths: s.Paths,
Excludes: s.Excludes,
Tags: s.Tags,
RetentionPolicy: s.RetentionPolicy,
Options: s.Options,
PreHook: s.PreHook,
PostHook: s.PostHook,
Enabled: s.Enabled,
Manual: s.Manual,
CreatedAt: s.CreatedAt.Format("2006-01-02T15:04:05.999999999Z07:00"),
UpdatedAt: s.UpdatedAt.Format("2006-01-02T15:04:05.999999999Z07:00"),
}
if out.Paths == nil {
out.Paths = []string{}
}
if out.Excludes == nil {
out.Excludes = []string{}
}
if out.Tags == nil {
out.Tags = []string{}
}
return out
writeJSONError(w, stdhttp.StatusNotImplemented, "redesign_in_progress",
"schedule REST API is being rebuilt — see P2 redesign Phase 3")
}