Post-brainstorm change after operator review: overnight-digest /
"don't ping me at 03:00, email me in the morning" use case is poorly
served by ntfy (push) and clumsy via webhook → email-gateway. SMTP joins
webhook + ntfy as the third v1 channel; Apprise stays deferred.
Spec updates:
- Decision 5 reworded: three channels in v1.
- Channel iface gains smtpChannel using net/smtp + crypto/tls. 10s
timeout vs 5s for HTTP — STARTTLS handshake + DATA over a slow link
legitimately needs the headroom.
- Migration 0014 CHECK now allows 'smtp'. New smtpConfig struct: host,
port, encryption (starttls/tls/none), username, password (AEAD), from,
to. One channel = one To-address; multi-recipient = multiple channels
(keeps failure attribution per-recipient).
- Body shape documented: hardcoded subject pattern
'[restic-manager] [<sev>] <host>: <kind>', Message-ID includes the
alert id so threading groups raised → ack → resolved cleanly. Plain
text only in v1.
- Encryption defaults to STARTTLS on 465/587; PLAIN auth over TLS, no
XOAUTH2 yet (app passwords recommended for Gmail / M365).
- Test plan adds MailHog step in the Playwright sweep.
- Non-goals expanded: HTML emails, OAuth2/XOAUTH2, multi-recipient
channels are explicitly out of v1.
Wireframe updates (_diag/p3-alerts-wireframe/wireframe.html):
- Kind picker grows from 2 cards to 3 (Webhook / Ntfy / SMTP @). SMTP
gets the --ok green colour family so it visually separates from
webhook (accent) and ntfy (warm).
- New SMTP variant section (3c): host+port+encryption row, user+pass
row, from+to row, test result, plus right-rail email shape preview
showing the RFC 5322 layout.
- Channel list grows a third row: 'overnight-digest · smtp://… →
ops-overnight@example.com'.
Phase 3 sub-spec covering the alerts engine, notification channels, and
UI (P3-05/06/07). Brainstorm ran 2026-05-04; all ten design decisions
locked before this spec was written.
Key decisions captured:
- Hardcoded rule set, no operator-tunable thresholds in v1. Six rules:
backup_failed, forget_failed, prune_failed, check_failed,
stale_schedule, agent_offline.
- Hybrid engine cadence: event hooks at MarkJobFinished + offline-sweeper
for immediate triggers; one 60s ticker for stale-schedule detection +
auto-resolution sweeps.
- Auto-resolve when underlying condition clears; manual Resolve any time;
Acknowledge as a separate I-have-seen-it intermediate state that does
NOT close the alert.
- v1 channels: native ntfy + webhook. Apprise + SMTP deferred. Channel
scope is global only — no per-host or per-severity routing.
- Webhook payload is one stable JSON envelope shape across raised /
acknowledged / resolved / test events; ntfy uses the standard publish
format with severity → priority mapping.
- Per-channel Send Test Notification button hits the real send path with
a synthetic info-severity event; inline green-tick / red-cross result.
- Dedup by (host_id, kind, resolved_at IS NULL); last_seen_at bumped on
every confirming tick so the UI can render still happening · Ns ago
without re-notifying.
- Top-level /alerts page; Settings shell with Notifications sub-tab.
Per-host vitals Open alerts cell deep-links into filtered list.
- Best-effort fire-and-forget delivery with 5s timeout; failures logged
to a new notification_log table but never retried. Alert row in the DB
is the source of truth.
Migrations:
- 0013 adds alerts.last_seen_at (column-level ALTER per CLAUDE.md)
- 0014 adds notification_channels + notification_log tables
Wireframe: _diag/p3-alerts-wireframe/wireframe.html
steve