From a1440719ae48c579c49fed0f75c55d6236fb169c Mon Sep 17 00:00:00 2001 From: Steve Cliff Date: Mon, 22 Jun 2026 00:19:05 +0100 Subject: [PATCH] docs: Phase 1 status report Co-Authored-By: Claude Opus 4.8 (1M context) --- specifications/PHASE1-STATUS.md | 92 +++++++++++++++++++++++++++++++++ 1 file changed, 92 insertions(+) create mode 100644 specifications/PHASE1-STATUS.md diff --git a/specifications/PHASE1-STATUS.md b/specifications/PHASE1-STATUS.md new file mode 100644 index 0000000..1209a81 --- /dev/null +++ b/specifications/PHASE1-STATUS.md @@ -0,0 +1,92 @@ +# emcli — Phase 1 Status Report + +**Date:** 2026-06-22 +**Branch:** `main` (pushed to `gitea.dcglab.co.uk/steve/emcli`, HEAD `6061bd2`) +**Phase 1 scope:** Foundation + read path (encrypted config, IMAP read, agent commands, password auth) + +## TL;DR + +**Phase 1 is complete and merged to `main`.** All 14 planned tasks were implemented test-first, each reviewed (spec compliance + code quality) by a separate agent, with fixes applied and re-verified. The binary builds as a single static CGO-free executable, `go vet` is clean, and the full unit-test suite passes. An end-to-end smoke test of the real binary confirms the agent (JSON) and admin (human-readable) surfaces behave per spec. + +``` +go build (CGO_ENABLED=0) → OK, single static binary +go vet ./... → clean +go test ./... → all packages pass +``` + +## What was built + +| Package | Responsibility | Status | +|---|---|---| +| `internal/version` | Build version string + `version` command | ✅ | +| `internal/crypto` | AES-256-GCM field encryption keyed from `EMCLI_KEY` (fail-closed) | ✅ | +| `internal/store` | Pure-Go SQLite: schema v1, accounts CRUD (encrypted secrets), whitelists, **seen-set read state** (floor + acked + compaction + UIDVALIDITY reset), audit log + retention purge | ✅ | +| `internal/policy` | Pure enforcement: case-insensitive address/domain matching, inbound whitelist + subject-regex filter | ✅ | +| `internal/mail` | RFC822 parsing (headers/body/attachments) + IMAP client (select/fetch/search, read-only, Peek) | ✅ | +| `internal/cli` | JSON envelope, agent commands `list`/`get`/`search`/`ack` (policy-gated), flag-based admin (`account`, `whitelist`), command router | ✅ | +| `cmd/emcli` | Entry point routing to `cli.Run` | ✅ | + +### Working commands (verified via smoke test) +- **Agent (JSON-only output, exit code mirrors error):** + - `emcli list --account [--new] [--before/--since ] [--limit N]` + - `emcli get --account --uid ` + - `emcli search --account [--from --subject-contains --text --since-date --before-date]` + - `emcli ack --account --uid-list 1,2,3` +- **Admin (human-readable):** + - `emcli account add|list`, `emcli whitelist in|out add|remove|list` + - `emcli version` + +### Security invariant — independently verified in the final review +Mail that fails the inbound policy (`whitelist_in` / `subject_regex`) is **invisible** on every path: excluded from `list`/`search`, returns generic `not_found` on `get` (body never fetched), and **cannot be `ack`ed** (no state manipulation for unseen mail). Filtered vs. genuinely-absent UIDs are indistinguishable to the agent. Credentials/secrets never appear in stdout, the JSON envelope, or the audit log; passwords are sealed at rest (ciphertext ≠ plaintext confirmed by test). + +## Process + +Executed via subagent-driven development: a fresh implementer per task (TDD, commit per task), a separate reviewer per task (spec + quality gates), fix sub-agents for Critical/Important findings, and a final whole-branch review on the most capable model. Durable progress tracked in `.superpowers/sdd/progress.md`. 20 commits on `main` (`04d3b61..6061bd2`). + +**Fixes applied during review (all re-verified):** +- `store`: pinned SQLite pool to one connection so `PRAGMA foreign_keys` (cascade deletes) stays effective; added a cascade test. +- `mail`: propagate `io.ReadAll` errors when parsing message parts and reading bodies off the network (prevents silent truncation); apply the `search` limit. +- `cli`: `AckCmd` reuses the folder UID-validity from `setup` (was swallowing a second `SelectFolder` error → epoch-0 acks); agent commands now **exit non-zero** when the JSON envelope reports an error (spec §8); `search` limit now counts *visible* results (filter before cap), consistent with `list`. + +## Known limitations / deferred (not defects) + +### Deliberately out of scope for Phase 1 (have their own future plans) +- **Phase 2 — Send path:** SMTP, MIME building, reply threading, outbound policy (RO/RW + whitelist-out), `send`. +- **Phase 3 — OAuth2:** Gmail loopback consent + token refresh (schema columns already present). +- **Phase 4 — Admin TUI + `doctor`:** interactive init/reconfigure, connectivity diagnostics. + +### Carry-over items to address (logged from reviews) +1. **[Important, Phase 2 priority] Header listing fetches full message bodies.** `mail.fetchByUIDSet` downloads each whole RFC822 message even for `list`/`search`, then discards the body. Functionally correct but slow/heavy on real mailboxes. Fix = request a header-only body section / `ENVELOPE` + `BODYSTRUCTURE` when `full=false`. **Deliberately not fixed yet** because the change to the IMAP fetch path could not be validated against a live server in this environment (see below); it should be done with a real/TLS-configured IMAP server in front of it. Phase 2's `--reply-to` should build that header-only fetch path too. +2. **[Minor] Audit completeness:** `list`/`search` don't write a per-filtered-message audit row; some IMAP-error envelope paths don't write an audit row; `imap_error` is an audit reason not enumerated in the spec. +3. **[Minor] CLI polish:** `account add` opens the store before parsing its own flags (`--help` touches the DB); invalid `--since-date`/`--before-date` are silently ignored rather than erroring; `whitelist add/remove` don't reject an empty `--address` at the CLI layer. +4. **[Minor] Performance:** `ack` fetches each UID's header in its own round-trip; `get` fetches the header then re-fetches the full message. Fine for a per-invocation CLI; batch ops in later phases may want to optimize. + +### Validation gap +- **Live IMAP integration was not exercised.** The `internal/mail` integration test (build tag `integration`) compiles and self-skips without a server. An attempt to validate against a GreenMail container failed on TLS/port friction: GreenMail 2.1.0's IMAP STARTTLS didn't negotiate here and the IMAPS port wasn't exposed, while our client (correctly) requires verified TLS and the test config has no skip-verify option. The IMAP code was reviewed by inspection and compiles under the integration tag, but has **not** been run end-to-end against a real server. **Recommended follow-up:** add a dev-only `--insecure`/skip-verify option (or point the integration test at a properly-certificated server) so the read path can be validated against live IMAP before relying on it operationally. + +## Suggested next steps + +1. **Validate the read path against a real IMAP account** (e.g. your own mailbox over TLS) — this is the highest-value confidence step and would also surface the full-body-fetch performance issue (#1) in practice. +2. Decide whether to fix the header-only fetch (#1) before or as part of Phase 2. +3. Proceed to **Phase 2 (send path)** — the structure is ready: `policy` is set up to host outbound checks, `Account.Mode` is loaded for RO rejection, and `MatchAddress` already implements the domain matching whitelist-out needs. + +## Repository layout + +``` +emcli/ +├── Makefile # build/test/vet (CGO_ENABLED=0) +├── go.mod / go.sum +├── cmd/emcli/main.go +├── internal/ +│ ├── version/ +│ ├── crypto/ +│ ├── store/ # store, schema, settings, account, whitelist, seenset, audit +│ ├── policy/ # policy (address), inbound +│ ├── mail/ # message, imap, testdata/*.eml +│ └── cli/ # envelope, agent, dispatch, run, admin +└── specifications/ + ├── PRD.md + ├── SPEC.md + ├── PHASE1-STATUS.md # this file + └── plans/2026-06-21-phase1-foundation-and-read-path.md +```