emcli/specifications/PHASE1-STATUS.md

emcli — Phase 1 Status Report

Date: 2026-06-22 Branch: main (pushed to gitea.dcglab.co.uk/steve/emcli) Phase 1 scope: Foundation + read path (encrypted config, IMAP read, agent commands, password auth)

Update (post-Phase-1 hardening): the read path has now been validated end-to-end against a live IMAP account, and the full-message-body-download issue has been resolved within Phase 1 (list/search fetch headers only). See "Live validation" and "Resolved" sections below.

TL;DR

Phase 1 is complete and merged to main. All 14 planned tasks were implemented test-first, each reviewed (spec compliance + code quality) by a separate agent, with fixes applied and re-verified. The binary builds as a single static CGO-free executable, go vet is clean, and the full unit-test suite passes (including under -race). Validated both via a real-binary smoke test and against a live IMAP mailbox.

Live validation (real IMAP account)

The read path was exercised against a live cPanel/MXlogin IMAP account over verified TLS (port 993, Let's Encrypt cert):

• Auth + connect: email-as-username login over implicit TLS succeeded.
• list: returned real INBOX headers; has_attachments correct.
• get: returned the full plain-text body and base64 attachments that decoded to exact byte sizes (a cPanel config mail with a PNG + 3 .mobileconfig files — all 4 matched).
• search: --from/--subject-contains correctly narrowed results.
• Seen-set: list --new correctly returned 0 on first contact (baseline floor = max UID); ack succeeded; folder_state/acked rows correct.
• Audit: every command wrote an audit_log row; password confirmed encrypted at rest (no plaintext in the DB).

Resolved within Phase 1: header-only fetch

The previously-flagged "header listing downloads full message bodies" issue is fixed (internal/mail/imap.go). list and search now fetch BODY.PEEK[HEADER] + BODYSTRUCTURE instead of the whole RFC822 message, so listing a large mailbox no longer downloads every body and attachment. Header parsing reuses the same go-message path (RFC2047 decoding and address formatting unchanged); has_attachments is derived from the body-structure tree. get still fetches the full message. Verified: output is identical to the prior full-fetch behaviour against the live account, new unit tests (TestParseHeaderBytes, TestHasAttachment) pass, and the fetch loops were hardened to drain the IMAP channel on error (race-tested).

go build (CGO_ENABLED=0)  → OK, single static binary
go vet ./...              → clean
go test ./...             → all packages pass

What was built

Package	Responsibility	Status
internal/version	Build version string + version command	✅
internal/crypto	AES-256-GCM field encryption keyed from EMCLI_KEY (fail-closed)	✅
internal/store	Pure-Go SQLite: schema v1, accounts CRUD (encrypted secrets), whitelists, seen-set read state (floor + acked + compaction + UIDVALIDITY reset), audit log + retention purge	✅
internal/policy	Pure enforcement: case-insensitive address/domain matching, inbound whitelist + subject-regex filter	✅
internal/mail	RFC822 parsing (headers/body/attachments) + IMAP client (select/fetch/search, read-only, Peek)	✅
internal/cli	JSON envelope, agent commands list/get/search/ack (policy-gated), flag-based admin (account, whitelist), command router	✅
cmd/emcli	Entry point routing to cli.Run	✅

Working commands (verified via smoke test)

• Agent (JSON-only output, exit code mirrors error):
  • emcli list --account <a> [--new] [--before/--since <uid>] [--limit N]
  • emcli get --account <a> --uid <uid>
  • emcli search --account <a> [--from --subject-contains --text --since-date --before-date]
  • emcli ack --account <a> --uid-list 1,2,3
• Admin (human-readable):
  • emcli account add|list, emcli whitelist in|out add|remove|list
  • emcli version

Security invariant — independently verified in the final review

Mail that fails the inbound policy (whitelist_in / subject_regex) is invisible on every path: excluded from list/search, returns generic not_found on get (body never fetched), and cannot be acked (no state manipulation for unseen mail). Filtered vs. genuinely-absent UIDs are indistinguishable to the agent. Credentials/secrets never appear in stdout, the JSON envelope, or the audit log; passwords are sealed at rest (ciphertext ≠ plaintext confirmed by test).

Process

Executed via subagent-driven development: a fresh implementer per task (TDD, commit per task), a separate reviewer per task (spec + quality gates), fix sub-agents for Critical/Important findings, and a final whole-branch review on the most capable model. Durable progress tracked in .superpowers/sdd/progress.md. 20 commits on main (04d3b61..6061bd2).

Fixes applied during review (all re-verified):

• store: pinned SQLite pool to one connection so PRAGMA foreign_keys (cascade deletes) stays effective; added a cascade test.
• mail: propagate io.ReadAll errors when parsing message parts and reading bodies off the network (prevents silent truncation); apply the search limit.
• cli: AckCmd reuses the folder UID-validity from setup (was swallowing a second SelectFolder error → epoch-0 acks); agent commands now exit non-zero when the JSON envelope reports an error (spec §8); search limit now counts visible results (filter before cap), consistent with list.

Known limitations / deferred (not defects)

Deliberately out of scope for Phase 1 (have their own future plans)

• Phase 2 — Send path: SMTP, MIME building, reply threading, outbound policy (RO/RW + whitelist-out), send.
• Phase 3 — OAuth2: Gmail loopback consent + token refresh (schema columns already present).
• Phase 4 — Admin TUI + doctor: interactive init/reconfigure, connectivity diagnostics.

Carry-over items to address (logged from reviews)

1. [Minor] Audit completeness: list/search don't write a per-filtered-message audit row; some IMAP-error envelope paths don't write an audit row; imap_error is an audit reason not enumerated in the spec.
2. [Minor] CLI polish: account add opens the store before parsing its own flags (--help touches the DB); invalid --since-date/--before-date are silently ignored rather than erroring; whitelist add/remove don't reject an empty --address at the CLI layer.
3. [Minor] Performance: ack fetches each UID's header in its own round-trip; get fetches the header then re-fetches the full message. Fine for a per-invocation CLI; batch ops in later phases may want to optimize.

Previously-open items now closed

• ✅ Live IMAP validation — done against a real mailbox over verified TLS (see "Live validation" above). The earlier GreenMail attempt had failed only on local self-signed-TLS friction, not a code issue.
• ✅ Header-only fetch — list/search no longer download message bodies (see "Resolved within Phase 1" above).
• A dev-only --insecure/skip-verify option is still worth adding later for local self-signed test servers, but is not needed for real providers (valid certs verified fine).

Suggested next steps

1. Proceed to Phase 2 (send path) — the structure is ready: policy is set up to host outbound checks, Account.Mode is loaded for RO rejection, and MatchAddress already implements the domain matching whitelist-out needs. --reply-to can reuse the new header-only fetch path to read Message-ID/References cheaply.
2. Optionally clean up the Minor carry-over items above as part of Phase 2 touch-ups.

Repository layout

emcli/
├── Makefile                 # build/test/vet (CGO_ENABLED=0)
├── go.mod / go.sum
├── cmd/emcli/main.go
├── internal/
│   ├── version/
│   ├── crypto/
│   ├── store/               # store, schema, settings, account, whitelist, seenset, audit
│   ├── policy/              # policy (address), inbound
│   ├── mail/                # message, imap, testdata/*.eml
│   └── cli/                 # envelope, agent, dispatch, run, admin
└── specifications/
    ├── PRD.md
    ├── SPEC.md
    ├── PHASE1-STATUS.md     # this file
    └── plans/2026-06-21-phase1-foundation-and-read-path.md