# emcli Phase 1 — Foundation & Read Path Implementation Plan > **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking. **Goal:** Build a working, testable read-only `emcli` — an encrypted-config CLI that lets an agent list, fetch, search, and acknowledge emails over IMAP, with all reads filtered by per-account policy and credentials never exposed. **Architecture:** A single Go binary organised into focused `internal/` packages: `crypto` (AES-256-GCM field encryption), `store` (pure-Go SQLite config + seen-set read state + audit), `policy` (pure enforcement functions), `mail` (IMAP read + message parsing), and `cli` (JSON-envelope command dispatch). Reads are stateless; the only state-mutating command is `ack`. This phase uses password auth only; OAuth2, SMTP send, and the TUI arrive in later phases. **Tech Stack:** Go (no CGO), `modernc.org/sqlite`, `github.com/emersion/go-imap` + `github.com/emersion/go-message`, Go stdlib `crypto/aes`+`crypto/cipher`. Tests use Go's `testing` package; IMAP integration tests run against a `greenmail` container. ## Global Constraints - Language: Go, **CGO disabled** (`CGO_ENABLED=0`) — the binary must be a single static cross-platform executable. - SQLite driver: `modernc.org/sqlite` (pure Go), registered driver name `"sqlite"`. Never `mattn/go-sqlite3`. - Encryption key source: env var `EMCLI_KEY`, a base64-standard-encoded 32-byte key (AES-256). Missing/malformed ⇒ fail closed, never a plaintext fallback. - DB path source: env var `EMCLI_DB`; default `~/.config/emcli/emcli.db` (`%AppData%\emcli\emcli.db` on Windows). - Secret columns (`enc_password`, `enc_oauth_*`) are stored as `crypto.Seal` output. Secrets never appear in stdout, error details, or the audit log. - Every agent command prints exactly one JSON object: `{"error":bool,"error_detail":obj,"data":obj}` and nothing else on stdout. - "New" = message exists in folder AND `uid > floor_uid` AND `uid ∉ acked`. Reads never mutate floor/ack state; only `ack` does. - Module path: `git.dcglab.co.uk/steve/emcli` (used in all import paths below). --- ### Task 1: Project scaffold & build **Files:** - Create: `go.mod` - Create: `cmd/emcli/main.go` - Create: `internal/version/version.go` - Create: `Makefile` - Test: `internal/version/version_test.go` **Interfaces:** - Consumes: nothing. - Produces: `version.String` (string constant); binary entrypoint `cmd/emcli`. - [ ] **Step 1: Initialise the module** Run: ```bash cd /home/steve/src/emcli go mod init git.dcglab.co.uk/steve/emcli go mod edit -go=1.22 ``` - [ ] **Step 2: Write the failing test** Create `internal/version/version_test.go`: ```go package version import "testing" func TestStringIsSemver(t *testing.T) { if String == "" { t.Fatal("version.String must not be empty") } if String[0] != 'v' { t.Fatalf("version %q must start with 'v'", String) } } ``` - [ ] **Step 3: Run test to verify it fails** Run: `go test ./internal/version/` Expected: FAIL — `undefined: String`. - [ ] **Step 4: Write minimal implementation** Create `internal/version/version.go`: ```go // Package version holds the build version string. package version // String is the emcli version. Overridden at release via -ldflags. var String = "v0.1.0-dev" ``` - [ ] **Step 5: Write the entrypoint** Create `cmd/emcli/main.go`: ```go package main import ( "fmt" "os" "git.dcglab.co.uk/steve/emcli/internal/version" ) func main() { if len(os.Args) >= 2 && os.Args[1] == "version" { fmt.Println(version.String) return } fmt.Fprintln(os.Stderr, "emcli: no command given") os.Exit(2) } ``` - [ ] **Step 6: Write the Makefile** Create `Makefile`: ```make BINARY := emcli LDFLAGS := -s -w .PHONY: build test vet build: CGO_ENABLED=0 go build -ldflags "$(LDFLAGS)" -o $(BINARY) ./cmd/emcli test: go test ./... vet: go vet ./... ``` - [ ] **Step 7: Run test and build to verify they pass** Run: `go test ./internal/version/ && make build && ./emcli version` Expected: test PASS; build succeeds; prints `v0.1.0-dev`. - [ ] **Step 8: Commit** ```bash git add go.mod cmd internal/version Makefile git commit -m "feat: project scaffold, version command, build" ``` --- ### Task 2: crypto — key loading & AES-256-GCM field encryption **Files:** - Create: `internal/crypto/crypto.go` - Test: `internal/crypto/crypto_test.go` **Interfaces:** - Consumes: nothing. - Produces: - `func KeyFromEnv() ([]byte, error)` — reads `EMCLI_KEY`, base64-decodes, validates 32 bytes. - `func Seal(key, plaintext []byte) ([]byte, error)` — returns `nonce||ciphertext` (AES-256-GCM). - `func Open(key, blob []byte) ([]byte, error)` — reverses Seal; auth failure ⇒ error. - `var ErrNoKey, ErrBadKey error`. - [ ] **Step 1: Write the failing tests** Create `internal/crypto/crypto_test.go`: ```go package crypto import ( "bytes" "encoding/base64" "testing" ) func testKey() []byte { k := make([]byte, 32) for i := range k { k[i] = byte(i) } return k } func TestSealOpenRoundTrip(t *testing.T) { key := testKey() msg := []byte("hunter2-the-password") blob, err := Seal(key, msg) if err != nil { t.Fatalf("Seal: %v", err) } if bytes.Contains(blob, msg) { t.Fatal("ciphertext must not contain plaintext") } got, err := Open(key, blob) if err != nil { t.Fatalf("Open: %v", err) } if !bytes.Equal(got, msg) { t.Fatalf("round-trip mismatch: %q", got) } } func TestSealUsesRandomNonce(t *testing.T) { key := testKey() a, _ := Seal(key, []byte("x")) b, _ := Seal(key, []byte("x")) if bytes.Equal(a, b) { t.Fatal("two seals of same plaintext must differ (random nonce)") } } func TestOpenWrongKeyFails(t *testing.T) { blob, _ := Seal(testKey(), []byte("secret")) wrong := make([]byte, 32) // all zeros if _, err := Open(wrong, blob); err == nil { t.Fatal("Open with wrong key must fail") } } func TestKeyFromEnv(t *testing.T) { t.Setenv("EMCLI_KEY", base64.StdEncoding.EncodeToString(testKey())) k, err := KeyFromEnv() if err != nil || len(k) != 32 { t.Fatalf("KeyFromEnv: key=%d err=%v", len(k), err) } t.Setenv("EMCLI_KEY", "") if _, err := KeyFromEnv(); err != ErrNoKey { t.Fatalf("empty key: want ErrNoKey, got %v", err) } t.Setenv("EMCLI_KEY", base64.StdEncoding.EncodeToString([]byte("tooshort"))) if _, err := KeyFromEnv(); err != ErrBadKey { t.Fatalf("short key: want ErrBadKey, got %v", err) } } ``` - [ ] **Step 2: Run tests to verify they fail** Run: `go test ./internal/crypto/` Expected: FAIL — undefined `Seal`, `Open`, `KeyFromEnv`. - [ ] **Step 3: Write the implementation** Create `internal/crypto/crypto.go`: ```go // Package crypto provides AES-256-GCM field encryption keyed from EMCLI_KEY. package crypto import ( "crypto/aes" "crypto/cipher" "crypto/rand" "encoding/base64" "errors" "io" "os" ) var ( ErrNoKey = errors.New("EMCLI_KEY is not set") ErrBadKey = errors.New("EMCLI_KEY must be base64 of exactly 32 bytes") ) // KeyFromEnv reads and validates the AES-256 key from EMCLI_KEY. func KeyFromEnv() ([]byte, error) { raw := os.Getenv("EMCLI_KEY") if raw == "" { return nil, ErrNoKey } key, err := base64.StdEncoding.DecodeString(raw) if err != nil || len(key) != 32 { return nil, ErrBadKey } return key, nil } func newGCM(key []byte) (cipher.AEAD, error) { block, err := aes.NewCipher(key) if err != nil { return nil, err } return cipher.NewGCM(block) } // Seal encrypts plaintext, returning nonce||ciphertext. func Seal(key, plaintext []byte) ([]byte, error) { gcm, err := newGCM(key) if err != nil { return nil, err } nonce := make([]byte, gcm.NonceSize()) if _, err := io.ReadFull(rand.Reader, nonce); err != nil { return nil, err } return gcm.Seal(nonce, nonce, plaintext, nil), nil } // Open reverses Seal. A wrong key or tampered blob returns an error. func Open(key, blob []byte) ([]byte, error) { gcm, err := newGCM(key) if err != nil { return nil, err } ns := gcm.NonceSize() if len(blob) < ns { return nil, errors.New("ciphertext too short") } nonce, ct := blob[:ns], blob[ns:] return gcm.Open(nil, nonce, ct, nil) } ``` - [ ] **Step 4: Run tests to verify they pass** Run: `go test ./internal/crypto/` Expected: PASS (4 tests). - [ ] **Step 5: Commit** ```bash git add internal/crypto git commit -m "feat(crypto): AES-256-GCM field encryption keyed from EMCLI_KEY" ``` --- ### Task 3: store — open DB, schema, migrations, settings **Files:** - Create: `internal/store/store.go` - Create: `internal/store/schema.go` - Create: `internal/store/settings.go` - Test: `internal/store/store_test.go` **Interfaces:** - Consumes: `crypto` (held for later tasks). - Produces: - `type Store struct { ... }` wrapping `*sql.DB` and the key. - `func Open(path string, key []byte) (*Store, error)` — opens/creates DB, runs migrations. - `func (s *Store) Close() error`. - `func (s *Store) GetSetting(name string) (string, error)` / `SetSetting(name, value string) error`. - `func DefaultDBPath() (string, error)` — resolves `EMCLI_DB` or the per-OS default. - Const `schemaVersion = 1`. - [ ] **Step 1: Add the SQLite dependency** Run: `go get modernc.org/sqlite@latest` - [ ] **Step 2: Write the failing test** Create `internal/store/store_test.go`: ```go package store import ( "path/filepath" "testing" ) func testKey() []byte { k := make([]byte, 32) for i := range k { k[i] = byte(i) } return k } // openTemp opens a fresh store in a temp dir. func openTemp(t *testing.T) *Store { t.Helper() p := filepath.Join(t.TempDir(), "emcli.db") s, err := Open(p, testKey()) if err != nil { t.Fatalf("Open: %v", err) } t.Cleanup(func() { s.Close() }) return s } func TestOpenCreatesSchemaAndIsIdempotent(t *testing.T) { p := filepath.Join(t.TempDir(), "emcli.db") s, err := Open(p, testKey()) if err != nil { t.Fatalf("first Open: %v", err) } v, err := s.GetSetting("schema_version") if err != nil || v != "1" { t.Fatalf("schema_version: %q err=%v", v, err) } s.Close() // Re-open: must not error or duplicate. s2, err := Open(p, testKey()) if err != nil { t.Fatalf("second Open: %v", err) } defer s2.Close() if v, _ := s2.GetSetting("schema_version"); v != "1" { t.Fatalf("schema_version after reopen: %q", v) } } func TestSettingsRoundTrip(t *testing.T) { s := openTemp(t) if err := s.SetSetting("audit_retention_days", "30"); err != nil { t.Fatalf("SetSetting: %v", err) } got, err := s.GetSetting("audit_retention_days") if err != nil || got != "30" { t.Fatalf("got %q err=%v", got, err) } // Upsert overwrites. _ = s.SetSetting("audit_retention_days", "7") if got, _ := s.GetSetting("audit_retention_days"); got != "7" { t.Fatalf("upsert failed: %q", got) } } ``` - [ ] **Step 3: Run test to verify it fails** Run: `go test ./internal/store/` Expected: FAIL — undefined `Open`. - [ ] **Step 4: Write the schema** Create `internal/store/schema.go`: ```go package store const schemaVersion = 1 // schemaSQL is the full v1 schema. All statements are idempotent via IF NOT EXISTS. const schemaSQL = ` CREATE TABLE IF NOT EXISTS settings ( key TEXT PRIMARY KEY, value TEXT NOT NULL ); CREATE TABLE IF NOT EXISTS accounts ( id INTEGER PRIMARY KEY AUTOINCREMENT, name TEXT UNIQUE NOT NULL, mode TEXT NOT NULL CHECK (mode IN ('RO','RW')), imap_host TEXT NOT NULL, imap_port INTEGER NOT NULL, imap_security TEXT NOT NULL CHECK (imap_security IN ('tls','starttls')), smtp_host TEXT, smtp_port INTEGER, smtp_security TEXT, auth_type TEXT NOT NULL CHECK (auth_type IN ('password','oauth2')), username TEXT NOT NULL, enc_password BLOB, enc_oauth_client_id BLOB, enc_oauth_client_secret BLOB, enc_oauth_refresh_token BLOB, whitelist_in_enabled INTEGER NOT NULL DEFAULT 0, whitelist_out_enabled INTEGER NOT NULL DEFAULT 0, subject_regex TEXT, process_backlog INTEGER NOT NULL DEFAULT 0 ); CREATE TABLE IF NOT EXISTS whitelist_in ( account_id INTEGER NOT NULL REFERENCES accounts(id) ON DELETE CASCADE, address TEXT NOT NULL, PRIMARY KEY (account_id, address) ); CREATE TABLE IF NOT EXISTS whitelist_out ( account_id INTEGER NOT NULL REFERENCES accounts(id) ON DELETE CASCADE, address TEXT NOT NULL, PRIMARY KEY (account_id, address) ); CREATE TABLE IF NOT EXISTS folder_state ( account_id INTEGER NOT NULL REFERENCES accounts(id) ON DELETE CASCADE, folder TEXT NOT NULL, uidvalidity INTEGER NOT NULL, floor_uid INTEGER NOT NULL, PRIMARY KEY (account_id, folder) ); CREATE TABLE IF NOT EXISTS acked ( account_id INTEGER NOT NULL REFERENCES accounts(id) ON DELETE CASCADE, folder TEXT NOT NULL, uidvalidity INTEGER NOT NULL, uid INTEGER NOT NULL, PRIMARY KEY (account_id, folder, uid) ); CREATE TABLE IF NOT EXISTS audit_log ( id INTEGER PRIMARY KEY AUTOINCREMENT, ts TEXT NOT NULL, account TEXT NOT NULL, action TEXT NOT NULL, target TEXT NOT NULL, result TEXT NOT NULL, reason TEXT ); ` ``` - [ ] **Step 5: Write the store core and settings** Create `internal/store/store.go`: ```go // Package store owns the encrypted SQLite config and read state. package store import ( "database/sql" "fmt" "os" "path/filepath" "runtime" "strconv" _ "modernc.org/sqlite" ) // Store wraps the database and the field-encryption key. type Store struct { db *sql.DB key []byte } // Open opens (creating if needed) the DB at path and applies the schema. func Open(path string, key []byte) (*Store, error) { if err := os.MkdirAll(filepath.Dir(path), 0o700); err != nil { return nil, fmt.Errorf("create db dir: %w", err) } db, err := sql.Open("sqlite", path) if err != nil { return nil, err } if _, err := db.Exec("PRAGMA foreign_keys = ON;"); err != nil { db.Close() return nil, err } if _, err := db.Exec(schemaSQL); err != nil { db.Close() return nil, fmt.Errorf("apply schema: %w", err) } s := &Store{db: db, key: key} if _, err := s.GetSetting("schema_version"); err != nil { if err := s.SetSetting("schema_version", strconv.Itoa(schemaVersion)); err != nil { db.Close() return nil, err } } return s, nil } func (s *Store) Close() error { return s.db.Close() } // DefaultDBPath resolves EMCLI_DB or the per-OS default location. func DefaultDBPath() (string, error) { if p := os.Getenv("EMCLI_DB"); p != "" { return p, nil } if runtime.GOOS == "windows" { if dir := os.Getenv("AppData"); dir != "" { return filepath.Join(dir, "emcli", "emcli.db"), nil } } home, err := os.UserHomeDir() if err != nil { return "", err } return filepath.Join(home, ".config", "emcli", "emcli.db"), nil } ``` Create `internal/store/settings.go`: ```go package store import "database/sql" // GetSetting returns a setting value or sql.ErrNoRows if absent. func (s *Store) GetSetting(name string) (string, error) { var v string err := s.db.QueryRow("SELECT value FROM settings WHERE key = ?", name).Scan(&v) return v, err } // SetSetting upserts a setting. func (s *Store) SetSetting(name, value string) error { _, err := s.db.Exec( "INSERT INTO settings(key,value) VALUES(?,?) "+ "ON CONFLICT(key) DO UPDATE SET value=excluded.value", name, value) return err } var _ = sql.ErrNoRows ``` - [ ] **Step 6: Run tests to verify they pass** Run: `go test ./internal/store/` Expected: PASS (2 tests). - [ ] **Step 7: Commit** ```bash git add internal/store go.mod go.sum git commit -m "feat(store): open encrypted SQLite, schema v1, settings" ``` --- ### Task 4: store — accounts CRUD with encrypted secrets **Files:** - Create: `internal/store/account.go` - Test: `internal/store/account_test.go` **Interfaces:** - Consumes: `crypto.Seal`/`crypto.Open`, `Store`. - Produces: - `type Account struct { ID int64; Name, Mode, IMAPHost string; IMAPPort int; IMAPSecurity, AuthType, Username string; Password string; WhitelistInEnabled, WhitelistOutEnabled bool; SubjectRegex string; ProcessBacklog bool }` - `func (s *Store) AddAccount(a Account) (int64, error)` — encrypts `Password` into `enc_password`. - `func (s *Store) GetAccount(name string) (Account, error)` — decrypts secrets; `ErrAccountNotFound` if absent. - `func (s *Store) ListAccounts() ([]Account, error)` — `Password` field left empty (never listed). - `func (s *Store) DeleteAccount(name string) error`. - `var ErrAccountNotFound error`. - [ ] **Step 1: Write the failing test** Create `internal/store/account_test.go`: ```go package store import ( "errors" "testing" ) func sampleAccount() Account { return Account{ Name: "work", Mode: "RO", IMAPHost: "imap.example.com", IMAPPort: 993, IMAPSecurity: "tls", AuthType: "password", Username: "me@example.com", Password: "s3cr3t", SubjectRegex: "", } } func TestAddGetAccountDecryptsSecret(t *testing.T) { s := openTemp(t) id, err := s.AddAccount(sampleAccount()) if err != nil { t.Fatalf("AddAccount: %v", err) } if id == 0 { t.Fatal("want non-zero id") } got, err := s.GetAccount("work") if err != nil { t.Fatalf("GetAccount: %v", err) } if got.Password != "s3cr3t" { t.Fatalf("password not decrypted: %q", got.Password) } if got.Mode != "RO" || got.IMAPPort != 993 { t.Fatalf("fields wrong: %+v", got) } } func TestPasswordStoredEncrypted(t *testing.T) { s := openTemp(t) _, _ = s.AddAccount(sampleAccount()) var blob []byte if err := s.db.QueryRow("SELECT enc_password FROM accounts WHERE name='work'").Scan(&blob); err != nil { t.Fatalf("query: %v", err) } if string(blob) == "s3cr3t" || len(blob) == 0 { t.Fatalf("password not encrypted at rest: %q", blob) } } func TestGetAccountNotFound(t *testing.T) { s := openTemp(t) if _, err := s.GetAccount("nope"); !errors.Is(err, ErrAccountNotFound) { t.Fatalf("want ErrAccountNotFound, got %v", err) } } func TestListAccountsOmitsSecrets(t *testing.T) { s := openTemp(t) _, _ = s.AddAccount(sampleAccount()) list, err := s.ListAccounts() if err != nil || len(list) != 1 { t.Fatalf("list: %v len=%d", err, len(list)) } if list[0].Password != "" { t.Fatal("ListAccounts must not return secrets") } } ``` - [ ] **Step 2: Run test to verify it fails** Run: `go test ./internal/store/ -run Account` Expected: FAIL — undefined `Account`/`AddAccount`. - [ ] **Step 3: Write the implementation** Create `internal/store/account.go`: ```go package store import ( "database/sql" "errors" "fmt" "git.dcglab.co.uk/steve/emcli/internal/crypto" ) var ErrAccountNotFound = errors.New("account not found") // Account is the decrypted, in-memory view of a configured account. type Account struct { ID int64 Name string Mode string // RO | RW IMAPHost string IMAPPort int IMAPSecurity string // tls | starttls AuthType string // password | oauth2 Username string Password string // decrypted; empty in ListAccounts WhitelistInEnabled bool WhitelistOutEnabled bool SubjectRegex string ProcessBacklog bool } func (s *Store) AddAccount(a Account) (int64, error) { var encPw []byte if a.Password != "" { b, err := crypto.Seal(s.key, []byte(a.Password)) if err != nil { return 0, err } encPw = b } res, err := s.db.Exec(` INSERT INTO accounts (name,mode,imap_host,imap_port,imap_security,auth_type,username, enc_password,whitelist_in_enabled,whitelist_out_enabled,subject_regex,process_backlog) VALUES (?,?,?,?,?,?,?,?,?,?,?,?)`, a.Name, a.Mode, a.IMAPHost, a.IMAPPort, a.IMAPSecurity, a.AuthType, a.Username, encPw, b2i(a.WhitelistInEnabled), b2i(a.WhitelistOutEnabled), nullStr(a.SubjectRegex), b2i(a.ProcessBacklog)) if err != nil { return 0, fmt.Errorf("insert account: %w", err) } return res.LastInsertId() } func (s *Store) GetAccount(name string) (Account, error) { row := s.db.QueryRow(` SELECT id,name,mode,imap_host,imap_port,imap_security,auth_type,username, enc_password,whitelist_in_enabled,whitelist_out_enabled,subject_regex,process_backlog FROM accounts WHERE name = ?`, name) a, encPw, err := scanAccount(row) if errors.Is(err, sql.ErrNoRows) { return Account{}, ErrAccountNotFound } if err != nil { return Account{}, err } if len(encPw) > 0 { pw, err := crypto.Open(s.key, encPw) if err != nil { return Account{}, fmt.Errorf("decrypt password (wrong EMCLI_KEY?): %w", err) } a.Password = string(pw) } return a, nil } func (s *Store) ListAccounts() ([]Account, error) { rows, err := s.db.Query(` SELECT id,name,mode,imap_host,imap_port,imap_security,auth_type,username, enc_password,whitelist_in_enabled,whitelist_out_enabled,subject_regex,process_backlog FROM accounts ORDER BY name`) if err != nil { return nil, err } defer rows.Close() var out []Account for rows.Next() { a, _, err := scanAccount(rows) // secrets discarded if err != nil { return nil, err } out = append(out, a) } return out, rows.Err() } func (s *Store) DeleteAccount(name string) error { res, err := s.db.Exec("DELETE FROM accounts WHERE name = ?", name) if err != nil { return err } if n, _ := res.RowsAffected(); n == 0 { return ErrAccountNotFound } return nil } // scanner is satisfied by *sql.Row and *sql.Rows. type scanner interface{ Scan(dest ...any) error } func scanAccount(sc scanner) (Account, []byte, error) { var ( a Account encPw []byte subj sql.NullString wlIn, wlOut int backlog int ) err := sc.Scan(&a.ID, &a.Name, &a.Mode, &a.IMAPHost, &a.IMAPPort, &a.IMAPSecurity, &a.AuthType, &a.Username, &encPw, &wlIn, &wlOut, &subj, &backlog) if err != nil { return Account{}, nil, err } a.WhitelistInEnabled = wlIn != 0 a.WhitelistOutEnabled = wlOut != 0 a.ProcessBacklog = backlog != 0 a.SubjectRegex = subj.String return a, encPw, nil } func b2i(b bool) int { if b { return 1 } return 0 } func nullStr(s string) any { if s == "" { return nil } return s } ``` - [ ] **Step 4: Run tests to verify they pass** Run: `go test ./internal/store/ -run Account` Expected: PASS (4 tests). - [ ] **Step 5: Commit** ```bash git add internal/store/account.go internal/store/account_test.go git commit -m "feat(store): accounts CRUD with encrypted password column" ``` --- ### Task 5: store — whitelist CRUD **Files:** - Create: `internal/store/whitelist.go` - Test: `internal/store/whitelist_test.go` **Interfaces:** - Consumes: `Store`, `GetAccount`. - Produces: - `type Direction string` with `DirIn Direction = "in"`, `DirOut Direction = "out"`. - `func (s *Store) AddWhitelist(account string, dir Direction, address string) error` - `func (s *Store) RemoveWhitelist(account string, dir Direction, address string) error` - `func (s *Store) ListWhitelist(account string, dir Direction) ([]string, error)` — addresses lower-cased. - [ ] **Step 1: Write the failing test** Create `internal/store/whitelist_test.go`: ```go package store import ( "reflect" "testing" ) func TestWhitelistAddListRemove(t *testing.T) { s := openTemp(t) _, _ = s.AddAccount(sampleAccount()) if err := s.AddWhitelist("work", DirIn, "Bob@Example.com"); err != nil { t.Fatalf("add: %v", err) } _ = s.AddWhitelist("work", DirIn, "@trusted.com") got, err := s.ListWhitelist("work", DirIn) if err != nil { t.Fatalf("list: %v", err) } want := []string{"@trusted.com", "bob@example.com"} // lower-cased, sorted if !reflect.DeepEqual(got, want) { t.Fatalf("got %v want %v", got, want) } // Out list is independent. if out, _ := s.ListWhitelist("work", DirOut); len(out) != 0 { t.Fatalf("out list should be empty, got %v", out) } if err := s.RemoveWhitelist("work", DirIn, "bob@example.com"); err != nil { t.Fatalf("remove: %v", err) } got, _ = s.ListWhitelist("work", DirIn) if !reflect.DeepEqual(got, []string{"@trusted.com"}) { t.Fatalf("after remove got %v", got) } } ``` - [ ] **Step 2: Run test to verify it fails** Run: `go test ./internal/store/ -run Whitelist` Expected: FAIL — undefined `DirIn`. - [ ] **Step 3: Write the implementation** Create `internal/store/whitelist.go`: ```go package store import ( "fmt" "strings" ) type Direction string const ( DirIn Direction = "in" DirOut Direction = "out" ) func (d Direction) table() (string, error) { switch d { case DirIn: return "whitelist_in", nil case DirOut: return "whitelist_out", nil default: return "", fmt.Errorf("invalid direction %q", d) } } func (s *Store) accountID(name string) (int64, error) { a, err := s.GetAccount(name) if err != nil { return 0, err } return a.ID, nil } func (s *Store) AddWhitelist(account string, dir Direction, address string) error { tbl, err := dir.table() if err != nil { return err } id, err := s.accountID(account) if err != nil { return err } _, err = s.db.Exec( fmt.Sprintf("INSERT OR IGNORE INTO %s(account_id,address) VALUES(?,?)", tbl), id, strings.ToLower(address)) return err } func (s *Store) RemoveWhitelist(account string, dir Direction, address string) error { tbl, err := dir.table() if err != nil { return err } id, err := s.accountID(account) if err != nil { return err } _, err = s.db.Exec( fmt.Sprintf("DELETE FROM %s WHERE account_id=? AND address=?", tbl), id, strings.ToLower(address)) return err } func (s *Store) ListWhitelist(account string, dir Direction) ([]string, error) { tbl, err := dir.table() if err != nil { return nil, err } id, err := s.accountID(account) if err != nil { return nil, err } rows, err := s.db.Query( fmt.Sprintf("SELECT address FROM %s WHERE account_id=? ORDER BY address", tbl), id) if err != nil { return nil, err } defer rows.Close() out := []string{} for rows.Next() { var a string if err := rows.Scan(&a); err != nil { return nil, err } out = append(out, a) } return out, rows.Err() } ``` - [ ] **Step 4: Run tests to verify they pass** Run: `go test ./internal/store/ -run Whitelist` Expected: PASS. - [ ] **Step 5: Commit** ```bash git add internal/store/whitelist.go internal/store/whitelist_test.go git commit -m "feat(store): per-account inbound/outbound whitelist CRUD" ``` --- ### Task 6: store — seen-set read state (floor, ack, compaction) This is the algorithmic core. A folder's read state is a `floor_uid` plus a set of acked UIDs above it. Compaction collapses contiguous acked runs immediately above the floor. **Files:** - Create: `internal/store/seenset.go` - Test: `internal/store/seenset_test.go` **Interfaces:** - Consumes: `Store`, `accountID`. - Produces: - `func (s *Store) EnsureFolderBaseline(account, folder string, uidvalidity, maxUID uint32) error` — on first contact creates `folder_state` with `floor_uid = maxUID` (or 0 if account.ProcessBacklog); on `uidvalidity` change, resets state and re-baselines. - `func (s *Store) IsNew(account, folder string, uid uint32) (bool, error)` — `uid > floor AND uid ∉ acked`. - `func (s *Store) FilterNew(account, folder string, uids []uint32) ([]uint32, error)` — convenience, preserves order. - `func (s *Store) Ack(account, folder string, uidvalidity uint32, uids ...uint32) error` — inserts into `acked` (ignoring ≤ floor), then compacts. - `func (s *Store) floor(account, folder string) (uint32, error)` — unexported helper used by tests via exported wrappers. - [ ] **Step 1: Write the failing tests** Create `internal/store/seenset_test.go`: ```go package store import "testing" func backlogAccount(t *testing.T, s *Store, backlog bool) { t.Helper() a := sampleAccount() a.ProcessBacklog = backlog if _, err := s.AddAccount(a); err != nil { t.Fatalf("AddAccount: %v", err) } } func TestBaselineIgnoresHistoryByDefault(t *testing.T) { s := openTemp(t) backlogAccount(t, s, false) if err := s.EnsureFolderBaseline("work", "INBOX", 1, 100); err != nil { t.Fatalf("baseline: %v", err) } // Existing mail (uid <= 100) is not new; 101 is. for _, tc := range []struct { uid uint32 want bool }{{50, false}, {100, false}, {101, true}} { got, _ := s.IsNew("work", "INBOX", tc.uid) if got != tc.want { t.Fatalf("IsNew(%d)=%v want %v", tc.uid, got, tc.want) } } } func TestBaselineBacklogProcessesAll(t *testing.T) { s := openTemp(t) backlogAccount(t, s, true) _ = s.EnsureFolderBaseline("work", "INBOX", 1, 100) if n, _ := s.IsNew("work", "INBOX", 1); !n { t.Fatal("with backlog, uid 1 must be new") } } func TestAckRemovesFromNewAndIsOutOfOrderSafe(t *testing.T) { s := openTemp(t) backlogAccount(t, s, false) _ = s.EnsureFolderBaseline("work", "INBOX", 1, 100) // New mail arrives: 101..105. Ack out of order: 103 then 101. _ = s.Ack("work", "INBOX", 1, 103) if n, _ := s.IsNew("work", "INBOX", 103); n { t.Fatal("103 acked, must not be new") } if n, _ := s.IsNew("work", "INBOX", 101); !n { t.Fatal("101 not acked yet, must still be new") } _ = s.Ack("work", "INBOX", 1, 101, 102) got, _ := s.FilterNew("work", "INBOX", []uint32{101, 102, 103, 104, 105}) want := []uint32{104, 105} if len(got) != 2 || got[0] != want[0] || got[1] != want[1] { t.Fatalf("FilterNew got %v want %v", got, want) } } func TestCompactionAdvancesFloor(t *testing.T) { s := openTemp(t) backlogAccount(t, s, false) _ = s.EnsureFolderBaseline("work", "INBOX", 1, 100) // Ack a contiguous run just above the floor: 101,102,103. _ = s.Ack("work", "INBOX", 1, 102, 101, 103) f, _ := s.floor("work", "INBOX") if f != 103 { t.Fatalf("floor should advance to 103, got %d", f) } // The acked rows for the collapsed run are gone. var n int _ = s.db.QueryRow("SELECT COUNT(*) FROM acked").Scan(&n) if n != 0 { t.Fatalf("acked rows should be compacted away, got %d", n) } // A hole remains uncompacted: ack 105 (gap at 104). _ = s.Ack("work", "INBOX", 1, 105) f, _ = s.floor("work", "INBOX") if f != 103 { t.Fatalf("floor must stay at 103 while 104 is a hole, got %d", f) } _ = s.db.QueryRow("SELECT COUNT(*) FROM acked").Scan(&n) if n != 1 { t.Fatalf("105 should remain in acked, got %d rows", n) } } func TestUIDValidityChangeResets(t *testing.T) { s := openTemp(t) backlogAccount(t, s, false) _ = s.EnsureFolderBaseline("work", "INBOX", 1, 100) _ = s.Ack("work", "INBOX", 1, 105) // Server reports a new UIDVALIDITY: state resets, re-baselines at new max. if err := s.EnsureFolderBaseline("work", "INBOX", 2, 10); err != nil { t.Fatalf("rebaseline: %v", err) } f, _ := s.floor("work", "INBOX") if f != 10 { t.Fatalf("floor should re-baseline to 10, got %d", f) } var n int _ = s.db.QueryRow("SELECT COUNT(*) FROM acked").Scan(&n) if n != 0 { t.Fatalf("acked must reset on uidvalidity change, got %d", n) } } ``` - [ ] **Step 2: Run tests to verify they fail** Run: `go test ./internal/store/ -run 'Baseline|Ack|Compaction|UIDValidity'` Expected: FAIL — undefined `EnsureFolderBaseline`. - [ ] **Step 3: Write the implementation** Create `internal/store/seenset.go`: ```go package store import "database/sql" // EnsureFolderBaseline initialises folder_state on first contact, or resets it // when the server's UIDVALIDITY differs from what we stored. func (s *Store) EnsureFolderBaseline(account, folder string, uidvalidity, maxUID uint32) error { a, err := s.GetAccount(account) if err != nil { return err } var ( curUIDValidity uint32 haveRow bool ) row := s.db.QueryRow( "SELECT uidvalidity FROM folder_state WHERE account_id=? AND folder=?", a.ID, folder) switch err := row.Scan(&curUIDValidity); err { case nil: haveRow = true case sql.ErrNoRows: haveRow = false default: return err } if haveRow && curUIDValidity == uidvalidity { return nil // already baselined, same validity } floor := maxUID if a.ProcessBacklog { floor = 0 } tx, err := s.db.Begin() if err != nil { return err } defer tx.Rollback() if _, err := tx.Exec("DELETE FROM acked WHERE account_id=? AND folder=?", a.ID, folder); err != nil { return err } if _, err := tx.Exec(` INSERT INTO folder_state(account_id,folder,uidvalidity,floor_uid) VALUES(?,?,?,?) ON CONFLICT(account_id,folder) DO UPDATE SET uidvalidity=excluded.uidvalidity, floor_uid=excluded.floor_uid`, a.ID, folder, uidvalidity, floor); err != nil { return err } return tx.Commit() } func (s *Store) floor(account, folder string) (uint32, error) { id, err := s.accountID(account) if err != nil { return 0, err } var f uint32 err = s.db.QueryRow( "SELECT floor_uid FROM folder_state WHERE account_id=? AND folder=?", id, folder).Scan(&f) return f, err } // IsNew reports whether uid is unread: above the floor and not acked. func (s *Store) IsNew(account, folder string, uid uint32) (bool, error) { id, err := s.accountID(account) if err != nil { return false, err } var floor uint32 if err := s.db.QueryRow( "SELECT floor_uid FROM folder_state WHERE account_id=? AND folder=?", id, folder).Scan(&floor); err != nil { return false, err } if uid <= floor { return false, nil } var one int err = s.db.QueryRow( "SELECT 1 FROM acked WHERE account_id=? AND folder=? AND uid=?", id, folder, uid).Scan(&one) if err == sql.ErrNoRows { return true, nil } if err != nil { return false, err } return false, nil // present in acked } // FilterNew returns the subset of uids that are new, preserving order. func (s *Store) FilterNew(account, folder string, uids []uint32) ([]uint32, error) { out := make([]uint32, 0, len(uids)) for _, u := range uids { n, err := s.IsNew(account, folder, u) if err != nil { return nil, err } if n { out = append(out, u) } } return out, nil } // Ack records uids as processed (ignoring any at or below the floor) then compacts. func (s *Store) Ack(account, folder string, uidvalidity uint32, uids ...uint32) error { id, err := s.accountID(account) if err != nil { return err } tx, err := s.db.Begin() if err != nil { return err } defer tx.Rollback() var floor uint32 if err := tx.QueryRow( "SELECT floor_uid FROM folder_state WHERE account_id=? AND folder=?", id, folder).Scan(&floor); err != nil { return err } for _, u := range uids { if u <= floor { continue } if _, err := tx.Exec( "INSERT OR IGNORE INTO acked(account_id,folder,uidvalidity,uid) VALUES(?,?,?,?)", id, folder, uidvalidity, u); err != nil { return err } } // Compact: while floor+1 is acked, advance floor and drop that row. for { next := floor + 1 var present int err := tx.QueryRow( "SELECT 1 FROM acked WHERE account_id=? AND folder=? AND uid=?", id, folder, next).Scan(&present) if err == sql.ErrNoRows { break } if err != nil { return err } if _, err := tx.Exec( "DELETE FROM acked WHERE account_id=? AND folder=? AND uid=?", id, folder, next); err != nil { return err } floor = next } if _, err := tx.Exec( "UPDATE folder_state SET floor_uid=? WHERE account_id=? AND folder=?", floor, id, folder); err != nil { return err } return tx.Commit() } ``` - [ ] **Step 4: Run tests to verify they pass** Run: `go test ./internal/store/ -run 'Baseline|Ack|Compaction|UIDValidity'` Expected: PASS (5 tests). - [ ] **Step 5: Commit** ```bash git add internal/store/seenset.go internal/store/seenset_test.go git commit -m "feat(store): seen-set read state with floor baseline and compaction" ``` --- ### Task 7: store — audit log & retention purge **Files:** - Create: `internal/store/audit.go` - Test: `internal/store/audit_test.go` **Interfaces:** - Consumes: `Store`, `SetSetting`/`GetSetting`. - Produces: - `type AuditEntry struct { TS, Account, Action, Target, Result, Reason string }` - `func (s *Store) Audit(now time.Time, e AuditEntry) error` — inserts; `now` injected for testability (no `time.Now()` inside). - `func (s *Store) PurgeAudit(now time.Time) (int64, error)` — deletes rows older than `audit_retention_days` (no-op if unset/0). - `func (s *Store) RecentAudit(limit int) ([]AuditEntry, error)`. - [ ] **Step 1: Write the failing test** Create `internal/store/audit_test.go`: ```go package store import ( "testing" "time" ) func TestAuditInsertAndRecent(t *testing.T) { s := openTemp(t) now := time.Date(2026, 6, 21, 12, 0, 0, 0, time.UTC) err := s.Audit(now, AuditEntry{ Account: "work", Action: "list", Target: "INBOX", Result: "allowed", }) if err != nil { t.Fatalf("Audit: %v", err) } got, err := s.RecentAudit(10) if err != nil || len(got) != 1 { t.Fatalf("recent: %v len=%d", err, len(got)) } if got[0].Action != "list" || got[0].Result != "allowed" { t.Fatalf("entry wrong: %+v", got[0]) } } func TestPurgeRespectsRetention(t *testing.T) { s := openTemp(t) _ = s.SetSetting("audit_retention_days", "30") now := time.Date(2026, 6, 21, 12, 0, 0, 0, time.UTC) old := now.AddDate(0, 0, -31) recent := now.AddDate(0, 0, -5) _ = s.Audit(old, AuditEntry{Account: "a", Action: "list", Target: "INBOX", Result: "allowed"}) _ = s.Audit(recent, AuditEntry{Account: "a", Action: "list", Target: "INBOX", Result: "allowed"}) n, err := s.PurgeAudit(now) if err != nil { t.Fatalf("purge: %v", err) } if n != 1 { t.Fatalf("want 1 purged, got %d", n) } got, _ := s.RecentAudit(10) if len(got) != 1 { t.Fatalf("want 1 remaining, got %d", len(got)) } } ``` - [ ] **Step 2: Run test to verify it fails** Run: `go test ./internal/store/ -run 'Audit|Purge'` Expected: FAIL — undefined `AuditEntry`. - [ ] **Step 3: Write the implementation** Create `internal/store/audit.go`: ```go package store import ( "strconv" "time" ) type AuditEntry struct { TS string Account string Action string Target string Result string Reason string } func (s *Store) Audit(now time.Time, e AuditEntry) error { var reason any if e.Reason != "" { reason = e.Reason } _, err := s.db.Exec( "INSERT INTO audit_log(ts,account,action,target,result,reason) VALUES(?,?,?,?,?,?)", now.UTC().Format(time.RFC3339), e.Account, e.Action, e.Target, e.Result, reason) return err } func (s *Store) PurgeAudit(now time.Time) (int64, error) { v, err := s.GetSetting("audit_retention_days") if err != nil { // unset => no retention policy return 0, nil } days, err := strconv.Atoi(v) if err != nil || days <= 0 { return 0, nil } cutoff := now.UTC().AddDate(0, 0, -days).Format(time.RFC3339) res, err := s.db.Exec("DELETE FROM audit_log WHERE ts < ?", cutoff) if err != nil { return 0, err } return res.RowsAffected() } func (s *Store) RecentAudit(limit int) ([]AuditEntry, error) { rows, err := s.db.Query( "SELECT ts,account,action,target,result,COALESCE(reason,'') FROM audit_log ORDER BY id DESC LIMIT ?", limit) if err != nil { return nil, err } defer rows.Close() var out []AuditEntry for rows.Next() { var e AuditEntry if err := rows.Scan(&e.TS, &e.Account, &e.Action, &e.Target, &e.Result, &e.Reason); err != nil { return nil, err } out = append(out, e) } return out, rows.Err() } ``` - [ ] **Step 4: Run tests to verify they pass** Run: `go test ./internal/store/ -run 'Audit|Purge'` Expected: PASS (2 tests). - [ ] **Step 5: Commit** ```bash git add internal/store/audit.go internal/store/audit_test.go git commit -m "feat(store): audit log with retention-based purge" ``` --- ### Task 8: policy — address matching **Files:** - Create: `internal/policy/policy.go` - Test: `internal/policy/address_test.go` **Interfaces:** - Consumes: nothing (pure). - Produces: - `func MatchAddress(entries []string, addr string) bool` — case-insensitive; `@domain.com` entry matches any address at that domain; otherwise exact match. `addr` may be a display-form address; only the `local@domain` part is compared. - `func NormalizeAddr(addr string) string` — lower-cases and strips display name / angle brackets. - [ ] **Step 1: Write the failing test** Create `internal/policy/address_test.go`: ```go package policy import "testing" func TestMatchAddress(t *testing.T) { wl := []string{"bob@example.com", "@trusted.com"} cases := []struct { addr string want bool }{ {"bob@example.com", true}, {"BOB@Example.com", true}, {`"Bob" `, true}, {"alice@trusted.com", true}, {"alice@untrusted.com", false}, {"eve@example.com", false}, {"", false}, } for _, c := range cases { if got := MatchAddress(wl, c.addr); got != c.want { t.Fatalf("MatchAddress(%q)=%v want %v", c.addr, got, c.want) } } } func TestNormalizeAddr(t *testing.T) { if got := NormalizeAddr(`"Bob Smith" `); got != "bob@example.com" { t.Fatalf("got %q", got) } } ``` - [ ] **Step 2: Run test to verify it fails** Run: `go test ./internal/policy/` Expected: FAIL — undefined `MatchAddress`. - [ ] **Step 3: Write the implementation** Create `internal/policy/policy.go`: ```go // Package policy holds pure enforcement functions: address matching, // inbound filtering, and (in a later phase) outbound checks. package policy import ( "net/mail" "strings" ) // NormalizeAddr lower-cases an address and strips any display name/brackets. func NormalizeAddr(addr string) string { addr = strings.TrimSpace(addr) if a, err := mail.ParseAddress(addr); err == nil { return strings.ToLower(a.Address) } return strings.ToLower(strings.Trim(addr, "<> ")) } // MatchAddress reports whether addr matches any whitelist entry. // Entries beginning with '@' match a whole domain; others match exactly. func MatchAddress(entries []string, addr string) bool { norm := NormalizeAddr(addr) if norm == "" { return false } at := strings.LastIndex(norm, "@") domain := "" if at >= 0 { domain = norm[at:] // includes '@' } for _, e := range entries { e = strings.ToLower(strings.TrimSpace(e)) if strings.HasPrefix(e, "@") { if e == domain { return true } continue } if e == norm { return true } } return false } ``` - [ ] **Step 4: Run tests to verify they pass** Run: `go test ./internal/policy/` Expected: PASS (2 tests). - [ ] **Step 5: Commit** ```bash git add internal/policy git commit -m "feat(policy): case-insensitive address and domain matching" ``` --- ### Task 9: policy — inbound filter **Files:** - Create: `internal/policy/inbound.go` - Test: `internal/policy/inbound_test.go` **Interfaces:** - Consumes: `MatchAddress`. - Produces: - `type InboundRule struct { WhitelistInEnabled bool; WhitelistIn []string; SubjectRegex *regexp.Regexp }` - `func (r InboundRule) Allows(from, subject string) bool` — true only if (whitelist disabled OR sender matches) AND (no regex OR subject matches). - `func CompileSubject(pattern string) (*regexp.Regexp, error)` — returns `(nil, nil)` for empty pattern. - [ ] **Step 1: Write the failing test** Create `internal/policy/inbound_test.go`: ```go package policy import "testing" func TestInboundAllows(t *testing.T) { re, err := CompileSubject("(?i)invoice") if err != nil { t.Fatalf("compile: %v", err) } rule := InboundRule{ WhitelistInEnabled: true, WhitelistIn: []string{"@trusted.com"}, SubjectRegex: re, } cases := []struct { from, subject string want bool }{ {"bob@trusted.com", "Your Invoice #5", true}, {"bob@trusted.com", "lunch?", false}, // subject fails {"eve@evil.com", "Invoice", false}, // sender fails {"eve@evil.com", "lunch?", false}, // both fail } for _, c := range cases { if got := rule.Allows(c.from, c.subject); got != c.want { t.Fatalf("Allows(%q,%q)=%v want %v", c.from, c.subject, got, c.want) } } } func TestInboundNoFiltersAllowsAll(t *testing.T) { rule := InboundRule{} // whitelist disabled, no regex if !rule.Allows("anyone@anywhere.com", "anything") { t.Fatal("empty rule must allow everything") } } func TestCompileSubjectEmpty(t *testing.T) { re, err := CompileSubject("") if err != nil || re != nil { t.Fatalf("empty pattern: re=%v err=%v", re, err) } } ``` - [ ] **Step 2: Run test to verify it fails** Run: `go test ./internal/policy/ -run Inbound` Expected: FAIL — undefined `InboundRule`. - [ ] **Step 3: Write the implementation** Create `internal/policy/inbound.go`: ```go package policy import "regexp" // InboundRule captures one account's read-side filtering. type InboundRule struct { WhitelistInEnabled bool WhitelistIn []string SubjectRegex *regexp.Regexp // nil = no subject filter } // CompileSubject compiles a subject filter; empty pattern => (nil, nil). func CompileSubject(pattern string) (*regexp.Regexp, error) { if pattern == "" { return nil, nil } return regexp.Compile(pattern) } // Allows reports whether a message with the given sender and subject is visible. func (r InboundRule) Allows(from, subject string) bool { if r.WhitelistInEnabled && !MatchAddress(r.WhitelistIn, from) { return false } if r.SubjectRegex != nil && !r.SubjectRegex.MatchString(subject) { return false } return true } ``` - [ ] **Step 4: Run tests to verify they pass** Run: `go test ./internal/policy/ -run Inbound` Expected: PASS. - [ ] **Step 5: Commit** ```bash git add internal/policy/inbound.go internal/policy/inbound_test.go git commit -m "feat(policy): inbound whitelist + subject-regex filter" ``` --- ### Task 10: mail — message parsing Parse a raw RFC822 message into headers, a plain-text body, and attachment metadata. This is pure (operates on `io.Reader`/bytes) so it is unit-testable without a server. **Files:** - Create: `internal/mail/message.go` - Test: `internal/mail/message_test.go` - Test fixture: `internal/mail/testdata/with_attachment.eml` **Interfaces:** - Consumes: `github.com/emersion/go-message/mail`. - Produces: - `type Header struct { UID uint32; From, To, Subject, Date, MessageID string; HasAttachments bool }` - `type Attachment struct { Name string; Size int; MIME string; Content []byte }` - `type Message struct { Header Header; BodyText string; Attachments []Attachment }` - `func ParseMessage(uid uint32, raw []byte) (Message, error)` — extracts the first `text/plain` part as `BodyText`; collects attachment parts (with `Content`); sets `HasAttachments`. - `func ParseHeaderOnly(uid uint32, raw []byte) (Header, error)` — header fields + `HasAttachments` without decoding attachment bodies. - [ ] **Step 1: Add the dependency and write the fixture** Run: `go get github.com/emersion/go-message@latest` Create `internal/mail/testdata/with_attachment.eml` (exact bytes, CRLF line endings not required for the parser): ``` From: "Bob" To: me@example.com Subject: Your Invoice #5 Date: Sat, 20 Jun 2026 10:00:00 +0000 Message-ID: MIME-Version: 1.0 Content-Type: multipart/mixed; boundary="BOUNDARY" --BOUNDARY Content-Type: text/plain; charset=utf-8 Hello, here is your invoice. --BOUNDARY Content-Type: text/plain; charset=utf-8 Content-Disposition: attachment; filename="invoice.txt" LINE-ITEM 1: 100.00 --BOUNDARY-- ``` - [ ] **Step 2: Write the failing test** Create `internal/mail/message_test.go`: ```go package mail import ( "os" "path/filepath" "testing" ) func loadFixture(t *testing.T, name string) []byte { t.Helper() b, err := os.ReadFile(filepath.Join("testdata", name)) if err != nil { t.Fatalf("read fixture: %v", err) } return b } func TestParseMessage(t *testing.T) { raw := loadFixture(t, "with_attachment.eml") m, err := ParseMessage(42, raw) if err != nil { t.Fatalf("ParseMessage: %v", err) } if m.Header.UID != 42 { t.Fatalf("uid: %d", m.Header.UID) } if m.Header.Subject != "Your Invoice #5" { t.Fatalf("subject: %q", m.Header.Subject) } if m.Header.From != `"Bob" ` && m.Header.From != "Bob " { t.Fatalf("from: %q", m.Header.From) } if m.Header.MessageID != "abc123@trusted.com" && m.Header.MessageID != "" { t.Fatalf("message-id: %q", m.Header.MessageID) } if want := "Hello, here is your invoice."; !contains(m.BodyText, want) { t.Fatalf("body=%q want contains %q", m.BodyText, want) } if !m.Header.HasAttachments { t.Fatal("HasAttachments should be true") } if len(m.Attachments) != 1 || m.Attachments[0].Name != "invoice.txt" { t.Fatalf("attachments: %+v", m.Attachments) } if !contains(string(m.Attachments[0].Content), "LINE-ITEM 1") { t.Fatalf("attachment content: %q", m.Attachments[0].Content) } } func TestParseHeaderOnly(t *testing.T) { raw := loadFixture(t, "with_attachment.eml") h, err := ParseHeaderOnly(7, raw) if err != nil { t.Fatalf("ParseHeaderOnly: %v", err) } if h.Subject != "Your Invoice #5" || !h.HasAttachments { t.Fatalf("header: %+v", h) } } func contains(s, sub string) bool { return len(s) >= len(sub) && (func() bool { for i := 0; i+len(sub) <= len(s); i++ { if s[i:i+len(sub)] == sub { return true } } return false }()) } ``` - [ ] **Step 3: Run test to verify it fails** Run: `go test ./internal/mail/` Expected: FAIL — undefined `ParseMessage`. - [ ] **Step 4: Write the implementation** Create `internal/mail/message.go`: ```go // Package mail provides IMAP reading and RFC822 message parsing. package mail import ( "bytes" "io" "strings" "github.com/emersion/go-message/mail" ) type Header struct { UID uint32 From string To string Subject string Date string MessageID string HasAttachments bool } type Attachment struct { Name string Size int MIME string Content []byte } type Message struct { Header Header BodyText string Attachments []Attachment } func readHeader(mr *mail.Reader, uid uint32) Header { h := Header{UID: uid} hd := mr.Header h.Subject, _ = hd.Subject() if addrs, err := hd.AddressList("From"); err == nil && len(addrs) > 0 { h.From = addrs[0].String() } if addrs, err := hd.AddressList("To"); err == nil && len(addrs) > 0 { h.To = addrs[0].String() } if d, err := hd.Date(); err == nil { h.Date = d.UTC().Format("Mon, 02 Jan 2006 15:04:05 -0700") } h.MessageID = strings.Trim(hd.Get("Message-Id"), "<> ") return h } // ParseMessage decodes the full message including attachment contents. func ParseMessage(uid uint32, raw []byte) (Message, error) { mr, err := mail.CreateReader(bytes.NewReader(raw)) if err != nil { return Message{}, err } m := Message{Header: readHeader(mr, uid)} for { part, err := mr.NextPart() if err == io.EOF { break } if err != nil { return Message{}, err } switch hdr := part.Header.(type) { case *mail.InlineHeader: ct, _, _ := hdr.ContentType() if strings.HasPrefix(ct, "text/plain") && m.BodyText == "" { b, _ := io.ReadAll(part.Body) m.BodyText = string(b) } case *mail.AttachmentHeader: name, _ := hdr.Filename() ct, _, _ := hdr.ContentType() b, _ := io.ReadAll(part.Body) m.Attachments = append(m.Attachments, Attachment{ Name: name, Size: len(b), MIME: ct, Content: b, }) } } m.Header.HasAttachments = len(m.Attachments) > 0 return m, nil } // ParseHeaderOnly decodes headers and detects attachments without keeping bodies. func ParseHeaderOnly(uid uint32, raw []byte) (Header, error) { m, err := ParseMessage(uid, raw) if err != nil { return Header{}, err } return m.Header, nil } ``` > Note: `ParseHeaderOnly` reuses `ParseMessage` for correctness in this phase. The IMAP layer (Task 11) avoids fetching full bodies for `list` by requesting only envelope + body-structure from the server, so this convenience does not cost a full download in practice. - [ ] **Step 5: Run tests to verify they pass** Run: `go test ./internal/mail/` Expected: PASS (2 tests). - [ ] **Step 6: Commit** ```bash git add internal/mail go.mod go.sum git commit -m "feat(mail): RFC822 message parsing (headers, body, attachments)" ``` --- ### Task 11: mail — IMAP client (integration) Connect, select a folder, fetch headers for new/ranged UIDs, fetch a full message, and search. Integration-tested against a `greenmail` container. If Docker is unavailable, the test self-skips. **Files:** - Create: `internal/mail/imap.go` - Test: `internal/mail/imap_integration_test.go` **Interfaces:** - Consumes: `github.com/emersion/go-imap`, `github.com/emersion/go-imap/client`, `ParseMessage`/`ParseHeaderOnly`. - Produces: - `type IMAPConfig struct { Host string; Port int; Security string; Username, Password string }` - `type Client struct { ... }` with `func Dial(cfg IMAPConfig) (*Client, error)` and `func (c *Client) Logout() error`. - `func (c *Client) SelectFolder(folder string) (uidValidity uint32, maxUID uint32, err error)`. - `func (c *Client) FetchHeaders(folder string, uids []uint32) ([]Header, error)` — empty `uids` ⇒ all in folder. - `func (c *Client) FetchHeadersRange(folder string, sinceUID, beforeUID uint32, limit int) ([]Header, error)`. - `func (c *Client) FetchFull(folder string, uid uint32) (Message, error)`. - `type SearchCriteria struct { From, SubjectContains, Text string; Since, Before time.Time }` - `func (c *Client) Search(folder string, sc SearchCriteria, limit int) ([]Header, error)`. - [ ] **Step 1: Add dependency and write the integration test** Run: `go get github.com/emersion/go-imap@v1` Create `internal/mail/imap_integration_test.go`: ```go //go:build integration package mail import ( "os" "testing" ) // These tests require a GreenMail server reachable via env vars: // EMCLI_TEST_IMAP_HOST, EMCLI_TEST_IMAP_PORT, EMCLI_TEST_USER, EMCLI_TEST_PASS // Run: go test -tags=integration ./internal/mail/ func testCfg(t *testing.T) IMAPConfig { t.Helper() host := os.Getenv("EMCLI_TEST_IMAP_HOST") if host == "" { t.Skip("EMCLI_TEST_IMAP_HOST not set; skipping IMAP integration test") } return IMAPConfig{ Host: host, Port: atoiEnv(t, "EMCLI_TEST_IMAP_PORT"), Security: "starttls", Username: os.Getenv("EMCLI_TEST_USER"), Password: os.Getenv("EMCLI_TEST_PASS"), } } func TestSelectAndFetch(t *testing.T) { c, err := Dial(testCfg(t)) if err != nil { t.Fatalf("Dial: %v", err) } defer c.Logout() uidv, maxUID, err := c.SelectFolder("INBOX") if err != nil { t.Fatalf("SelectFolder: %v", err) } if uidv == 0 { t.Fatal("uidvalidity should be non-zero") } headers, err := c.FetchHeaders("INBOX", nil) if err != nil { t.Fatalf("FetchHeaders: %v", err) } t.Logf("inbox has %d messages, maxUID=%d", len(headers), maxUID) } ``` Create the helper in the same file: ```go func atoiEnv(t *testing.T, k string) int { t.Helper() var n int if _, err := fmtSscan(os.Getenv(k), &n); err != nil { t.Fatalf("bad int env %s: %v", k, err) } return n } ``` > Note: define `fmtSscan` as a tiny wrapper `func fmtSscan(s string, v *int) (int, error){ return fmt.Sscan(s, v) }` in the test file, importing `fmt`. (Kept indirect only to avoid an unused-import lint if the body changes.) - [ ] **Step 2: Write the IMAP client implementation** Create `internal/mail/imap.go`: ```go package mail import ( "fmt" "io" "sort" "time" "github.com/emersion/go-imap" "github.com/emersion/go-imap/client" ) type IMAPConfig struct { Host string Port int Security string // tls | starttls Username string Password string } type Client struct { c *client.Client } func Dial(cfg IMAPConfig) (*Client, error) { addr := fmt.Sprintf("%s:%d", cfg.Host, cfg.Port) var ( ic *client.Client err error ) switch cfg.Security { case "tls": ic, err = client.DialTLS(addr, nil) case "starttls": ic, err = client.Dial(addr) if err == nil { err = ic.StartTLS(nil) } default: return nil, fmt.Errorf("unknown imap security %q", cfg.Security) } if err != nil { return nil, err } if err := ic.Login(cfg.Username, cfg.Password); err != nil { ic.Logout() return nil, err } return &Client{c: ic}, nil } func (c *Client) Logout() error { return c.c.Logout() } func (c *Client) SelectFolder(folder string) (uint32, uint32, error) { mbox, err := c.c.Select(folder, true) // read-only select if err != nil { return 0, 0, err } var maxUID uint32 if mbox.Messages > 0 { // UIDNext-1 is an upper bound for the highest existing UID. if mbox.UidNext > 0 { maxUID = mbox.UidNext - 1 } } return mbox.UidValidity, maxUID, nil } func (c *Client) fetchByUIDSet(folder string, set *imap.SeqSet, full bool) ([]Message, error) { if _, err := c.c.Select(folder, true); err != nil { return nil, err } section := &imap.BodySectionName{} // entire message items := []imap.FetchItem{imap.FetchUid, section.FetchItem()} msgCh := make(chan *imap.Message, 16) done := make(chan error, 1) go func() { done <- c.c.UidFetch(set, items, msgCh) }() var out []Message for m := range msgCh { r := m.GetBody(section) if r == nil { continue } raw, _ := io.ReadAll(r) parsed, err := ParseMessage(m.Uid, raw) if err != nil { return nil, err } if !full { parsed.Attachments = nil parsed.BodyText = "" } out = append(out, parsed) } if err := <-done; err != nil { return nil, err } sort.Slice(out, func(i, j int) bool { return out[i].Header.UID > out[j].Header.UID }) return out, nil } func (c *Client) FetchHeaders(folder string, uids []uint32) ([]Header, error) { set := new(imap.SeqSet) if len(uids) == 0 { set.AddRange(1, 0) // 1:* — all } else { for _, u := range uids { set.AddNum(u) } } msgs, err := c.fetchByUIDSet(folder, set, false) if err != nil { return nil, err } return headersOf(msgs), nil } func (c *Client) FetchHeadersRange(folder string, sinceUID, beforeUID uint32, limit int) ([]Header, error) { set := new(imap.SeqSet) lo := sinceUID + 1 if sinceUID == 0 { lo = 1 } hi := uint32(0) // '*' if beforeUID > 1 { hi = beforeUID - 1 } set.AddRange(lo, hi) msgs, err := c.fetchByUIDSet(folder, set, false) if err != nil { return nil, err } h := headersOf(msgs) if limit > 0 && len(h) > limit { h = h[:limit] } return h, nil } func (c *Client) FetchFull(folder string, uid uint32) (Message, error) { set := new(imap.SeqSet) set.AddNum(uid) msgs, err := c.fetchByUIDSet(folder, set, true) if err != nil { return Message{}, err } if len(msgs) == 0 { return Message{}, fmt.Errorf("uid %d not found in %s", uid, folder) } return msgs[0], nil } type SearchCriteria struct { From string SubjectContains string Text string Since time.Time Before time.Time } func (c *Client) Search(folder string, sc SearchCriteria, limit int) ([]Header, error) { if _, err := c.c.Select(folder, true); err != nil { return nil, err } crit := imap.NewSearchCriteria() if sc.From != "" { crit.Header.Add("From", sc.From) } if sc.SubjectContains != "" { crit.Header.Add("Subject", sc.SubjectContains) } if sc.Text != "" { crit.Text = []string{sc.Text} } if !sc.Since.IsZero() { crit.Since = sc.Since } if !sc.Before.IsZero() { crit.Before = sc.Before } uids, err := c.c.UidSearch(crit) if err != nil { return nil, err } if len(uids) == 0 { return nil, nil } return c.FetchHeaders(folder, uids) } func headersOf(msgs []Message) []Header { out := make([]Header, 0, len(msgs)) for _, m := range msgs { out = append(out, m.Header) } return out } ``` - [ ] **Step 3: Run unit tests (integration self-skips without Docker)** Run: `go build ./internal/mail/ && go test ./internal/mail/` Expected: PASS — parsing tests pass; integration test is excluded (build tag `integration` not set). - [ ] **Step 4: Run the integration test against GreenMail (optional but recommended)** Run: ```bash docker run -d --rm --name emcli-greenmail \ -e GREENMAIL_OPTS='-Dgreenmail.setup.test.all -Dgreenmail.users=me:pass@example.com -Dgreenmail.verbose' \ -p 3143:3143 -p 3025:3025 greenmail/standalone:2.1.0 EMCLI_TEST_IMAP_HOST=localhost EMCLI_TEST_IMAP_PORT=3143 \ EMCLI_TEST_USER=me@example.com EMCLI_TEST_PASS=pass \ go test -tags=integration ./internal/mail/ -run TestSelectAndFetch -v docker stop emcli-greenmail ``` Expected: PASS (selects INBOX, logs message count). Skips cleanly if Docker is absent. - [ ] **Step 5: Commit** ```bash git add internal/mail/imap.go internal/mail/imap_integration_test.go go.mod go.sum git commit -m "feat(mail): IMAP client — select, fetch headers/full, search" ``` --- ### Task 12: cli — JSON envelope **Files:** - Create: `internal/cli/envelope.go` - Test: `internal/cli/envelope_test.go` **Interfaces:** - Consumes: nothing. - Produces: - `type Envelope struct { Error bool; ErrorDetail map[string]any; Data any }` with JSON tags `error`, `error_detail`, `data`. - `func Success(data any) Envelope` - `func Failure(code, message string) Envelope` - `func (e Envelope) Write(w io.Writer) error` — writes compact JSON + newline. - Error code constants: `CodeConfig, CodeDB, CodeNetwork, CodeAuth, CodePolicy, CodeNotFound, CodeUsage`. - [ ] **Step 1: Write the failing test** Create `internal/cli/envelope_test.go`: ```go package cli import ( "bytes" "encoding/json" "testing" ) func TestSuccessEnvelope(t *testing.T) { var buf bytes.Buffer if err := Success(map[string]any{"count": 2}).Write(&buf); err != nil { t.Fatalf("write: %v", err) } var got map[string]any if err := json.Unmarshal(buf.Bytes(), &got); err != nil { t.Fatalf("unmarshal: %v", err) } if got["error"] != false { t.Fatalf("error should be false: %v", got["error"]) } if _, ok := got["data"]; !ok { t.Fatal("data key missing") } if ed, ok := got["error_detail"].(map[string]any); !ok || len(ed) != 0 { t.Fatalf("error_detail should be empty object: %v", got["error_detail"]) } } func TestFailureEnvelope(t *testing.T) { var buf bytes.Buffer _ = Failure(CodeNotFound, "uid 7 not found").Write(&buf) var got struct { Error bool `json:"error"` ErrorDetail struct { Code string `json:"code"` Message string `json:"message"` } `json:"error_detail"` } _ = json.Unmarshal(buf.Bytes(), &got) if !got.Error || got.ErrorDetail.Code != "not_found" { t.Fatalf("bad failure envelope: %+v", got) } } ``` - [ ] **Step 2: Run test to verify it fails** Run: `go test ./internal/cli/` Expected: FAIL — undefined `Success`. - [ ] **Step 3: Write the implementation** Create `internal/cli/envelope.go`: ```go // Package cli implements command dispatch and the agent JSON envelope. package cli import ( "encoding/json" "io" ) const ( CodeConfig = "config" CodeDB = "db" CodeNetwork = "network" CodeAuth = "auth" CodePolicy = "policy" CodeNotFound = "not_found" CodeUsage = "usage" ) // Envelope is the single JSON object every agent command emits. type Envelope struct { Error bool `json:"error"` ErrorDetail map[string]any `json:"error_detail"` Data any `json:"data"` } func Success(data any) Envelope { if data == nil { data = map[string]any{} } return Envelope{Error: false, ErrorDetail: map[string]any{}, Data: data} } func Failure(code, message string) Envelope { return Envelope{ Error: true, ErrorDetail: map[string]any{"code": code, "message": message}, Data: map[string]any{}, } } func (e Envelope) Write(w io.Writer) error { b, err := json.Marshal(e) if err != nil { return err } b = append(b, '\n') _, err = w.Write(b) return err } ``` - [ ] **Step 4: Run tests to verify they pass** Run: `go test ./internal/cli/` Expected: PASS (2 tests). - [ ] **Step 5: Commit** ```bash git add internal/cli/envelope.go internal/cli/envelope_test.go git commit -m "feat(cli): JSON output envelope with stable error codes" ``` --- ### Task 13: cli — agent read commands wired end-to-end Wire `list`, `get`, `search`, and `ack` together: open store, load account, apply inbound policy, talk to IMAP, write the envelope, and audit. Uses a `mailDialer` seam so the command logic is testable with a fake. **Files:** - Create: `internal/cli/agent.go` - Create: `internal/cli/dispatch.go` - Modify: `cmd/emcli/main.go` (route agent subcommands into `cli.Run`) - Test: `internal/cli/agent_test.go` **Interfaces:** - Consumes: `store`, `policy`, `mail` (via the `Mailer` interface), `Envelope`. - Produces: - `type Mailer interface { SelectFolder(folder string)(uint32,uint32,error); FetchHeaders(folder string, uids []uint32)([]mail.Header,error); FetchHeadersRange(folder string, since,before uint32, limit int)([]mail.Header,error); FetchFull(folder string, uid uint32)(mail.Message,error); Search(folder string, sc mail.SearchCriteria, limit int)([]mail.Header,error); Logout() error }` - `type Deps struct { Store *store.Store; Dial func(store.Account)(Mailer,error); Now func() time.Time; Out io.Writer }` - `func ListCmd(d Deps, account, folder string, onlyNew bool, beforeUID, sinceUID uint32, limit int) error` - `func GetCmd(d Deps, account, folder string, uid uint32) error` - `func SearchCmd(d Deps, account, folder string, sc mail.SearchCriteria, limit int) error` - `func AckCmd(d Deps, account, folder string, uids []uint32) error` - [ ] **Step 1: Write the failing test (fake Mailer, real store)** Create `internal/cli/agent_test.go`: ```go package cli import ( "bytes" "encoding/json" "path/filepath" "testing" "time" "git.dcglab.co.uk/steve/emcli/internal/mail" "git.dcglab.co.uk/steve/emcli/internal/store" ) type fakeMailer struct { uidValidity uint32 maxUID uint32 headers []mail.Header full map[uint32]mail.Message } func (f *fakeMailer) SelectFolder(string) (uint32, uint32, error) { return f.uidValidity, f.maxUID, nil } func (f *fakeMailer) FetchHeaders(_ string, uids []uint32) ([]mail.Header, error) { if len(uids) == 0 { return f.headers, nil } want := map[uint32]bool{} for _, u := range uids { want[u] = true } var out []mail.Header for _, h := range f.headers { if want[h.UID] { out = append(out, h) } } return out, nil } func (f *fakeMailer) FetchHeadersRange(string, uint32, uint32, int) ([]mail.Header, error) { return f.headers, nil } func (f *fakeMailer) FetchFull(_ string, uid uint32) (mail.Message, error) { return f.full[uid], nil } func (f *fakeMailer) Search(string, mail.SearchCriteria, int) ([]mail.Header, error) { return f.headers, nil } func (f *fakeMailer) Logout() error { return nil } func testKey() []byte { k := make([]byte, 32) for i := range k { k[i] = byte(i) } return k } func newDeps(t *testing.T, fm *fakeMailer) (Deps, *bytes.Buffer) { t.Helper() st, err := store.Open(filepath.Join(t.TempDir(), "e.db"), testKey()) if err != nil { t.Fatalf("store: %v", err) } t.Cleanup(func() { st.Close() }) _, err = st.AddAccount(store.Account{ Name: "work", Mode: "RO", IMAPHost: "h", IMAPPort: 993, IMAPSecurity: "tls", AuthType: "password", Username: "me@example.com", Password: "pw", WhitelistInEnabled: true, }) if err != nil { t.Fatalf("AddAccount: %v", err) } _ = st.AddWhitelist("work", store.DirIn, "@trusted.com") var buf bytes.Buffer d := Deps{ Store: st, Dial: func(store.Account) (Mailer, error) { return fm, nil }, Now: func() time.Time { return time.Date(2026, 6, 21, 0, 0, 0, 0, time.UTC) }, Out: &buf, } return d, &buf } func decode(t *testing.T, b []byte) map[string]any { t.Helper() var m map[string]any if err := json.Unmarshal(b, &m); err != nil { t.Fatalf("unmarshal %q: %v", b, err) } return m } func TestListNewFiltersBySenderAndState(t *testing.T) { fm := &fakeMailer{ uidValidity: 1, maxUID: 100, headers: []mail.Header{ {UID: 101, From: "bob@trusted.com", Subject: "hi"}, {UID: 102, From: "eve@evil.com", Subject: "spam"}, // filtered out by whitelist {UID: 103, From: "ann@trusted.com", Subject: "yo"}, }, } d, buf := newDeps(t, fm) if err := ListCmd(d, "work", "INBOX", true, 0, 0, 50); err != nil { t.Fatalf("ListCmd: %v", err) } res := decode(t, buf.Bytes()) if res["error"] != false { t.Fatalf("unexpected error: %v", res) } data := res["data"].(map[string]any) msgs := data["messages"].([]any) if len(msgs) != 2 { // 101 and 103; 102 filtered t.Fatalf("want 2 messages, got %d: %v", len(msgs), msgs) } } func TestGetFilteredReturnsNotFound(t *testing.T) { fm := &fakeMailer{ uidValidity: 1, maxUID: 100, headers: []mail.Header{{UID: 102, From: "eve@evil.com", Subject: "spam"}}, full: map[uint32]mail.Message{ 102: {Header: mail.Header{UID: 102, From: "eve@evil.com", Subject: "spam"}, BodyText: "secret"}, }, } d, buf := newDeps(t, fm) if err := GetCmd(d, "work", "INBOX", 102); err != nil { t.Fatalf("GetCmd: %v", err) } res := decode(t, buf.Bytes()) if res["error"] != true { t.Fatal("filtered get must return error envelope") } ed := res["error_detail"].(map[string]any) if ed["code"] != "not_found" { t.Fatalf("want not_found, got %v", ed["code"]) } } func TestAckAdvancesStateAndFiltered(t *testing.T) { fm := &fakeMailer{ uidValidity: 1, maxUID: 100, headers: []mail.Header{ {UID: 101, From: "bob@trusted.com", Subject: "hi"}, {UID: 102, From: "eve@evil.com", Subject: "spam"}, }, } d, buf := newDeps(t, fm) // Acking a filtered uid (102) must be rejected as not-found. if err := AckCmd(d, "work", "INBOX", []uint32{102}); err != nil { t.Fatalf("AckCmd: %v", err) } if decode(t, buf.Bytes())["error"] != true { t.Fatal("acking filtered uid must fail") } // Acking a visible uid (101) succeeds and removes it from list --new. buf.Reset() if err := AckCmd(d, "work", "INBOX", []uint32{101}); err != nil { t.Fatalf("AckCmd 101: %v", err) } if decode(t, buf.Bytes())["error"] != false { t.Fatal("ack of visible uid should succeed") } buf.Reset() _ = ListCmd(d, "work", "INBOX", true, 0, 0, 50) data := decode(t, buf.Bytes())["data"].(map[string]any) msgs := data["messages"].([]any) if len(msgs) != 0 { // 101 acked, 102 filtered t.Fatalf("want 0 new messages, got %d", len(msgs)) } } ``` - [ ] **Step 2: Run test to verify it fails** Run: `go test ./internal/cli/ -run 'List|Get|Ack'` Expected: FAIL — undefined `Deps`/`ListCmd`. - [ ] **Step 3: Write the agent command implementation** Create `internal/cli/agent.go`: ```go package cli import ( "io" "time" "git.dcglab.co.uk/steve/emcli/internal/mail" "git.dcglab.co.uk/steve/emcli/internal/policy" "git.dcglab.co.uk/steve/emcli/internal/store" ) // Mailer is the subset of the IMAP client the agent commands use. type Mailer interface { SelectFolder(folder string) (uint32, uint32, error) FetchHeaders(folder string, uids []uint32) ([]mail.Header, error) FetchHeadersRange(folder string, since, before uint32, limit int) ([]mail.Header, error) FetchFull(folder string, uid uint32) (mail.Message, error) Search(folder string, sc mail.SearchCriteria, limit int) ([]mail.Header, error) Logout() error } type Deps struct { Store *store.Store Dial func(store.Account) (Mailer, error) Now func() time.Time Out io.Writer } func (d Deps) emit(e Envelope) error { return e.Write(d.Out) } func (d Deps) audit(account, action, target, result, reason string) { _ = d.Store.Audit(d.Now(), store.AuditEntry{ Account: account, Action: action, Target: target, Result: result, Reason: reason, }) } // setup loads the account, builds the inbound rule, dials IMAP, and selects the // folder (establishing the baseline). Returns a cleanup func. func (d Deps) setup(account, folder string) (store.Account, policy.InboundRule, Mailer, func(), *Envelope) { acc, err := d.Store.GetAccount(account) if err != nil { e := Failure(CodeNotFound, "account not found: "+account) return acc, policy.InboundRule{}, nil, nil, &e } re, err := policy.CompileSubject(acc.SubjectRegex) if err != nil { e := Failure(CodeConfig, "invalid subject_regex: "+err.Error()) return acc, policy.InboundRule{}, nil, nil, &e } wlIn, _ := d.Store.ListWhitelist(account, store.DirIn) rule := policy.InboundRule{ WhitelistInEnabled: acc.WhitelistInEnabled, WhitelistIn: wlIn, SubjectRegex: re, } m, err := d.Dial(acc) if err != nil { e := Failure(CodeNetwork, "imap connect failed: "+err.Error()) return acc, rule, nil, nil, &e } uidv, maxUID, err := m.SelectFolder(folder) if err != nil { m.Logout() e := Failure(CodeNetwork, "select folder failed: "+err.Error()) return acc, rule, nil, nil, &e } if err := d.Store.EnsureFolderBaseline(account, folder, uidv, maxUID); err != nil { m.Logout() e := Failure(CodeDB, err.Error()) return acc, rule, nil, nil, &e } return acc, rule, m, func() { m.Logout() }, nil } func headerMap(h mail.Header) map[string]any { return map[string]any{ "uid": h.UID, "from": h.From, "to": h.To, "subject": h.Subject, "date": h.Date, "message_id": h.MessageID, "has_attachments": h.HasAttachments, } } func ListCmd(d Deps, account, folder string, onlyNew bool, beforeUID, sinceUID uint32, limit int) error { _, rule, m, done, fail := d.setup(account, folder) if fail != nil { return d.emit(*fail) } defer done() var headers []mail.Header var err error if beforeUID > 0 || sinceUID > 0 { headers, err = m.FetchHeadersRange(folder, sinceUID, beforeUID, limit) } else { headers, err = m.FetchHeaders(folder, nil) } if err != nil { d.audit(account, "list", folder, "blocked", "imap_error") return d.emit(Failure(CodeNetwork, err.Error())) } out := make([]map[string]any, 0, len(headers)) for _, h := range headers { if !rule.Allows(h.From, h.Subject) { continue // invisible } if onlyNew { isNew, err := d.Store.IsNew(account, folder, h.UID) if err != nil { return d.emit(Failure(CodeDB, err.Error())) } if !isNew { continue } } out = append(out, headerMap(h)) if limit > 0 && len(out) >= limit { break } } d.audit(account, "list", folder, "allowed", "") return d.emit(Success(map[string]any{"messages": out})) } // visible fetches a UID's header and reports whether policy allows it. func (d Deps) visible(m Mailer, rule policy.InboundRule, folder string, uid uint32) (bool, error) { hs, err := m.FetchHeaders(folder, []uint32{uid}) if err != nil { return false, err } if len(hs) == 0 { return false, nil } return rule.Allows(hs[0].From, hs[0].Subject), nil } func GetCmd(d Deps, account, folder string, uid uint32) error { _, rule, m, done, fail := d.setup(account, folder) if fail != nil { return d.emit(*fail) } defer done() ok, err := d.visible(m, rule, folder, uid) if err != nil { return d.emit(Failure(CodeNetwork, err.Error())) } if !ok { d.audit(account, "get", uitoa(uid), "blocked", "filtered") return d.emit(Failure(CodeNotFound, "message not found")) } msg, err := m.FetchFull(folder, uid) if err != nil { return d.emit(Failure(CodeNetwork, err.Error())) } atts := make([]map[string]any, 0, len(msg.Attachments)) for _, a := range msg.Attachments { atts = append(atts, map[string]any{ "name": a.Name, "size": a.Size, "mime": a.MIME, "content_b64": b64(a.Content), }) } d.audit(account, "get", uitoa(uid), "allowed", "") return d.emit(Success(map[string]any{ "uid": msg.Header.UID, "from": msg.Header.From, "to": msg.Header.To, "subject": msg.Header.Subject, "date": msg.Header.Date, "message_id": msg.Header.MessageID, "body_text": msg.BodyText, "attachments": atts, })) } func SearchCmd(d Deps, account, folder string, sc mail.SearchCriteria, limit int) error { _, rule, m, done, fail := d.setup(account, folder) if fail != nil { return d.emit(*fail) } defer done() headers, err := m.Search(folder, sc, limit) if err != nil { return d.emit(Failure(CodeNetwork, err.Error())) } out := make([]map[string]any, 0, len(headers)) for _, h := range headers { if !rule.Allows(h.From, h.Subject) { continue } out = append(out, headerMap(h)) if limit > 0 && len(out) >= limit { break } } d.audit(account, "search", folder, "allowed", "") return d.emit(Success(map[string]any{"messages": out})) } func AckCmd(d Deps, account, folder string, uids []uint32) error { _, rule, m, done, fail := d.setup(account, folder) if fail != nil { return d.emit(*fail) } defer done() uidv, _, _ := m.SelectFolder(folder) for _, uid := range uids { ok, err := d.visible(m, rule, folder, uid) if err != nil { return d.emit(Failure(CodeNetwork, err.Error())) } if !ok { d.audit(account, "ack", uitoa(uid), "blocked", "filtered") return d.emit(Failure(CodeNotFound, "message not found")) } } if err := d.Store.Ack(account, folder, uidv, uids...); err != nil { return d.emit(Failure(CodeDB, err.Error())) } d.audit(account, "ack", folder, "allowed", "") return d.emit(Success(map[string]any{"acked": uintSlice(uids)})) } ``` Create `internal/cli/dispatch.go` (helpers + the `Run` router skeleton used by `main`): ```go package cli import ( "encoding/base64" "strconv" ) func b64(b []byte) string { return base64.StdEncoding.EncodeToString(b) } func uitoa(u uint32) string { return strconv.FormatUint(uint64(u), 10) } func uintSlice(us []uint32) []uint64 { out := make([]uint64, len(us)) for i, u := range us { out[i] = uint64(u) } return out } ``` - [ ] **Step 4: Run tests to verify they pass** Run: `go test ./internal/cli/` Expected: PASS (envelope + agent tests). - [ ] **Step 5: Commit** ```bash git add internal/cli/agent.go internal/cli/dispatch.go internal/cli/agent_test.go git commit -m "feat(cli): agent read commands (list/get/search/ack) with policy filtering" ``` --- ### Task 14: cli — flag parsing, real IMAP wiring & minimal admin Wire the real IMAP client to the `Mailer` interface, parse flags for each agent command, add minimal flag-based admin (`account add`, `whitelist add`) so the tool is configurable end-to-end, and route everything from `main`. The interactive TUI is Phase 4; this gives a usable non-interactive admin surface now. **Files:** - Create: `internal/cli/run.go` - Create: `internal/cli/admin.go` - Modify: `cmd/emcli/main.go` - Test: `internal/cli/run_test.go` **Interfaces:** - Consumes: everything above. - Produces: - `func Run(args []string, out, errOut io.Writer) int` — top-level router; returns process exit code. - `func realMailer(acc store.Account) (Mailer, error)` — adapts `mail.Dial` to `Mailer`. - Admin handlers writing human-readable output: `accountAdd`, `whitelistAdd`, `accountList`. - [ ] **Step 1: Write the failing test** Create `internal/cli/run_test.go`: ```go package cli import ( "bytes" "encoding/json" "strings" "testing" ) func TestRunUnknownCommand(t *testing.T) { var out, errOut bytes.Buffer code := Run([]string{"frobnicate"}, &out, &errOut) if code == 0 { t.Fatal("unknown command should be non-zero exit") } if !strings.Contains(errOut.String(), "unknown") { t.Fatalf("stderr should mention unknown command: %q", errOut.String()) } } func TestRunVersionIsJSONForAgentButTextHere(t *testing.T) { // `account list` with no DB key should fail closed with a usage/config error, // proving the key check happens before any DB work. var out, errOut bytes.Buffer t.Setenv("EMCLI_KEY", "") code := Run([]string{"account", "list"}, &out, &errOut) if code == 0 { t.Fatal("missing EMCLI_KEY must fail") } if !strings.Contains(out.String()+errOut.String(), "EMCLI_KEY") { t.Fatalf("should mention EMCLI_KEY, got out=%q err=%q", out.String(), errOut.String()) } } func TestListUsageErrorIsJSON(t *testing.T) { // Agent command with a missing required flag emits a JSON error envelope. var out, errOut bytes.Buffer t.Setenv("EMCLI_KEY", b64Key()) t.Setenv("EMCLI_DB", "") // default path is fine; command fails before connecting code := Run([]string{"list"}, &out, &errOut) // missing --account if code == 0 { t.Fatal("missing --account should be non-zero") } var env map[string]any if err := json.Unmarshal(out.Bytes(), &env); err != nil { t.Fatalf("agent usage error must be JSON, got %q", out.String()) } if env["error"] != true { t.Fatalf("want error envelope: %v", env) } } func b64Key() string { // 32 zero bytes, base64. return "AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA=" } ``` - [ ] **Step 2: Run test to verify it fails** Run: `go test ./internal/cli/ -run Run` Expected: FAIL — undefined `Run`. - [ ] **Step 3: Write the router and admin** Create `internal/cli/run.go`: ```go package cli import ( "flag" "fmt" "io" "strconv" "strings" "time" "git.dcglab.co.uk/steve/emcli/internal/crypto" "git.dcglab.co.uk/steve/emcli/internal/mail" "git.dcglab.co.uk/steve/emcli/internal/store" ) func realMailer(acc store.Account) (Mailer, error) { c, err := mail.Dial(mail.IMAPConfig{ Host: acc.IMAPHost, Port: acc.IMAPPort, Security: acc.IMAPSecurity, Username: acc.Username, Password: acc.Password, }) if err != nil { return nil, err } return c, nil } // openStore loads the key and opens the DB, returning a human-readable error string. func openStore() (*store.Store, error) { key, err := crypto.KeyFromEnv() if err != nil { return nil, err } path, err := store.DefaultDBPath() if err != nil { return nil, err } return store.Open(path, key) } func newDepsLive(st *store.Store, out io.Writer) Deps { return Deps{Store: st, Dial: realMailer, Now: time.Now, Out: out} } // Run routes a command line and returns an exit code. func Run(args []string, out, errOut io.Writer) int { if len(args) == 0 { fmt.Fprintln(errOut, "emcli: no command given") return 2 } cmd, rest := args[0], args[1:] switch cmd { case "list", "get", "search", "ack": return runAgent(cmd, rest, out, errOut) case "account": return runAccount(rest, out, errOut) case "whitelist": return runWhitelist(rest, out, errOut) default: fmt.Fprintf(errOut, "emcli: unknown command %q\n", cmd) return 2 } } // runAgent handles JSON-emitting commands. Errors are emitted as JSON envelopes. func runAgent(cmd string, args []string, out, errOut io.Writer) int { fs := flag.NewFlagSet(cmd, flag.ContinueOnError) fs.SetOutput(errOut) account := fs.String("account", "", "account name") folder := fs.String("folder", "INBOX", "folder/mailbox") onlyNew := fs.Bool("new", false, "only new (unacked) messages") limit := fs.Int("limit", 50, "max results (cap 500)") uid := fs.Uint("uid", 0, "message UID (get)") before := fs.Uint("before", 0, "list: UID cursor, older than") since := fs.Uint("since", 0, "list: UID cursor, newer than") from := fs.String("from", "", "search: sender") subjectContains := fs.String("subject-contains", "", "search: subject substring") text := fs.String("text", "", "search: full-text") sinceDate := fs.String("since-date", "", "search: RFC3339 date lower bound") beforeDate := fs.String("before-date", "", "search: RFC3339 date upper bound") ackUIDs := fs.String("uid-list", "", "ack: comma-separated UIDs") if err := fs.Parse(args); err != nil { _ = Failure(CodeUsage, err.Error()).Write(out) return 2 } if *limit > 500 { *limit = 500 } if *account == "" { _ = Failure(CodeUsage, "--account is required").Write(out) return 2 } st, err := openStore() if err != nil { _ = Failure(CodeConfig, err.Error()).Write(out) return 1 } defer st.Close() _, _ = st.PurgeAudit(time.Now()) d := newDepsLive(st, out) switch cmd { case "list": if err := ListCmd(d, *account, *folder, *onlyNew, u32(*before), u32(*since), *limit); err != nil { return 1 } case "get": if *uid == 0 { _ = Failure(CodeUsage, "--uid is required").Write(out) return 2 } if err := GetCmd(d, *account, *folder, u32(*uid)); err != nil { return 1 } case "search": sc := mail.SearchCriteria{From: *from, SubjectContains: *subjectContains, Text: *text} if *sinceDate != "" { if tm, err := time.Parse(time.RFC3339, *sinceDate); err == nil { sc.Since = tm } } if *beforeDate != "" { if tm, err := time.Parse(time.RFC3339, *beforeDate); err == nil { sc.Before = tm } } if err := SearchCmd(d, *account, *folder, sc, *limit); err != nil { return 1 } case "ack": uids, err := parseUIDList(*ackUIDs) if err != nil { _ = Failure(CodeUsage, err.Error()).Write(out) return 2 } if err := AckCmd(d, *account, *folder, uids); err != nil { return 1 } } return 0 } func u32(u uint) uint32 { return uint32(u) } func parseUIDList(s string) ([]uint32, error) { if strings.TrimSpace(s) == "" { return nil, fmt.Errorf("--uid-list is required") } var out []uint32 for _, part := range strings.Split(s, ",") { n, err := strconv.ParseUint(strings.TrimSpace(part), 10, 32) if err != nil { return nil, fmt.Errorf("invalid uid %q", part) } out = append(out, uint32(n)) } return out, nil } ``` Create `internal/cli/admin.go`: ```go package cli import ( "flag" "fmt" "io" "git.dcglab.co.uk/steve/emcli/internal/store" ) // runAccount handles `account add|list`. Human-readable output (never JSON). func runAccount(args []string, out, errOut io.Writer) int { if len(args) == 0 { fmt.Fprintln(errOut, "usage: emcli account ") return 2 } sub, rest := args[0], args[1:] st, err := openStore() if err != nil { fmt.Fprintf(errOut, "emcli: %v\n", err) return 1 } defer st.Close() switch sub { case "add": fs := flag.NewFlagSet("account add", flag.ContinueOnError) fs.SetOutput(errOut) name := fs.String("name", "", "account name") mode := fs.String("mode", "RO", "RO|RW") host := fs.String("imap-host", "", "IMAP host") port := fs.Int("imap-port", 993, "IMAP port") sec := fs.String("imap-security", "tls", "tls|starttls") user := fs.String("username", "", "login username") pass := fs.String("password", "", "login password") subj := fs.String("subject-regex", "", "inbound subject filter") wlIn := fs.Bool("whitelist-in", false, "enable inbound whitelist") wlOut := fs.Bool("whitelist-out", false, "enable outbound whitelist") backlog := fs.Bool("process-backlog", false, "treat existing mail as new") if err := fs.Parse(rest); err != nil { return 2 } if *name == "" || *host == "" || *user == "" { fmt.Fprintln(errOut, "name, imap-host, and username are required") return 2 } _, err := st.AddAccount(store.Account{ Name: *name, Mode: *mode, IMAPHost: *host, IMAPPort: *port, IMAPSecurity: *sec, AuthType: "password", Username: *user, Password: *pass, SubjectRegex: *subj, WhitelistInEnabled: *wlIn, WhitelistOutEnabled: *wlOut, ProcessBacklog: *backlog, }) if err != nil { fmt.Fprintf(errOut, "add account: %v\n", err) return 1 } fmt.Fprintf(out, "account %q added (%s)\n", *name, *mode) return 0 case "list": accs, err := st.ListAccounts() if err != nil { fmt.Fprintf(errOut, "list: %v\n", err) return 1 } fmt.Fprintf(out, "%-16s %-4s %-28s %s\n", "NAME", "MODE", "IMAP", "USER") for _, a := range accs { fmt.Fprintf(out, "%-16s %-4s %-28s %s\n", a.Name, a.Mode, fmt.Sprintf("%s:%d", a.IMAPHost, a.IMAPPort), a.Username) } return 0 default: fmt.Fprintf(errOut, "unknown account subcommand %q\n", sub) return 2 } } // runWhitelist handles `whitelist add --account NAME --address A`. func runWhitelist(args []string, out, errOut io.Writer) int { if len(args) < 2 { fmt.Fprintln(errOut, "usage: emcli whitelist [flags]") return 2 } dir := store.Direction(args[0]) sub, rest := args[1], args[2:] fs := flag.NewFlagSet("whitelist", flag.ContinueOnError) fs.SetOutput(errOut) account := fs.String("account", "", "account name") address := fs.String("address", "", "email or @domain") if err := fs.Parse(rest); err != nil { return 2 } if *account == "" { fmt.Fprintln(errOut, "--account is required") return 2 } st, err := openStore() if err != nil { fmt.Fprintf(errOut, "emcli: %v\n", err) return 1 } defer st.Close() switch sub { case "add": if err := st.AddWhitelist(*account, dir, *address); err != nil { fmt.Fprintf(errOut, "add: %v\n", err) return 1 } fmt.Fprintf(out, "added %s to %s whitelist of %q\n", *address, dir, *account) case "remove": if err := st.RemoveWhitelist(*account, dir, *address); err != nil { fmt.Fprintf(errOut, "remove: %v\n", err) return 1 } fmt.Fprintf(out, "removed %s\n", *address) case "list": addrs, err := st.ListWhitelist(*account, dir) if err != nil { fmt.Fprintf(errOut, "list: %v\n", err) return 1 } for _, a := range addrs { fmt.Fprintln(out, a) } default: fmt.Fprintf(errOut, "unknown whitelist subcommand %q\n", sub) return 2 } return 0 } ``` - [ ] **Step 4: Replace main to route through Run** Replace `cmd/emcli/main.go` with: ```go package main import ( "fmt" "os" "git.dcglab.co.uk/steve/emcli/internal/cli" "git.dcglab.co.uk/steve/emcli/internal/version" ) func main() { if len(os.Args) >= 2 && os.Args[1] == "version" { fmt.Println(version.String) return } os.Exit(cli.Run(os.Args[1:], os.Stdout, os.Stderr)) } ``` - [ ] **Step 5: Run all tests and build** Run: `go test ./... && make build` Expected: PASS across all packages; `emcli` builds. - [ ] **Step 6: Manual smoke test** Run: ```bash export EMCLI_KEY=$(head -c32 /dev/zero | base64) export EMCLI_DB=/tmp/emcli-smoke.db ./emcli account add --name demo --imap-host imap.example.com --username me@example.com --password x --whitelist-in ./emcli whitelist in add --account demo --address @trusted.com ./emcli account list ./emcli list --account demo --new # emits a JSON error envelope (cannot connect to example.com) — proves JSON-only output rm -f /tmp/emcli-smoke.db ``` Expected: admin commands print human-readable lines; `list` prints a single JSON object with `"error": true` and a `network` code. - [ ] **Step 7: Commit** ```bash git add internal/cli/run.go internal/cli/admin.go internal/cli/run_test.go cmd/emcli/main.go git commit -m "feat(cli): command router, real IMAP wiring, flag-based admin" ``` --- ## Phase 1 Self-Review - **Spec coverage (read path):** encrypted store ✓ (Tasks 2–4), per-account RO/RW field stored ✓ (Task 4; send-side enforcement is Phase 2), whitelist-in + subject filter ✓ (Tasks 5, 8, 9, 13), seen-set "new" + ack + compaction ✓ (Task 6), folder-specified reads + pointer-per-folder ✓ (Tasks 6, 11, 13), `list`/`get`/`search`/`ack` JSON ✓ (Tasks 12–14), headers-on-list / body+attachments-on-get ✓ (Tasks 10, 13), invisible inbound filtering incl. un-ackable filtered UIDs ✓ (Task 13), audit log + retention purge ✓ (Tasks 7, 14), single static binary + pure-Go SQLite ✓ (Tasks 1, 3, global constraints), `EMCLI_KEY`/`EMCLI_DB` handling and fail-closed ✓ (Tasks 2, 3, 14). - **Deferred to later phases (intentional, not gaps):** SMTP `send` + outbound policy (Phase 2), OAuth2 loopback + token refresh and the `enc_oauth_*` columns wiring (Phase 3), interactive TUI `init`/reconfigure and `doctor` (Phase 4). The schema already includes the OAuth columns so no migration is needed later. - **Placeholder scan:** no TBD/TODO; every code step is complete and compilable in sequence. - **Type consistency:** `Mailer` interface in Task 13 matches the concrete `*mail.Client` methods in Task 11 (`SelectFolder`, `FetchHeaders`, `FetchHeadersRange`, `FetchFull`, `Search`, `Logout`); `store` method names (`EnsureFolderBaseline`, `IsNew`, `Ack`, `Audit`, `ListWhitelist`) are used consistently across Tasks 6, 7, 13. ## Notes for the implementer - Go-imap v1's `client.Client` is not safe for concurrent use; each command dials its own connection, which is correct for a per-invocation CLI. - Keep `time.Now()` out of `store`; it is always injected (`Deps.Now`, `Audit(now, …)`) so tests are deterministic — the workflow runtime forbids wall-clock calls in some contexts and determinism aids review. - When in doubt about an emersion API detail, prefer the smallest change that keeps the `Mailer` interface stable; the CLI tests depend only on the interface, not the concrete client.