04d3b61bb0
TDD task-by-task plan for the read-only emcli: crypto, encrypted store, seen-set read state, policy filtering, IMAP read, and the agent list/get/search/ack commands with flag-based admin. Phases 2-4 (send, OAuth2, TUI) to follow as their own plans. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
3320 lines
90 KiB
Markdown
3320 lines
90 KiB
Markdown
# emcli Phase 1 — Foundation & Read Path Implementation Plan
|
||
|
||
> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking.
|
||
|
||
**Goal:** Build a working, testable read-only `emcli` — an encrypted-config CLI that lets an agent list, fetch, search, and acknowledge emails over IMAP, with all reads filtered by per-account policy and credentials never exposed.
|
||
|
||
**Architecture:** A single Go binary organised into focused `internal/` packages: `crypto` (AES-256-GCM field encryption), `store` (pure-Go SQLite config + seen-set read state + audit), `policy` (pure enforcement functions), `mail` (IMAP read + message parsing), and `cli` (JSON-envelope command dispatch). Reads are stateless; the only state-mutating command is `ack`. This phase uses password auth only; OAuth2, SMTP send, and the TUI arrive in later phases.
|
||
|
||
**Tech Stack:** Go (no CGO), `modernc.org/sqlite`, `github.com/emersion/go-imap` + `github.com/emersion/go-message`, Go stdlib `crypto/aes`+`crypto/cipher`. Tests use Go's `testing` package; IMAP integration tests run against a `greenmail` container.
|
||
|
||
## Global Constraints
|
||
|
||
- Language: Go, **CGO disabled** (`CGO_ENABLED=0`) — the binary must be a single static cross-platform executable.
|
||
- SQLite driver: `modernc.org/sqlite` (pure Go), registered driver name `"sqlite"`. Never `mattn/go-sqlite3`.
|
||
- Encryption key source: env var `EMCLI_KEY`, a base64-standard-encoded 32-byte key (AES-256). Missing/malformed ⇒ fail closed, never a plaintext fallback.
|
||
- DB path source: env var `EMCLI_DB`; default `~/.config/emcli/emcli.db` (`%AppData%\emcli\emcli.db` on Windows).
|
||
- Secret columns (`enc_password`, `enc_oauth_*`) are stored as `crypto.Seal` output. Secrets never appear in stdout, error details, or the audit log.
|
||
- Every agent command prints exactly one JSON object: `{"error":bool,"error_detail":obj,"data":obj}` and nothing else on stdout.
|
||
- "New" = message exists in folder AND `uid > floor_uid` AND `uid ∉ acked`. Reads never mutate floor/ack state; only `ack` does.
|
||
- Module path: `git.dcglab.co.uk/steve/emcli` (used in all import paths below).
|
||
|
||
---
|
||
|
||
### Task 1: Project scaffold & build
|
||
|
||
**Files:**
|
||
- Create: `go.mod`
|
||
- Create: `cmd/emcli/main.go`
|
||
- Create: `internal/version/version.go`
|
||
- Create: `Makefile`
|
||
- Test: `internal/version/version_test.go`
|
||
|
||
**Interfaces:**
|
||
- Consumes: nothing.
|
||
- Produces: `version.String` (string constant); binary entrypoint `cmd/emcli`.
|
||
|
||
- [ ] **Step 1: Initialise the module**
|
||
|
||
Run:
|
||
```bash
|
||
cd /home/steve/src/emcli
|
||
go mod init git.dcglab.co.uk/steve/emcli
|
||
go mod edit -go=1.22
|
||
```
|
||
|
||
- [ ] **Step 2: Write the failing test**
|
||
|
||
Create `internal/version/version_test.go`:
|
||
```go
|
||
package version
|
||
|
||
import "testing"
|
||
|
||
func TestStringIsSemver(t *testing.T) {
|
||
if String == "" {
|
||
t.Fatal("version.String must not be empty")
|
||
}
|
||
if String[0] != 'v' {
|
||
t.Fatalf("version %q must start with 'v'", String)
|
||
}
|
||
}
|
||
```
|
||
|
||
- [ ] **Step 3: Run test to verify it fails**
|
||
|
||
Run: `go test ./internal/version/`
|
||
Expected: FAIL — `undefined: String`.
|
||
|
||
- [ ] **Step 4: Write minimal implementation**
|
||
|
||
Create `internal/version/version.go`:
|
||
```go
|
||
// Package version holds the build version string.
|
||
package version
|
||
|
||
// String is the emcli version. Overridden at release via -ldflags.
|
||
var String = "v0.1.0-dev"
|
||
```
|
||
|
||
- [ ] **Step 5: Write the entrypoint**
|
||
|
||
Create `cmd/emcli/main.go`:
|
||
```go
|
||
package main
|
||
|
||
import (
|
||
"fmt"
|
||
"os"
|
||
|
||
"git.dcglab.co.uk/steve/emcli/internal/version"
|
||
)
|
||
|
||
func main() {
|
||
if len(os.Args) >= 2 && os.Args[1] == "version" {
|
||
fmt.Println(version.String)
|
||
return
|
||
}
|
||
fmt.Fprintln(os.Stderr, "emcli: no command given")
|
||
os.Exit(2)
|
||
}
|
||
```
|
||
|
||
- [ ] **Step 6: Write the Makefile**
|
||
|
||
Create `Makefile`:
|
||
```make
|
||
BINARY := emcli
|
||
LDFLAGS := -s -w
|
||
|
||
.PHONY: build test vet
|
||
build:
|
||
CGO_ENABLED=0 go build -ldflags "$(LDFLAGS)" -o $(BINARY) ./cmd/emcli
|
||
|
||
test:
|
||
go test ./...
|
||
|
||
vet:
|
||
go vet ./...
|
||
```
|
||
|
||
- [ ] **Step 7: Run test and build to verify they pass**
|
||
|
||
Run: `go test ./internal/version/ && make build && ./emcli version`
|
||
Expected: test PASS; build succeeds; prints `v0.1.0-dev`.
|
||
|
||
- [ ] **Step 8: Commit**
|
||
|
||
```bash
|
||
git add go.mod cmd internal/version Makefile
|
||
git commit -m "feat: project scaffold, version command, build"
|
||
```
|
||
|
||
---
|
||
|
||
### Task 2: crypto — key loading & AES-256-GCM field encryption
|
||
|
||
**Files:**
|
||
- Create: `internal/crypto/crypto.go`
|
||
- Test: `internal/crypto/crypto_test.go`
|
||
|
||
**Interfaces:**
|
||
- Consumes: nothing.
|
||
- Produces:
|
||
- `func KeyFromEnv() ([]byte, error)` — reads `EMCLI_KEY`, base64-decodes, validates 32 bytes.
|
||
- `func Seal(key, plaintext []byte) ([]byte, error)` — returns `nonce||ciphertext` (AES-256-GCM).
|
||
- `func Open(key, blob []byte) ([]byte, error)` — reverses Seal; auth failure ⇒ error.
|
||
- `var ErrNoKey, ErrBadKey error`.
|
||
|
||
- [ ] **Step 1: Write the failing tests**
|
||
|
||
Create `internal/crypto/crypto_test.go`:
|
||
```go
|
||
package crypto
|
||
|
||
import (
|
||
"bytes"
|
||
"encoding/base64"
|
||
"testing"
|
||
)
|
||
|
||
func testKey() []byte {
|
||
k := make([]byte, 32)
|
||
for i := range k {
|
||
k[i] = byte(i)
|
||
}
|
||
return k
|
||
}
|
||
|
||
func TestSealOpenRoundTrip(t *testing.T) {
|
||
key := testKey()
|
||
msg := []byte("hunter2-the-password")
|
||
blob, err := Seal(key, msg)
|
||
if err != nil {
|
||
t.Fatalf("Seal: %v", err)
|
||
}
|
||
if bytes.Contains(blob, msg) {
|
||
t.Fatal("ciphertext must not contain plaintext")
|
||
}
|
||
got, err := Open(key, blob)
|
||
if err != nil {
|
||
t.Fatalf("Open: %v", err)
|
||
}
|
||
if !bytes.Equal(got, msg) {
|
||
t.Fatalf("round-trip mismatch: %q", got)
|
||
}
|
||
}
|
||
|
||
func TestSealUsesRandomNonce(t *testing.T) {
|
||
key := testKey()
|
||
a, _ := Seal(key, []byte("x"))
|
||
b, _ := Seal(key, []byte("x"))
|
||
if bytes.Equal(a, b) {
|
||
t.Fatal("two seals of same plaintext must differ (random nonce)")
|
||
}
|
||
}
|
||
|
||
func TestOpenWrongKeyFails(t *testing.T) {
|
||
blob, _ := Seal(testKey(), []byte("secret"))
|
||
wrong := make([]byte, 32) // all zeros
|
||
if _, err := Open(wrong, blob); err == nil {
|
||
t.Fatal("Open with wrong key must fail")
|
||
}
|
||
}
|
||
|
||
func TestKeyFromEnv(t *testing.T) {
|
||
t.Setenv("EMCLI_KEY", base64.StdEncoding.EncodeToString(testKey()))
|
||
k, err := KeyFromEnv()
|
||
if err != nil || len(k) != 32 {
|
||
t.Fatalf("KeyFromEnv: key=%d err=%v", len(k), err)
|
||
}
|
||
|
||
t.Setenv("EMCLI_KEY", "")
|
||
if _, err := KeyFromEnv(); err != ErrNoKey {
|
||
t.Fatalf("empty key: want ErrNoKey, got %v", err)
|
||
}
|
||
|
||
t.Setenv("EMCLI_KEY", base64.StdEncoding.EncodeToString([]byte("tooshort")))
|
||
if _, err := KeyFromEnv(); err != ErrBadKey {
|
||
t.Fatalf("short key: want ErrBadKey, got %v", err)
|
||
}
|
||
}
|
||
```
|
||
|
||
- [ ] **Step 2: Run tests to verify they fail**
|
||
|
||
Run: `go test ./internal/crypto/`
|
||
Expected: FAIL — undefined `Seal`, `Open`, `KeyFromEnv`.
|
||
|
||
- [ ] **Step 3: Write the implementation**
|
||
|
||
Create `internal/crypto/crypto.go`:
|
||
```go
|
||
// Package crypto provides AES-256-GCM field encryption keyed from EMCLI_KEY.
|
||
package crypto
|
||
|
||
import (
|
||
"crypto/aes"
|
||
"crypto/cipher"
|
||
"crypto/rand"
|
||
"encoding/base64"
|
||
"errors"
|
||
"io"
|
||
"os"
|
||
)
|
||
|
||
var (
|
||
ErrNoKey = errors.New("EMCLI_KEY is not set")
|
||
ErrBadKey = errors.New("EMCLI_KEY must be base64 of exactly 32 bytes")
|
||
)
|
||
|
||
// KeyFromEnv reads and validates the AES-256 key from EMCLI_KEY.
|
||
func KeyFromEnv() ([]byte, error) {
|
||
raw := os.Getenv("EMCLI_KEY")
|
||
if raw == "" {
|
||
return nil, ErrNoKey
|
||
}
|
||
key, err := base64.StdEncoding.DecodeString(raw)
|
||
if err != nil || len(key) != 32 {
|
||
return nil, ErrBadKey
|
||
}
|
||
return key, nil
|
||
}
|
||
|
||
func newGCM(key []byte) (cipher.AEAD, error) {
|
||
block, err := aes.NewCipher(key)
|
||
if err != nil {
|
||
return nil, err
|
||
}
|
||
return cipher.NewGCM(block)
|
||
}
|
||
|
||
// Seal encrypts plaintext, returning nonce||ciphertext.
|
||
func Seal(key, plaintext []byte) ([]byte, error) {
|
||
gcm, err := newGCM(key)
|
||
if err != nil {
|
||
return nil, err
|
||
}
|
||
nonce := make([]byte, gcm.NonceSize())
|
||
if _, err := io.ReadFull(rand.Reader, nonce); err != nil {
|
||
return nil, err
|
||
}
|
||
return gcm.Seal(nonce, nonce, plaintext, nil), nil
|
||
}
|
||
|
||
// Open reverses Seal. A wrong key or tampered blob returns an error.
|
||
func Open(key, blob []byte) ([]byte, error) {
|
||
gcm, err := newGCM(key)
|
||
if err != nil {
|
||
return nil, err
|
||
}
|
||
ns := gcm.NonceSize()
|
||
if len(blob) < ns {
|
||
return nil, errors.New("ciphertext too short")
|
||
}
|
||
nonce, ct := blob[:ns], blob[ns:]
|
||
return gcm.Open(nil, nonce, ct, nil)
|
||
}
|
||
```
|
||
|
||
- [ ] **Step 4: Run tests to verify they pass**
|
||
|
||
Run: `go test ./internal/crypto/`
|
||
Expected: PASS (4 tests).
|
||
|
||
- [ ] **Step 5: Commit**
|
||
|
||
```bash
|
||
git add internal/crypto
|
||
git commit -m "feat(crypto): AES-256-GCM field encryption keyed from EMCLI_KEY"
|
||
```
|
||
|
||
---
|
||
|
||
### Task 3: store — open DB, schema, migrations, settings
|
||
|
||
**Files:**
|
||
- Create: `internal/store/store.go`
|
||
- Create: `internal/store/schema.go`
|
||
- Create: `internal/store/settings.go`
|
||
- Test: `internal/store/store_test.go`
|
||
|
||
**Interfaces:**
|
||
- Consumes: `crypto` (held for later tasks).
|
||
- Produces:
|
||
- `type Store struct { ... }` wrapping `*sql.DB` and the key.
|
||
- `func Open(path string, key []byte) (*Store, error)` — opens/creates DB, runs migrations.
|
||
- `func (s *Store) Close() error`.
|
||
- `func (s *Store) GetSetting(name string) (string, error)` / `SetSetting(name, value string) error`.
|
||
- `func DefaultDBPath() (string, error)` — resolves `EMCLI_DB` or the per-OS default.
|
||
- Const `schemaVersion = 1`.
|
||
|
||
- [ ] **Step 1: Add the SQLite dependency**
|
||
|
||
Run: `go get modernc.org/sqlite@latest`
|
||
|
||
- [ ] **Step 2: Write the failing test**
|
||
|
||
Create `internal/store/store_test.go`:
|
||
```go
|
||
package store
|
||
|
||
import (
|
||
"path/filepath"
|
||
"testing"
|
||
)
|
||
|
||
func testKey() []byte {
|
||
k := make([]byte, 32)
|
||
for i := range k {
|
||
k[i] = byte(i)
|
||
}
|
||
return k
|
||
}
|
||
|
||
// openTemp opens a fresh store in a temp dir.
|
||
func openTemp(t *testing.T) *Store {
|
||
t.Helper()
|
||
p := filepath.Join(t.TempDir(), "emcli.db")
|
||
s, err := Open(p, testKey())
|
||
if err != nil {
|
||
t.Fatalf("Open: %v", err)
|
||
}
|
||
t.Cleanup(func() { s.Close() })
|
||
return s
|
||
}
|
||
|
||
func TestOpenCreatesSchemaAndIsIdempotent(t *testing.T) {
|
||
p := filepath.Join(t.TempDir(), "emcli.db")
|
||
s, err := Open(p, testKey())
|
||
if err != nil {
|
||
t.Fatalf("first Open: %v", err)
|
||
}
|
||
v, err := s.GetSetting("schema_version")
|
||
if err != nil || v != "1" {
|
||
t.Fatalf("schema_version: %q err=%v", v, err)
|
||
}
|
||
s.Close()
|
||
|
||
// Re-open: must not error or duplicate.
|
||
s2, err := Open(p, testKey())
|
||
if err != nil {
|
||
t.Fatalf("second Open: %v", err)
|
||
}
|
||
defer s2.Close()
|
||
if v, _ := s2.GetSetting("schema_version"); v != "1" {
|
||
t.Fatalf("schema_version after reopen: %q", v)
|
||
}
|
||
}
|
||
|
||
func TestSettingsRoundTrip(t *testing.T) {
|
||
s := openTemp(t)
|
||
if err := s.SetSetting("audit_retention_days", "30"); err != nil {
|
||
t.Fatalf("SetSetting: %v", err)
|
||
}
|
||
got, err := s.GetSetting("audit_retention_days")
|
||
if err != nil || got != "30" {
|
||
t.Fatalf("got %q err=%v", got, err)
|
||
}
|
||
// Upsert overwrites.
|
||
_ = s.SetSetting("audit_retention_days", "7")
|
||
if got, _ := s.GetSetting("audit_retention_days"); got != "7" {
|
||
t.Fatalf("upsert failed: %q", got)
|
||
}
|
||
}
|
||
```
|
||
|
||
- [ ] **Step 3: Run test to verify it fails**
|
||
|
||
Run: `go test ./internal/store/`
|
||
Expected: FAIL — undefined `Open`.
|
||
|
||
- [ ] **Step 4: Write the schema**
|
||
|
||
Create `internal/store/schema.go`:
|
||
```go
|
||
package store
|
||
|
||
const schemaVersion = 1
|
||
|
||
// schemaSQL is the full v1 schema. All statements are idempotent via IF NOT EXISTS.
|
||
const schemaSQL = `
|
||
CREATE TABLE IF NOT EXISTS settings (
|
||
key TEXT PRIMARY KEY,
|
||
value TEXT NOT NULL
|
||
);
|
||
|
||
CREATE TABLE IF NOT EXISTS accounts (
|
||
id INTEGER PRIMARY KEY AUTOINCREMENT,
|
||
name TEXT UNIQUE NOT NULL,
|
||
mode TEXT NOT NULL CHECK (mode IN ('RO','RW')),
|
||
imap_host TEXT NOT NULL,
|
||
imap_port INTEGER NOT NULL,
|
||
imap_security TEXT NOT NULL CHECK (imap_security IN ('tls','starttls')),
|
||
smtp_host TEXT,
|
||
smtp_port INTEGER,
|
||
smtp_security TEXT,
|
||
auth_type TEXT NOT NULL CHECK (auth_type IN ('password','oauth2')),
|
||
username TEXT NOT NULL,
|
||
enc_password BLOB,
|
||
enc_oauth_client_id BLOB,
|
||
enc_oauth_client_secret BLOB,
|
||
enc_oauth_refresh_token BLOB,
|
||
whitelist_in_enabled INTEGER NOT NULL DEFAULT 0,
|
||
whitelist_out_enabled INTEGER NOT NULL DEFAULT 0,
|
||
subject_regex TEXT,
|
||
process_backlog INTEGER NOT NULL DEFAULT 0
|
||
);
|
||
|
||
CREATE TABLE IF NOT EXISTS whitelist_in (
|
||
account_id INTEGER NOT NULL REFERENCES accounts(id) ON DELETE CASCADE,
|
||
address TEXT NOT NULL,
|
||
PRIMARY KEY (account_id, address)
|
||
);
|
||
|
||
CREATE TABLE IF NOT EXISTS whitelist_out (
|
||
account_id INTEGER NOT NULL REFERENCES accounts(id) ON DELETE CASCADE,
|
||
address TEXT NOT NULL,
|
||
PRIMARY KEY (account_id, address)
|
||
);
|
||
|
||
CREATE TABLE IF NOT EXISTS folder_state (
|
||
account_id INTEGER NOT NULL REFERENCES accounts(id) ON DELETE CASCADE,
|
||
folder TEXT NOT NULL,
|
||
uidvalidity INTEGER NOT NULL,
|
||
floor_uid INTEGER NOT NULL,
|
||
PRIMARY KEY (account_id, folder)
|
||
);
|
||
|
||
CREATE TABLE IF NOT EXISTS acked (
|
||
account_id INTEGER NOT NULL REFERENCES accounts(id) ON DELETE CASCADE,
|
||
folder TEXT NOT NULL,
|
||
uidvalidity INTEGER NOT NULL,
|
||
uid INTEGER NOT NULL,
|
||
PRIMARY KEY (account_id, folder, uid)
|
||
);
|
||
|
||
CREATE TABLE IF NOT EXISTS audit_log (
|
||
id INTEGER PRIMARY KEY AUTOINCREMENT,
|
||
ts TEXT NOT NULL,
|
||
account TEXT NOT NULL,
|
||
action TEXT NOT NULL,
|
||
target TEXT NOT NULL,
|
||
result TEXT NOT NULL,
|
||
reason TEXT
|
||
);
|
||
`
|
||
```
|
||
|
||
- [ ] **Step 5: Write the store core and settings**
|
||
|
||
Create `internal/store/store.go`:
|
||
```go
|
||
// Package store owns the encrypted SQLite config and read state.
|
||
package store
|
||
|
||
import (
|
||
"database/sql"
|
||
"fmt"
|
||
"os"
|
||
"path/filepath"
|
||
"runtime"
|
||
"strconv"
|
||
|
||
_ "modernc.org/sqlite"
|
||
)
|
||
|
||
// Store wraps the database and the field-encryption key.
|
||
type Store struct {
|
||
db *sql.DB
|
||
key []byte
|
||
}
|
||
|
||
// Open opens (creating if needed) the DB at path and applies the schema.
|
||
func Open(path string, key []byte) (*Store, error) {
|
||
if err := os.MkdirAll(filepath.Dir(path), 0o700); err != nil {
|
||
return nil, fmt.Errorf("create db dir: %w", err)
|
||
}
|
||
db, err := sql.Open("sqlite", path)
|
||
if err != nil {
|
||
return nil, err
|
||
}
|
||
if _, err := db.Exec("PRAGMA foreign_keys = ON;"); err != nil {
|
||
db.Close()
|
||
return nil, err
|
||
}
|
||
if _, err := db.Exec(schemaSQL); err != nil {
|
||
db.Close()
|
||
return nil, fmt.Errorf("apply schema: %w", err)
|
||
}
|
||
s := &Store{db: db, key: key}
|
||
if _, err := s.GetSetting("schema_version"); err != nil {
|
||
if err := s.SetSetting("schema_version", strconv.Itoa(schemaVersion)); err != nil {
|
||
db.Close()
|
||
return nil, err
|
||
}
|
||
}
|
||
return s, nil
|
||
}
|
||
|
||
func (s *Store) Close() error { return s.db.Close() }
|
||
|
||
// DefaultDBPath resolves EMCLI_DB or the per-OS default location.
|
||
func DefaultDBPath() (string, error) {
|
||
if p := os.Getenv("EMCLI_DB"); p != "" {
|
||
return p, nil
|
||
}
|
||
if runtime.GOOS == "windows" {
|
||
if dir := os.Getenv("AppData"); dir != "" {
|
||
return filepath.Join(dir, "emcli", "emcli.db"), nil
|
||
}
|
||
}
|
||
home, err := os.UserHomeDir()
|
||
if err != nil {
|
||
return "", err
|
||
}
|
||
return filepath.Join(home, ".config", "emcli", "emcli.db"), nil
|
||
}
|
||
```
|
||
|
||
Create `internal/store/settings.go`:
|
||
```go
|
||
package store
|
||
|
||
import "database/sql"
|
||
|
||
// GetSetting returns a setting value or sql.ErrNoRows if absent.
|
||
func (s *Store) GetSetting(name string) (string, error) {
|
||
var v string
|
||
err := s.db.QueryRow("SELECT value FROM settings WHERE key = ?", name).Scan(&v)
|
||
return v, err
|
||
}
|
||
|
||
// SetSetting upserts a setting.
|
||
func (s *Store) SetSetting(name, value string) error {
|
||
_, err := s.db.Exec(
|
||
"INSERT INTO settings(key,value) VALUES(?,?) "+
|
||
"ON CONFLICT(key) DO UPDATE SET value=excluded.value",
|
||
name, value)
|
||
return err
|
||
}
|
||
|
||
var _ = sql.ErrNoRows
|
||
```
|
||
|
||
- [ ] **Step 6: Run tests to verify they pass**
|
||
|
||
Run: `go test ./internal/store/`
|
||
Expected: PASS (2 tests).
|
||
|
||
- [ ] **Step 7: Commit**
|
||
|
||
```bash
|
||
git add internal/store go.mod go.sum
|
||
git commit -m "feat(store): open encrypted SQLite, schema v1, settings"
|
||
```
|
||
|
||
---
|
||
|
||
### Task 4: store — accounts CRUD with encrypted secrets
|
||
|
||
**Files:**
|
||
- Create: `internal/store/account.go`
|
||
- Test: `internal/store/account_test.go`
|
||
|
||
**Interfaces:**
|
||
- Consumes: `crypto.Seal`/`crypto.Open`, `Store`.
|
||
- Produces:
|
||
- `type Account struct { ID int64; Name, Mode, IMAPHost string; IMAPPort int; IMAPSecurity, AuthType, Username string; Password string; WhitelistInEnabled, WhitelistOutEnabled bool; SubjectRegex string; ProcessBacklog bool }`
|
||
- `func (s *Store) AddAccount(a Account) (int64, error)` — encrypts `Password` into `enc_password`.
|
||
- `func (s *Store) GetAccount(name string) (Account, error)` — decrypts secrets; `ErrAccountNotFound` if absent.
|
||
- `func (s *Store) ListAccounts() ([]Account, error)` — `Password` field left empty (never listed).
|
||
- `func (s *Store) DeleteAccount(name string) error`.
|
||
- `var ErrAccountNotFound error`.
|
||
|
||
- [ ] **Step 1: Write the failing test**
|
||
|
||
Create `internal/store/account_test.go`:
|
||
```go
|
||
package store
|
||
|
||
import (
|
||
"errors"
|
||
"testing"
|
||
)
|
||
|
||
func sampleAccount() Account {
|
||
return Account{
|
||
Name: "work", Mode: "RO",
|
||
IMAPHost: "imap.example.com", IMAPPort: 993, IMAPSecurity: "tls",
|
||
AuthType: "password", Username: "me@example.com",
|
||
Password: "s3cr3t", SubjectRegex: "",
|
||
}
|
||
}
|
||
|
||
func TestAddGetAccountDecryptsSecret(t *testing.T) {
|
||
s := openTemp(t)
|
||
id, err := s.AddAccount(sampleAccount())
|
||
if err != nil {
|
||
t.Fatalf("AddAccount: %v", err)
|
||
}
|
||
if id == 0 {
|
||
t.Fatal("want non-zero id")
|
||
}
|
||
got, err := s.GetAccount("work")
|
||
if err != nil {
|
||
t.Fatalf("GetAccount: %v", err)
|
||
}
|
||
if got.Password != "s3cr3t" {
|
||
t.Fatalf("password not decrypted: %q", got.Password)
|
||
}
|
||
if got.Mode != "RO" || got.IMAPPort != 993 {
|
||
t.Fatalf("fields wrong: %+v", got)
|
||
}
|
||
}
|
||
|
||
func TestPasswordStoredEncrypted(t *testing.T) {
|
||
s := openTemp(t)
|
||
_, _ = s.AddAccount(sampleAccount())
|
||
var blob []byte
|
||
if err := s.db.QueryRow("SELECT enc_password FROM accounts WHERE name='work'").Scan(&blob); err != nil {
|
||
t.Fatalf("query: %v", err)
|
||
}
|
||
if string(blob) == "s3cr3t" || len(blob) == 0 {
|
||
t.Fatalf("password not encrypted at rest: %q", blob)
|
||
}
|
||
}
|
||
|
||
func TestGetAccountNotFound(t *testing.T) {
|
||
s := openTemp(t)
|
||
if _, err := s.GetAccount("nope"); !errors.Is(err, ErrAccountNotFound) {
|
||
t.Fatalf("want ErrAccountNotFound, got %v", err)
|
||
}
|
||
}
|
||
|
||
func TestListAccountsOmitsSecrets(t *testing.T) {
|
||
s := openTemp(t)
|
||
_, _ = s.AddAccount(sampleAccount())
|
||
list, err := s.ListAccounts()
|
||
if err != nil || len(list) != 1 {
|
||
t.Fatalf("list: %v len=%d", err, len(list))
|
||
}
|
||
if list[0].Password != "" {
|
||
t.Fatal("ListAccounts must not return secrets")
|
||
}
|
||
}
|
||
```
|
||
|
||
- [ ] **Step 2: Run test to verify it fails**
|
||
|
||
Run: `go test ./internal/store/ -run Account`
|
||
Expected: FAIL — undefined `Account`/`AddAccount`.
|
||
|
||
- [ ] **Step 3: Write the implementation**
|
||
|
||
Create `internal/store/account.go`:
|
||
```go
|
||
package store
|
||
|
||
import (
|
||
"database/sql"
|
||
"errors"
|
||
"fmt"
|
||
|
||
"git.dcglab.co.uk/steve/emcli/internal/crypto"
|
||
)
|
||
|
||
var ErrAccountNotFound = errors.New("account not found")
|
||
|
||
// Account is the decrypted, in-memory view of a configured account.
|
||
type Account struct {
|
||
ID int64
|
||
Name string
|
||
Mode string // RO | RW
|
||
IMAPHost string
|
||
IMAPPort int
|
||
IMAPSecurity string // tls | starttls
|
||
AuthType string // password | oauth2
|
||
Username string
|
||
Password string // decrypted; empty in ListAccounts
|
||
WhitelistInEnabled bool
|
||
WhitelistOutEnabled bool
|
||
SubjectRegex string
|
||
ProcessBacklog bool
|
||
}
|
||
|
||
func (s *Store) AddAccount(a Account) (int64, error) {
|
||
var encPw []byte
|
||
if a.Password != "" {
|
||
b, err := crypto.Seal(s.key, []byte(a.Password))
|
||
if err != nil {
|
||
return 0, err
|
||
}
|
||
encPw = b
|
||
}
|
||
res, err := s.db.Exec(`
|
||
INSERT INTO accounts
|
||
(name,mode,imap_host,imap_port,imap_security,auth_type,username,
|
||
enc_password,whitelist_in_enabled,whitelist_out_enabled,subject_regex,process_backlog)
|
||
VALUES (?,?,?,?,?,?,?,?,?,?,?,?)`,
|
||
a.Name, a.Mode, a.IMAPHost, a.IMAPPort, a.IMAPSecurity, a.AuthType, a.Username,
|
||
encPw, b2i(a.WhitelistInEnabled), b2i(a.WhitelistOutEnabled),
|
||
nullStr(a.SubjectRegex), b2i(a.ProcessBacklog))
|
||
if err != nil {
|
||
return 0, fmt.Errorf("insert account: %w", err)
|
||
}
|
||
return res.LastInsertId()
|
||
}
|
||
|
||
func (s *Store) GetAccount(name string) (Account, error) {
|
||
row := s.db.QueryRow(`
|
||
SELECT id,name,mode,imap_host,imap_port,imap_security,auth_type,username,
|
||
enc_password,whitelist_in_enabled,whitelist_out_enabled,subject_regex,process_backlog
|
||
FROM accounts WHERE name = ?`, name)
|
||
a, encPw, err := scanAccount(row)
|
||
if errors.Is(err, sql.ErrNoRows) {
|
||
return Account{}, ErrAccountNotFound
|
||
}
|
||
if err != nil {
|
||
return Account{}, err
|
||
}
|
||
if len(encPw) > 0 {
|
||
pw, err := crypto.Open(s.key, encPw)
|
||
if err != nil {
|
||
return Account{}, fmt.Errorf("decrypt password (wrong EMCLI_KEY?): %w", err)
|
||
}
|
||
a.Password = string(pw)
|
||
}
|
||
return a, nil
|
||
}
|
||
|
||
func (s *Store) ListAccounts() ([]Account, error) {
|
||
rows, err := s.db.Query(`
|
||
SELECT id,name,mode,imap_host,imap_port,imap_security,auth_type,username,
|
||
enc_password,whitelist_in_enabled,whitelist_out_enabled,subject_regex,process_backlog
|
||
FROM accounts ORDER BY name`)
|
||
if err != nil {
|
||
return nil, err
|
||
}
|
||
defer rows.Close()
|
||
var out []Account
|
||
for rows.Next() {
|
||
a, _, err := scanAccount(rows) // secrets discarded
|
||
if err != nil {
|
||
return nil, err
|
||
}
|
||
out = append(out, a)
|
||
}
|
||
return out, rows.Err()
|
||
}
|
||
|
||
func (s *Store) DeleteAccount(name string) error {
|
||
res, err := s.db.Exec("DELETE FROM accounts WHERE name = ?", name)
|
||
if err != nil {
|
||
return err
|
||
}
|
||
if n, _ := res.RowsAffected(); n == 0 {
|
||
return ErrAccountNotFound
|
||
}
|
||
return nil
|
||
}
|
||
|
||
// scanner is satisfied by *sql.Row and *sql.Rows.
|
||
type scanner interface{ Scan(dest ...any) error }
|
||
|
||
func scanAccount(sc scanner) (Account, []byte, error) {
|
||
var (
|
||
a Account
|
||
encPw []byte
|
||
subj sql.NullString
|
||
wlIn, wlOut int
|
||
backlog int
|
||
)
|
||
err := sc.Scan(&a.ID, &a.Name, &a.Mode, &a.IMAPHost, &a.IMAPPort, &a.IMAPSecurity,
|
||
&a.AuthType, &a.Username, &encPw, &wlIn, &wlOut, &subj, &backlog)
|
||
if err != nil {
|
||
return Account{}, nil, err
|
||
}
|
||
a.WhitelistInEnabled = wlIn != 0
|
||
a.WhitelistOutEnabled = wlOut != 0
|
||
a.ProcessBacklog = backlog != 0
|
||
a.SubjectRegex = subj.String
|
||
return a, encPw, nil
|
||
}
|
||
|
||
func b2i(b bool) int {
|
||
if b {
|
||
return 1
|
||
}
|
||
return 0
|
||
}
|
||
|
||
func nullStr(s string) any {
|
||
if s == "" {
|
||
return nil
|
||
}
|
||
return s
|
||
}
|
||
```
|
||
|
||
- [ ] **Step 4: Run tests to verify they pass**
|
||
|
||
Run: `go test ./internal/store/ -run Account`
|
||
Expected: PASS (4 tests).
|
||
|
||
- [ ] **Step 5: Commit**
|
||
|
||
```bash
|
||
git add internal/store/account.go internal/store/account_test.go
|
||
git commit -m "feat(store): accounts CRUD with encrypted password column"
|
||
```
|
||
|
||
---
|
||
|
||
### Task 5: store — whitelist CRUD
|
||
|
||
**Files:**
|
||
- Create: `internal/store/whitelist.go`
|
||
- Test: `internal/store/whitelist_test.go`
|
||
|
||
**Interfaces:**
|
||
- Consumes: `Store`, `GetAccount`.
|
||
- Produces:
|
||
- `type Direction string` with `DirIn Direction = "in"`, `DirOut Direction = "out"`.
|
||
- `func (s *Store) AddWhitelist(account string, dir Direction, address string) error`
|
||
- `func (s *Store) RemoveWhitelist(account string, dir Direction, address string) error`
|
||
- `func (s *Store) ListWhitelist(account string, dir Direction) ([]string, error)` — addresses lower-cased.
|
||
|
||
- [ ] **Step 1: Write the failing test**
|
||
|
||
Create `internal/store/whitelist_test.go`:
|
||
```go
|
||
package store
|
||
|
||
import (
|
||
"reflect"
|
||
"testing"
|
||
)
|
||
|
||
func TestWhitelistAddListRemove(t *testing.T) {
|
||
s := openTemp(t)
|
||
_, _ = s.AddAccount(sampleAccount())
|
||
|
||
if err := s.AddWhitelist("work", DirIn, "Bob@Example.com"); err != nil {
|
||
t.Fatalf("add: %v", err)
|
||
}
|
||
_ = s.AddWhitelist("work", DirIn, "@trusted.com")
|
||
|
||
got, err := s.ListWhitelist("work", DirIn)
|
||
if err != nil {
|
||
t.Fatalf("list: %v", err)
|
||
}
|
||
want := []string{"@trusted.com", "bob@example.com"} // lower-cased, sorted
|
||
if !reflect.DeepEqual(got, want) {
|
||
t.Fatalf("got %v want %v", got, want)
|
||
}
|
||
|
||
// Out list is independent.
|
||
if out, _ := s.ListWhitelist("work", DirOut); len(out) != 0 {
|
||
t.Fatalf("out list should be empty, got %v", out)
|
||
}
|
||
|
||
if err := s.RemoveWhitelist("work", DirIn, "bob@example.com"); err != nil {
|
||
t.Fatalf("remove: %v", err)
|
||
}
|
||
got, _ = s.ListWhitelist("work", DirIn)
|
||
if !reflect.DeepEqual(got, []string{"@trusted.com"}) {
|
||
t.Fatalf("after remove got %v", got)
|
||
}
|
||
}
|
||
```
|
||
|
||
- [ ] **Step 2: Run test to verify it fails**
|
||
|
||
Run: `go test ./internal/store/ -run Whitelist`
|
||
Expected: FAIL — undefined `DirIn`.
|
||
|
||
- [ ] **Step 3: Write the implementation**
|
||
|
||
Create `internal/store/whitelist.go`:
|
||
```go
|
||
package store
|
||
|
||
import (
|
||
"fmt"
|
||
"strings"
|
||
)
|
||
|
||
type Direction string
|
||
|
||
const (
|
||
DirIn Direction = "in"
|
||
DirOut Direction = "out"
|
||
)
|
||
|
||
func (d Direction) table() (string, error) {
|
||
switch d {
|
||
case DirIn:
|
||
return "whitelist_in", nil
|
||
case DirOut:
|
||
return "whitelist_out", nil
|
||
default:
|
||
return "", fmt.Errorf("invalid direction %q", d)
|
||
}
|
||
}
|
||
|
||
func (s *Store) accountID(name string) (int64, error) {
|
||
a, err := s.GetAccount(name)
|
||
if err != nil {
|
||
return 0, err
|
||
}
|
||
return a.ID, nil
|
||
}
|
||
|
||
func (s *Store) AddWhitelist(account string, dir Direction, address string) error {
|
||
tbl, err := dir.table()
|
||
if err != nil {
|
||
return err
|
||
}
|
||
id, err := s.accountID(account)
|
||
if err != nil {
|
||
return err
|
||
}
|
||
_, err = s.db.Exec(
|
||
fmt.Sprintf("INSERT OR IGNORE INTO %s(account_id,address) VALUES(?,?)", tbl),
|
||
id, strings.ToLower(address))
|
||
return err
|
||
}
|
||
|
||
func (s *Store) RemoveWhitelist(account string, dir Direction, address string) error {
|
||
tbl, err := dir.table()
|
||
if err != nil {
|
||
return err
|
||
}
|
||
id, err := s.accountID(account)
|
||
if err != nil {
|
||
return err
|
||
}
|
||
_, err = s.db.Exec(
|
||
fmt.Sprintf("DELETE FROM %s WHERE account_id=? AND address=?", tbl),
|
||
id, strings.ToLower(address))
|
||
return err
|
||
}
|
||
|
||
func (s *Store) ListWhitelist(account string, dir Direction) ([]string, error) {
|
||
tbl, err := dir.table()
|
||
if err != nil {
|
||
return nil, err
|
||
}
|
||
id, err := s.accountID(account)
|
||
if err != nil {
|
||
return nil, err
|
||
}
|
||
rows, err := s.db.Query(
|
||
fmt.Sprintf("SELECT address FROM %s WHERE account_id=? ORDER BY address", tbl), id)
|
||
if err != nil {
|
||
return nil, err
|
||
}
|
||
defer rows.Close()
|
||
out := []string{}
|
||
for rows.Next() {
|
||
var a string
|
||
if err := rows.Scan(&a); err != nil {
|
||
return nil, err
|
||
}
|
||
out = append(out, a)
|
||
}
|
||
return out, rows.Err()
|
||
}
|
||
```
|
||
|
||
- [ ] **Step 4: Run tests to verify they pass**
|
||
|
||
Run: `go test ./internal/store/ -run Whitelist`
|
||
Expected: PASS.
|
||
|
||
- [ ] **Step 5: Commit**
|
||
|
||
```bash
|
||
git add internal/store/whitelist.go internal/store/whitelist_test.go
|
||
git commit -m "feat(store): per-account inbound/outbound whitelist CRUD"
|
||
```
|
||
|
||
---
|
||
|
||
### Task 6: store — seen-set read state (floor, ack, compaction)
|
||
|
||
This is the algorithmic core. A folder's read state is a `floor_uid` plus a set of acked UIDs above it. Compaction collapses contiguous acked runs immediately above the floor.
|
||
|
||
**Files:**
|
||
- Create: `internal/store/seenset.go`
|
||
- Test: `internal/store/seenset_test.go`
|
||
|
||
**Interfaces:**
|
||
- Consumes: `Store`, `accountID`.
|
||
- Produces:
|
||
- `func (s *Store) EnsureFolderBaseline(account, folder string, uidvalidity, maxUID uint32) error` — on first contact creates `folder_state` with `floor_uid = maxUID` (or 0 if account.ProcessBacklog); on `uidvalidity` change, resets state and re-baselines.
|
||
- `func (s *Store) IsNew(account, folder string, uid uint32) (bool, error)` — `uid > floor AND uid ∉ acked`.
|
||
- `func (s *Store) FilterNew(account, folder string, uids []uint32) ([]uint32, error)` — convenience, preserves order.
|
||
- `func (s *Store) Ack(account, folder string, uidvalidity uint32, uids ...uint32) error` — inserts into `acked` (ignoring ≤ floor), then compacts.
|
||
- `func (s *Store) floor(account, folder string) (uint32, error)` — unexported helper used by tests via exported wrappers.
|
||
|
||
- [ ] **Step 1: Write the failing tests**
|
||
|
||
Create `internal/store/seenset_test.go`:
|
||
```go
|
||
package store
|
||
|
||
import "testing"
|
||
|
||
func backlogAccount(t *testing.T, s *Store, backlog bool) {
|
||
t.Helper()
|
||
a := sampleAccount()
|
||
a.ProcessBacklog = backlog
|
||
if _, err := s.AddAccount(a); err != nil {
|
||
t.Fatalf("AddAccount: %v", err)
|
||
}
|
||
}
|
||
|
||
func TestBaselineIgnoresHistoryByDefault(t *testing.T) {
|
||
s := openTemp(t)
|
||
backlogAccount(t, s, false)
|
||
if err := s.EnsureFolderBaseline("work", "INBOX", 1, 100); err != nil {
|
||
t.Fatalf("baseline: %v", err)
|
||
}
|
||
// Existing mail (uid <= 100) is not new; 101 is.
|
||
for _, tc := range []struct {
|
||
uid uint32
|
||
want bool
|
||
}{{50, false}, {100, false}, {101, true}} {
|
||
got, _ := s.IsNew("work", "INBOX", tc.uid)
|
||
if got != tc.want {
|
||
t.Fatalf("IsNew(%d)=%v want %v", tc.uid, got, tc.want)
|
||
}
|
||
}
|
||
}
|
||
|
||
func TestBaselineBacklogProcessesAll(t *testing.T) {
|
||
s := openTemp(t)
|
||
backlogAccount(t, s, true)
|
||
_ = s.EnsureFolderBaseline("work", "INBOX", 1, 100)
|
||
if n, _ := s.IsNew("work", "INBOX", 1); !n {
|
||
t.Fatal("with backlog, uid 1 must be new")
|
||
}
|
||
}
|
||
|
||
func TestAckRemovesFromNewAndIsOutOfOrderSafe(t *testing.T) {
|
||
s := openTemp(t)
|
||
backlogAccount(t, s, false)
|
||
_ = s.EnsureFolderBaseline("work", "INBOX", 1, 100)
|
||
// New mail arrives: 101..105. Ack out of order: 103 then 101.
|
||
_ = s.Ack("work", "INBOX", 1, 103)
|
||
if n, _ := s.IsNew("work", "INBOX", 103); n {
|
||
t.Fatal("103 acked, must not be new")
|
||
}
|
||
if n, _ := s.IsNew("work", "INBOX", 101); !n {
|
||
t.Fatal("101 not acked yet, must still be new")
|
||
}
|
||
_ = s.Ack("work", "INBOX", 1, 101, 102)
|
||
got, _ := s.FilterNew("work", "INBOX", []uint32{101, 102, 103, 104, 105})
|
||
want := []uint32{104, 105}
|
||
if len(got) != 2 || got[0] != want[0] || got[1] != want[1] {
|
||
t.Fatalf("FilterNew got %v want %v", got, want)
|
||
}
|
||
}
|
||
|
||
func TestCompactionAdvancesFloor(t *testing.T) {
|
||
s := openTemp(t)
|
||
backlogAccount(t, s, false)
|
||
_ = s.EnsureFolderBaseline("work", "INBOX", 1, 100)
|
||
// Ack a contiguous run just above the floor: 101,102,103.
|
||
_ = s.Ack("work", "INBOX", 1, 102, 101, 103)
|
||
f, _ := s.floor("work", "INBOX")
|
||
if f != 103 {
|
||
t.Fatalf("floor should advance to 103, got %d", f)
|
||
}
|
||
// The acked rows for the collapsed run are gone.
|
||
var n int
|
||
_ = s.db.QueryRow("SELECT COUNT(*) FROM acked").Scan(&n)
|
||
if n != 0 {
|
||
t.Fatalf("acked rows should be compacted away, got %d", n)
|
||
}
|
||
// A hole remains uncompacted: ack 105 (gap at 104).
|
||
_ = s.Ack("work", "INBOX", 1, 105)
|
||
f, _ = s.floor("work", "INBOX")
|
||
if f != 103 {
|
||
t.Fatalf("floor must stay at 103 while 104 is a hole, got %d", f)
|
||
}
|
||
_ = s.db.QueryRow("SELECT COUNT(*) FROM acked").Scan(&n)
|
||
if n != 1 {
|
||
t.Fatalf("105 should remain in acked, got %d rows", n)
|
||
}
|
||
}
|
||
|
||
func TestUIDValidityChangeResets(t *testing.T) {
|
||
s := openTemp(t)
|
||
backlogAccount(t, s, false)
|
||
_ = s.EnsureFolderBaseline("work", "INBOX", 1, 100)
|
||
_ = s.Ack("work", "INBOX", 1, 105)
|
||
// Server reports a new UIDVALIDITY: state resets, re-baselines at new max.
|
||
if err := s.EnsureFolderBaseline("work", "INBOX", 2, 10); err != nil {
|
||
t.Fatalf("rebaseline: %v", err)
|
||
}
|
||
f, _ := s.floor("work", "INBOX")
|
||
if f != 10 {
|
||
t.Fatalf("floor should re-baseline to 10, got %d", f)
|
||
}
|
||
var n int
|
||
_ = s.db.QueryRow("SELECT COUNT(*) FROM acked").Scan(&n)
|
||
if n != 0 {
|
||
t.Fatalf("acked must reset on uidvalidity change, got %d", n)
|
||
}
|
||
}
|
||
```
|
||
|
||
- [ ] **Step 2: Run tests to verify they fail**
|
||
|
||
Run: `go test ./internal/store/ -run 'Baseline|Ack|Compaction|UIDValidity'`
|
||
Expected: FAIL — undefined `EnsureFolderBaseline`.
|
||
|
||
- [ ] **Step 3: Write the implementation**
|
||
|
||
Create `internal/store/seenset.go`:
|
||
```go
|
||
package store
|
||
|
||
import "database/sql"
|
||
|
||
// EnsureFolderBaseline initialises folder_state on first contact, or resets it
|
||
// when the server's UIDVALIDITY differs from what we stored.
|
||
func (s *Store) EnsureFolderBaseline(account, folder string, uidvalidity, maxUID uint32) error {
|
||
a, err := s.GetAccount(account)
|
||
if err != nil {
|
||
return err
|
||
}
|
||
var (
|
||
curUIDValidity uint32
|
||
haveRow bool
|
||
)
|
||
row := s.db.QueryRow(
|
||
"SELECT uidvalidity FROM folder_state WHERE account_id=? AND folder=?", a.ID, folder)
|
||
switch err := row.Scan(&curUIDValidity); err {
|
||
case nil:
|
||
haveRow = true
|
||
case sql.ErrNoRows:
|
||
haveRow = false
|
||
default:
|
||
return err
|
||
}
|
||
if haveRow && curUIDValidity == uidvalidity {
|
||
return nil // already baselined, same validity
|
||
}
|
||
|
||
floor := maxUID
|
||
if a.ProcessBacklog {
|
||
floor = 0
|
||
}
|
||
tx, err := s.db.Begin()
|
||
if err != nil {
|
||
return err
|
||
}
|
||
defer tx.Rollback()
|
||
if _, err := tx.Exec("DELETE FROM acked WHERE account_id=? AND folder=?", a.ID, folder); err != nil {
|
||
return err
|
||
}
|
||
if _, err := tx.Exec(`
|
||
INSERT INTO folder_state(account_id,folder,uidvalidity,floor_uid)
|
||
VALUES(?,?,?,?)
|
||
ON CONFLICT(account_id,folder) DO UPDATE SET uidvalidity=excluded.uidvalidity, floor_uid=excluded.floor_uid`,
|
||
a.ID, folder, uidvalidity, floor); err != nil {
|
||
return err
|
||
}
|
||
return tx.Commit()
|
||
}
|
||
|
||
func (s *Store) floor(account, folder string) (uint32, error) {
|
||
id, err := s.accountID(account)
|
||
if err != nil {
|
||
return 0, err
|
||
}
|
||
var f uint32
|
||
err = s.db.QueryRow(
|
||
"SELECT floor_uid FROM folder_state WHERE account_id=? AND folder=?", id, folder).Scan(&f)
|
||
return f, err
|
||
}
|
||
|
||
// IsNew reports whether uid is unread: above the floor and not acked.
|
||
func (s *Store) IsNew(account, folder string, uid uint32) (bool, error) {
|
||
id, err := s.accountID(account)
|
||
if err != nil {
|
||
return false, err
|
||
}
|
||
var floor uint32
|
||
if err := s.db.QueryRow(
|
||
"SELECT floor_uid FROM folder_state WHERE account_id=? AND folder=?", id, folder).Scan(&floor); err != nil {
|
||
return false, err
|
||
}
|
||
if uid <= floor {
|
||
return false, nil
|
||
}
|
||
var one int
|
||
err = s.db.QueryRow(
|
||
"SELECT 1 FROM acked WHERE account_id=? AND folder=? AND uid=?", id, folder, uid).Scan(&one)
|
||
if err == sql.ErrNoRows {
|
||
return true, nil
|
||
}
|
||
if err != nil {
|
||
return false, err
|
||
}
|
||
return false, nil // present in acked
|
||
}
|
||
|
||
// FilterNew returns the subset of uids that are new, preserving order.
|
||
func (s *Store) FilterNew(account, folder string, uids []uint32) ([]uint32, error) {
|
||
out := make([]uint32, 0, len(uids))
|
||
for _, u := range uids {
|
||
n, err := s.IsNew(account, folder, u)
|
||
if err != nil {
|
||
return nil, err
|
||
}
|
||
if n {
|
||
out = append(out, u)
|
||
}
|
||
}
|
||
return out, nil
|
||
}
|
||
|
||
// Ack records uids as processed (ignoring any at or below the floor) then compacts.
|
||
func (s *Store) Ack(account, folder string, uidvalidity uint32, uids ...uint32) error {
|
||
id, err := s.accountID(account)
|
||
if err != nil {
|
||
return err
|
||
}
|
||
tx, err := s.db.Begin()
|
||
if err != nil {
|
||
return err
|
||
}
|
||
defer tx.Rollback()
|
||
|
||
var floor uint32
|
||
if err := tx.QueryRow(
|
||
"SELECT floor_uid FROM folder_state WHERE account_id=? AND folder=?", id, folder).Scan(&floor); err != nil {
|
||
return err
|
||
}
|
||
for _, u := range uids {
|
||
if u <= floor {
|
||
continue
|
||
}
|
||
if _, err := tx.Exec(
|
||
"INSERT OR IGNORE INTO acked(account_id,folder,uidvalidity,uid) VALUES(?,?,?,?)",
|
||
id, folder, uidvalidity, u); err != nil {
|
||
return err
|
||
}
|
||
}
|
||
// Compact: while floor+1 is acked, advance floor and drop that row.
|
||
for {
|
||
next := floor + 1
|
||
var present int
|
||
err := tx.QueryRow(
|
||
"SELECT 1 FROM acked WHERE account_id=? AND folder=? AND uid=?", id, folder, next).Scan(&present)
|
||
if err == sql.ErrNoRows {
|
||
break
|
||
}
|
||
if err != nil {
|
||
return err
|
||
}
|
||
if _, err := tx.Exec(
|
||
"DELETE FROM acked WHERE account_id=? AND folder=? AND uid=?", id, folder, next); err != nil {
|
||
return err
|
||
}
|
||
floor = next
|
||
}
|
||
if _, err := tx.Exec(
|
||
"UPDATE folder_state SET floor_uid=? WHERE account_id=? AND folder=?", floor, id, folder); err != nil {
|
||
return err
|
||
}
|
||
return tx.Commit()
|
||
}
|
||
```
|
||
|
||
- [ ] **Step 4: Run tests to verify they pass**
|
||
|
||
Run: `go test ./internal/store/ -run 'Baseline|Ack|Compaction|UIDValidity'`
|
||
Expected: PASS (5 tests).
|
||
|
||
- [ ] **Step 5: Commit**
|
||
|
||
```bash
|
||
git add internal/store/seenset.go internal/store/seenset_test.go
|
||
git commit -m "feat(store): seen-set read state with floor baseline and compaction"
|
||
```
|
||
|
||
---
|
||
|
||
### Task 7: store — audit log & retention purge
|
||
|
||
**Files:**
|
||
- Create: `internal/store/audit.go`
|
||
- Test: `internal/store/audit_test.go`
|
||
|
||
**Interfaces:**
|
||
- Consumes: `Store`, `SetSetting`/`GetSetting`.
|
||
- Produces:
|
||
- `type AuditEntry struct { TS, Account, Action, Target, Result, Reason string }`
|
||
- `func (s *Store) Audit(now time.Time, e AuditEntry) error` — inserts; `now` injected for testability (no `time.Now()` inside).
|
||
- `func (s *Store) PurgeAudit(now time.Time) (int64, error)` — deletes rows older than `audit_retention_days` (no-op if unset/0).
|
||
- `func (s *Store) RecentAudit(limit int) ([]AuditEntry, error)`.
|
||
|
||
- [ ] **Step 1: Write the failing test**
|
||
|
||
Create `internal/store/audit_test.go`:
|
||
```go
|
||
package store
|
||
|
||
import (
|
||
"testing"
|
||
"time"
|
||
)
|
||
|
||
func TestAuditInsertAndRecent(t *testing.T) {
|
||
s := openTemp(t)
|
||
now := time.Date(2026, 6, 21, 12, 0, 0, 0, time.UTC)
|
||
err := s.Audit(now, AuditEntry{
|
||
Account: "work", Action: "list", Target: "INBOX", Result: "allowed",
|
||
})
|
||
if err != nil {
|
||
t.Fatalf("Audit: %v", err)
|
||
}
|
||
got, err := s.RecentAudit(10)
|
||
if err != nil || len(got) != 1 {
|
||
t.Fatalf("recent: %v len=%d", err, len(got))
|
||
}
|
||
if got[0].Action != "list" || got[0].Result != "allowed" {
|
||
t.Fatalf("entry wrong: %+v", got[0])
|
||
}
|
||
}
|
||
|
||
func TestPurgeRespectsRetention(t *testing.T) {
|
||
s := openTemp(t)
|
||
_ = s.SetSetting("audit_retention_days", "30")
|
||
now := time.Date(2026, 6, 21, 12, 0, 0, 0, time.UTC)
|
||
old := now.AddDate(0, 0, -31)
|
||
recent := now.AddDate(0, 0, -5)
|
||
_ = s.Audit(old, AuditEntry{Account: "a", Action: "list", Target: "INBOX", Result: "allowed"})
|
||
_ = s.Audit(recent, AuditEntry{Account: "a", Action: "list", Target: "INBOX", Result: "allowed"})
|
||
|
||
n, err := s.PurgeAudit(now)
|
||
if err != nil {
|
||
t.Fatalf("purge: %v", err)
|
||
}
|
||
if n != 1 {
|
||
t.Fatalf("want 1 purged, got %d", n)
|
||
}
|
||
got, _ := s.RecentAudit(10)
|
||
if len(got) != 1 {
|
||
t.Fatalf("want 1 remaining, got %d", len(got))
|
||
}
|
||
}
|
||
```
|
||
|
||
- [ ] **Step 2: Run test to verify it fails**
|
||
|
||
Run: `go test ./internal/store/ -run 'Audit|Purge'`
|
||
Expected: FAIL — undefined `AuditEntry`.
|
||
|
||
- [ ] **Step 3: Write the implementation**
|
||
|
||
Create `internal/store/audit.go`:
|
||
```go
|
||
package store
|
||
|
||
import (
|
||
"strconv"
|
||
"time"
|
||
)
|
||
|
||
type AuditEntry struct {
|
||
TS string
|
||
Account string
|
||
Action string
|
||
Target string
|
||
Result string
|
||
Reason string
|
||
}
|
||
|
||
func (s *Store) Audit(now time.Time, e AuditEntry) error {
|
||
var reason any
|
||
if e.Reason != "" {
|
||
reason = e.Reason
|
||
}
|
||
_, err := s.db.Exec(
|
||
"INSERT INTO audit_log(ts,account,action,target,result,reason) VALUES(?,?,?,?,?,?)",
|
||
now.UTC().Format(time.RFC3339), e.Account, e.Action, e.Target, e.Result, reason)
|
||
return err
|
||
}
|
||
|
||
func (s *Store) PurgeAudit(now time.Time) (int64, error) {
|
||
v, err := s.GetSetting("audit_retention_days")
|
||
if err != nil { // unset => no retention policy
|
||
return 0, nil
|
||
}
|
||
days, err := strconv.Atoi(v)
|
||
if err != nil || days <= 0 {
|
||
return 0, nil
|
||
}
|
||
cutoff := now.UTC().AddDate(0, 0, -days).Format(time.RFC3339)
|
||
res, err := s.db.Exec("DELETE FROM audit_log WHERE ts < ?", cutoff)
|
||
if err != nil {
|
||
return 0, err
|
||
}
|
||
return res.RowsAffected()
|
||
}
|
||
|
||
func (s *Store) RecentAudit(limit int) ([]AuditEntry, error) {
|
||
rows, err := s.db.Query(
|
||
"SELECT ts,account,action,target,result,COALESCE(reason,'') FROM audit_log ORDER BY id DESC LIMIT ?", limit)
|
||
if err != nil {
|
||
return nil, err
|
||
}
|
||
defer rows.Close()
|
||
var out []AuditEntry
|
||
for rows.Next() {
|
||
var e AuditEntry
|
||
if err := rows.Scan(&e.TS, &e.Account, &e.Action, &e.Target, &e.Result, &e.Reason); err != nil {
|
||
return nil, err
|
||
}
|
||
out = append(out, e)
|
||
}
|
||
return out, rows.Err()
|
||
}
|
||
```
|
||
|
||
- [ ] **Step 4: Run tests to verify they pass**
|
||
|
||
Run: `go test ./internal/store/ -run 'Audit|Purge'`
|
||
Expected: PASS (2 tests).
|
||
|
||
- [ ] **Step 5: Commit**
|
||
|
||
```bash
|
||
git add internal/store/audit.go internal/store/audit_test.go
|
||
git commit -m "feat(store): audit log with retention-based purge"
|
||
```
|
||
|
||
---
|
||
|
||
### Task 8: policy — address matching
|
||
|
||
**Files:**
|
||
- Create: `internal/policy/policy.go`
|
||
- Test: `internal/policy/address_test.go`
|
||
|
||
**Interfaces:**
|
||
- Consumes: nothing (pure).
|
||
- Produces:
|
||
- `func MatchAddress(entries []string, addr string) bool` — case-insensitive; `@domain.com` entry matches any address at that domain; otherwise exact match. `addr` may be a display-form address; only the `local@domain` part is compared.
|
||
- `func NormalizeAddr(addr string) string` — lower-cases and strips display name / angle brackets.
|
||
|
||
- [ ] **Step 1: Write the failing test**
|
||
|
||
Create `internal/policy/address_test.go`:
|
||
```go
|
||
package policy
|
||
|
||
import "testing"
|
||
|
||
func TestMatchAddress(t *testing.T) {
|
||
wl := []string{"bob@example.com", "@trusted.com"}
|
||
cases := []struct {
|
||
addr string
|
||
want bool
|
||
}{
|
||
{"bob@example.com", true},
|
||
{"BOB@Example.com", true},
|
||
{`"Bob" <bob@example.com>`, true},
|
||
{"alice@trusted.com", true},
|
||
{"alice@untrusted.com", false},
|
||
{"eve@example.com", false},
|
||
{"", false},
|
||
}
|
||
for _, c := range cases {
|
||
if got := MatchAddress(wl, c.addr); got != c.want {
|
||
t.Fatalf("MatchAddress(%q)=%v want %v", c.addr, got, c.want)
|
||
}
|
||
}
|
||
}
|
||
|
||
func TestNormalizeAddr(t *testing.T) {
|
||
if got := NormalizeAddr(`"Bob Smith" <Bob@Example.COM>`); got != "bob@example.com" {
|
||
t.Fatalf("got %q", got)
|
||
}
|
||
}
|
||
```
|
||
|
||
- [ ] **Step 2: Run test to verify it fails**
|
||
|
||
Run: `go test ./internal/policy/`
|
||
Expected: FAIL — undefined `MatchAddress`.
|
||
|
||
- [ ] **Step 3: Write the implementation**
|
||
|
||
Create `internal/policy/policy.go`:
|
||
```go
|
||
// Package policy holds pure enforcement functions: address matching,
|
||
// inbound filtering, and (in a later phase) outbound checks.
|
||
package policy
|
||
|
||
import (
|
||
"net/mail"
|
||
"strings"
|
||
)
|
||
|
||
// NormalizeAddr lower-cases an address and strips any display name/brackets.
|
||
func NormalizeAddr(addr string) string {
|
||
addr = strings.TrimSpace(addr)
|
||
if a, err := mail.ParseAddress(addr); err == nil {
|
||
return strings.ToLower(a.Address)
|
||
}
|
||
return strings.ToLower(strings.Trim(addr, "<> "))
|
||
}
|
||
|
||
// MatchAddress reports whether addr matches any whitelist entry.
|
||
// Entries beginning with '@' match a whole domain; others match exactly.
|
||
func MatchAddress(entries []string, addr string) bool {
|
||
norm := NormalizeAddr(addr)
|
||
if norm == "" {
|
||
return false
|
||
}
|
||
at := strings.LastIndex(norm, "@")
|
||
domain := ""
|
||
if at >= 0 {
|
||
domain = norm[at:] // includes '@'
|
||
}
|
||
for _, e := range entries {
|
||
e = strings.ToLower(strings.TrimSpace(e))
|
||
if strings.HasPrefix(e, "@") {
|
||
if e == domain {
|
||
return true
|
||
}
|
||
continue
|
||
}
|
||
if e == norm {
|
||
return true
|
||
}
|
||
}
|
||
return false
|
||
}
|
||
```
|
||
|
||
- [ ] **Step 4: Run tests to verify they pass**
|
||
|
||
Run: `go test ./internal/policy/`
|
||
Expected: PASS (2 tests).
|
||
|
||
- [ ] **Step 5: Commit**
|
||
|
||
```bash
|
||
git add internal/policy
|
||
git commit -m "feat(policy): case-insensitive address and domain matching"
|
||
```
|
||
|
||
---
|
||
|
||
### Task 9: policy — inbound filter
|
||
|
||
**Files:**
|
||
- Create: `internal/policy/inbound.go`
|
||
- Test: `internal/policy/inbound_test.go`
|
||
|
||
**Interfaces:**
|
||
- Consumes: `MatchAddress`.
|
||
- Produces:
|
||
- `type InboundRule struct { WhitelistInEnabled bool; WhitelistIn []string; SubjectRegex *regexp.Regexp }`
|
||
- `func (r InboundRule) Allows(from, subject string) bool` — true only if (whitelist disabled OR sender matches) AND (no regex OR subject matches).
|
||
- `func CompileSubject(pattern string) (*regexp.Regexp, error)` — returns `(nil, nil)` for empty pattern.
|
||
|
||
- [ ] **Step 1: Write the failing test**
|
||
|
||
Create `internal/policy/inbound_test.go`:
|
||
```go
|
||
package policy
|
||
|
||
import "testing"
|
||
|
||
func TestInboundAllows(t *testing.T) {
|
||
re, err := CompileSubject("(?i)invoice")
|
||
if err != nil {
|
||
t.Fatalf("compile: %v", err)
|
||
}
|
||
rule := InboundRule{
|
||
WhitelistInEnabled: true,
|
||
WhitelistIn: []string{"@trusted.com"},
|
||
SubjectRegex: re,
|
||
}
|
||
cases := []struct {
|
||
from, subject string
|
||
want bool
|
||
}{
|
||
{"bob@trusted.com", "Your Invoice #5", true},
|
||
{"bob@trusted.com", "lunch?", false}, // subject fails
|
||
{"eve@evil.com", "Invoice", false}, // sender fails
|
||
{"eve@evil.com", "lunch?", false}, // both fail
|
||
}
|
||
for _, c := range cases {
|
||
if got := rule.Allows(c.from, c.subject); got != c.want {
|
||
t.Fatalf("Allows(%q,%q)=%v want %v", c.from, c.subject, got, c.want)
|
||
}
|
||
}
|
||
}
|
||
|
||
func TestInboundNoFiltersAllowsAll(t *testing.T) {
|
||
rule := InboundRule{} // whitelist disabled, no regex
|
||
if !rule.Allows("anyone@anywhere.com", "anything") {
|
||
t.Fatal("empty rule must allow everything")
|
||
}
|
||
}
|
||
|
||
func TestCompileSubjectEmpty(t *testing.T) {
|
||
re, err := CompileSubject("")
|
||
if err != nil || re != nil {
|
||
t.Fatalf("empty pattern: re=%v err=%v", re, err)
|
||
}
|
||
}
|
||
```
|
||
|
||
- [ ] **Step 2: Run test to verify it fails**
|
||
|
||
Run: `go test ./internal/policy/ -run Inbound`
|
||
Expected: FAIL — undefined `InboundRule`.
|
||
|
||
- [ ] **Step 3: Write the implementation**
|
||
|
||
Create `internal/policy/inbound.go`:
|
||
```go
|
||
package policy
|
||
|
||
import "regexp"
|
||
|
||
// InboundRule captures one account's read-side filtering.
|
||
type InboundRule struct {
|
||
WhitelistInEnabled bool
|
||
WhitelistIn []string
|
||
SubjectRegex *regexp.Regexp // nil = no subject filter
|
||
}
|
||
|
||
// CompileSubject compiles a subject filter; empty pattern => (nil, nil).
|
||
func CompileSubject(pattern string) (*regexp.Regexp, error) {
|
||
if pattern == "" {
|
||
return nil, nil
|
||
}
|
||
return regexp.Compile(pattern)
|
||
}
|
||
|
||
// Allows reports whether a message with the given sender and subject is visible.
|
||
func (r InboundRule) Allows(from, subject string) bool {
|
||
if r.WhitelistInEnabled && !MatchAddress(r.WhitelistIn, from) {
|
||
return false
|
||
}
|
||
if r.SubjectRegex != nil && !r.SubjectRegex.MatchString(subject) {
|
||
return false
|
||
}
|
||
return true
|
||
}
|
||
```
|
||
|
||
- [ ] **Step 4: Run tests to verify they pass**
|
||
|
||
Run: `go test ./internal/policy/ -run Inbound`
|
||
Expected: PASS.
|
||
|
||
- [ ] **Step 5: Commit**
|
||
|
||
```bash
|
||
git add internal/policy/inbound.go internal/policy/inbound_test.go
|
||
git commit -m "feat(policy): inbound whitelist + subject-regex filter"
|
||
```
|
||
|
||
---
|
||
|
||
### Task 10: mail — message parsing
|
||
|
||
Parse a raw RFC822 message into headers, a plain-text body, and attachment metadata. This is pure (operates on `io.Reader`/bytes) so it is unit-testable without a server.
|
||
|
||
**Files:**
|
||
- Create: `internal/mail/message.go`
|
||
- Test: `internal/mail/message_test.go`
|
||
- Test fixture: `internal/mail/testdata/with_attachment.eml`
|
||
|
||
**Interfaces:**
|
||
- Consumes: `github.com/emersion/go-message/mail`.
|
||
- Produces:
|
||
- `type Header struct { UID uint32; From, To, Subject, Date, MessageID string; HasAttachments bool }`
|
||
- `type Attachment struct { Name string; Size int; MIME string; Content []byte }`
|
||
- `type Message struct { Header Header; BodyText string; Attachments []Attachment }`
|
||
- `func ParseMessage(uid uint32, raw []byte) (Message, error)` — extracts the first `text/plain` part as `BodyText`; collects attachment parts (with `Content`); sets `HasAttachments`.
|
||
- `func ParseHeaderOnly(uid uint32, raw []byte) (Header, error)` — header fields + `HasAttachments` without decoding attachment bodies.
|
||
|
||
- [ ] **Step 1: Add the dependency and write the fixture**
|
||
|
||
Run: `go get github.com/emersion/go-message@latest`
|
||
|
||
Create `internal/mail/testdata/with_attachment.eml` (exact bytes, CRLF line endings not required for the parser):
|
||
```
|
||
From: "Bob" <bob@trusted.com>
|
||
To: me@example.com
|
||
Subject: Your Invoice #5
|
||
Date: Sat, 20 Jun 2026 10:00:00 +0000
|
||
Message-ID: <abc123@trusted.com>
|
||
MIME-Version: 1.0
|
||
Content-Type: multipart/mixed; boundary="BOUNDARY"
|
||
|
||
--BOUNDARY
|
||
Content-Type: text/plain; charset=utf-8
|
||
|
||
Hello, here is your invoice.
|
||
--BOUNDARY
|
||
Content-Type: text/plain; charset=utf-8
|
||
Content-Disposition: attachment; filename="invoice.txt"
|
||
|
||
LINE-ITEM 1: 100.00
|
||
--BOUNDARY--
|
||
```
|
||
|
||
- [ ] **Step 2: Write the failing test**
|
||
|
||
Create `internal/mail/message_test.go`:
|
||
```go
|
||
package mail
|
||
|
||
import (
|
||
"os"
|
||
"path/filepath"
|
||
"testing"
|
||
)
|
||
|
||
func loadFixture(t *testing.T, name string) []byte {
|
||
t.Helper()
|
||
b, err := os.ReadFile(filepath.Join("testdata", name))
|
||
if err != nil {
|
||
t.Fatalf("read fixture: %v", err)
|
||
}
|
||
return b
|
||
}
|
||
|
||
func TestParseMessage(t *testing.T) {
|
||
raw := loadFixture(t, "with_attachment.eml")
|
||
m, err := ParseMessage(42, raw)
|
||
if err != nil {
|
||
t.Fatalf("ParseMessage: %v", err)
|
||
}
|
||
if m.Header.UID != 42 {
|
||
t.Fatalf("uid: %d", m.Header.UID)
|
||
}
|
||
if m.Header.Subject != "Your Invoice #5" {
|
||
t.Fatalf("subject: %q", m.Header.Subject)
|
||
}
|
||
if m.Header.From != `"Bob" <bob@trusted.com>` && m.Header.From != "Bob <bob@trusted.com>" {
|
||
t.Fatalf("from: %q", m.Header.From)
|
||
}
|
||
if m.Header.MessageID != "abc123@trusted.com" && m.Header.MessageID != "<abc123@trusted.com>" {
|
||
t.Fatalf("message-id: %q", m.Header.MessageID)
|
||
}
|
||
if want := "Hello, here is your invoice."; !contains(m.BodyText, want) {
|
||
t.Fatalf("body=%q want contains %q", m.BodyText, want)
|
||
}
|
||
if !m.Header.HasAttachments {
|
||
t.Fatal("HasAttachments should be true")
|
||
}
|
||
if len(m.Attachments) != 1 || m.Attachments[0].Name != "invoice.txt" {
|
||
t.Fatalf("attachments: %+v", m.Attachments)
|
||
}
|
||
if !contains(string(m.Attachments[0].Content), "LINE-ITEM 1") {
|
||
t.Fatalf("attachment content: %q", m.Attachments[0].Content)
|
||
}
|
||
}
|
||
|
||
func TestParseHeaderOnly(t *testing.T) {
|
||
raw := loadFixture(t, "with_attachment.eml")
|
||
h, err := ParseHeaderOnly(7, raw)
|
||
if err != nil {
|
||
t.Fatalf("ParseHeaderOnly: %v", err)
|
||
}
|
||
if h.Subject != "Your Invoice #5" || !h.HasAttachments {
|
||
t.Fatalf("header: %+v", h)
|
||
}
|
||
}
|
||
|
||
func contains(s, sub string) bool {
|
||
return len(s) >= len(sub) && (func() bool {
|
||
for i := 0; i+len(sub) <= len(s); i++ {
|
||
if s[i:i+len(sub)] == sub {
|
||
return true
|
||
}
|
||
}
|
||
return false
|
||
}())
|
||
}
|
||
```
|
||
|
||
- [ ] **Step 3: Run test to verify it fails**
|
||
|
||
Run: `go test ./internal/mail/`
|
||
Expected: FAIL — undefined `ParseMessage`.
|
||
|
||
- [ ] **Step 4: Write the implementation**
|
||
|
||
Create `internal/mail/message.go`:
|
||
```go
|
||
// Package mail provides IMAP reading and RFC822 message parsing.
|
||
package mail
|
||
|
||
import (
|
||
"bytes"
|
||
"io"
|
||
"strings"
|
||
|
||
"github.com/emersion/go-message/mail"
|
||
)
|
||
|
||
type Header struct {
|
||
UID uint32
|
||
From string
|
||
To string
|
||
Subject string
|
||
Date string
|
||
MessageID string
|
||
HasAttachments bool
|
||
}
|
||
|
||
type Attachment struct {
|
||
Name string
|
||
Size int
|
||
MIME string
|
||
Content []byte
|
||
}
|
||
|
||
type Message struct {
|
||
Header Header
|
||
BodyText string
|
||
Attachments []Attachment
|
||
}
|
||
|
||
func readHeader(mr *mail.Reader, uid uint32) Header {
|
||
h := Header{UID: uid}
|
||
hd := mr.Header
|
||
h.Subject, _ = hd.Subject()
|
||
if addrs, err := hd.AddressList("From"); err == nil && len(addrs) > 0 {
|
||
h.From = addrs[0].String()
|
||
}
|
||
if addrs, err := hd.AddressList("To"); err == nil && len(addrs) > 0 {
|
||
h.To = addrs[0].String()
|
||
}
|
||
if d, err := hd.Date(); err == nil {
|
||
h.Date = d.UTC().Format("Mon, 02 Jan 2006 15:04:05 -0700")
|
||
}
|
||
h.MessageID = strings.Trim(hd.Get("Message-Id"), "<> ")
|
||
return h
|
||
}
|
||
|
||
// ParseMessage decodes the full message including attachment contents.
|
||
func ParseMessage(uid uint32, raw []byte) (Message, error) {
|
||
mr, err := mail.CreateReader(bytes.NewReader(raw))
|
||
if err != nil {
|
||
return Message{}, err
|
||
}
|
||
m := Message{Header: readHeader(mr, uid)}
|
||
for {
|
||
part, err := mr.NextPart()
|
||
if err == io.EOF {
|
||
break
|
||
}
|
||
if err != nil {
|
||
return Message{}, err
|
||
}
|
||
switch hdr := part.Header.(type) {
|
||
case *mail.InlineHeader:
|
||
ct, _, _ := hdr.ContentType()
|
||
if strings.HasPrefix(ct, "text/plain") && m.BodyText == "" {
|
||
b, _ := io.ReadAll(part.Body)
|
||
m.BodyText = string(b)
|
||
}
|
||
case *mail.AttachmentHeader:
|
||
name, _ := hdr.Filename()
|
||
ct, _, _ := hdr.ContentType()
|
||
b, _ := io.ReadAll(part.Body)
|
||
m.Attachments = append(m.Attachments, Attachment{
|
||
Name: name, Size: len(b), MIME: ct, Content: b,
|
||
})
|
||
}
|
||
}
|
||
m.Header.HasAttachments = len(m.Attachments) > 0
|
||
return m, nil
|
||
}
|
||
|
||
// ParseHeaderOnly decodes headers and detects attachments without keeping bodies.
|
||
func ParseHeaderOnly(uid uint32, raw []byte) (Header, error) {
|
||
m, err := ParseMessage(uid, raw)
|
||
if err != nil {
|
||
return Header{}, err
|
||
}
|
||
return m.Header, nil
|
||
}
|
||
```
|
||
|
||
> Note: `ParseHeaderOnly` reuses `ParseMessage` for correctness in this phase. The IMAP layer (Task 11) avoids fetching full bodies for `list` by requesting only envelope + body-structure from the server, so this convenience does not cost a full download in practice.
|
||
|
||
- [ ] **Step 5: Run tests to verify they pass**
|
||
|
||
Run: `go test ./internal/mail/`
|
||
Expected: PASS (2 tests).
|
||
|
||
- [ ] **Step 6: Commit**
|
||
|
||
```bash
|
||
git add internal/mail go.mod go.sum
|
||
git commit -m "feat(mail): RFC822 message parsing (headers, body, attachments)"
|
||
```
|
||
|
||
---
|
||
|
||
### Task 11: mail — IMAP client (integration)
|
||
|
||
Connect, select a folder, fetch headers for new/ranged UIDs, fetch a full message, and search. Integration-tested against a `greenmail` container. If Docker is unavailable, the test self-skips.
|
||
|
||
**Files:**
|
||
- Create: `internal/mail/imap.go`
|
||
- Test: `internal/mail/imap_integration_test.go`
|
||
|
||
**Interfaces:**
|
||
- Consumes: `github.com/emersion/go-imap`, `github.com/emersion/go-imap/client`, `ParseMessage`/`ParseHeaderOnly`.
|
||
- Produces:
|
||
- `type IMAPConfig struct { Host string; Port int; Security string; Username, Password string }`
|
||
- `type Client struct { ... }` with `func Dial(cfg IMAPConfig) (*Client, error)` and `func (c *Client) Logout() error`.
|
||
- `func (c *Client) SelectFolder(folder string) (uidValidity uint32, maxUID uint32, err error)`.
|
||
- `func (c *Client) FetchHeaders(folder string, uids []uint32) ([]Header, error)` — empty `uids` ⇒ all in folder.
|
||
- `func (c *Client) FetchHeadersRange(folder string, sinceUID, beforeUID uint32, limit int) ([]Header, error)`.
|
||
- `func (c *Client) FetchFull(folder string, uid uint32) (Message, error)`.
|
||
- `type SearchCriteria struct { From, SubjectContains, Text string; Since, Before time.Time }`
|
||
- `func (c *Client) Search(folder string, sc SearchCriteria, limit int) ([]Header, error)`.
|
||
|
||
- [ ] **Step 1: Add dependency and write the integration test**
|
||
|
||
Run: `go get github.com/emersion/go-imap@v1`
|
||
|
||
Create `internal/mail/imap_integration_test.go`:
|
||
```go
|
||
//go:build integration
|
||
|
||
package mail
|
||
|
||
import (
|
||
"os"
|
||
"testing"
|
||
)
|
||
|
||
// These tests require a GreenMail server reachable via env vars:
|
||
// EMCLI_TEST_IMAP_HOST, EMCLI_TEST_IMAP_PORT, EMCLI_TEST_USER, EMCLI_TEST_PASS
|
||
// Run: go test -tags=integration ./internal/mail/
|
||
func testCfg(t *testing.T) IMAPConfig {
|
||
t.Helper()
|
||
host := os.Getenv("EMCLI_TEST_IMAP_HOST")
|
||
if host == "" {
|
||
t.Skip("EMCLI_TEST_IMAP_HOST not set; skipping IMAP integration test")
|
||
}
|
||
return IMAPConfig{
|
||
Host: host, Port: atoiEnv(t, "EMCLI_TEST_IMAP_PORT"), Security: "starttls",
|
||
Username: os.Getenv("EMCLI_TEST_USER"), Password: os.Getenv("EMCLI_TEST_PASS"),
|
||
}
|
||
}
|
||
|
||
func TestSelectAndFetch(t *testing.T) {
|
||
c, err := Dial(testCfg(t))
|
||
if err != nil {
|
||
t.Fatalf("Dial: %v", err)
|
||
}
|
||
defer c.Logout()
|
||
|
||
uidv, maxUID, err := c.SelectFolder("INBOX")
|
||
if err != nil {
|
||
t.Fatalf("SelectFolder: %v", err)
|
||
}
|
||
if uidv == 0 {
|
||
t.Fatal("uidvalidity should be non-zero")
|
||
}
|
||
headers, err := c.FetchHeaders("INBOX", nil)
|
||
if err != nil {
|
||
t.Fatalf("FetchHeaders: %v", err)
|
||
}
|
||
t.Logf("inbox has %d messages, maxUID=%d", len(headers), maxUID)
|
||
}
|
||
```
|
||
|
||
Create the helper in the same file:
|
||
```go
|
||
func atoiEnv(t *testing.T, k string) int {
|
||
t.Helper()
|
||
var n int
|
||
if _, err := fmtSscan(os.Getenv(k), &n); err != nil {
|
||
t.Fatalf("bad int env %s: %v", k, err)
|
||
}
|
||
return n
|
||
}
|
||
```
|
||
|
||
> Note: define `fmtSscan` as a tiny wrapper `func fmtSscan(s string, v *int) (int, error){ return fmt.Sscan(s, v) }` in the test file, importing `fmt`. (Kept indirect only to avoid an unused-import lint if the body changes.)
|
||
|
||
- [ ] **Step 2: Write the IMAP client implementation**
|
||
|
||
Create `internal/mail/imap.go`:
|
||
```go
|
||
package mail
|
||
|
||
import (
|
||
"fmt"
|
||
"io"
|
||
"sort"
|
||
"time"
|
||
|
||
"github.com/emersion/go-imap"
|
||
"github.com/emersion/go-imap/client"
|
||
)
|
||
|
||
type IMAPConfig struct {
|
||
Host string
|
||
Port int
|
||
Security string // tls | starttls
|
||
Username string
|
||
Password string
|
||
}
|
||
|
||
type Client struct {
|
||
c *client.Client
|
||
}
|
||
|
||
func Dial(cfg IMAPConfig) (*Client, error) {
|
||
addr := fmt.Sprintf("%s:%d", cfg.Host, cfg.Port)
|
||
var (
|
||
ic *client.Client
|
||
err error
|
||
)
|
||
switch cfg.Security {
|
||
case "tls":
|
||
ic, err = client.DialTLS(addr, nil)
|
||
case "starttls":
|
||
ic, err = client.Dial(addr)
|
||
if err == nil {
|
||
err = ic.StartTLS(nil)
|
||
}
|
||
default:
|
||
return nil, fmt.Errorf("unknown imap security %q", cfg.Security)
|
||
}
|
||
if err != nil {
|
||
return nil, err
|
||
}
|
||
if err := ic.Login(cfg.Username, cfg.Password); err != nil {
|
||
ic.Logout()
|
||
return nil, err
|
||
}
|
||
return &Client{c: ic}, nil
|
||
}
|
||
|
||
func (c *Client) Logout() error { return c.c.Logout() }
|
||
|
||
func (c *Client) SelectFolder(folder string) (uint32, uint32, error) {
|
||
mbox, err := c.c.Select(folder, true) // read-only select
|
||
if err != nil {
|
||
return 0, 0, err
|
||
}
|
||
var maxUID uint32
|
||
if mbox.Messages > 0 {
|
||
// UIDNext-1 is an upper bound for the highest existing UID.
|
||
if mbox.UidNext > 0 {
|
||
maxUID = mbox.UidNext - 1
|
||
}
|
||
}
|
||
return mbox.UidValidity, maxUID, nil
|
||
}
|
||
|
||
func (c *Client) fetchByUIDSet(folder string, set *imap.SeqSet, full bool) ([]Message, error) {
|
||
if _, err := c.c.Select(folder, true); err != nil {
|
||
return nil, err
|
||
}
|
||
section := &imap.BodySectionName{} // entire message
|
||
items := []imap.FetchItem{imap.FetchUid, section.FetchItem()}
|
||
|
||
msgCh := make(chan *imap.Message, 16)
|
||
done := make(chan error, 1)
|
||
go func() { done <- c.c.UidFetch(set, items, msgCh) }()
|
||
|
||
var out []Message
|
||
for m := range msgCh {
|
||
r := m.GetBody(section)
|
||
if r == nil {
|
||
continue
|
||
}
|
||
raw, _ := io.ReadAll(r)
|
||
parsed, err := ParseMessage(m.Uid, raw)
|
||
if err != nil {
|
||
return nil, err
|
||
}
|
||
if !full {
|
||
parsed.Attachments = nil
|
||
parsed.BodyText = ""
|
||
}
|
||
out = append(out, parsed)
|
||
}
|
||
if err := <-done; err != nil {
|
||
return nil, err
|
||
}
|
||
sort.Slice(out, func(i, j int) bool { return out[i].Header.UID > out[j].Header.UID })
|
||
return out, nil
|
||
}
|
||
|
||
func (c *Client) FetchHeaders(folder string, uids []uint32) ([]Header, error) {
|
||
set := new(imap.SeqSet)
|
||
if len(uids) == 0 {
|
||
set.AddRange(1, 0) // 1:* — all
|
||
} else {
|
||
for _, u := range uids {
|
||
set.AddNum(u)
|
||
}
|
||
}
|
||
msgs, err := c.fetchByUIDSet(folder, set, false)
|
||
if err != nil {
|
||
return nil, err
|
||
}
|
||
return headersOf(msgs), nil
|
||
}
|
||
|
||
func (c *Client) FetchHeadersRange(folder string, sinceUID, beforeUID uint32, limit int) ([]Header, error) {
|
||
set := new(imap.SeqSet)
|
||
lo := sinceUID + 1
|
||
if sinceUID == 0 {
|
||
lo = 1
|
||
}
|
||
hi := uint32(0) // '*'
|
||
if beforeUID > 1 {
|
||
hi = beforeUID - 1
|
||
}
|
||
set.AddRange(lo, hi)
|
||
msgs, err := c.fetchByUIDSet(folder, set, false)
|
||
if err != nil {
|
||
return nil, err
|
||
}
|
||
h := headersOf(msgs)
|
||
if limit > 0 && len(h) > limit {
|
||
h = h[:limit]
|
||
}
|
||
return h, nil
|
||
}
|
||
|
||
func (c *Client) FetchFull(folder string, uid uint32) (Message, error) {
|
||
set := new(imap.SeqSet)
|
||
set.AddNum(uid)
|
||
msgs, err := c.fetchByUIDSet(folder, set, true)
|
||
if err != nil {
|
||
return Message{}, err
|
||
}
|
||
if len(msgs) == 0 {
|
||
return Message{}, fmt.Errorf("uid %d not found in %s", uid, folder)
|
||
}
|
||
return msgs[0], nil
|
||
}
|
||
|
||
type SearchCriteria struct {
|
||
From string
|
||
SubjectContains string
|
||
Text string
|
||
Since time.Time
|
||
Before time.Time
|
||
}
|
||
|
||
func (c *Client) Search(folder string, sc SearchCriteria, limit int) ([]Header, error) {
|
||
if _, err := c.c.Select(folder, true); err != nil {
|
||
return nil, err
|
||
}
|
||
crit := imap.NewSearchCriteria()
|
||
if sc.From != "" {
|
||
crit.Header.Add("From", sc.From)
|
||
}
|
||
if sc.SubjectContains != "" {
|
||
crit.Header.Add("Subject", sc.SubjectContains)
|
||
}
|
||
if sc.Text != "" {
|
||
crit.Text = []string{sc.Text}
|
||
}
|
||
if !sc.Since.IsZero() {
|
||
crit.Since = sc.Since
|
||
}
|
||
if !sc.Before.IsZero() {
|
||
crit.Before = sc.Before
|
||
}
|
||
uids, err := c.c.UidSearch(crit)
|
||
if err != nil {
|
||
return nil, err
|
||
}
|
||
if len(uids) == 0 {
|
||
return nil, nil
|
||
}
|
||
return c.FetchHeaders(folder, uids)
|
||
}
|
||
|
||
func headersOf(msgs []Message) []Header {
|
||
out := make([]Header, 0, len(msgs))
|
||
for _, m := range msgs {
|
||
out = append(out, m.Header)
|
||
}
|
||
return out
|
||
}
|
||
```
|
||
|
||
- [ ] **Step 3: Run unit tests (integration self-skips without Docker)**
|
||
|
||
Run: `go build ./internal/mail/ && go test ./internal/mail/`
|
||
Expected: PASS — parsing tests pass; integration test is excluded (build tag `integration` not set).
|
||
|
||
- [ ] **Step 4: Run the integration test against GreenMail (optional but recommended)**
|
||
|
||
Run:
|
||
```bash
|
||
docker run -d --rm --name emcli-greenmail \
|
||
-e GREENMAIL_OPTS='-Dgreenmail.setup.test.all -Dgreenmail.users=me:pass@example.com -Dgreenmail.verbose' \
|
||
-p 3143:3143 -p 3025:3025 greenmail/standalone:2.1.0
|
||
EMCLI_TEST_IMAP_HOST=localhost EMCLI_TEST_IMAP_PORT=3143 \
|
||
EMCLI_TEST_USER=me@example.com EMCLI_TEST_PASS=pass \
|
||
go test -tags=integration ./internal/mail/ -run TestSelectAndFetch -v
|
||
docker stop emcli-greenmail
|
||
```
|
||
Expected: PASS (selects INBOX, logs message count). Skips cleanly if Docker is absent.
|
||
|
||
- [ ] **Step 5: Commit**
|
||
|
||
```bash
|
||
git add internal/mail/imap.go internal/mail/imap_integration_test.go go.mod go.sum
|
||
git commit -m "feat(mail): IMAP client — select, fetch headers/full, search"
|
||
```
|
||
|
||
---
|
||
|
||
### Task 12: cli — JSON envelope
|
||
|
||
**Files:**
|
||
- Create: `internal/cli/envelope.go`
|
||
- Test: `internal/cli/envelope_test.go`
|
||
|
||
**Interfaces:**
|
||
- Consumes: nothing.
|
||
- Produces:
|
||
- `type Envelope struct { Error bool; ErrorDetail map[string]any; Data any }` with JSON tags `error`, `error_detail`, `data`.
|
||
- `func Success(data any) Envelope`
|
||
- `func Failure(code, message string) Envelope`
|
||
- `func (e Envelope) Write(w io.Writer) error` — writes compact JSON + newline.
|
||
- Error code constants: `CodeConfig, CodeDB, CodeNetwork, CodeAuth, CodePolicy, CodeNotFound, CodeUsage`.
|
||
|
||
- [ ] **Step 1: Write the failing test**
|
||
|
||
Create `internal/cli/envelope_test.go`:
|
||
```go
|
||
package cli
|
||
|
||
import (
|
||
"bytes"
|
||
"encoding/json"
|
||
"testing"
|
||
)
|
||
|
||
func TestSuccessEnvelope(t *testing.T) {
|
||
var buf bytes.Buffer
|
||
if err := Success(map[string]any{"count": 2}).Write(&buf); err != nil {
|
||
t.Fatalf("write: %v", err)
|
||
}
|
||
var got map[string]any
|
||
if err := json.Unmarshal(buf.Bytes(), &got); err != nil {
|
||
t.Fatalf("unmarshal: %v", err)
|
||
}
|
||
if got["error"] != false {
|
||
t.Fatalf("error should be false: %v", got["error"])
|
||
}
|
||
if _, ok := got["data"]; !ok {
|
||
t.Fatal("data key missing")
|
||
}
|
||
if ed, ok := got["error_detail"].(map[string]any); !ok || len(ed) != 0 {
|
||
t.Fatalf("error_detail should be empty object: %v", got["error_detail"])
|
||
}
|
||
}
|
||
|
||
func TestFailureEnvelope(t *testing.T) {
|
||
var buf bytes.Buffer
|
||
_ = Failure(CodeNotFound, "uid 7 not found").Write(&buf)
|
||
var got struct {
|
||
Error bool `json:"error"`
|
||
ErrorDetail struct {
|
||
Code string `json:"code"`
|
||
Message string `json:"message"`
|
||
} `json:"error_detail"`
|
||
}
|
||
_ = json.Unmarshal(buf.Bytes(), &got)
|
||
if !got.Error || got.ErrorDetail.Code != "not_found" {
|
||
t.Fatalf("bad failure envelope: %+v", got)
|
||
}
|
||
}
|
||
```
|
||
|
||
- [ ] **Step 2: Run test to verify it fails**
|
||
|
||
Run: `go test ./internal/cli/`
|
||
Expected: FAIL — undefined `Success`.
|
||
|
||
- [ ] **Step 3: Write the implementation**
|
||
|
||
Create `internal/cli/envelope.go`:
|
||
```go
|
||
// Package cli implements command dispatch and the agent JSON envelope.
|
||
package cli
|
||
|
||
import (
|
||
"encoding/json"
|
||
"io"
|
||
)
|
||
|
||
const (
|
||
CodeConfig = "config"
|
||
CodeDB = "db"
|
||
CodeNetwork = "network"
|
||
CodeAuth = "auth"
|
||
CodePolicy = "policy"
|
||
CodeNotFound = "not_found"
|
||
CodeUsage = "usage"
|
||
)
|
||
|
||
// Envelope is the single JSON object every agent command emits.
|
||
type Envelope struct {
|
||
Error bool `json:"error"`
|
||
ErrorDetail map[string]any `json:"error_detail"`
|
||
Data any `json:"data"`
|
||
}
|
||
|
||
func Success(data any) Envelope {
|
||
if data == nil {
|
||
data = map[string]any{}
|
||
}
|
||
return Envelope{Error: false, ErrorDetail: map[string]any{}, Data: data}
|
||
}
|
||
|
||
func Failure(code, message string) Envelope {
|
||
return Envelope{
|
||
Error: true,
|
||
ErrorDetail: map[string]any{"code": code, "message": message},
|
||
Data: map[string]any{},
|
||
}
|
||
}
|
||
|
||
func (e Envelope) Write(w io.Writer) error {
|
||
b, err := json.Marshal(e)
|
||
if err != nil {
|
||
return err
|
||
}
|
||
b = append(b, '\n')
|
||
_, err = w.Write(b)
|
||
return err
|
||
}
|
||
```
|
||
|
||
- [ ] **Step 4: Run tests to verify they pass**
|
||
|
||
Run: `go test ./internal/cli/`
|
||
Expected: PASS (2 tests).
|
||
|
||
- [ ] **Step 5: Commit**
|
||
|
||
```bash
|
||
git add internal/cli/envelope.go internal/cli/envelope_test.go
|
||
git commit -m "feat(cli): JSON output envelope with stable error codes"
|
||
```
|
||
|
||
---
|
||
|
||
### Task 13: cli — agent read commands wired end-to-end
|
||
|
||
Wire `list`, `get`, `search`, and `ack` together: open store, load account, apply inbound policy, talk to IMAP, write the envelope, and audit. Uses a `mailDialer` seam so the command logic is testable with a fake.
|
||
|
||
**Files:**
|
||
- Create: `internal/cli/agent.go`
|
||
- Create: `internal/cli/dispatch.go`
|
||
- Modify: `cmd/emcli/main.go` (route agent subcommands into `cli.Run`)
|
||
- Test: `internal/cli/agent_test.go`
|
||
|
||
**Interfaces:**
|
||
- Consumes: `store`, `policy`, `mail` (via the `Mailer` interface), `Envelope`.
|
||
- Produces:
|
||
- `type Mailer interface { SelectFolder(folder string)(uint32,uint32,error); FetchHeaders(folder string, uids []uint32)([]mail.Header,error); FetchHeadersRange(folder string, since,before uint32, limit int)([]mail.Header,error); FetchFull(folder string, uid uint32)(mail.Message,error); Search(folder string, sc mail.SearchCriteria, limit int)([]mail.Header,error); Logout() error }`
|
||
- `type Deps struct { Store *store.Store; Dial func(store.Account)(Mailer,error); Now func() time.Time; Out io.Writer }`
|
||
- `func ListCmd(d Deps, account, folder string, onlyNew bool, beforeUID, sinceUID uint32, limit int) error`
|
||
- `func GetCmd(d Deps, account, folder string, uid uint32) error`
|
||
- `func SearchCmd(d Deps, account, folder string, sc mail.SearchCriteria, limit int) error`
|
||
- `func AckCmd(d Deps, account, folder string, uids []uint32) error`
|
||
|
||
- [ ] **Step 1: Write the failing test (fake Mailer, real store)**
|
||
|
||
Create `internal/cli/agent_test.go`:
|
||
```go
|
||
package cli
|
||
|
||
import (
|
||
"bytes"
|
||
"encoding/json"
|
||
"path/filepath"
|
||
"testing"
|
||
"time"
|
||
|
||
"git.dcglab.co.uk/steve/emcli/internal/mail"
|
||
"git.dcglab.co.uk/steve/emcli/internal/store"
|
||
)
|
||
|
||
type fakeMailer struct {
|
||
uidValidity uint32
|
||
maxUID uint32
|
||
headers []mail.Header
|
||
full map[uint32]mail.Message
|
||
}
|
||
|
||
func (f *fakeMailer) SelectFolder(string) (uint32, uint32, error) { return f.uidValidity, f.maxUID, nil }
|
||
func (f *fakeMailer) FetchHeaders(_ string, uids []uint32) ([]mail.Header, error) {
|
||
if len(uids) == 0 {
|
||
return f.headers, nil
|
||
}
|
||
want := map[uint32]bool{}
|
||
for _, u := range uids {
|
||
want[u] = true
|
||
}
|
||
var out []mail.Header
|
||
for _, h := range f.headers {
|
||
if want[h.UID] {
|
||
out = append(out, h)
|
||
}
|
||
}
|
||
return out, nil
|
||
}
|
||
func (f *fakeMailer) FetchHeadersRange(string, uint32, uint32, int) ([]mail.Header, error) {
|
||
return f.headers, nil
|
||
}
|
||
func (f *fakeMailer) FetchFull(_ string, uid uint32) (mail.Message, error) {
|
||
return f.full[uid], nil
|
||
}
|
||
func (f *fakeMailer) Search(string, mail.SearchCriteria, int) ([]mail.Header, error) {
|
||
return f.headers, nil
|
||
}
|
||
func (f *fakeMailer) Logout() error { return nil }
|
||
|
||
func testKey() []byte {
|
||
k := make([]byte, 32)
|
||
for i := range k {
|
||
k[i] = byte(i)
|
||
}
|
||
return k
|
||
}
|
||
|
||
func newDeps(t *testing.T, fm *fakeMailer) (Deps, *bytes.Buffer) {
|
||
t.Helper()
|
||
st, err := store.Open(filepath.Join(t.TempDir(), "e.db"), testKey())
|
||
if err != nil {
|
||
t.Fatalf("store: %v", err)
|
||
}
|
||
t.Cleanup(func() { st.Close() })
|
||
_, err = st.AddAccount(store.Account{
|
||
Name: "work", Mode: "RO", IMAPHost: "h", IMAPPort: 993, IMAPSecurity: "tls",
|
||
AuthType: "password", Username: "me@example.com", Password: "pw",
|
||
WhitelistInEnabled: true,
|
||
})
|
||
if err != nil {
|
||
t.Fatalf("AddAccount: %v", err)
|
||
}
|
||
_ = st.AddWhitelist("work", store.DirIn, "@trusted.com")
|
||
var buf bytes.Buffer
|
||
d := Deps{
|
||
Store: st,
|
||
Dial: func(store.Account) (Mailer, error) { return fm, nil },
|
||
Now: func() time.Time { return time.Date(2026, 6, 21, 0, 0, 0, 0, time.UTC) },
|
||
Out: &buf,
|
||
}
|
||
return d, &buf
|
||
}
|
||
|
||
func decode(t *testing.T, b []byte) map[string]any {
|
||
t.Helper()
|
||
var m map[string]any
|
||
if err := json.Unmarshal(b, &m); err != nil {
|
||
t.Fatalf("unmarshal %q: %v", b, err)
|
||
}
|
||
return m
|
||
}
|
||
|
||
func TestListNewFiltersBySenderAndState(t *testing.T) {
|
||
fm := &fakeMailer{
|
||
uidValidity: 1, maxUID: 100,
|
||
headers: []mail.Header{
|
||
{UID: 101, From: "bob@trusted.com", Subject: "hi"},
|
||
{UID: 102, From: "eve@evil.com", Subject: "spam"}, // filtered out by whitelist
|
||
{UID: 103, From: "ann@trusted.com", Subject: "yo"},
|
||
},
|
||
}
|
||
d, buf := newDeps(t, fm)
|
||
if err := ListCmd(d, "work", "INBOX", true, 0, 0, 50); err != nil {
|
||
t.Fatalf("ListCmd: %v", err)
|
||
}
|
||
res := decode(t, buf.Bytes())
|
||
if res["error"] != false {
|
||
t.Fatalf("unexpected error: %v", res)
|
||
}
|
||
data := res["data"].(map[string]any)
|
||
msgs := data["messages"].([]any)
|
||
if len(msgs) != 2 { // 101 and 103; 102 filtered
|
||
t.Fatalf("want 2 messages, got %d: %v", len(msgs), msgs)
|
||
}
|
||
}
|
||
|
||
func TestGetFilteredReturnsNotFound(t *testing.T) {
|
||
fm := &fakeMailer{
|
||
uidValidity: 1, maxUID: 100,
|
||
headers: []mail.Header{{UID: 102, From: "eve@evil.com", Subject: "spam"}},
|
||
full: map[uint32]mail.Message{
|
||
102: {Header: mail.Header{UID: 102, From: "eve@evil.com", Subject: "spam"}, BodyText: "secret"},
|
||
},
|
||
}
|
||
d, buf := newDeps(t, fm)
|
||
if err := GetCmd(d, "work", "INBOX", 102); err != nil {
|
||
t.Fatalf("GetCmd: %v", err)
|
||
}
|
||
res := decode(t, buf.Bytes())
|
||
if res["error"] != true {
|
||
t.Fatal("filtered get must return error envelope")
|
||
}
|
||
ed := res["error_detail"].(map[string]any)
|
||
if ed["code"] != "not_found" {
|
||
t.Fatalf("want not_found, got %v", ed["code"])
|
||
}
|
||
}
|
||
|
||
func TestAckAdvancesStateAndFiltered(t *testing.T) {
|
||
fm := &fakeMailer{
|
||
uidValidity: 1, maxUID: 100,
|
||
headers: []mail.Header{
|
||
{UID: 101, From: "bob@trusted.com", Subject: "hi"},
|
||
{UID: 102, From: "eve@evil.com", Subject: "spam"},
|
||
},
|
||
}
|
||
d, buf := newDeps(t, fm)
|
||
// Acking a filtered uid (102) must be rejected as not-found.
|
||
if err := AckCmd(d, "work", "INBOX", []uint32{102}); err != nil {
|
||
t.Fatalf("AckCmd: %v", err)
|
||
}
|
||
if decode(t, buf.Bytes())["error"] != true {
|
||
t.Fatal("acking filtered uid must fail")
|
||
}
|
||
// Acking a visible uid (101) succeeds and removes it from list --new.
|
||
buf.Reset()
|
||
if err := AckCmd(d, "work", "INBOX", []uint32{101}); err != nil {
|
||
t.Fatalf("AckCmd 101: %v", err)
|
||
}
|
||
if decode(t, buf.Bytes())["error"] != false {
|
||
t.Fatal("ack of visible uid should succeed")
|
||
}
|
||
buf.Reset()
|
||
_ = ListCmd(d, "work", "INBOX", true, 0, 0, 50)
|
||
data := decode(t, buf.Bytes())["data"].(map[string]any)
|
||
msgs := data["messages"].([]any)
|
||
if len(msgs) != 0 { // 101 acked, 102 filtered
|
||
t.Fatalf("want 0 new messages, got %d", len(msgs))
|
||
}
|
||
}
|
||
```
|
||
|
||
- [ ] **Step 2: Run test to verify it fails**
|
||
|
||
Run: `go test ./internal/cli/ -run 'List|Get|Ack'`
|
||
Expected: FAIL — undefined `Deps`/`ListCmd`.
|
||
|
||
- [ ] **Step 3: Write the agent command implementation**
|
||
|
||
Create `internal/cli/agent.go`:
|
||
```go
|
||
package cli
|
||
|
||
import (
|
||
"io"
|
||
"time"
|
||
|
||
"git.dcglab.co.uk/steve/emcli/internal/mail"
|
||
"git.dcglab.co.uk/steve/emcli/internal/policy"
|
||
"git.dcglab.co.uk/steve/emcli/internal/store"
|
||
)
|
||
|
||
// Mailer is the subset of the IMAP client the agent commands use.
|
||
type Mailer interface {
|
||
SelectFolder(folder string) (uint32, uint32, error)
|
||
FetchHeaders(folder string, uids []uint32) ([]mail.Header, error)
|
||
FetchHeadersRange(folder string, since, before uint32, limit int) ([]mail.Header, error)
|
||
FetchFull(folder string, uid uint32) (mail.Message, error)
|
||
Search(folder string, sc mail.SearchCriteria, limit int) ([]mail.Header, error)
|
||
Logout() error
|
||
}
|
||
|
||
type Deps struct {
|
||
Store *store.Store
|
||
Dial func(store.Account) (Mailer, error)
|
||
Now func() time.Time
|
||
Out io.Writer
|
||
}
|
||
|
||
func (d Deps) emit(e Envelope) error { return e.Write(d.Out) }
|
||
|
||
func (d Deps) audit(account, action, target, result, reason string) {
|
||
_ = d.Store.Audit(d.Now(), store.AuditEntry{
|
||
Account: account, Action: action, Target: target, Result: result, Reason: reason,
|
||
})
|
||
}
|
||
|
||
// setup loads the account, builds the inbound rule, dials IMAP, and selects the
|
||
// folder (establishing the baseline). Returns a cleanup func.
|
||
func (d Deps) setup(account, folder string) (store.Account, policy.InboundRule, Mailer, func(), *Envelope) {
|
||
acc, err := d.Store.GetAccount(account)
|
||
if err != nil {
|
||
e := Failure(CodeNotFound, "account not found: "+account)
|
||
return acc, policy.InboundRule{}, nil, nil, &e
|
||
}
|
||
re, err := policy.CompileSubject(acc.SubjectRegex)
|
||
if err != nil {
|
||
e := Failure(CodeConfig, "invalid subject_regex: "+err.Error())
|
||
return acc, policy.InboundRule{}, nil, nil, &e
|
||
}
|
||
wlIn, _ := d.Store.ListWhitelist(account, store.DirIn)
|
||
rule := policy.InboundRule{
|
||
WhitelistInEnabled: acc.WhitelistInEnabled,
|
||
WhitelistIn: wlIn,
|
||
SubjectRegex: re,
|
||
}
|
||
m, err := d.Dial(acc)
|
||
if err != nil {
|
||
e := Failure(CodeNetwork, "imap connect failed: "+err.Error())
|
||
return acc, rule, nil, nil, &e
|
||
}
|
||
uidv, maxUID, err := m.SelectFolder(folder)
|
||
if err != nil {
|
||
m.Logout()
|
||
e := Failure(CodeNetwork, "select folder failed: "+err.Error())
|
||
return acc, rule, nil, nil, &e
|
||
}
|
||
if err := d.Store.EnsureFolderBaseline(account, folder, uidv, maxUID); err != nil {
|
||
m.Logout()
|
||
e := Failure(CodeDB, err.Error())
|
||
return acc, rule, nil, nil, &e
|
||
}
|
||
return acc, rule, m, func() { m.Logout() }, nil
|
||
}
|
||
|
||
func headerMap(h mail.Header) map[string]any {
|
||
return map[string]any{
|
||
"uid": h.UID, "from": h.From, "to": h.To, "subject": h.Subject,
|
||
"date": h.Date, "message_id": h.MessageID, "has_attachments": h.HasAttachments,
|
||
}
|
||
}
|
||
|
||
func ListCmd(d Deps, account, folder string, onlyNew bool, beforeUID, sinceUID uint32, limit int) error {
|
||
_, rule, m, done, fail := d.setup(account, folder)
|
||
if fail != nil {
|
||
return d.emit(*fail)
|
||
}
|
||
defer done()
|
||
|
||
var headers []mail.Header
|
||
var err error
|
||
if beforeUID > 0 || sinceUID > 0 {
|
||
headers, err = m.FetchHeadersRange(folder, sinceUID, beforeUID, limit)
|
||
} else {
|
||
headers, err = m.FetchHeaders(folder, nil)
|
||
}
|
||
if err != nil {
|
||
d.audit(account, "list", folder, "blocked", "imap_error")
|
||
return d.emit(Failure(CodeNetwork, err.Error()))
|
||
}
|
||
|
||
out := make([]map[string]any, 0, len(headers))
|
||
for _, h := range headers {
|
||
if !rule.Allows(h.From, h.Subject) {
|
||
continue // invisible
|
||
}
|
||
if onlyNew {
|
||
isNew, err := d.Store.IsNew(account, folder, h.UID)
|
||
if err != nil {
|
||
return d.emit(Failure(CodeDB, err.Error()))
|
||
}
|
||
if !isNew {
|
||
continue
|
||
}
|
||
}
|
||
out = append(out, headerMap(h))
|
||
if limit > 0 && len(out) >= limit {
|
||
break
|
||
}
|
||
}
|
||
d.audit(account, "list", folder, "allowed", "")
|
||
return d.emit(Success(map[string]any{"messages": out}))
|
||
}
|
||
|
||
// visible fetches a UID's header and reports whether policy allows it.
|
||
func (d Deps) visible(m Mailer, rule policy.InboundRule, folder string, uid uint32) (bool, error) {
|
||
hs, err := m.FetchHeaders(folder, []uint32{uid})
|
||
if err != nil {
|
||
return false, err
|
||
}
|
||
if len(hs) == 0 {
|
||
return false, nil
|
||
}
|
||
return rule.Allows(hs[0].From, hs[0].Subject), nil
|
||
}
|
||
|
||
func GetCmd(d Deps, account, folder string, uid uint32) error {
|
||
_, rule, m, done, fail := d.setup(account, folder)
|
||
if fail != nil {
|
||
return d.emit(*fail)
|
||
}
|
||
defer done()
|
||
|
||
ok, err := d.visible(m, rule, folder, uid)
|
||
if err != nil {
|
||
return d.emit(Failure(CodeNetwork, err.Error()))
|
||
}
|
||
if !ok {
|
||
d.audit(account, "get", uitoa(uid), "blocked", "filtered")
|
||
return d.emit(Failure(CodeNotFound, "message not found"))
|
||
}
|
||
msg, err := m.FetchFull(folder, uid)
|
||
if err != nil {
|
||
return d.emit(Failure(CodeNetwork, err.Error()))
|
||
}
|
||
atts := make([]map[string]any, 0, len(msg.Attachments))
|
||
for _, a := range msg.Attachments {
|
||
atts = append(atts, map[string]any{
|
||
"name": a.Name, "size": a.Size, "mime": a.MIME,
|
||
"content_b64": b64(a.Content),
|
||
})
|
||
}
|
||
d.audit(account, "get", uitoa(uid), "allowed", "")
|
||
return d.emit(Success(map[string]any{
|
||
"uid": msg.Header.UID, "from": msg.Header.From, "to": msg.Header.To,
|
||
"subject": msg.Header.Subject, "date": msg.Header.Date,
|
||
"message_id": msg.Header.MessageID, "body_text": msg.BodyText,
|
||
"attachments": atts,
|
||
}))
|
||
}
|
||
|
||
func SearchCmd(d Deps, account, folder string, sc mail.SearchCriteria, limit int) error {
|
||
_, rule, m, done, fail := d.setup(account, folder)
|
||
if fail != nil {
|
||
return d.emit(*fail)
|
||
}
|
||
defer done()
|
||
headers, err := m.Search(folder, sc, limit)
|
||
if err != nil {
|
||
return d.emit(Failure(CodeNetwork, err.Error()))
|
||
}
|
||
out := make([]map[string]any, 0, len(headers))
|
||
for _, h := range headers {
|
||
if !rule.Allows(h.From, h.Subject) {
|
||
continue
|
||
}
|
||
out = append(out, headerMap(h))
|
||
if limit > 0 && len(out) >= limit {
|
||
break
|
||
}
|
||
}
|
||
d.audit(account, "search", folder, "allowed", "")
|
||
return d.emit(Success(map[string]any{"messages": out}))
|
||
}
|
||
|
||
func AckCmd(d Deps, account, folder string, uids []uint32) error {
|
||
_, rule, m, done, fail := d.setup(account, folder)
|
||
if fail != nil {
|
||
return d.emit(*fail)
|
||
}
|
||
defer done()
|
||
uidv, _, _ := m.SelectFolder(folder)
|
||
for _, uid := range uids {
|
||
ok, err := d.visible(m, rule, folder, uid)
|
||
if err != nil {
|
||
return d.emit(Failure(CodeNetwork, err.Error()))
|
||
}
|
||
if !ok {
|
||
d.audit(account, "ack", uitoa(uid), "blocked", "filtered")
|
||
return d.emit(Failure(CodeNotFound, "message not found"))
|
||
}
|
||
}
|
||
if err := d.Store.Ack(account, folder, uidv, uids...); err != nil {
|
||
return d.emit(Failure(CodeDB, err.Error()))
|
||
}
|
||
d.audit(account, "ack", folder, "allowed", "")
|
||
return d.emit(Success(map[string]any{"acked": uintSlice(uids)}))
|
||
}
|
||
```
|
||
|
||
Create `internal/cli/dispatch.go` (helpers + the `Run` router skeleton used by `main`):
|
||
```go
|
||
package cli
|
||
|
||
import (
|
||
"encoding/base64"
|
||
"strconv"
|
||
)
|
||
|
||
func b64(b []byte) string { return base64.StdEncoding.EncodeToString(b) }
|
||
|
||
func uitoa(u uint32) string { return strconv.FormatUint(uint64(u), 10) }
|
||
|
||
func uintSlice(us []uint32) []uint64 {
|
||
out := make([]uint64, len(us))
|
||
for i, u := range us {
|
||
out[i] = uint64(u)
|
||
}
|
||
return out
|
||
}
|
||
```
|
||
|
||
- [ ] **Step 4: Run tests to verify they pass**
|
||
|
||
Run: `go test ./internal/cli/`
|
||
Expected: PASS (envelope + agent tests).
|
||
|
||
- [ ] **Step 5: Commit**
|
||
|
||
```bash
|
||
git add internal/cli/agent.go internal/cli/dispatch.go internal/cli/agent_test.go
|
||
git commit -m "feat(cli): agent read commands (list/get/search/ack) with policy filtering"
|
||
```
|
||
|
||
---
|
||
|
||
### Task 14: cli — flag parsing, real IMAP wiring & minimal admin
|
||
|
||
Wire the real IMAP client to the `Mailer` interface, parse flags for each agent command, add minimal flag-based admin (`account add`, `whitelist add`) so the tool is configurable end-to-end, and route everything from `main`. The interactive TUI is Phase 4; this gives a usable non-interactive admin surface now.
|
||
|
||
**Files:**
|
||
- Create: `internal/cli/run.go`
|
||
- Create: `internal/cli/admin.go`
|
||
- Modify: `cmd/emcli/main.go`
|
||
- Test: `internal/cli/run_test.go`
|
||
|
||
**Interfaces:**
|
||
- Consumes: everything above.
|
||
- Produces:
|
||
- `func Run(args []string, out, errOut io.Writer) int` — top-level router; returns process exit code.
|
||
- `func realMailer(acc store.Account) (Mailer, error)` — adapts `mail.Dial` to `Mailer`.
|
||
- Admin handlers writing human-readable output: `accountAdd`, `whitelistAdd`, `accountList`.
|
||
|
||
- [ ] **Step 1: Write the failing test**
|
||
|
||
Create `internal/cli/run_test.go`:
|
||
```go
|
||
package cli
|
||
|
||
import (
|
||
"bytes"
|
||
"encoding/json"
|
||
"strings"
|
||
"testing"
|
||
)
|
||
|
||
func TestRunUnknownCommand(t *testing.T) {
|
||
var out, errOut bytes.Buffer
|
||
code := Run([]string{"frobnicate"}, &out, &errOut)
|
||
if code == 0 {
|
||
t.Fatal("unknown command should be non-zero exit")
|
||
}
|
||
if !strings.Contains(errOut.String(), "unknown") {
|
||
t.Fatalf("stderr should mention unknown command: %q", errOut.String())
|
||
}
|
||
}
|
||
|
||
func TestRunVersionIsJSONForAgentButTextHere(t *testing.T) {
|
||
// `account list` with no DB key should fail closed with a usage/config error,
|
||
// proving the key check happens before any DB work.
|
||
var out, errOut bytes.Buffer
|
||
t.Setenv("EMCLI_KEY", "")
|
||
code := Run([]string{"account", "list"}, &out, &errOut)
|
||
if code == 0 {
|
||
t.Fatal("missing EMCLI_KEY must fail")
|
||
}
|
||
if !strings.Contains(out.String()+errOut.String(), "EMCLI_KEY") {
|
||
t.Fatalf("should mention EMCLI_KEY, got out=%q err=%q", out.String(), errOut.String())
|
||
}
|
||
}
|
||
|
||
func TestListUsageErrorIsJSON(t *testing.T) {
|
||
// Agent command with a missing required flag emits a JSON error envelope.
|
||
var out, errOut bytes.Buffer
|
||
t.Setenv("EMCLI_KEY", b64Key())
|
||
t.Setenv("EMCLI_DB", "") // default path is fine; command fails before connecting
|
||
code := Run([]string{"list"}, &out, &errOut) // missing --account
|
||
if code == 0 {
|
||
t.Fatal("missing --account should be non-zero")
|
||
}
|
||
var env map[string]any
|
||
if err := json.Unmarshal(out.Bytes(), &env); err != nil {
|
||
t.Fatalf("agent usage error must be JSON, got %q", out.String())
|
||
}
|
||
if env["error"] != true {
|
||
t.Fatalf("want error envelope: %v", env)
|
||
}
|
||
}
|
||
|
||
func b64Key() string {
|
||
// 32 zero bytes, base64.
|
||
return "AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA="
|
||
}
|
||
```
|
||
|
||
- [ ] **Step 2: Run test to verify it fails**
|
||
|
||
Run: `go test ./internal/cli/ -run Run`
|
||
Expected: FAIL — undefined `Run`.
|
||
|
||
- [ ] **Step 3: Write the router and admin**
|
||
|
||
Create `internal/cli/run.go`:
|
||
```go
|
||
package cli
|
||
|
||
import (
|
||
"flag"
|
||
"fmt"
|
||
"io"
|
||
"strconv"
|
||
"strings"
|
||
"time"
|
||
|
||
"git.dcglab.co.uk/steve/emcli/internal/crypto"
|
||
"git.dcglab.co.uk/steve/emcli/internal/mail"
|
||
"git.dcglab.co.uk/steve/emcli/internal/store"
|
||
)
|
||
|
||
func realMailer(acc store.Account) (Mailer, error) {
|
||
c, err := mail.Dial(mail.IMAPConfig{
|
||
Host: acc.IMAPHost, Port: acc.IMAPPort, Security: acc.IMAPSecurity,
|
||
Username: acc.Username, Password: acc.Password,
|
||
})
|
||
if err != nil {
|
||
return nil, err
|
||
}
|
||
return c, nil
|
||
}
|
||
|
||
// openStore loads the key and opens the DB, returning a human-readable error string.
|
||
func openStore() (*store.Store, error) {
|
||
key, err := crypto.KeyFromEnv()
|
||
if err != nil {
|
||
return nil, err
|
||
}
|
||
path, err := store.DefaultDBPath()
|
||
if err != nil {
|
||
return nil, err
|
||
}
|
||
return store.Open(path, key)
|
||
}
|
||
|
||
func newDepsLive(st *store.Store, out io.Writer) Deps {
|
||
return Deps{Store: st, Dial: realMailer, Now: time.Now, Out: out}
|
||
}
|
||
|
||
// Run routes a command line and returns an exit code.
|
||
func Run(args []string, out, errOut io.Writer) int {
|
||
if len(args) == 0 {
|
||
fmt.Fprintln(errOut, "emcli: no command given")
|
||
return 2
|
||
}
|
||
cmd, rest := args[0], args[1:]
|
||
switch cmd {
|
||
case "list", "get", "search", "ack":
|
||
return runAgent(cmd, rest, out, errOut)
|
||
case "account":
|
||
return runAccount(rest, out, errOut)
|
||
case "whitelist":
|
||
return runWhitelist(rest, out, errOut)
|
||
default:
|
||
fmt.Fprintf(errOut, "emcli: unknown command %q\n", cmd)
|
||
return 2
|
||
}
|
||
}
|
||
|
||
// runAgent handles JSON-emitting commands. Errors are emitted as JSON envelopes.
|
||
func runAgent(cmd string, args []string, out, errOut io.Writer) int {
|
||
fs := flag.NewFlagSet(cmd, flag.ContinueOnError)
|
||
fs.SetOutput(errOut)
|
||
account := fs.String("account", "", "account name")
|
||
folder := fs.String("folder", "INBOX", "folder/mailbox")
|
||
onlyNew := fs.Bool("new", false, "only new (unacked) messages")
|
||
limit := fs.Int("limit", 50, "max results (cap 500)")
|
||
uid := fs.Uint("uid", 0, "message UID (get)")
|
||
before := fs.Uint("before", 0, "list: UID cursor, older than")
|
||
since := fs.Uint("since", 0, "list: UID cursor, newer than")
|
||
from := fs.String("from", "", "search: sender")
|
||
subjectContains := fs.String("subject-contains", "", "search: subject substring")
|
||
text := fs.String("text", "", "search: full-text")
|
||
sinceDate := fs.String("since-date", "", "search: RFC3339 date lower bound")
|
||
beforeDate := fs.String("before-date", "", "search: RFC3339 date upper bound")
|
||
ackUIDs := fs.String("uid-list", "", "ack: comma-separated UIDs")
|
||
if err := fs.Parse(args); err != nil {
|
||
_ = Failure(CodeUsage, err.Error()).Write(out)
|
||
return 2
|
||
}
|
||
if *limit > 500 {
|
||
*limit = 500
|
||
}
|
||
if *account == "" {
|
||
_ = Failure(CodeUsage, "--account is required").Write(out)
|
||
return 2
|
||
}
|
||
st, err := openStore()
|
||
if err != nil {
|
||
_ = Failure(CodeConfig, err.Error()).Write(out)
|
||
return 1
|
||
}
|
||
defer st.Close()
|
||
_, _ = st.PurgeAudit(time.Now())
|
||
d := newDepsLive(st, out)
|
||
|
||
switch cmd {
|
||
case "list":
|
||
if err := ListCmd(d, *account, *folder, *onlyNew, u32(*before), u32(*since), *limit); err != nil {
|
||
return 1
|
||
}
|
||
case "get":
|
||
if *uid == 0 {
|
||
_ = Failure(CodeUsage, "--uid is required").Write(out)
|
||
return 2
|
||
}
|
||
if err := GetCmd(d, *account, *folder, u32(*uid)); err != nil {
|
||
return 1
|
||
}
|
||
case "search":
|
||
sc := mail.SearchCriteria{From: *from, SubjectContains: *subjectContains, Text: *text}
|
||
if *sinceDate != "" {
|
||
if tm, err := time.Parse(time.RFC3339, *sinceDate); err == nil {
|
||
sc.Since = tm
|
||
}
|
||
}
|
||
if *beforeDate != "" {
|
||
if tm, err := time.Parse(time.RFC3339, *beforeDate); err == nil {
|
||
sc.Before = tm
|
||
}
|
||
}
|
||
if err := SearchCmd(d, *account, *folder, sc, *limit); err != nil {
|
||
return 1
|
||
}
|
||
case "ack":
|
||
uids, err := parseUIDList(*ackUIDs)
|
||
if err != nil {
|
||
_ = Failure(CodeUsage, err.Error()).Write(out)
|
||
return 2
|
||
}
|
||
if err := AckCmd(d, *account, *folder, uids); err != nil {
|
||
return 1
|
||
}
|
||
}
|
||
return 0
|
||
}
|
||
|
||
func u32(u uint) uint32 { return uint32(u) }
|
||
|
||
func parseUIDList(s string) ([]uint32, error) {
|
||
if strings.TrimSpace(s) == "" {
|
||
return nil, fmt.Errorf("--uid-list is required")
|
||
}
|
||
var out []uint32
|
||
for _, part := range strings.Split(s, ",") {
|
||
n, err := strconv.ParseUint(strings.TrimSpace(part), 10, 32)
|
||
if err != nil {
|
||
return nil, fmt.Errorf("invalid uid %q", part)
|
||
}
|
||
out = append(out, uint32(n))
|
||
}
|
||
return out, nil
|
||
}
|
||
```
|
||
|
||
Create `internal/cli/admin.go`:
|
||
```go
|
||
package cli
|
||
|
||
import (
|
||
"flag"
|
||
"fmt"
|
||
"io"
|
||
|
||
"git.dcglab.co.uk/steve/emcli/internal/store"
|
||
)
|
||
|
||
// runAccount handles `account add|list`. Human-readable output (never JSON).
|
||
func runAccount(args []string, out, errOut io.Writer) int {
|
||
if len(args) == 0 {
|
||
fmt.Fprintln(errOut, "usage: emcli account <add|list>")
|
||
return 2
|
||
}
|
||
sub, rest := args[0], args[1:]
|
||
st, err := openStore()
|
||
if err != nil {
|
||
fmt.Fprintf(errOut, "emcli: %v\n", err)
|
||
return 1
|
||
}
|
||
defer st.Close()
|
||
|
||
switch sub {
|
||
case "add":
|
||
fs := flag.NewFlagSet("account add", flag.ContinueOnError)
|
||
fs.SetOutput(errOut)
|
||
name := fs.String("name", "", "account name")
|
||
mode := fs.String("mode", "RO", "RO|RW")
|
||
host := fs.String("imap-host", "", "IMAP host")
|
||
port := fs.Int("imap-port", 993, "IMAP port")
|
||
sec := fs.String("imap-security", "tls", "tls|starttls")
|
||
user := fs.String("username", "", "login username")
|
||
pass := fs.String("password", "", "login password")
|
||
subj := fs.String("subject-regex", "", "inbound subject filter")
|
||
wlIn := fs.Bool("whitelist-in", false, "enable inbound whitelist")
|
||
wlOut := fs.Bool("whitelist-out", false, "enable outbound whitelist")
|
||
backlog := fs.Bool("process-backlog", false, "treat existing mail as new")
|
||
if err := fs.Parse(rest); err != nil {
|
||
return 2
|
||
}
|
||
if *name == "" || *host == "" || *user == "" {
|
||
fmt.Fprintln(errOut, "name, imap-host, and username are required")
|
||
return 2
|
||
}
|
||
_, err := st.AddAccount(store.Account{
|
||
Name: *name, Mode: *mode, IMAPHost: *host, IMAPPort: *port, IMAPSecurity: *sec,
|
||
AuthType: "password", Username: *user, Password: *pass,
|
||
SubjectRegex: *subj, WhitelistInEnabled: *wlIn, WhitelistOutEnabled: *wlOut,
|
||
ProcessBacklog: *backlog,
|
||
})
|
||
if err != nil {
|
||
fmt.Fprintf(errOut, "add account: %v\n", err)
|
||
return 1
|
||
}
|
||
fmt.Fprintf(out, "account %q added (%s)\n", *name, *mode)
|
||
return 0
|
||
case "list":
|
||
accs, err := st.ListAccounts()
|
||
if err != nil {
|
||
fmt.Fprintf(errOut, "list: %v\n", err)
|
||
return 1
|
||
}
|
||
fmt.Fprintf(out, "%-16s %-4s %-28s %s\n", "NAME", "MODE", "IMAP", "USER")
|
||
for _, a := range accs {
|
||
fmt.Fprintf(out, "%-16s %-4s %-28s %s\n",
|
||
a.Name, a.Mode, fmt.Sprintf("%s:%d", a.IMAPHost, a.IMAPPort), a.Username)
|
||
}
|
||
return 0
|
||
default:
|
||
fmt.Fprintf(errOut, "unknown account subcommand %q\n", sub)
|
||
return 2
|
||
}
|
||
}
|
||
|
||
// runWhitelist handles `whitelist <in|out> add --account NAME --address A`.
|
||
func runWhitelist(args []string, out, errOut io.Writer) int {
|
||
if len(args) < 2 {
|
||
fmt.Fprintln(errOut, "usage: emcli whitelist <in|out> <add|remove|list> [flags]")
|
||
return 2
|
||
}
|
||
dir := store.Direction(args[0])
|
||
sub, rest := args[1], args[2:]
|
||
fs := flag.NewFlagSet("whitelist", flag.ContinueOnError)
|
||
fs.SetOutput(errOut)
|
||
account := fs.String("account", "", "account name")
|
||
address := fs.String("address", "", "email or @domain")
|
||
if err := fs.Parse(rest); err != nil {
|
||
return 2
|
||
}
|
||
if *account == "" {
|
||
fmt.Fprintln(errOut, "--account is required")
|
||
return 2
|
||
}
|
||
st, err := openStore()
|
||
if err != nil {
|
||
fmt.Fprintf(errOut, "emcli: %v\n", err)
|
||
return 1
|
||
}
|
||
defer st.Close()
|
||
|
||
switch sub {
|
||
case "add":
|
||
if err := st.AddWhitelist(*account, dir, *address); err != nil {
|
||
fmt.Fprintf(errOut, "add: %v\n", err)
|
||
return 1
|
||
}
|
||
fmt.Fprintf(out, "added %s to %s whitelist of %q\n", *address, dir, *account)
|
||
case "remove":
|
||
if err := st.RemoveWhitelist(*account, dir, *address); err != nil {
|
||
fmt.Fprintf(errOut, "remove: %v\n", err)
|
||
return 1
|
||
}
|
||
fmt.Fprintf(out, "removed %s\n", *address)
|
||
case "list":
|
||
addrs, err := st.ListWhitelist(*account, dir)
|
||
if err != nil {
|
||
fmt.Fprintf(errOut, "list: %v\n", err)
|
||
return 1
|
||
}
|
||
for _, a := range addrs {
|
||
fmt.Fprintln(out, a)
|
||
}
|
||
default:
|
||
fmt.Fprintf(errOut, "unknown whitelist subcommand %q\n", sub)
|
||
return 2
|
||
}
|
||
return 0
|
||
}
|
||
```
|
||
|
||
- [ ] **Step 4: Replace main to route through Run**
|
||
|
||
Replace `cmd/emcli/main.go` with:
|
||
```go
|
||
package main
|
||
|
||
import (
|
||
"fmt"
|
||
"os"
|
||
|
||
"git.dcglab.co.uk/steve/emcli/internal/cli"
|
||
"git.dcglab.co.uk/steve/emcli/internal/version"
|
||
)
|
||
|
||
func main() {
|
||
if len(os.Args) >= 2 && os.Args[1] == "version" {
|
||
fmt.Println(version.String)
|
||
return
|
||
}
|
||
os.Exit(cli.Run(os.Args[1:], os.Stdout, os.Stderr))
|
||
}
|
||
```
|
||
|
||
- [ ] **Step 5: Run all tests and build**
|
||
|
||
Run: `go test ./... && make build`
|
||
Expected: PASS across all packages; `emcli` builds.
|
||
|
||
- [ ] **Step 6: Manual smoke test**
|
||
|
||
Run:
|
||
```bash
|
||
export EMCLI_KEY=$(head -c32 /dev/zero | base64)
|
||
export EMCLI_DB=/tmp/emcli-smoke.db
|
||
./emcli account add --name demo --imap-host imap.example.com --username me@example.com --password x --whitelist-in
|
||
./emcli whitelist in add --account demo --address @trusted.com
|
||
./emcli account list
|
||
./emcli list --account demo --new # emits a JSON error envelope (cannot connect to example.com) — proves JSON-only output
|
||
rm -f /tmp/emcli-smoke.db
|
||
```
|
||
Expected: admin commands print human-readable lines; `list` prints a single JSON object with `"error": true` and a `network` code.
|
||
|
||
- [ ] **Step 7: Commit**
|
||
|
||
```bash
|
||
git add internal/cli/run.go internal/cli/admin.go internal/cli/run_test.go cmd/emcli/main.go
|
||
git commit -m "feat(cli): command router, real IMAP wiring, flag-based admin"
|
||
```
|
||
|
||
---
|
||
|
||
## Phase 1 Self-Review
|
||
|
||
- **Spec coverage (read path):** encrypted store ✓ (Tasks 2–4), per-account RO/RW field stored ✓ (Task 4; send-side enforcement is Phase 2), whitelist-in + subject filter ✓ (Tasks 5, 8, 9, 13), seen-set "new" + ack + compaction ✓ (Task 6), folder-specified reads + pointer-per-folder ✓ (Tasks 6, 11, 13), `list`/`get`/`search`/`ack` JSON ✓ (Tasks 12–14), headers-on-list / body+attachments-on-get ✓ (Tasks 10, 13), invisible inbound filtering incl. un-ackable filtered UIDs ✓ (Task 13), audit log + retention purge ✓ (Tasks 7, 14), single static binary + pure-Go SQLite ✓ (Tasks 1, 3, global constraints), `EMCLI_KEY`/`EMCLI_DB` handling and fail-closed ✓ (Tasks 2, 3, 14).
|
||
- **Deferred to later phases (intentional, not gaps):** SMTP `send` + outbound policy (Phase 2), OAuth2 loopback + token refresh and the `enc_oauth_*` columns wiring (Phase 3), interactive TUI `init`/reconfigure and `doctor` (Phase 4). The schema already includes the OAuth columns so no migration is needed later.
|
||
- **Placeholder scan:** no TBD/TODO; every code step is complete and compilable in sequence.
|
||
- **Type consistency:** `Mailer` interface in Task 13 matches the concrete `*mail.Client` methods in Task 11 (`SelectFolder`, `FetchHeaders`, `FetchHeadersRange`, `FetchFull`, `Search`, `Logout`); `store` method names (`EnsureFolderBaseline`, `IsNew`, `Ack`, `Audit`, `ListWhitelist`) are used consistently across Tasks 6, 7, 13.
|
||
|
||
## Notes for the implementer
|
||
|
||
- Go-imap v1's `client.Client` is not safe for concurrent use; each command dials its own connection, which is correct for a per-invocation CLI.
|
||
- Keep `time.Now()` out of `store`; it is always injected (`Deps.Now`, `Audit(now, …)`) so tests are deterministic — the workflow runtime forbids wall-clock calls in some contexts and determinism aids review.
|
||
- When in doubt about an emersion API detail, prefer the smallest change that keeps the `Mailer` interface stable; the CLI tests depend only on the interface, not the concrete client.
|