feat(send): Phase 2 send path — SMTP, MIME, reply threading, outbound policy

Adds the `send` agent command and everything behind it:

- store: Account carries SMTP host/port/security (NULL-safe scan/insert/select);
  admin `account add` gains --smtp-* flags (applied for RW accounts).
- policy: OutboundRule.Check(recipients) → (ok, reason); RO ⇒ ro_mode,
  whitelist-out blocks the whole send if any recipient fails (no partial send).
- mail: Header.References; OutgoingMessage + BuildMIME (plain text + attachments,
  In-Reply-To/References threading, Bcc envelope-only); SendSMTP (tls/starttls,
  SASL PLAIN, envelope send) via emersion/go-smtp.
- cli: SendCmd gates outbound, resolves --reply-to under the inbound filter
  (filtered/absent source ⇒ not_found), reads attachments, audits, emits the
  JSON envelope; repeatable --to/--cc/--bcc/--attach flags wired into the router.

Implemented test-first; full suite passes incl -race. Validated live against
friday.mxlogin.com: real send to me@stevecliff.com, RO + whitelist-out blocks,
and --reply-to threading off a live INBOX message. test-creds.md gitignored.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This commit is contained in:
2026-06-22 17:39:07 +01:00
parent 3224a87b6e
commit c99eaedafd
17 changed files with 923 additions and 15 deletions
+1
View File
@@ -14,6 +14,7 @@ coverage.*
*.sqlite
.env
.env.*
test-creds.md
# Editor / OS
.DS_Store
+2 -1
View File
@@ -5,12 +5,13 @@ go 1.25.0
require (
github.com/emersion/go-imap v1.2.1
github.com/emersion/go-message v0.18.2
github.com/emersion/go-sasl v0.0.0-20241020182733-b788ff22d5a6
github.com/emersion/go-smtp v0.24.0
modernc.org/sqlite v1.53.0
)
require (
github.com/dustin/go-humanize v1.0.1 // indirect
github.com/emersion/go-sasl v0.0.0-20200509203442-7bfe0ed36a21 // indirect
github.com/google/uuid v1.6.0 // indirect
github.com/mattn/go-isatty v0.0.20 // indirect
github.com/ncruces/go-strftime v1.0.0 // indirect
+4 -1
View File
@@ -5,8 +5,11 @@ github.com/emersion/go-imap v1.2.1/go.mod h1:Qlx1FSx2FTxjnjWpIlVNEuX+ylerZQNFE5N
github.com/emersion/go-message v0.15.0/go.mod h1:wQUEfE+38+7EW8p8aZ96ptg6bAb1iwdgej19uXASlE4=
github.com/emersion/go-message v0.18.2 h1:rl55SQdjd9oJcIoQNhubD2Acs1E6IzlZISRTK7x/Lpg=
github.com/emersion/go-message v0.18.2/go.mod h1:XpJyL70LwRvq2a8rVbHXikPgKj8+aI0kGdHlg16ibYA=
github.com/emersion/go-sasl v0.0.0-20200509203442-7bfe0ed36a21 h1:OJyUGMJTzHTd1XQp98QTaHernxMYzRaOasRir9hUlFQ=
github.com/emersion/go-sasl v0.0.0-20200509203442-7bfe0ed36a21/go.mod h1:iL2twTeMvZnrg54ZoPDNfJaJaqy0xIQFuBdrLsmspwQ=
github.com/emersion/go-sasl v0.0.0-20241020182733-b788ff22d5a6 h1:oP4q0fw+fOSWn3DfFi4EXdT+B+gTtzx8GC9xsc26Znk=
github.com/emersion/go-sasl v0.0.0-20241020182733-b788ff22d5a6/go.mod h1:iL2twTeMvZnrg54ZoPDNfJaJaqy0xIQFuBdrLsmspwQ=
github.com/emersion/go-smtp v0.24.0 h1:g6AfoF140mvW0vLNPD/LuCBLEAdlxOjIXqbIkJIS6Wk=
github.com/emersion/go-smtp v0.24.0/go.mod h1:ZtRRkbTyp2XTHCA+BmyTFTrj8xY4I+b4McvHxCU2gsQ=
github.com/emersion/go-textwrapper v0.0.0-20200911093747-65d896831594/go.mod h1:aqO8z8wPrjkscevZJFVE1wXJrLpC5LtJG7fqLOsPb2U=
github.com/google/pprof v0.0.0-20250317173921-a4b03ec1a45e h1:ijClszYn+mADRFY17kjQEVQ1XRhq2/JR1M3sGqeJoxs=
github.com/google/pprof v0.0.0-20250317173921-a4b03ec1a45e/go.mod h1:boTsfXsheKC2y+lKOCMpSfarhxDeIzfZG1jqGcPl3cA=
+9 -2
View File
@@ -31,6 +31,9 @@ func runAccount(args []string, out, errOut io.Writer) int {
host := fs.String("imap-host", "", "IMAP host")
port := fs.Int("imap-port", 993, "IMAP port")
sec := fs.String("imap-security", "tls", "tls|starttls")
smtpHost := fs.String("smtp-host", "", "SMTP host (RW accounts)")
smtpPort := fs.Int("smtp-port", 465, "SMTP port")
smtpSec := fs.String("smtp-security", "tls", "tls|starttls")
user := fs.String("username", "", "login username")
pass := fs.String("password", "", "login password")
subj := fs.String("subject-regex", "", "inbound subject filter")
@@ -44,12 +47,16 @@ func runAccount(args []string, out, errOut io.Writer) int {
fmt.Fprintln(errOut, "name, imap-host, and username are required")
return 2
}
_, err := st.AddAccount(store.Account{
acc := store.Account{
Name: *name, Mode: *mode, IMAPHost: *host, IMAPPort: *port, IMAPSecurity: *sec,
AuthType: "password", Username: *user, Password: *pass,
SubjectRegex: *subj, WhitelistInEnabled: *wlIn, WhitelistOutEnabled: *wlOut,
ProcessBacklog: *backlog,
})
}
if *mode == "RW" {
acc.SMTPHost, acc.SMTPPort, acc.SMTPSecurity = *smtpHost, *smtpPort, *smtpSec
}
_, err := st.AddAccount(acc)
if err != nil {
fmt.Fprintf(errOut, "add account: %v\n", err)
return 1
+1
View File
@@ -27,6 +27,7 @@ type Mailer interface {
type Deps struct {
Store *store.Store
Dial func(store.Account) (Mailer, error)
Send func(store.Account, mail.OutgoingMessage) error
Now func() time.Time
Out io.Writer
}
+60 -1
View File
@@ -37,8 +37,15 @@ func openStore() (*store.Store, error) {
return store.Open(path, key)
}
func realSender(acc store.Account, m mail.OutgoingMessage) error {
return mail.SendSMTP(mail.SMTPConfig{
Host: acc.SMTPHost, Port: acc.SMTPPort, Security: acc.SMTPSecurity,
Username: acc.Username, Password: acc.Password,
}, m)
}
func newDepsLive(st *store.Store, out io.Writer) Deps {
return Deps{Store: st, Dial: realMailer, Now: time.Now, Out: out}
return Deps{Store: st, Dial: realMailer, Send: realSender, Now: time.Now, Out: out}
}
// Run routes a command line and returns an exit code.
@@ -51,6 +58,8 @@ func Run(args []string, out, errOut io.Writer) int {
switch cmd {
case "list", "get", "search", "ack":
return runAgent(cmd, rest, out, errOut)
case "send":
return runSend(rest, out, errOut)
case "account":
return runAccount(rest, out, errOut)
case "whitelist":
@@ -139,6 +148,56 @@ func runAgent(cmd string, args []string, out, errOut io.Writer) int {
return 0
}
// stringSlice is a repeatable string flag that also splits comma-separated
// values, so `--to a@x --to b@x` and `--to a@x,b@x` both yield two recipients.
type stringSlice []string
func (s *stringSlice) String() string { return strings.Join(*s, ",") }
func (s *stringSlice) Set(v string) error {
for _, part := range strings.Split(v, ",") {
if p := strings.TrimSpace(part); p != "" {
*s = append(*s, p)
}
}
return nil
}
// runSend handles the `send` agent command (JSON envelope output).
func runSend(args []string, out, errOut io.Writer) int {
fs := flag.NewFlagSet("send", flag.ContinueOnError)
fs.SetOutput(errOut)
account := fs.String("account", "", "account name")
var to, cc, bcc, attach stringSlice
fs.Var(&to, "to", "recipient (repeatable / comma-separated)")
fs.Var(&cc, "cc", "cc recipient (repeatable / comma-separated)")
fs.Var(&bcc, "bcc", "bcc recipient (repeatable / comma-separated)")
fs.Var(&attach, "attach", "attachment file path (repeatable)")
subject := fs.String("subject", "", "subject")
body := fs.String("body", "", "plain-text body")
replyTo := fs.Uint("reply-to", 0, "source UID to reply to (threading)")
folder := fs.String("folder", "INBOX", "folder of the reply source")
if err := fs.Parse(args); err != nil {
_ = Failure(CodeUsage, err.Error()).Write(out)
return 2
}
if *account == "" {
_ = Failure(CodeUsage, "--account is required").Write(out)
return 2
}
st, err := openStore()
if err != nil {
_ = Failure(CodeConfig, err.Error()).Write(out)
return 1
}
defer st.Close()
_, _ = st.PurgeAudit(time.Now())
d := newDepsLive(st, out)
if err := SendCmd(d, *account, to, cc, bcc, *subject, *body, attach, u32(*replyTo), *folder); err != nil {
return 1
}
return 0
}
func u32(u uint) uint32 { return uint32(u) }
func parseUIDList(s string) ([]uint32, error) {
+92
View File
@@ -0,0 +1,92 @@
package cli
import (
"mime"
"os"
"path/filepath"
"strings"
"git.dcglab.co.uk/steve/emcli/internal/mail"
"git.dcglab.co.uk/steve/emcli/internal/policy"
"git.dcglab.co.uk/steve/emcli/internal/store"
)
// SendCmd sends a plain-text message (with optional attachments) via the
// account's SMTP endpoint, enforcing outbound policy (RO rejection +
// whitelist-out). When replyToUID > 0, the source message is read from
// replyFolder (subject to inbound filtering) and its Message-ID/References are
// used to thread the reply.
func SendCmd(d Deps, account string, to, cc, bcc []string, subject, body string, attachPaths []string, replyToUID uint32, replyFolder string) error {
acc, err := d.Store.GetAccount(account)
if err != nil {
return d.emit(Failure(CodeNotFound, "account not found: "+account))
}
msg := mail.OutgoingMessage{
From: acc.Username, To: to, Cc: cc, Bcc: bcc,
Subject: subject, BodyText: body,
}
recipients := msg.Recipients()
if len(recipients) == 0 {
return d.emit(Failure(CodeUsage, "at least one recipient is required"))
}
// Outbound enforcement first — block before any network I/O.
wlOut, _ := d.Store.ListWhitelist(account, store.DirOut)
rule := policy.OutboundRule{
Mode: acc.Mode,
WhitelistOutEnabled: acc.WhitelistOutEnabled,
WhitelistOut: wlOut,
}
if ok, reason := rule.Check(recipients); !ok {
d.audit(account, "send", strings.Join(recipients, ","), "blocked", reason)
return d.emit(Failure(CodePolicy, "send blocked: "+reason))
}
// Reply threading: resolve the source's Message-ID/References, applying the
// inbound filter so a hidden source cannot be replied to.
if replyToUID > 0 {
_, irule, m, _, done, fail := d.setup(account, replyFolder)
if fail != nil {
return d.emit(*fail)
}
defer done()
hs, err := m.FetchHeaders(replyFolder, []uint32{replyToUID})
if err != nil {
return d.emit(Failure(CodeNetwork, err.Error()))
}
if len(hs) == 0 || !irule.Allows(hs[0].From, hs[0].Subject) {
d.audit(account, "send", uitoa(replyToUID), "blocked", "filtered")
return d.emit(Failure(CodeNotFound, "reply source not found"))
}
src := hs[0]
if src.MessageID != "" {
msg.InReplyTo = src.MessageID
msg.References = append(append([]string{}, src.References...), src.MessageID)
}
}
// Read attachments from disk.
for _, p := range attachPaths {
content, err := os.ReadFile(p)
if err != nil {
return d.emit(Failure(CodeUsage, "read attachment: "+err.Error()))
}
mimeType := mime.TypeByExtension(filepath.Ext(p))
if mimeType == "" {
mimeType = "application/octet-stream"
}
msg.Attachments = append(msg.Attachments, mail.Attachment{
Name: filepath.Base(p), Size: len(content), MIME: mimeType, Content: content,
})
}
if err := d.Send(acc, msg); err != nil {
d.audit(account, "send", strings.Join(recipients, ","), "blocked", "smtp_error")
return d.emit(Failure(CodeNetwork, err.Error()))
}
d.audit(account, "send", strings.Join(recipients, ","), "allowed", "")
return d.emit(Success(map[string]any{
"sent": true, "recipients": recipients,
}))
}
+162
View File
@@ -0,0 +1,162 @@
package cli
import (
"path/filepath"
"testing"
"time"
"git.dcglab.co.uk/steve/emcli/internal/mail"
"git.dcglab.co.uk/steve/emcli/internal/store"
)
// sendDeps builds Deps wired to a recording fake sender and an optional fake
// mailer (for reply-to). The named account is created per the supplied template.
func sendDeps(t *testing.T, acc store.Account, fm *fakeMailer) (Deps, *[]mail.OutgoingMessage, *[]byte) {
t.Helper()
st, err := store.Open(filepath.Join(t.TempDir(), "e.db"), testKey())
if err != nil {
t.Fatalf("store: %v", err)
}
t.Cleanup(func() { st.Close() })
if _, err := st.AddAccount(acc); err != nil {
t.Fatalf("AddAccount: %v", err)
}
var sent []mail.OutgoingMessage
buf := &[]byte{}
d := Deps{
Store: st,
Dial: func(store.Account) (Mailer, error) { return fm, nil },
Now: func() time.Time { return time.Date(2026, 6, 22, 0, 0, 0, 0, time.UTC) },
Out: bufWriter{buf},
Send: func(_ store.Account, m mail.OutgoingMessage) error {
sent = append(sent, m)
return nil
},
}
return d, &sent, buf
}
type bufWriter struct{ b *[]byte }
func (w bufWriter) Write(p []byte) (int, error) { *w.b = append(*w.b, p...); return len(p), nil }
func rwAccount() store.Account {
return store.Account{
Name: "send", Mode: "RW", IMAPHost: "h", IMAPPort: 993, IMAPSecurity: "tls",
SMTPHost: "h", SMTPPort: 465, SMTPSecurity: "tls",
AuthType: "password", Username: "emcli@stevecliff.com", Password: "pw",
}
}
func TestSendHappyPath(t *testing.T) {
d, sent, buf := sendDeps(t, rwAccount(), nil)
err := SendCmd(d, "send", []string{"me@stevecliff.com"}, nil, nil, "hi", "body", nil, 0, "INBOX")
if err != nil {
t.Fatalf("SendCmd: %v", err)
}
res := decode(t, *buf)
if res["error"] != false {
t.Fatalf("unexpected error: %v", res)
}
if len(*sent) != 1 {
t.Fatalf("want 1 send, got %d", len(*sent))
}
m := (*sent)[0]
if len(m.To) != 1 || m.To[0] != "me@stevecliff.com" || m.Subject != "hi" {
t.Fatalf("wrong outgoing message: %+v", m)
}
}
func TestSendROBlocked(t *testing.T) {
acc := rwAccount()
acc.Mode = "RO"
d, sent, buf := sendDeps(t, acc, nil)
_ = SendCmd(d, "send", []string{"me@stevecliff.com"}, nil, nil, "hi", "body", nil, 0, "INBOX")
res := decode(t, *buf)
if res["error"] != true {
t.Fatal("RO send must be blocked")
}
if len(*sent) != 0 {
t.Fatal("sender must not be called when blocked")
}
}
func TestSendWhitelistOutBlocked(t *testing.T) {
acc := rwAccount()
acc.WhitelistOutEnabled = true
d, sent, buf := sendDeps(t, acc, nil)
_ = d.Store.AddWhitelist("send", store.DirOut, "@stevecliff.com")
// recipient not on the whitelist -> blocked
_ = SendCmd(d, "send", []string{"stranger@elsewhere.com"}, nil, nil, "hi", "body", nil, 0, "INBOX")
res := decode(t, *buf)
if res["error"] != true {
t.Fatal("non-whitelisted recipient must be blocked")
}
if len(*sent) != 0 {
t.Fatal("sender must not be called when blocked")
}
}
func TestSendWhitelistOutAllowed(t *testing.T) {
acc := rwAccount()
acc.WhitelistOutEnabled = true
d, sent, _ := sendDeps(t, acc, nil)
_ = d.Store.AddWhitelist("send", store.DirOut, "@stevecliff.com")
if err := SendCmd(d, "send", []string{"me@stevecliff.com"}, nil, nil, "hi", "body", nil, 0, "INBOX"); err != nil {
t.Fatalf("SendCmd: %v", err)
}
if len(*sent) != 1 {
t.Fatalf("whitelisted recipient should be sent, got %d", len(*sent))
}
}
func TestSendReplyToThreadsHeaders(t *testing.T) {
acc := rwAccount()
fm := &fakeMailer{
uidValidity: 1, maxUID: 100,
headers: []mail.Header{
{UID: 42, From: "me@stevecliff.com", Subject: "orig",
MessageID: "orig@stevecliff.com", References: []string{"root@stevecliff.com"}},
},
}
d, sent, _ := sendDeps(t, acc, fm)
if err := SendCmd(d, "send", []string{"me@stevecliff.com"}, nil, nil, "re: orig", "body", nil, 42, "INBOX"); err != nil {
t.Fatalf("SendCmd: %v", err)
}
if len(*sent) != 1 {
t.Fatalf("want 1 send, got %d", len(*sent))
}
m := (*sent)[0]
if m.InReplyTo != "orig@stevecliff.com" {
t.Fatalf("InReplyTo=%q", m.InReplyTo)
}
// References should chain the source's references then the source itself.
if len(m.References) != 2 || m.References[0] != "root@stevecliff.com" || m.References[1] != "orig@stevecliff.com" {
t.Fatalf("References=%v", m.References)
}
}
func TestSendReplyToFilteredSourceNotFound(t *testing.T) {
acc := rwAccount()
acc.WhitelistInEnabled = true // inbound filter active
fm := &fakeMailer{
uidValidity: 1, maxUID: 100,
headers: []mail.Header{
{UID: 42, From: "eve@evil.com", Subject: "spam", MessageID: "spam@evil.com"},
},
}
d, sent, buf := sendDeps(t, acc, fm)
_ = d.Store.AddWhitelist("send", store.DirIn, "@stevecliff.com") // 42 is from evil.com => filtered
_ = SendCmd(d, "send", []string{"me@stevecliff.com"}, nil, nil, "re", "body", nil, 42, "INBOX")
res := decode(t, *buf)
if res["error"] != true {
t.Fatal("reply to a filtered source must fail")
}
ed := res["error_detail"].(map[string]any)
if ed["code"] != "not_found" {
t.Fatalf("want not_found, got %v", ed["code"])
}
if len(*sent) != 0 {
t.Fatal("must not send when reply source is filtered")
}
}
+4
View File
@@ -17,6 +17,7 @@ type Header struct {
Subject string
Date string
MessageID string
References []string // parsed Message-IDs (no angle brackets), oldest first
HasAttachments bool
}
@@ -51,6 +52,9 @@ func readHeader(mr *mail.Reader, uid uint32) Header {
} else {
h.MessageID = strings.Trim(hd.Get("Message-Id"), "<> ")
}
if refs, err := hd.MsgIDList("References"); err == nil && len(refs) > 0 {
h.References = refs
}
return h
}
+170
View File
@@ -0,0 +1,170 @@
package mail
import (
"bytes"
"crypto/tls"
"fmt"
"io"
"time"
gomail "github.com/emersion/go-message/mail"
"github.com/emersion/go-sasl"
"github.com/emersion/go-smtp"
)
// SMTPConfig is the connection/auth detail for an account's send endpoint.
type SMTPConfig struct {
Host string
Port int
Security string // tls | starttls
Username string
Password string
}
// OutgoingMessage is a plain-text message (with optional attachments) to send.
// InReplyTo and References hold bare Message-IDs (no angle brackets) and, when
// set, produce threading headers so a reply chains onto its source.
type OutgoingMessage struct {
From string
To []string
Cc []string
Bcc []string
Subject string
BodyText string
Attachments []Attachment
InReplyTo string
References []string
Date time.Time
}
// Recipients returns the full envelope recipient set: to + cc + bcc.
func (m OutgoingMessage) Recipients() []string {
out := make([]string, 0, len(m.To)+len(m.Cc)+len(m.Bcc))
out = append(out, m.To...)
out = append(out, m.Cc...)
out = append(out, m.Bcc...)
return out
}
func addrList(addrs []string) []*gomail.Address {
out := make([]*gomail.Address, 0, len(addrs))
for _, a := range addrs {
if parsed, err := gomail.ParseAddress(a); err == nil {
out = append(out, parsed)
} else {
out = append(out, &gomail.Address{Address: a})
}
}
return out
}
// BuildMIME renders an OutgoingMessage to RFC822 bytes: a plain-text inline part
// plus any attachments. Bcc recipients are intentionally omitted from the headers
// (they travel only in the SMTP envelope).
func BuildMIME(m OutgoingMessage) ([]byte, error) {
var h gomail.Header
if m.Date.IsZero() {
m.Date = time.Now()
}
h.SetDate(m.Date)
h.SetAddressList("From", addrList([]string{m.From}))
if len(m.To) > 0 {
h.SetAddressList("To", addrList(m.To))
}
if len(m.Cc) > 0 {
h.SetAddressList("Cc", addrList(m.Cc))
}
h.SetSubject(m.Subject)
if m.InReplyTo != "" {
h.SetMsgIDList("In-Reply-To", []string{m.InReplyTo})
}
if len(m.References) > 0 {
h.SetMsgIDList("References", m.References)
}
if err := h.GenerateMessageID(); err != nil {
return nil, fmt.Errorf("generate message-id: %w", err)
}
var buf bytes.Buffer
w, err := gomail.CreateWriter(&buf, h)
if err != nil {
return nil, fmt.Errorf("create mail writer: %w", err)
}
tw, err := w.CreateInline()
if err != nil {
return nil, fmt.Errorf("create inline: %w", err)
}
var th gomail.InlineHeader
th.Set("Content-Type", "text/plain; charset=utf-8")
pw, err := tw.CreatePart(th)
if err != nil {
return nil, fmt.Errorf("create text part: %w", err)
}
if _, err := io.WriteString(pw, m.BodyText); err != nil {
return nil, fmt.Errorf("write body: %w", err)
}
if err := pw.Close(); err != nil {
return nil, err
}
if err := tw.Close(); err != nil {
return nil, err
}
for _, a := range m.Attachments {
var ah gomail.AttachmentHeader
if a.MIME != "" {
ah.Set("Content-Type", a.MIME)
}
ah.SetFilename(a.Name)
aw, err := w.CreateAttachment(ah)
if err != nil {
return nil, fmt.Errorf("create attachment %q: %w", a.Name, err)
}
if _, err := aw.Write(a.Content); err != nil {
return nil, fmt.Errorf("write attachment %q: %w", a.Name, err)
}
if err := aw.Close(); err != nil {
return nil, err
}
}
if err := w.Close(); err != nil {
return nil, err
}
return buf.Bytes(), nil
}
// SendSMTP builds the MIME body and delivers it via the account's SMTP endpoint,
// authenticating with SASL PLAIN. The envelope sender is m.From and the envelope
// recipients are m.Recipients() (to + cc + bcc).
func SendSMTP(cfg SMTPConfig, m OutgoingMessage) error {
raw, err := BuildMIME(m)
if err != nil {
return err
}
addr := fmt.Sprintf("%s:%d", cfg.Host, cfg.Port)
tlsConf := &tls.Config{ServerName: cfg.Host}
var c *smtp.Client
switch cfg.Security {
case "tls":
c, err = smtp.DialTLS(addr, tlsConf)
case "starttls":
c, err = smtp.DialStartTLS(addr, tlsConf)
default:
return fmt.Errorf("unknown smtp security %q", cfg.Security)
}
if err != nil {
return fmt.Errorf("smtp connect: %w", err)
}
defer c.Close()
auth := sasl.NewPlainClient("", cfg.Username, cfg.Password)
if err := c.Auth(auth); err != nil {
return fmt.Errorf("smtp auth: %w", err)
}
if err := c.SendMail(m.From, m.Recipients(), bytes.NewReader(raw)); err != nil {
return fmt.Errorf("smtp send: %w", err)
}
return c.Quit()
}
+117
View File
@@ -0,0 +1,117 @@
package mail
import (
"strings"
"testing"
"time"
)
func TestBuildMIMERoundTrip(t *testing.T) {
msg := OutgoingMessage{
From: "emcli@stevecliff.com",
To: []string{"me@stevecliff.com"},
Cc: []string{"cc@stevecliff.com"},
Subject: "hello from emcli",
BodyText: "this is the body\nwith two lines",
Date: time.Date(2026, 6, 22, 12, 0, 0, 0, time.UTC),
InReplyTo: "abc123@origin.example",
References: []string{"root@origin.example", "abc123@origin.example"},
Attachments: []Attachment{
{Name: "note.txt", MIME: "text/plain", Content: []byte("attached bytes")},
},
}
raw, err := BuildMIME(msg)
if err != nil {
t.Fatalf("BuildMIME: %v", err)
}
// Headers present in the raw bytes.
rawStr := string(raw)
for _, want := range []string{
"From:", "emcli@stevecliff.com",
"To:", "me@stevecliff.com",
"Cc:", "cc@stevecliff.com",
"Subject:", "hello from emcli",
"In-Reply-To:", "abc123@origin.example",
"References:", "root@origin.example",
} {
if !strings.Contains(rawStr, want) {
t.Fatalf("MIME missing %q in:\n%s", want, rawStr)
}
}
// Round-trips back through the parser: body and attachment survive.
parsed, err := ParseMessage(0, raw)
if err != nil {
t.Fatalf("ParseMessage: %v", err)
}
gotBody := strings.ReplaceAll(strings.TrimSpace(parsed.BodyText), "\r\n", "\n")
if gotBody != "this is the body\nwith two lines" {
t.Fatalf("body not preserved: %q", parsed.BodyText)
}
if len(parsed.Attachments) != 1 {
t.Fatalf("want 1 attachment, got %d", len(parsed.Attachments))
}
if parsed.Attachments[0].Name != "note.txt" || string(parsed.Attachments[0].Content) != "attached bytes" {
t.Fatalf("attachment not preserved: %+v", parsed.Attachments[0])
}
}
func TestBuildMIMENoAttachments(t *testing.T) {
msg := OutgoingMessage{
From: "emcli@stevecliff.com",
To: []string{"me@stevecliff.com"},
Subject: "plain",
BodyText: "just text",
Date: time.Date(2026, 6, 22, 12, 0, 0, 0, time.UTC),
}
raw, err := BuildMIME(msg)
if err != nil {
t.Fatalf("BuildMIME: %v", err)
}
parsed, err := ParseMessage(0, raw)
if err != nil {
t.Fatalf("ParseMessage: %v", err)
}
if strings.TrimSpace(parsed.BodyText) != "just text" {
t.Fatalf("body not preserved: %q", parsed.BodyText)
}
if len(parsed.Attachments) != 0 {
t.Fatalf("want 0 attachments, got %d", len(parsed.Attachments))
}
}
func TestRecipientsCombinesAllFields(t *testing.T) {
msg := OutgoingMessage{
To: []string{"a@x.com"},
Cc: []string{"b@x.com"},
Bcc: []string{"c@x.com"},
}
got := msg.Recipients()
want := []string{"a@x.com", "b@x.com", "c@x.com"}
if len(got) != len(want) {
t.Fatalf("Recipients()=%v want %v", got, want)
}
for i := range want {
if got[i] != want[i] {
t.Fatalf("Recipients()=%v want %v", got, want)
}
}
}
func TestReadHeaderParsesReferences(t *testing.T) {
raw := "From: a@x.com\r\n" +
"To: b@x.com\r\n" +
"Subject: re: hi\r\n" +
"Message-Id: <reply@x.com>\r\n" +
"References: <root@x.com> <mid@x.com>\r\n" +
"\r\nbody\r\n"
h, err := ParseHeaderBytes(7, []byte(raw))
if err != nil {
t.Fatalf("ParseHeaderBytes: %v", err)
}
if len(h.References) != 2 || h.References[0] != "root@x.com" || h.References[1] != "mid@x.com" {
t.Fatalf("References not parsed: %v", h.References)
}
}
+26
View File
@@ -0,0 +1,26 @@
package policy
// OutboundRule captures one account's send-side enforcement.
type OutboundRule struct {
Mode string // RO | RW
WhitelistOutEnabled bool
WhitelistOut []string
}
// Check reports whether a send to the given recipient set is permitted. On a
// block it returns a stable reason: "ro_mode" (account is read-only) or
// "whitelist_out" (a recipient is not whitelisted). The whole send is blocked
// if any single recipient fails — there is no partial send.
func (r OutboundRule) Check(recipients []string) (bool, string) {
if r.Mode == "RO" {
return false, "ro_mode"
}
if r.WhitelistOutEnabled {
for _, addr := range recipients {
if !MatchAddress(r.WhitelistOut, addr) {
return false, "whitelist_out"
}
}
}
return true, ""
}
+64
View File
@@ -0,0 +1,64 @@
package policy
import "testing"
func TestOutboundRuleCheck(t *testing.T) {
cases := []struct {
name string
rule OutboundRule
recipients []string
wantOK bool
wantReason string
}{
{
name: "RO mode always blocked",
rule: OutboundRule{Mode: "RO"},
recipients: []string{"a@x.com"},
wantOK: false,
wantReason: "ro_mode",
},
{
name: "RW no whitelist allows anything",
rule: OutboundRule{Mode: "RW"},
recipients: []string{"anyone@anywhere.com"},
wantOK: true,
},
{
name: "whitelist-out all match allows",
rule: OutboundRule{Mode: "RW", WhitelistOutEnabled: true, WhitelistOut: []string{"bob@x.com", "@trusted.com"}},
recipients: []string{"bob@x.com", "ann@trusted.com"},
wantOK: true,
},
{
name: "whitelist-out domain match",
rule: OutboundRule{Mode: "RW", WhitelistOutEnabled: true, WhitelistOut: []string{"@trusted.com"}},
recipients: []string{"ANN@Trusted.com"},
wantOK: true,
},
{
name: "one bad recipient blocks whole send",
rule: OutboundRule{Mode: "RW", WhitelistOutEnabled: true, WhitelistOut: []string{"@trusted.com"}},
recipients: []string{"ann@trusted.com", "eve@evil.com"},
wantOK: false,
wantReason: "whitelist_out",
},
{
name: "RO takes precedence over whitelist pass",
rule: OutboundRule{Mode: "RO", WhitelistOutEnabled: true, WhitelistOut: []string{"@trusted.com"}},
recipients: []string{"ann@trusted.com"},
wantOK: false,
wantReason: "ro_mode",
},
}
for _, c := range cases {
t.Run(c.name, func(t *testing.T) {
ok, reason := c.rule.Check(c.recipients)
if ok != c.wantOK {
t.Fatalf("Check ok=%v want %v", ok, c.wantOK)
}
if !ok && reason != c.wantReason {
t.Fatalf("reason=%q want %q", reason, c.wantReason)
}
})
}
}
+30 -10
View File
@@ -18,6 +18,9 @@ type Account struct {
IMAPHost string
IMAPPort int
IMAPSecurity string // tls | starttls
SMTPHost string // nullable for RO accounts
SMTPPort int
SMTPSecurity string // tls | starttls
AuthType string // password | oauth2
Username string
Password string // decrypted; empty in ListAccounts
@@ -38,10 +41,13 @@ func (s *Store) AddAccount(a Account) (int64, error) {
}
res, err := s.db.Exec(`
INSERT INTO accounts
(name,mode,imap_host,imap_port,imap_security,auth_type,username,
(name,mode,imap_host,imap_port,imap_security,smtp_host,smtp_port,smtp_security,
auth_type,username,
enc_password,whitelist_in_enabled,whitelist_out_enabled,subject_regex,process_backlog)
VALUES (?,?,?,?,?,?,?,?,?,?,?,?)`,
a.Name, a.Mode, a.IMAPHost, a.IMAPPort, a.IMAPSecurity, a.AuthType, a.Username,
VALUES (?,?,?,?,?,?,?,?,?,?,?,?,?,?,?)`,
a.Name, a.Mode, a.IMAPHost, a.IMAPPort, a.IMAPSecurity,
nullStr(a.SMTPHost), nullInt(a.SMTPPort), nullStr(a.SMTPSecurity),
a.AuthType, a.Username,
encPw, b2i(a.WhitelistInEnabled), b2i(a.WhitelistOutEnabled),
nullStr(a.SubjectRegex), b2i(a.ProcessBacklog))
if err != nil {
@@ -52,7 +58,8 @@ func (s *Store) AddAccount(a Account) (int64, error) {
func (s *Store) GetAccount(name string) (Account, error) {
row := s.db.QueryRow(`
SELECT id,name,mode,imap_host,imap_port,imap_security,auth_type,username,
SELECT id,name,mode,imap_host,imap_port,imap_security,smtp_host,smtp_port,smtp_security,
auth_type,username,
enc_password,whitelist_in_enabled,whitelist_out_enabled,subject_regex,process_backlog
FROM accounts WHERE name = ?`, name)
a, encPw, err := scanAccount(row)
@@ -74,7 +81,8 @@ func (s *Store) GetAccount(name string) (Account, error) {
func (s *Store) ListAccounts() ([]Account, error) {
rows, err := s.db.Query(`
SELECT id,name,mode,imap_host,imap_port,imap_security,auth_type,username,
SELECT id,name,mode,imap_host,imap_port,imap_security,smtp_host,smtp_port,smtp_security,
auth_type,username,
enc_password,whitelist_in_enabled,whitelist_out_enabled,subject_regex,process_backlog
FROM accounts ORDER BY name`)
if err != nil {
@@ -108,17 +116,22 @@ type scanner interface{ Scan(dest ...any) error }
func scanAccount(sc scanner) (Account, []byte, error) {
var (
a Account
encPw []byte
subj sql.NullString
wlIn, wlOut int
backlog int
a Account
encPw []byte
subj, smtpHost, smtpSec sql.NullString
smtpPort sql.NullInt64
wlIn, wlOut int
backlog int
)
err := sc.Scan(&a.ID, &a.Name, &a.Mode, &a.IMAPHost, &a.IMAPPort, &a.IMAPSecurity,
&smtpHost, &smtpPort, &smtpSec,
&a.AuthType, &a.Username, &encPw, &wlIn, &wlOut, &subj, &backlog)
if err != nil {
return Account{}, nil, err
}
a.SMTPHost = smtpHost.String
a.SMTPPort = int(smtpPort.Int64)
a.SMTPSecurity = smtpSec.String
a.WhitelistInEnabled = wlIn != 0
a.WhitelistOutEnabled = wlOut != 0
a.ProcessBacklog = backlog != 0
@@ -139,3 +152,10 @@ func nullStr(s string) any {
}
return s
}
func nullInt(n int) any {
if n == 0 {
return nil
}
return n
}
+19
View File
@@ -35,6 +35,25 @@ func TestAddGetAccountDecryptsSecret(t *testing.T) {
}
}
func TestAddGetAccountRoundTripsSMTP(t *testing.T) {
s := openTemp(t)
a := sampleAccount()
a.Mode = "RW"
a.SMTPHost = "smtp.example.com"
a.SMTPPort = 465
a.SMTPSecurity = "tls"
if _, err := s.AddAccount(a); err != nil {
t.Fatalf("AddAccount: %v", err)
}
got, err := s.GetAccount("work")
if err != nil {
t.Fatalf("GetAccount: %v", err)
}
if got.SMTPHost != "smtp.example.com" || got.SMTPPort != 465 || got.SMTPSecurity != "tls" {
t.Fatalf("SMTP fields not round-tripped: %+v", got)
}
}
func TestPasswordStoredEncrypted(t *testing.T) {
s := openTemp(t)
_, _ = s.AddAccount(sampleAccount())
+85
View File
@@ -0,0 +1,85 @@
# emcli — Phase 2 Status Report
**Date:** 2026-06-22
**Branch:** `main`
**Phase 2 scope:** Send path — SMTP, MIME building, reply threading, outbound policy
(RO rejection + whitelist-out), the `send` agent command, and admin SMTP account fields.
## TL;DR
**Phase 2 is complete and validated end-to-end against a live SMTP account.** All slices were
implemented test-first (TDD: failing test → minimal code → green). The binary builds as a single
static CGO-free executable, `go vet` is clean, and the full unit-test suite passes including under
`-race`. A real message was sent to `me@stevecliff.com` over the live provider, and outbound
enforcement (RO, whitelist-out) plus reply threading were exercised against the live mailbox.
## Live validation (real SMTP/IMAP account)
Against the live cPanel/MXlogin account over implicit TLS (`friday.mxlogin.com:465` for SMTP,
`:993` for IMAP):
- **`send`:** delivered a plain-text message to `me@stevecliff.com` — `{"sent":true,
"recipients":["me@stevecliff.com"]}`, exit 0. SASL PLAIN auth over implicit TLS succeeded.
- **RO rejection:** a send from an `RO` account was blocked with
`{"code":"policy","message":"send blocked: ro_mode"}`, exit 1 — sender never dialed.
- **whitelist-out:** with `@stevecliff.com` whitelisted, a send to `stranger@elsewhere.com` was
blocked (`whitelist_out`); a send to `me@stevecliff.com` succeeded.
- **`--reply-to` threading:** replying to a real INBOX UID read the source's
`Message-ID`/`References` over IMAP and set `In-Reply-To`/`References` on the outgoing message
(subject to the inbound filter — a filtered/absent source returns `not_found`).
- **Audit:** every send (allowed or blocked) wrote an `audit_log` row with the correct
`result`/`reason`. Password confirmed **encrypted at rest** (no plaintext in the DB).
## What was built
| Package | Change | Status |
|---|---|---|
| `internal/store` | `Account` now carries `SMTPHost/SMTPPort/SMTPSecurity`; threaded through insert/select/scan (NULL-safe). | ✅ |
| `internal/policy` | `OutboundRule.Check(recipients) (ok, reason)` — RO ⇒ `ro_mode`; whitelist-out, any recipient fails ⇒ `whitelist_out` (no partial send). Reuses `MatchAddress` (case-insensitive, `@domain`). | ✅ |
| `internal/mail` | `Header.References` (parsed Message-IDs); `OutgoingMessage`, `BuildMIME` (plain-text + attachments + threading headers, Bcc envelope-only), `SendSMTP` (tls/starttls dial, SASL PLAIN, envelope send). | ✅ |
| `internal/cli` | `Deps.Send`; `SendCmd` (outbound gating → reply-to resolution w/ inbound filter → attachment reads → send → audit); `send` wired into the router with repeatable `--to/--cc/--bcc/--attach` flags. | ✅ |
| `internal/cli` (admin) | `account add` gains `--smtp-host/--smtp-port/--smtp-security` (applied for `RW`). | ✅ |
### `send` command
```
emcli send --account <a> --to <addr>… [--cc <addr>…] [--bcc <addr>…]
--subject <s> --body <text> [--attach <path>]… [--reply-to <uid> [--folder <f>]]
```
- Emits the standard JSON envelope; exit code mirrors `error`.
- `--to/--cc/--bcc/--attach` are repeatable and also accept comma-separated values.
### Enforcement (SPEC §9 Outbound) — verified
- `RO` accounts cannot send (blocked before any network I/O).
- With whitelist-out enabled, **every** recipient (to+cc+bcc) must match or the whole send is
blocked — no partial send.
- A `--reply-to` source that fails the **inbound** filter returns `not_found`: the agent cannot
thread off, or confirm the existence of, mail it isn't allowed to see.
## Verification
```
CGO_ENABLED=0 go build ./... → OK, single static binary
go vet ./... → clean
go test ./... → all packages pass
go test -race ./... → all packages pass
```
New tests: `policy` outbound matrix (RO, all-must-pass, domain, case, one-bad-blocks-all);
`mail` `BuildMIME` round-trip (body + attachment survive, threading headers present) and
`References` parsing; `cli` send gating (RO block, whitelist-out block/allow, happy path,
reply-to threading, filtered-source `not_found`).
## Process
Test-first per slice, tracked via the task list. Plan:
`specifications/plans/2026-06-22-phase2-send-path.md`.
## Known limitations / deferred (not defects)
- **Phase 3 — OAuth2:** SMTP auth here is password/SASL-PLAIN only; XOAUTH2 follows in Phase 3
(schema columns already present).
- **Phase 4 — Admin TUI + `doctor`.**
- `From` is taken from the account `username` (email-as-username, as in Phase 1). A distinct
display-name/from-address is a future nicety.
- No attachment size cap (SPEC §13 open item) — unchanged from Phase 1.
- Carry-over Minor items from Phase 1 (audit-row completeness, CLI polish) remain open; none were
in the Phase 2 path.
@@ -0,0 +1,77 @@
# emcli — Phase 2 Plan: Send Path
**Date:** 2026-06-22
**Depends on:** Phase 1 (read path, encrypted store, policy package, JSON envelope) — complete.
**Scope (SPEC §7.1 `send`, §9 Outbound, §10 threading headers):** SMTP send, MIME building,
reply threading, outbound policy (RO rejection + whitelist-out), the `send` agent command,
and admin support for SMTP account fields.
## Approach
Test-first (TDD), one slice at a time, mirroring Phase 1's package boundaries. Pure logic
(`policy`, MIME building) is unit-tested directly; the live SMTP path is validated against the
real provider at the end (as Phase 1 did for IMAP), since a unit SMTP server is out of scope.
## Tasks
### 1. `store`: load/save SMTP fields on `Account`
The schema already has `smtp_host/smtp_port/smtp_security`, but `Account` and its
scan/insert/select don't carry them. Add `SMTPHost string`, `SMTPPort int`,
`SMTPSecurity string`; thread through `AddAccount`, `GetAccount`, `ListAccounts`,
`scanAccount`. Admin `account add` gains `--smtp-host/--smtp-port/--smtp-security`.
**Test:** add an account with SMTP fields, read it back, assert equality.
### 2. `policy`: outbound rule
`OutboundRule{ Mode string; WhitelistOutEnabled bool; WhitelistOut []string }` with
`Check(recipients []string) (ok bool, reason string)`:
- `Mode == "RO"``(false, "ro_mode")`.
- `WhitelistOutEnabled` and any recipient fails `MatchAddress``(false, "whitelist_out")`
(whole send blocked — no partial).
- else `(true, "")`.
**Tests:** table-driven — RW allows; RO blocks; all-recipients-must-pass; domain `@x.com`
match; case-insensitivity; one bad recipient among good ones blocks all.
### 3. `mail`: SMTP send + MIME building
- Add `References` to `Header`; populate in `readHeader` from the `References` header so a
reply can chain correctly.
- `SMTPConfig{Host,Port,Security,Username,Password}`.
- `OutgoingMessage{From,To,Cc,Bcc,Subject,BodyText,Attachments,InReplyTo,References,Date}`.
- `BuildMIME(OutgoingMessage) ([]byte, error)` via `go-message/mail` writer: a plain-text
inline part plus any attachments; sets From/To/Cc/Subject/Date and, when present,
`In-Reply-To`/`References`. (Bcc recipients go in the SMTP envelope, not the headers.)
- `SendSMTP(SMTPConfig, OutgoingMessage) error`: dial (`tls``DialTLS`, `starttls`
`DialStartTLS`), `Auth` with SASL PLAIN, then `SendMail(from, allRecipients, mime)`.
**Tests:** `BuildMIME` round-trips through `ParseMessage` (body + attachment survive);
threading headers present when set; recipients assembled correctly. Live send deferred to task 6.
### 4. `cli`: `send` command + dispatch wiring
- Add `Send func(store.Account, mail.OutgoingMessage) error` to `Deps` (live impl wraps
`mail.SendSMTP`).
- `SendCmd(d, account, to, cc, bcc, subject, body, attachPaths, replyToUID, replyFolder)`:
1. Load account. Build `OutboundRule`; `Check(to+cc+bcc)` → on block, audit
`send`/`blocked`/reason and emit policy error.
2. If `replyToUID > 0`: dial IMAP, apply **inbound** filter (a filtered/invisible source
UID → `not_found`), fetch its `Message-ID`+`References`, set `InReplyTo`/`References`.
3. Read `--attach` files from disk into attachments.
4. `d.Send(acc, msg)`; on success audit `send`/`allowed`, emit `{ "sent": true,
"recipients": [...] }`.
- `run.go`: route `send`; repeatable `--to/--cc/--bcc/--attach` (custom `stringSlice`
flag.Value, also comma-splitting), `--subject/--body/--reply-to/--folder`.
**Tests (fake Sender + fake Mailer):** RO blocks (sender never called); whitelist-out blocks;
happy path calls sender with right recipients and emits success; reply-to on a filtered
source returns `not_found`.
### 5. Build / vet / full test suite (incl `-race`)
`CGO_ENABLED=0 go build`, `go vet ./...`, `go test ./...` and `-race`. Pristine output.
### 6. Live send validation
Configure an RW account with the test SMTP creds (`emcli@stevecliff.com` @
`friday.mxlogin.com`), send a real message to `me@stevecliff.com`, confirm the success
envelope. Spot-check enforcement live: RO account rejects send; whitelist-out blocks a
non-listed recipient.
## Out of scope (later phases)
- OAuth2 send auth (Phase 3) — password/SASL-PLAIN only here.
- Admin TUI / `doctor` (Phase 4).
- Carry-over Minor items from Phase 1 (audit-row completeness, CLI polish) — touch only if
directly in the way.