diff --git a/README.md b/README.md index 7f380ac..ffa9c32 100644 --- a/README.md +++ b/README.md @@ -1 +1,20 @@ # emcli + +A single command-line program that mediates all email access for an AI agent. The agent never +holds your email password and never talks to the mail server directly — every read and send goes +through `emcli`, which enforces the limits you configure (read-only/read-write, sender and +recipient whitelists, subject filters). Even with faulty instructions, the agent can't read mail +it isn't permitted to see or send mail to people it isn't permitted to contact. + +## Getting started + +```bash +export EMCLI_KEY="$(head -c 32 /dev/urandom | base64)" # one-time: generate & save a key +emcli init # create the DB, add your first account +emcli doctor # confirm it connects and authenticates +``` + +## Documentation + +See the **[User Manual](USER-MANUAL.md)** for full setup, account configuration (including Gmail +app passwords), the agent and admin command reference, the JSON output format, and troubleshooting. diff --git a/USER-MANUAL.md b/USER-MANUAL.md new file mode 100644 index 0000000..75a6c87 --- /dev/null +++ b/USER-MANUAL.md @@ -0,0 +1,547 @@ +# emcli — User Manual + +`emcli` is a single command-line program that mediates all email access for an AI agent. The +agent never holds your email password and never talks to the mail server directly — every read +and send goes through `emcli`, which enforces the limits you configure. Even if the agent is given +faulty instructions, it cannot read mail it isn't permitted to see or send mail to people it isn't +permitted to contact. + +This manual is for **using and administering** `emcli`. It assumes you have the `emcli` binary. + +--- + +## Contents + +1. [Key concepts](#1-key-concepts) +2. [Setup: the encryption key and database](#2-setup-the-encryption-key-and-database) +3. [Quick start](#3-quick-start) +4. [Adding accounts](#4-adding-accounts) + - [Gmail (app password)](#gmail-app-password) + - [Other providers (cPanel, etc.)](#other-providers) +5. [Administering accounts](#5-administering-accounts) +6. [Whitelists, modes, and filters](#6-whitelists-modes-and-filters) +7. [Agent commands (reading and sending)](#7-agent-commands) +8. [The JSON envelope](#8-the-json-envelope) +9. [Diagnostics: `doctor`](#9-diagnostics-doctor) +10. [Audit log and settings](#10-audit-log-and-settings) +11. [Troubleshooting](#11-troubleshooting) +12. [Command cheat sheet](#12-command-cheat-sheet) + +--- + +## 1. Key concepts + +**Two kinds of commands.** +- **Admin commands** (`init`, `account`, `whitelist`, `config`, `audit`, `doctor`) are for *you*, + the human. They print human-readable text or open an interactive form. +- **Agent commands** (`list`, `get`, `search`, `ack`, `send`) are for the *agent*. They print one + line of JSON and nothing else, so a program can consume them reliably. + +**Accounts** are named (e.g. `gmail`, `work`). The agent refers to an account by name and never +sees its password. + +**Read-only vs read-write (mode).** +- `RO` — the agent can read this account but **cannot send**. Any `send` is rejected. +- `RW` — the agent can read and send. + +**Whitelists** (optional, per account): +- **Inbound** (`whitelist in`) — if enabled, the agent only sees mail from listed senders. + Everything else is invisible: it won't appear in `list`/`search`, can't be opened with `get`, + and can't be acknowledged. +- **Outbound** (`whitelist out`) — if enabled, every recipient of a `send` must be listed, or the + whole send is blocked. + +**Subject filter** (optional, per account) — a regular expression; if set, the agent only sees mail +whose subject matches. + +**"New" mail and acknowledging.** `emcli` tracks which messages an account has already processed. +`list --new` shows only messages not yet acknowledged. The agent calls `ack` to mark messages +done; they then drop out of `--new`. Reading a message (`get`) does **not** acknowledge it — +acking is a deliberate, separate step. + +--- + +## 2. Setup: the encryption key and database + +`emcli` reads two environment variables: + +| Variable | Purpose | Default | +|---|---|---| +| `EMCLI_KEY` | **Required.** Base64-encoded 32-byte key (AES-256) used to encrypt passwords at rest. | none — commands fail without it | +| `EMCLI_DB` | Path to the database file. | `~/.config/emcli/emcli.db` (Linux/macOS), `%AppData%\emcli\emcli.db` (Windows) | + +**Generate a key once** and keep it safe (store it the way the program/orchestrator that launches +`emcli` expects — e.g. a secrets manager or your shell profile): + +```bash +head -c 32 /dev/urandom | base64 +``` + +Set it in your environment before running any command: + +```bash +export EMCLI_KEY='paste-the-base64-key-here' +``` + +> **Important:** the key encrypts your account passwords. If you lose it, the stored passwords +> can't be decrypted and you'll have to re-add accounts. If you change it, the same applies. +> `emcli` never falls back to plaintext — a missing or wrong key makes every command fail safely. + +Account passwords are stored **encrypted**; they never appear in command output, error messages, +or the audit log. + +--- + +## 3. Quick start + +```bash +# 1. Set your key (see section 2) +export EMCLI_KEY='…' + +# 2. Create the database and add your first account (interactive form) +emcli init + +# 3. Check it connects and authenticates +emcli doctor + +# 4. The agent can now read +emcli list --account gmail --folder INBOX --limit 10 +``` + +`emcli init` opens an interactive form. Use **Tab**/**Shift+Tab** to move between fields, **Enter** +to save, **Esc** to cancel. Boolean fields take `y`/`n`. + +Prefer flags? You can do everything non-interactively too — see the next section. + +--- + +## 4. Adding accounts + +Interactive (recommended for first time): + +```bash +emcli account add # opens the form +``` + +Or with flags (good for scripting): + +```bash +emcli account add --name work --mode RW \ + --imap-host imap.example.com --imap-port 993 --imap-security tls \ + --smtp-host smtp.example.com --smtp-port 465 --smtp-security tls \ + --username you@example.com --password 'your-password' +``` + +**`account add` flags:** + +| Flag | Default | Notes | +|---|---|---| +| `--name` | — | Account name the agent will use (required) | +| `--mode` | `RO` | `RO` (read-only) or `RW` (read + send) | +| `--imap-host` | — | IMAP server (required) | +| `--imap-port` | `993` | | +| `--imap-security` | `tls` | `tls` or `starttls` | +| `--smtp-host` | — | SMTP server (used for `RW` accounts) | +| `--smtp-port` | `465` | | +| `--smtp-security` | `tls` | `tls` or `starttls` | +| `--username` | — | Login username, usually your full email (required) | +| `--password` | — | Login password or app password | +| `--subject-regex` | — | Inbound subject filter (optional) | +| `--whitelist-in` | off | Enable inbound whitelist | +| `--whitelist-out` | off | Enable outbound whitelist | +| `--process-backlog` | off | Treat existing mail as "new" (see below) | + +**`--process-backlog`.** When `emcli` first sees a folder: +- **off (default):** existing mail is treated as already handled — `list --new` starts empty and + only mail that arrives *after* this point counts as new. +- **on:** all existing mail in the folder is treated as new for the agent to process. + +### Gmail (app password) + +Gmail needs an **app password**, not your normal Google password. + +1. Turn on **2-Step Verification** on your Google account (required — the app-passwords page is + hidden otherwise). +2. Create a 16-character app password at (name it, + e.g., "emcli"). +3. Enable IMAP in Gmail: **Settings → See all settings → Forwarding and POP/IMAP → Enable IMAP**. +4. Add the account (paste the app password; the spaces Google shows are optional): + +```bash +emcli account add --name gmail --mode RW \ + --imap-host imap.gmail.com --imap-port 993 --imap-security tls \ + --smtp-host smtp.gmail.com --smtp-port 465 --smtp-security tls \ + --username you@gmail.com --password 'xxxxxxxxxxxxxxxx' +``` + +An app password keeps working until you revoke it, change your main Google password, or turn off +2-Step Verification. If that happens, generate a new one and update the account (section 5). + +### Other providers + +Most IMAP/SMTP providers work the same way. A typical cPanel-style host: + +```bash +emcli account add --name work --mode RW \ + --imap-host mail.yourdomain.com --imap-port 993 --imap-security tls \ + --smtp-host mail.yourdomain.com --smtp-port 465 --smtp-security tls \ + --username you@yourdomain.com --password 'your-password' +``` + +If `465`/`tls` doesn't connect for SMTP, try `587`/`starttls`. Use `emcli doctor` (section 9) to +confirm. + +--- + +## 5. Administering accounts + +**List accounts** (never shows secrets): + +```bash +emcli account list +``` + +**Edit an account** — interactive (opens the form pre-filled): + +```bash +emcli account edit --name gmail +``` + +**Edit with flags** — only the flags you pass are changed; everything else is preserved. Leaving +`--password` out keeps the existing password. + +```bash +emcli account edit --name work --mode RW --smtp-host smtp.example.com --smtp-port 587 --smtp-security starttls +emcli account edit --name gmail --password 'new-app-password' # rotate the app password +``` + +> Note: the flag form of `account edit` covers connection/auth fields and `--subject-regex`. To +> toggle whitelists or `process-backlog`, use the interactive form (`emcli account edit --name X` +> with no other flags), or the `whitelist` commands in section 6. + +**Remove an account** (requires `--yes`): + +```bash +emcli account remove --name work --yes +``` + +--- + +## 6. Whitelists, modes, and filters + +### Modes + +Set with `--mode RO|RW` on `account add`/`edit`. `RO` accounts reject every `send`. + +### Inbound whitelist + +Enable it on the account (`--whitelist-in`, or the interactive form), then manage entries: + +```bash +emcli whitelist in add --account gmail --address boss@example.com +emcli whitelist in add --account gmail --address @partner.com +emcli whitelist in list --account gmail +emcli whitelist in remove --account gmail --address boss@example.com +``` + +When enabled, the agent only sees mail from listed senders. Everything else is invisible. + +### Outbound whitelist + +Enable it (`--whitelist-out`), then manage entries the same way with `whitelist out`: + +```bash +emcli whitelist out add --account gmail --address @example.com +emcli whitelist out list --account gmail +``` + +When enabled, **every** recipient of a `send` (to + cc + bcc) must match an entry or the whole +send is blocked. + +### Address matching (both directions) + +- Case-insensitive. +- An entry like `@example.com` matches **any** address at that domain. +- Any other entry matches that **exact** address. + +### Subject filter + +A regular expression on the account. If set, the agent only sees mail whose subject matches: + +```bash +emcli account edit --name gmail --subject-regex '^\[ticket\]' +``` + +Clear it by setting it empty in the interactive form. + +--- + +## 7. Agent commands + +These are what the agent runs. Each prints exactly one JSON object (see section 8). They all take +`--account` and most take `--folder` (default `INBOX`). + +> **Reading never changes state.** `list`, `get`, and `search` are read-only. Only `ack` advances +> "what's been processed." + +### `list` — message headers + +```bash +emcli list --account gmail --folder INBOX --limit 20 +emcli list --account gmail --new # only un-acked messages +emcli list --account gmail --before 1000 # older than UID 1000 (paging) +emcli list --account gmail --since 1000 # newer than UID 1000 +``` + +| Flag | Default | Meaning | +|---|---|---| +| `--folder` | `INBOX` | Mailbox/folder | +| `--new` | off | Only messages not yet acknowledged | +| `--limit` | `50` | Max results (capped at `500`) | +| `--before ` | — | Only messages with a lower UID (page to older mail) | +| `--since ` | — | Only messages with a higher UID (page to newer mail) | + +Returns headers only (no body): `uid`, `from`, `to`, `subject`, `date`, `message_id`, +`has_attachments`. Newest first. + +### `get` — one full message + +```bash +emcli get --account gmail --folder INBOX --uid 70314 +``` + +Returns the full message: headers, decoded plain-text body, and attachments (each as +`name`, `size`, `mime`, and `content_b64` — base64-encoded contents). Does **not** acknowledge it. + +### `search` — find mail server-side + +```bash +emcli search --account gmail --from boss@example.com +emcli search --account gmail --subject-contains invoice +emcli search --account gmail --text "quarterly report" +emcli search --account gmail --since-date 2026-01-01T00:00:00Z --before-date 2026-02-01T00:00:00Z +``` + +| Flag | Meaning | +|---|---| +| `--from` | Sender contains | +| `--subject-contains` | Subject contains | +| `--text` | Full-text search | +| `--since-date` / `--before-date` | Date bounds, RFC 3339 (e.g. `2026-06-01T00:00:00Z`) | +| `--limit` | Max results (default `50`) | + +Returns the same headers-only shape as `list`. Searches the whole folder, regardless of +new/acked state. Filtered (whitelisted-out) mail never appears. + +### `ack` — mark messages processed + +```bash +emcli ack --account gmail --folder INBOX --uid-list 70314,70315,70320 +``` + +Marks one or more UIDs as processed; they stop appearing under `list --new`. Safe to call more +than once, and the order doesn't matter. You can't ack a message you aren't allowed to see. + +### `send` — send a message (RW accounts only) + +```bash +emcli send --account gmail \ + --to alice@example.com --cc bob@example.com \ + --subject "Hello" --body "Plain-text body here." + +# multiple recipients (repeat the flag or comma-separate) +emcli send --account gmail --to a@x.com --to b@x.com --subject Hi --body Yo +emcli send --account gmail --to 'a@x.com,b@x.com' --subject Hi --body Yo + +# attachments +emcli send --account gmail --to a@x.com --subject Report --body "see attached" \ + --attach ./report.pdf --attach ./data.csv + +# reply (threads correctly off an existing message you can see) +emcli send --account gmail --to a@x.com --subject "Re: Hi" --body "thanks" \ + --reply-to 70314 --folder INBOX +``` + +| Flag | Meaning | +|---|---| +| `--to` / `--cc` / `--bcc` | Recipients (repeatable, or comma-separated) | +| `--subject` | Subject | +| `--body` | Plain-text body | +| `--attach` | File to attach (repeatable) | +| `--reply-to ` | Thread the reply onto this source message | +| `--folder` | Folder of the `--reply-to` source (default `INBOX`) | + +`--reply-to` reads the source message's identifiers so the reply threads in the recipient's mail +client. The source is subject to the inbound whitelist — you can't reply to mail you aren't allowed +to see. + +--- + +## 8. The JSON envelope + +Every agent command prints exactly one JSON object: + +```json +{ "error": false, "error_detail": {}, "data": { } } +``` + +- `error` — `true` or `false`. +- `error_detail` — empty `{}` on success; on failure `{ "code": "...", "message": "..." }`. +- `data` — the result; shape depends on the command. + +The process **exit code** mirrors `error` (0 on success, non-zero on failure), so scripts can check +either. + +**Examples.** + +`list` / `search` success: +```json +{ "error": false, "error_detail": {}, + "data": { "messages": [ + { "uid": 70314, "from": "\"Boss\" ", "to": "", + "subject": "Hello", "date": "Mon, 22 Jun 2026 17:00:30 +0000", + "message_id": "abc@example.com", "has_attachments": false } ] } } +``` + +`send` success: +```json +{ "error": false, "error_detail": {}, "data": { "sent": true, "recipients": ["alice@example.com"] } } +``` + +A blocked send (read-only account): +```json +{ "error": true, "error_detail": { "code": "policy", "message": "send blocked: ro_mode" }, "data": {} } +``` + +**Error codes** you may see in `error_detail.code`: + +| Code | Meaning | +|---|---| +| `config` | Missing/invalid `EMCLI_KEY` or configuration problem | +| `db` | Database error | +| `network` | Connection problem reaching the mail server | +| `auth` | Authentication failed | +| `policy` | Blocked by a rule (`ro_mode`, `whitelist_out`) | +| `not_found` | Message/account not found — also returned for filtered (invisible) mail | +| `usage` | A required flag was missing or invalid | + +> A message hidden by the inbound whitelist or subject filter returns `not_found` — the same as a +> message that doesn't exist. This is deliberate: the agent can't tell the difference, so it can't +> learn that filtered mail exists. + +--- + +## 9. Diagnostics: `doctor` + +`doctor` checks that each account can actually connect and authenticate — it logs in and out but +sends and reads nothing. + +```bash +emcli doctor # check every account +emcli doctor --account gmail # check just one +``` + +Example output: + +``` +gmail (RW) + IMAP ok + SMTP ok +work (RO) + IMAP ok + SMTP n/a (read-only) +badpass (RO) + IMAP FAIL: Invalid credentials + SMTP n/a (read-only) +``` + +`doctor` exits non-zero if any check fails. SMTP is only checked for `RW` accounts with an SMTP +host configured. Run `doctor` after adding or editing an account to confirm the credentials and +server settings are right. + +--- + +## 10. Audit log and settings + +### Audit log + +Every agent action (`list`, `get`, `search`, `ack`, `send`) — allowed or blocked — is recorded. + +```bash +emcli audit list # most recent 50 +emcli audit list --account gmail # filter to one account +emcli audit list --limit 200 +``` + +Each row shows the time, account, action, result (`allowed`/`blocked`), target, and (for blocks) +the reason. Secrets never appear here. + +### Settings + +```bash +emcli config set audit_retention_days 90 +emcli config get audit_retention_days +``` + +- `audit_retention_days` — how long to keep audit rows. On each run, entries older than this are + purged. Must be a whole number ≥ 0. `0` or unset means no automatic purging. + +--- + +## 11. Troubleshooting + +**"EMCLI_KEY is not set" / "must be base64 of exactly 32 bytes".** Set `EMCLI_KEY` to a valid +base64-encoded 32-byte key (section 2). Every command that touches the database needs it. + +**A command fails to decrypt / "wrong EMCLI_KEY?".** The key doesn't match the one used when the +account was added. Restore the original key, or re-add the account with the current key. + +**`doctor` shows `IMAP FAIL` / `SMTP FAIL`.** +- *Invalid credentials / authentication failed* — wrong username or password. For Gmail, make sure + you're using an **app password** (section 4) and that IMAP is enabled in Gmail settings. +- *Connection errors* — wrong host/port/security. Try the provider's documented settings; for SMTP, + `587`/`starttls` is a common alternative to `465`/`tls`. + +**The agent can't see a message you know exists.** It's probably filtered: check the account's +inbound whitelist (`emcli whitelist in list --account NAME`) and subject filter. Filtered mail is +invisible by design. + +**`send` is blocked.** +- `ro_mode` — the account is read-only. Change it: `emcli account edit --name NAME --mode RW` + (and set SMTP details). +- `whitelist_out` — a recipient isn't on the outbound whitelist. Add it, or review the rule. + +**`list --new` is empty but there's mail.** By default, mail that existed before you added the +account is treated as already handled. Add the account with `--process-backlog` to treat existing +mail as new, or just use `list`/`search` without `--new`. + +**The interactive form won't open ("could not open a new TTY").** Interactive commands (`init`, +`account add`/`edit` without flags) need a real terminal. Use the flag-based forms instead when +running non-interactively. + +--- + +## 12. Command cheat sheet + +``` +# Admin +emcli init # create DB + add first account (form) +emcli account add [flags | none for form] # add an account +emcli account list # list accounts (no secrets) +emcli account edit --name N [flags | none for form] # change an account +emcli account remove --name N --yes # delete an account +emcli whitelist in|out add|remove|list --account N [--address A] +emcli config set|get [value] # e.g. audit_retention_days +emcli audit list [--account N] [--limit K] +emcli doctor [--account N] # connectivity/auth check +emcli version + +# Agent (one line of JSON each) +emcli list --account N [--folder F] [--new] [--limit K] [--before U] [--since U] +emcli get --account N [--folder F] --uid U +emcli search --account N [--folder F] [--from A] [--subject-contains S] [--text S] [--since-date D] [--before-date D] [--limit K] +emcli ack --account N [--folder F] --uid-list U1,U2,U3 +emcli send --account N --to A [--cc A] [--bcc A] --subject S --body B [--attach P]… [--reply-to U [--folder F]] +``` + +Environment: `EMCLI_KEY` (required, base64 32-byte key), `EMCLI_DB` (optional DB path).