emcli/USER-MANUAL.md

# emcli — User Manual

`emcli` is a single command-line program that mediates all email access for an AI agent. The
agent never holds your email password and never talks to the mail server directly — every read
and send goes through `emcli`, which enforces the limits you configure. Even if the agent is given
faulty instructions, it cannot read mail it isn't permitted to see or send mail to people it isn't
permitted to contact.

This manual is for **using and administering** `emcli`. It assumes you have the `emcli` binary.

---

## Contents

1. [Key concepts](#1-key-concepts)
2. [Setup: the encryption key and database](#2-setup-the-encryption-key-and-database)
3. [Quick start](#3-quick-start)
4. [Adding accounts](#4-adding-accounts)
   - [Gmail (app password)](#gmail-app-password)
   - [Other providers (cPanel, etc.)](#other-providers)
5. [Administering accounts](#5-administering-accounts)
6. [Whitelists, modes, and filters](#6-whitelists-modes-and-filters)
7. [Agent commands (reading and sending)](#7-agent-commands)
8. [The JSON envelope](#8-the-json-envelope)
9. [Diagnostics: `doctor`](#9-diagnostics-doctor)
10. [Audit log and settings](#10-audit-log-and-settings)
11. [Troubleshooting](#11-troubleshooting)
12. [Command cheat sheet](#12-command-cheat-sheet)

---

## 1. Key concepts

**Two kinds of commands.**
- **Admin commands** (`init`, `account`, `whitelist`, `config`, `audit`, `doctor`) are for *you*,
  the human. They print human-readable text or open an interactive form.
- **Agent commands** (`list`, `get`, `search`, `ack`, `send`) are for the *agent*. They print one
  line of JSON and nothing else, so a program can consume them reliably.

**Accounts** are named (e.g. `gmail`, `work`). The agent refers to an account by name and never
sees its password.

**Read-only vs read-write (mode).**
- `RO` — the agent can read this account but **cannot send**. Any `send` is rejected.
- `RW` — the agent can read and send.

**Whitelists** (optional, per account):
- **Inbound** (`whitelist in`) — if enabled, the agent only sees mail from listed senders.
  Everything else is invisible: it won't appear in `list`/`search`, can't be opened with `get`,
  and can't be acknowledged.
- **Outbound** (`whitelist out`) — if enabled, every recipient of a `send` must be listed, or the
  whole send is blocked.

**Subject filter** (optional, per account) — a regular expression; if set, the agent only sees mail
whose subject matches.

**"New" mail and acknowledging.** `emcli` tracks which messages an account has already processed.
`list --new` shows only messages not yet acknowledged. The agent calls `ack` to mark messages
done; they then drop out of `--new`. Reading a message (`get`) does **not** acknowledge it —
acking is a deliberate, separate step.

---

## 2. Setup: the encryption key and database

`emcli` reads two environment variables:

| Variable | Purpose | Default |
|---|---|---|
| `EMCLI_KEY` | **Required.** Base64-encoded 32-byte key (AES-256) used to encrypt passwords at rest. | none — commands fail without it |
| `EMCLI_DB` | Path to the database file. | `~/.config/emcli/emcli.db` (Linux/macOS), `%AppData%\emcli\emcli.db` (Windows) |

**Generate a key once** and keep it safe (store it the way the program/orchestrator that launches
`emcli` expects — e.g. a secrets manager or your shell profile):

```bash
head -c 32 /dev/urandom | base64
```

Set it in your environment before running any command:

```bash
export EMCLI_KEY='paste-the-base64-key-here'
```

> **Important:** the key encrypts your account passwords. If you lose it, the stored passwords
> can't be decrypted and you'll have to re-add accounts. If you change it, the same applies.
> `emcli` never falls back to plaintext — a missing or wrong key makes every command fail safely.

Account passwords are stored **encrypted**; they never appear in command output, error messages,
or the audit log.

---

## 3. Quick start

```bash
# 1. Set your key (see section 2)
export EMCLI_KEY='…'

# 2. Create the database and add your first account (interactive form)
emcli init

# 3. Check it connects and authenticates
emcli doctor

# 4. The agent can now read
emcli list --account gmail --folder INBOX --limit 10
```

`emcli init` opens an interactive form. Use **Tab**/**Shift+Tab** to move between fields, **Enter**
to save, **Esc** to cancel. Boolean fields take `y`/`n`.

Prefer flags? You can do everything non-interactively too — see the next section.

---

## 4. Adding accounts

Interactive (recommended for first time):

```bash
emcli account add          # opens the form
```

Or with flags (good for scripting):

```bash
emcli account add --name work --mode RW \
  --imap-host imap.example.com --imap-port 993 --imap-security tls \
  --smtp-host smtp.example.com --smtp-port 465 --smtp-security tls \
  --username you@example.com --password 'your-password'
```

**`account add` flags:**

| Flag | Default | Notes |
|---|---|---|
| `--name` | — | Account name the agent will use (required) |
| `--mode` | `RO` | `RO` (read-only) or `RW` (read + send) |
| `--imap-host` | — | IMAP server (required) |
| `--imap-port` | `993` | |
| `--imap-security` | `tls` | `tls` or `starttls` |
| `--smtp-host` | — | SMTP server (used for `RW` accounts) |
| `--smtp-port` | `465` | |
| `--smtp-security` | `tls` | `tls` or `starttls` |
| `--username` | — | Login username, usually your full email (required) |
| `--password` | — | Login password or app password |
| `--subject-regex` | — | Inbound subject filter (optional) |
| `--whitelist-in` | off | Enable inbound whitelist |
| `--whitelist-out` | off | Enable outbound whitelist |
| `--process-backlog` | off | Treat existing mail as "new" (see below) |

**`--process-backlog`.** When `emcli` first sees a folder:
- **off (default):** existing mail is treated as already handled — `list --new` starts empty and
  only mail that arrives *after* this point counts as new.
- **on:** all existing mail in the folder is treated as new for the agent to process.

### Gmail (app password)

Gmail needs an **app password**, not your normal Google password.

1. Turn on **2-Step Verification** on your Google account (required — the app-passwords page is
   hidden otherwise).
2. Create a 16-character app password at <https://myaccount.google.com/apppasswords> (name it,
   e.g., "emcli").
3. Enable IMAP in Gmail: **Settings → See all settings → Forwarding and POP/IMAP → Enable IMAP**.
4. Add the account (paste the app password; the spaces Google shows are optional):

```bash
emcli account add --name gmail --mode RW \
  --imap-host imap.gmail.com --imap-port 993 --imap-security tls \
  --smtp-host smtp.gmail.com --smtp-port 465 --smtp-security tls \
  --username you@gmail.com --password 'xxxxxxxxxxxxxxxx'
```

An app password keeps working until you revoke it, change your main Google password, or turn off
2-Step Verification. If that happens, generate a new one and update the account (section 5).

### Other providers

Most IMAP/SMTP providers work the same way. A typical cPanel-style host:

```bash
emcli account add --name work --mode RW \
  --imap-host mail.yourdomain.com --imap-port 993 --imap-security tls \
  --smtp-host mail.yourdomain.com --smtp-port 465 --smtp-security tls \
  --username you@yourdomain.com --password 'your-password'
```

If `465`/`tls` doesn't connect for SMTP, try `587`/`starttls`. Use `emcli doctor` (section 9) to
confirm.

---

## 5. Administering accounts

**List accounts** (never shows secrets):

```bash
emcli account list
```

**Edit an account** — interactive (opens the form pre-filled):

```bash
emcli account edit --name gmail
```

**Edit with flags** — only the flags you pass are changed; everything else is preserved. Leaving
`--password` out keeps the existing password.

```bash
emcli account edit --name work --mode RW --smtp-host smtp.example.com --smtp-port 587 --smtp-security starttls
emcli account edit --name gmail --password 'new-app-password'   # rotate the app password
```

> Note: the flag form of `account edit` covers connection/auth fields and `--subject-regex`. To
> toggle whitelists or `process-backlog`, use the interactive form (`emcli account edit --name X`
> with no other flags), or the `whitelist` commands in section 6.

**Remove an account** (requires `--yes`):

```bash
emcli account remove --name work --yes
```

---

## 6. Whitelists, modes, and filters

### Modes

Set with `--mode RO|RW` on `account add`/`edit`. `RO` accounts reject every `send`.

### Inbound whitelist

Enable it on the account (`--whitelist-in`, or the interactive form), then manage entries:

```bash
emcli whitelist in add    --account gmail --address boss@example.com
emcli whitelist in add    --account gmail --address @partner.com
emcli whitelist in list   --account gmail
emcli whitelist in remove --account gmail --address boss@example.com
```

When enabled, the agent only sees mail from listed senders. Everything else is invisible.

### Outbound whitelist

Enable it (`--whitelist-out`), then manage entries the same way with `whitelist out`:

```bash
emcli whitelist out add  --account gmail --address @example.com
emcli whitelist out list --account gmail
```

When enabled, **every** recipient of a `send` (to + cc + bcc) must match an entry or the whole
send is blocked.

### Address matching (both directions)

- Case-insensitive.
- An entry like `@example.com` matches **any** address at that domain.
- Any other entry matches that **exact** address.

### Subject filter

A regular expression on the account. If set, the agent only sees mail whose subject matches:

```bash
emcli account edit --name gmail --subject-regex '^\[ticket\]'
```

Clear it by setting it empty in the interactive form.

---

## 7. Agent commands

These are what the agent runs. Each prints exactly one JSON object (see section 8). They all take
`--account` and most take `--folder` (default `INBOX`).

> **Reading never changes state.** `list`, `get`, and `search` are read-only. Only `ack` advances
> "what's been processed."

### `list` — message headers

```bash
emcli list --account gmail --folder INBOX --limit 20
emcli list --account gmail --new                 # only un-acked messages
emcli list --account gmail --before 1000         # older than UID 1000 (paging)
emcli list --account gmail --since 1000          # newer than UID 1000
```

| Flag | Default | Meaning |
|---|---|---|
| `--folder` | `INBOX` | Mailbox/folder |
| `--new` | off | Only messages not yet acknowledged |
| `--limit` | `50` | Max results (capped at `500`) |
| `--before <uid>` | — | Only messages with a lower UID (page to older mail) |
| `--since <uid>` | — | Only messages with a higher UID (page to newer mail) |

Returns headers only (no body): `uid`, `from`, `to`, `subject`, `date`, `message_id`,
`has_attachments`. Newest first.

### `get` — one full message

```bash
emcli get --account gmail --folder INBOX --uid 70314
```

Returns the full message: headers, decoded plain-text body, and attachments (each as
`name`, `size`, `mime`, and `content_b64` — base64-encoded contents). Does **not** acknowledge it.

### `search` — find mail server-side

```bash
emcli search --account gmail --from boss@example.com
emcli search --account gmail --subject-contains invoice
emcli search --account gmail --text "quarterly report"
emcli search --account gmail --since-date 2026-01-01T00:00:00Z --before-date 2026-02-01T00:00:00Z
```

| Flag | Meaning |
|---|---|
| `--from` | Sender contains |
| `--subject-contains` | Subject contains |
| `--text` | Full-text search |
| `--since-date` / `--before-date` | Date bounds, RFC 3339 (e.g. `2026-06-01T00:00:00Z`) |
| `--limit` | Max results (default `50`) |

Returns the same headers-only shape as `list`. Searches the whole folder, regardless of
new/acked state. Filtered (whitelisted-out) mail never appears.

### `ack` — mark messages processed

```bash
emcli ack --account gmail --folder INBOX --uid-list 70314,70315,70320
```

Marks one or more UIDs as processed; they stop appearing under `list --new`. Safe to call more
than once, and the order doesn't matter. You can't ack a message you aren't allowed to see.

### `send` — send a message (RW accounts only)

```bash
emcli send --account gmail \
  --to alice@example.com --cc bob@example.com \
  --subject "Hello" --body "Plain-text body here."

# multiple recipients (repeat the flag or comma-separate)
emcli send --account gmail --to a@x.com --to b@x.com --subject Hi --body Yo
emcli send --account gmail --to 'a@x.com,b@x.com' --subject Hi --body Yo

# attachments
emcli send --account gmail --to a@x.com --subject Report --body "see attached" \
  --attach ./report.pdf --attach ./data.csv

# reply (threads correctly off an existing message you can see)
emcli send --account gmail --to a@x.com --subject "Re: Hi" --body "thanks" \
  --reply-to 70314 --folder INBOX
```

| Flag | Meaning |
|---|---|
| `--to` / `--cc` / `--bcc` | Recipients (repeatable, or comma-separated) |
| `--subject` | Subject |
| `--body` | Plain-text body |
| `--attach` | File to attach (repeatable) |
| `--reply-to <uid>` | Thread the reply onto this source message |
| `--folder` | Folder of the `--reply-to` source (default `INBOX`) |

`--reply-to` reads the source message's identifiers so the reply threads in the recipient's mail
client. The source is subject to the inbound whitelist — you can't reply to mail you aren't allowed
to see.

---

## 8. The JSON envelope

Every agent command prints exactly one JSON object:

```json
{ "error": false, "error_detail": {}, "data": { } }
```

- `error` — `true` or `false`.
- `error_detail` — empty `{}` on success; on failure `{ "code": "...", "message": "..." }`.
- `data` — the result; shape depends on the command.

The process **exit code** mirrors `error` (0 on success, non-zero on failure), so scripts can check
either.

**Examples.**

`list` / `search` success:
```json
{ "error": false, "error_detail": {},
  "data": { "messages": [
    { "uid": 70314, "from": "\"Boss\" <boss@example.com>", "to": "<you@example.com>",
      "subject": "Hello", "date": "Mon, 22 Jun 2026 17:00:30 +0000",
      "message_id": "abc@example.com", "has_attachments": false } ] } }
```

`send` success:
```json
{ "error": false, "error_detail": {}, "data": { "sent": true, "recipients": ["alice@example.com"] } }
```

A blocked send (read-only account):
```json
{ "error": true, "error_detail": { "code": "policy", "message": "send blocked: ro_mode" }, "data": {} }
```

**Error codes** you may see in `error_detail.code`:

| Code | Meaning |
|---|---|
| `config` | Missing/invalid `EMCLI_KEY` or configuration problem |
| `db` | Database error |
| `network` | Connection problem reaching the mail server |
| `auth` | Authentication failed |
| `policy` | Blocked by a rule (`ro_mode`, `whitelist_out`) |
| `not_found` | Message/account not found — also returned for filtered (invisible) mail |
| `usage` | A required flag was missing or invalid |

> A message hidden by the inbound whitelist or subject filter returns `not_found` — the same as a
> message that doesn't exist. This is deliberate: the agent can't tell the difference, so it can't
> learn that filtered mail exists.

---

## 9. Diagnostics: `doctor`

`doctor` checks that each account can actually connect and authenticate — it logs in and out but
sends and reads nothing.

```bash
emcli doctor                    # check every account
emcli doctor --account gmail    # check just one
```

Example output:

```
gmail (RW)
  IMAP  ok
  SMTP  ok
work (RO)
  IMAP  ok
  SMTP  n/a (read-only)
badpass (RO)
  IMAP  FAIL: Invalid credentials
  SMTP  n/a (read-only)
```

`doctor` exits non-zero if any check fails. SMTP is only checked for `RW` accounts with an SMTP
host configured. Run `doctor` after adding or editing an account to confirm the credentials and
server settings are right.

---

## 10. Audit log and settings

### Audit log

Every agent action (`list`, `get`, `search`, `ack`, `send`) — allowed or blocked — is recorded.

```bash
emcli audit list                       # most recent 50
emcli audit list --account gmail        # filter to one account
emcli audit list --limit 200
```

Each row shows the time, account, action, result (`allowed`/`blocked`), target, and (for blocks)
the reason. Secrets never appear here.

### Settings

```bash
emcli config set audit_retention_days 90
emcli config get audit_retention_days
```

- `audit_retention_days` — how long to keep audit rows. On each run, entries older than this are
  purged. Must be a whole number ≥ 0. `0` or unset means no automatic purging.

---

## 11. Troubleshooting

**"EMCLI_KEY is not set" / "must be base64 of exactly 32 bytes".** Set `EMCLI_KEY` to a valid
base64-encoded 32-byte key (section 2). Every command that touches the database needs it.

**A command fails to decrypt / "wrong EMCLI_KEY?".** The key doesn't match the one used when the
account was added. Restore the original key, or re-add the account with the current key.

**`doctor` shows `IMAP FAIL` / `SMTP FAIL`.**
- *Invalid credentials / authentication failed* — wrong username or password. For Gmail, make sure
  you're using an **app password** (section 4) and that IMAP is enabled in Gmail settings.
- *Connection errors* — wrong host/port/security. Try the provider's documented settings; for SMTP,
  `587`/`starttls` is a common alternative to `465`/`tls`.

**The agent can't see a message you know exists.** It's probably filtered: check the account's
inbound whitelist (`emcli whitelist in list --account NAME`) and subject filter. Filtered mail is
invisible by design.

**`send` is blocked.**
- `ro_mode` — the account is read-only. Change it: `emcli account edit --name NAME --mode RW`
  (and set SMTP details).
- `whitelist_out` — a recipient isn't on the outbound whitelist. Add it, or review the rule.

**`list --new` is empty but there's mail.** By default, mail that existed before you added the
account is treated as already handled. Add the account with `--process-backlog` to treat existing
mail as new, or just use `list`/`search` without `--new`.

**The interactive form won't open ("could not open a new TTY").** Interactive commands (`init`,
`account add`/`edit` without flags) need a real terminal. Use the flag-based forms instead when
running non-interactively.

---

## 12. Command cheat sheet

```
# Help
emcli                  # or: emcli help / emcli --help — list all commands
emcli <command> --help # usage and flags for one command

# Admin
emcli init                                            # create DB + add first account (form)
emcli account add [flags | none for form]            # add an account
emcli account list                                    # list accounts (no secrets)
emcli account edit --name N [flags | none for form]  # change an account
emcli account remove --name N --yes                   # delete an account
emcli whitelist in|out add|remove|list --account N [--address A]
emcli config set|get <key> [value]                    # e.g. audit_retention_days
emcli audit list [--account N] [--limit K]
emcli doctor [--account N]                             # connectivity/auth check
emcli version

# Agent (one line of JSON each)
emcli list   --account N [--folder F] [--new] [--limit K] [--before U] [--since U]
emcli get    --account N [--folder F] --uid U
emcli search --account N [--folder F] [--from A] [--subject-contains S] [--text S] [--since-date D] [--before-date D] [--limit K]
emcli ack    --account N [--folder F] --uid-list U1,U2,U3
emcli send   --account N --to A [--cc A] [--bcc A] --subject S --body B [--attach P]… [--reply-to U [--folder F]]
```

Environment: `EMCLI_KEY` (required, base64 32-byte key), `EMCLI_DB` (optional DB path).