Reindex command, implicit note shorthand, add→addfile rename

- Add `kb reindex` command with confirmation prompt and --yes flag - Add implicit note shorthand: `kb "my note"` submits a note directly - Rename `add` to `addfile`, remove --note/--title/--type flags - Add client-side file extension validation before upload - Add `kb examples` command for common usage patterns - Update README, SKILL.md, and main specs - Archive completed changes and sync delta specs BREAKING: `kb add` renamed to `kb addfile`, `kb add --note` replaced by `kb "text"` Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-29 13:58:04 +01:00
parent 528a09ca90
commit 7f4decee26
26 changed files with 786 additions and 142 deletions
@@ -18,6 +18,73 @@ Go CLI (kb) ──HTTP──▶ FastAPI Engine (Docker) ──▶ SQLite + GPU

 ### 1. Start the engine

+**From pre-built images** (recommended):
+
+```bash
+# NVIDIA GPU
+docker run -d --name kb-engine \
+  --gpus all \
+  -p 8000:8000 \
+  -v ~/kb-data:/data \
+  -e KB_MODEL=all-MiniLM-L6-v2 \
+  -e KB_DEVICE=auto \
+  -e KB_API_KEY=your-secret-key \
+  --restart unless-stopped \
+  docker.dcglab.co.uk/dcg/kb/engine:latest-nvidia
+
+# AMD GPU (ROCm)
+docker run -d --name kb-engine \
+  --device /dev/kfd --device /dev/dri \
+  --group-add video \
+  -p 8000:8000 \
+  -v ~/kb-data:/data \
+  -e KB_MODEL=all-MiniLM-L6-v2 \
+  -e KB_DEVICE=auto \
+  -e KB_API_KEY=your-secret-key \
+  --restart unless-stopped \
+  docker.dcglab.co.uk/dcg/kb/engine:latest-rocm
+```
+
+Or use a compose file — create `compose.yaml`:
+
+```yaml
+services:
+  kb-engine:
+    image: docker.dcglab.co.uk/dcg/kb/engine:latest-nvidia  # or latest-rocm
+    runtime: nvidia  # remove for ROCm
+    deploy:
+      resources:
+        reservations:
+          devices:
+            - driver: nvidia
+              count: 1
+              capabilities: [gpu]
+    # For ROCm, replace the above runtime/deploy block with:
+    # devices:
+    #   - "/dev/kfd"
+    #   - "/dev/dri"
+    # group_add:
+    #   - "video"
+    ports:
+      - "${KB_PORT:-8000}:8000"
+    volumes:
+      - ${KB_DATA_PATH:-./data}:/data
+    environment:
+      - KB_MODEL=${KB_MODEL:-all-MiniLM-L6-v2}
+      - KB_DEVICE=${KB_DEVICE:-auto}
+      - KB_INGEST_DEVICE=${KB_INGEST_DEVICE:-auto}
+      - KB_API_KEY=${KB_API_KEY:-}
+      - KB_SEARCH_THRESHOLD=${KB_SEARCH_THRESHOLD:-0.01}
+      - HF_HUB_OFFLINE=${HF_HUB_OFFLINE:-}
+    restart: unless-stopped
+```
+
+```bash
+KB_DATA_PATH=~/kb-data docker compose up -d
+```
+
+**From source** (for development):
+
 ```bash
 cd engine

@@ -65,10 +132,13 @@ Override via environment variables (`KB_ENGINE_URL`, `KB_API_KEY`) or CLI flags
 ### 4. Use it

 ```bash
-# Add documents (async — uploads and exits immediately)
-kb add ~/docs/manual.pdf --tags admin
-kb add ~/notes/ --recursive
-kb add --note "Always restart nginx after config changes" --tags ops
+# Quick notes (shorthand — no subcommand needed)
+kb "Always restart nginx after config changes"
+kb "Server room is building 3, floor 2" --tags ops
+
+# Add files (async — uploads and exits immediately)
+kb addfile ~/docs/manual.pdf --tags admin
+kb addfile ~/notes/ --recursive

 # Check ingestion progress
 kb jobs
@@ -82,6 +152,7 @@ kb list
 kb info 1
 kb tags
 kb tag 1 --add important
+kb export 1 -o manual.pdf    # download original file
 kb remove 3 --yes
 kb status
 ```
@@ -100,12 +171,14 @@ The engine is configured via environment variables (set in the compose file or v
 |---|---|---|
 | `KB_DATA_DIR` | `/data` | Data directory inside the container (bind-mounted) |
 | `KB_MODEL` | `all-MiniLM-L6-v2` | HuggingFace embedding model name |
-| `KB_DEVICE` | `auto` | Embedding device: `auto`, `cpu`, or `cuda` |
-| `KB_INGEST_DEVICE` | `auto` | Docling layout detection device |
+| `KB_DEVICE` | `auto` | Embedding/search device: `auto`, `cpu`, or `cuda` |
+| `KB_INGEST_DEVICE` | `auto` | Docling layout detection device: `auto`, `cpu`, or `cuda` |
 | `KB_API_KEY` | (none) | Optional Bearer token for API authentication |
 | `KB_SEARCH_THRESHOLD` | `0.01` | Minimum score for search results (filters noise) |
 | `KB_PORT` | `8000` | Port to expose |
-| `KB_DATA_PATH` | `./data` | Host path for bind mount (compose variable) |
+| `KB_HOST` | `0.0.0.0` | Host to bind to |
+| `HF_HUB_OFFLINE` | (none) | Set to `1` to prevent model downloads (use cached only) |
+| `KB_DATA_PATH` | `./data` | Host path for bind mount (compose variable, not used by engine) |

 ## Data portability

@@ -134,7 +207,8 @@ All endpoints are under `/api/v1/`. Requires `Authorization: Bearer <key>` heade
 | `GET` | `/jobs/{id}` | Job details |
 | `GET` | `/documents` | List documents |
 | `GET` | `/documents/{id}` | Document details with chunks |
-| `DELETE` | `/documents/{id}` | Remove a document |
+| `GET` | `/documents/{id}/file` | Download original file |
+| `DELETE` | `/documents/{id}` | Remove a document (and stored file) |
 | `PUT` | `/documents/{id}/tags` | Add/remove tags |
 | `GET` | `/tags` | List all tags |
 | `GET` | `/status` | Engine status, GPU info, DB stats |
@@ -142,26 +216,31 @@ All endpoints are under `/api/v1/`. Requires `Authorization: Bearer <key>` heade

 ## Building and releasing

-Versioning is managed via `client/VERSION` and `engine/VERSION` files. The release script bumps these, builds all artifacts, tags, and publishes in one step.
+Client and engine are versioned independently via `client/VERSION` and `engine/VERSION`. Each has its own release script and git tag prefix.

-### Release
+### Release client

 ```bash
-./release.sh --gitea              # patch bump (e.g. 2.0.0 → 2.0.1), release via Gitea
-./release.sh --github --minor     # minor bump (e.g. 2.0.1 → 2.1.0), release via GitHub
-./release.sh --gitea --major      # major bump (e.g. 2.1.0 → 3.0.0)
-./release.sh --gitea --no-increment  # release current version as-is
-./release.sh --gitea --dry-run    # preview without doing anything
+./release-client.sh --gitea              # patch bump, release via Gitea
+./release-client.sh --github --minor     # minor bump, release via GitHub
+./release-client.sh --gitea --no-increment  # release current version as-is
+./release-client.sh --gitea --dry-run    # preview without doing anything
 ```

-The script will:
+Creates tag `client-vX.Y.Z`, builds Go binaries for all platforms, and creates a Gitea/GitHub release with binaries attached.

-1. Bump the version in both `client/VERSION` and `engine/VERSION` (unless `--no-increment`)
-2. Build Go client binaries for all platforms (linux/darwin/windows, amd64/arm64)
-3. Build Docker engine images for NVIDIA and ROCm
-4. Commit the version bump, create an annotated git tag, and push
-5. Create a release (with client binaries attached) via `tea` or `gh`
-6. Push Docker images to the registry
+The client embeds a `MinEngineVersion` (from `client/MIN_ENGINE_VERSION`) and will hard-fail if the connected engine is too old.
+
+### Release engine
+
+```bash
+./release-engine.sh --gitea              # patch bump, release via Gitea
+./release-engine.sh --github --minor     # minor bump, release via GitHub
+./release-engine.sh --gitea --no-increment  # release current version as-is
+./release-engine.sh --gitea --dry-run    # preview without doing anything
+```
+
+Creates tag `engine-vX.Y.Z`, builds NVIDIA and ROCm Docker images, creates a Gitea/GitHub release, and pushes images to the registry.

 ### Checking versions

@@ -177,13 +256,13 @@ curl http://localhost:8000/api/v1/status | jq .version

 Images are pushed to `docker.dcglab.co.uk/dcg/kb/engine` with tags:

- `v2.1.0-nvidia` / `v2.1.0-rocm` — versioned
+- `engine-v2.0.6-nvidia` / `engine-v2.0.6-rocm` — versioned
 - `latest-nvidia` / `latest-rocm` — latest release

 Override the registry and org via environment variables:

 ```bash
-REGISTRY=ghcr.io IMAGE_ORG=myorg ./release.sh --github
+REGISTRY=ghcr.io IMAGE_ORG=myorg ./release-engine.sh --github
 ```

 ## Future: ROCm runtime migration