43 Commits

Author SHA1 Message Date
steve 75e4a0cf73 Bump engine version to 3.2.4 engine-v3.2.4 2026-05-15 18:37:12 +01:00
steve 753c641e72 Switch container registry to Gitea built-in (gitea.dcglab.co.uk/steve/kb)
The standalone Registry v2 host (docker.dcglab.co.uk, briefly registry.dcglab.co.uk)
is being scrapped. Move all kb images to Gitea's built-in container registry.
2026-05-15 18:35:20 +01:00
steve 45e2c5ce91 Bump engine version to 3.2.3 engine-v3.2.3 2026-05-15 18:22:08 +01:00
steve e6e91f1d5c Clarify hybrid semantic + full-text search in MCP descriptions
Agents were misreading kb_search as keyword-only because the vector/semantic
component was only mentioned in the negative ("fts_only: no vector similarity").
Lead with hybrid semantic + BM25 + RRF in the server instructions, kb_search
docstring, and MCP.md so agents recognise it as a vector search tool.
2026-05-15 18:19:42 +01:00
steve 9eccc527ae Add next-steps.md with UX improvement ideas for kb CLI
Captures pain points found while trying to locate an uploaded PDF: kb
list silently ignores positional args, kb search results lack
document_id, kb info dumps all chunks with no summary mode, and
scan-heavy PDFs produce noisy single-char chunk hits.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-13 20:34:55 +01:00
steve d44d11e4fe Bump engine version to 3.2.2 engine-v3.2.2 2026-04-14 21:48:55 +01:00
steve 574370e8d1 Remove AMD ROCm support — CPU and NVIDIA only
BREAKING: Remove Dockerfile.rocm, compose.rocm.yaml, and ROCm image
build/push from the release pipeline. Remove AMD quick-start and ROCm
references from README and DEVELOPER docs. Update docker-deployment
and developer-docs specs to reflect CPU + NVIDIA only.

The ROCm variant added significant complexity (4.2GB torch wheel,
>20GB container) with limited usage. Users on AMD GPUs should stay
on engine v3.2.x or switch to CPU mode.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
engine-v3.2.1
2026-04-06 16:39:37 +01:00
steve 17b19999de Switch nvidia and rocm Dockerfiles from onnxruntime to torch
Nvidia: install torch+torchvision from PyTorch cu130 index, drop
onnxruntime-gpu. ROCm: use local torch wheel with rocm6.4 index for
torchvision, clean up nvidia remnants from the venv.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-06 16:13:41 +01:00
steve bb78f4ea80 Fix 500 error on notes with slashes in title, bump engine to 3.2.1
Sanitize / and \ in note titles and filenames when writing to the
staging directory — a title like "/reset skill" was interpreted as a
path separator, causing a FileNotFoundError and a 500 from the jobs
endpoint. Also add PRAGMA busy_timeout=5000 to SQLite connections to
prevent immediate failure under concurrent write load.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-06 16:12:58 +01:00
steve 223ff2cf5d Latest changes all archived 2026-04-04 22:50:19 +01:00
steve e9a282ddb1 Document KB_BULK_SAFETY_PERCENT in README, DEVELOPER, MCP, and SKILL docs
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-04 22:43:42 +01:00
steve b5a203d2aa Add bulk operations and remove collections abstraction
- Add bulk delete, bulk tags, and bulk set-tags engine endpoints
  (POST /api/v1/bulk/delete, /bulk/tags, /bulk/set-tags)
- Filter-based selection: by tags, doc_type, ID list, ID range
- Safety threshold (KB_BULK_SAFETY_PERCENT, default 70%) prevents
  accidental mass operations unless force=true
- Synchronous execution with audit trail via jobs table
- Add kb_bulk_delete, kb_bulk_tags, kb_bulk_set_tags MCP tools
- Add kb bulk-remove, bulk-tag, bulk-set-tags CLI commands
- Remove collection abstraction from MCP server (use tags instead)
- Remove kb_set_collection MCP tool
- Update SKILL.md, MCP.md, README.md documentation

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
engine-v3.2.0 client-v3.2.0
2026-04-04 22:34:47 +01:00
steve 0c124c4ab7 Bump engine version to 3.0.1 engine-v3.0.1 2026-04-04 12:42:32 +01:00
steve da5b8435bc Add configurable allowed hosts for MCP remote access (KB_MCP_ALLOWED_HOSTS)
The MCP SDK's DNS rebinding protection rejects remote clients with 421
when the Host header isn't in the allowlist. Add KB_MCP_ALLOWED_HOSTS env
var (comma-separated IPs/FQDNs) to configure additional allowed hosts
while keeping localhost always permitted.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-04 12:39:43 +01:00
steve e39e00a2c0 Add MCP auth status to kb_status and update server instructions
- kb_status now returns authenticated: true/false so clients can verify auth
- Server instructions mention Bearer token auth requirement
- Add .env, .venv/, test_mcp_client.py to .gitignore

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
engine-v3.0.0
2026-04-04 12:04:12 +01:00
steve d078af9ad3 Split MCP docs into MCP.md with AI tool setup examples
Move MCP server documentation from README into dedicated MCP.md.
Add configuration examples for Claude Code, VS Code, Cursor,
Windsurf, and JetBrains IDEs.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-02 22:03:41 +01:00
steve b3dce188e1 Fix version check failing on non-200 status responses
When the engine returns 401 (auth required) or other non-200 responses,
the version check was parsing the error body, getting an empty version
string, and fatally exiting. Now skips the check on non-200 responses
and lets the actual API call surface the real error.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-02 21:52:24 +01:00
steve 0dc3065979 Update README for v3.0.0 — add MCP server docs, updatenote, fix version refs
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-02 21:45:31 +01:00
steve e7136a4a20 Add MCP server, note mutation endpoint, and updated_at tracking (v3.0.0)
New MCP server (mcp/) exposes kb operations as native MCP tools over
Streamable HTTP with Bearer token auth. Supports collections via tag
conventions, chunked file uploads, and agent-side search patterns.

Engine gains PATCH /api/v1/notes/{id} for in-place note updates with
transactional re-chunk/re-embed, and updated_at column on documents.

Go client adds updatenote command and Patch HTTP method.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
client-v3.0.0
2026-04-02 21:34:55 +01:00
steve adeba21712 Bump client version to 2.2.1 client-v2.2.1 2026-04-02 16:18:06 +01:00
steve 2d179af557 Fix search human-mode output to match engine API response
The Go client struct expected a nested document object and top-level
page/section fields, but the engine returns flat results with metadata
in chunk_metadata. This caused empty display for title, type, tags,
page, and section in human output mode.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-02 16:17:35 +01:00
steve a6bab5e55e Add CPU-only Docker image and fix release tag naming
- Add Dockerfile.cpu and compose.cpu.yaml for CPU-only deployments
- Use sentence-transformers[onnx] + CPU-only torch for ~4x smaller image
- Fix release script: separate git tags (engine-v*) from Docker tags (v*)
- Add CPU image to release build/push pipeline
- Update README with CPU deployment instructions

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-02 16:02:00 +01:00
steve c5191df9c0 Bump client version to 2.2.0 client-v2.2.0 2026-03-31 20:50:17 +01:00
steve afbe270181 Replace implicit note shorthand with explicit addnote command and split README
Two changes:

1. structured-add-commands: The implicit note shorthand (kb "text") caused
   accidental note creation from mistyped commands. Replaced with explicit
   kb addnote <text> command. Root command reverts to standard Cobra
   behaviour. Updated examples, tests, SKILL.md, and specs.

2. split-readme-developer-docs: Moved build-from-source instructions, release
   process, API reference, and ROCm migration notes from README.md into a
   new DEVELOPER.md. README now links to DEVELOPER.md for dev workflows.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-31 20:48:22 +01:00
steve 9e957f1a9a Added pycache to gitignore 2026-03-30 07:26:16 +01:00
steve bbe6a5e909 Add dev-up script and archive kb-title-in-chunks change
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-30 07:25:22 +01:00
steve 743102aee4 Bump client version to 2.1.1 client-v2.1.1 2026-03-29 21:09:55 +01:00
steve 0f3b3be59f Bump engine version to 2.1.0 engine-v2.1.0 2026-03-29 21:06:04 +01:00
steve 2fa2ac1134 Reject single bare word as implicit note shorthand
Single unrecognized words now print an error with usage hint instead of
being submitted as a note. Prevents typos from creating junk notes.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-29 21:03:52 +01:00
steve b2176c36ea Chunk enrichment: prepend document title to embeddings
Adds enriched_text column to chunks table that prepends document title
(and section header when present) to chunk text. Embeddings and FTS now
use enriched text for better search relevance. Includes schema migration
with backfill for existing data.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-29 21:03:48 +01:00
steve 5f9946efc9 Added manual download to README 2026-03-29 14:03:35 +01:00
steve ea3d5707e1 Bump client version to 2.1.0 client-v2.1.0 2026-03-29 13:58:42 +01:00
steve 7f4decee26 Reindex command, implicit note shorthand, add→addfile rename
- Add `kb reindex` command with confirmation prompt and --yes flag
- Add implicit note shorthand: `kb "my note"` submits a note directly
- Rename `add` to `addfile`, remove --note/--title/--type flags
- Add client-side file extension validation before upload
- Add `kb examples` command for common usage patterns
- Update README, SKILL.md, and main specs
- Archive completed changes and sync delta specs

BREAKING: `kb add` renamed to `kb addfile`, `kb add --note` replaced by `kb "text"`

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-29 13:58:04 +01:00
steve 528a09ca90 Independent client/engine versioning with compatibility check
Split release.sh into release-client.sh and release-engine.sh for
independent release cadences. Client checks engine version on first
API call and hard-fails if engine is below MinEngineVersion.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
client-v2.0.6 engine-v2.0.6
2026-03-28 15:59:16 +00:00
steve b04823e67b Store original documents for download after ingestion
Persist uploaded files to {data_dir}/documents/{content_hash}{ext} after
successful ingestion. Add GET /documents/{id}/file endpoint for retrieval,
delete stored files on document deletion, and add `kb export` client command.
Includes schema migration, tests, and spec updates.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-28 15:16:27 +00:00
steve 6a4bce4659 Bump version to 2.0.5 v2.0.5 2026-03-26 23:08:48 +00:00
steve 4590c124ad Merge pull request 'Upload-time dedup, FTS5 query sanitization, release guard' (#1) from 2.0.5 into main
Reviewed-on: #1
2026-03-26 23:06:08 +00:00
steve 6fec627503 Upload-time duplicate detection, FTS5 query sanitization, release guard
- Reject duplicate uploads at the API boundary (HTTP 409) instead of
  silently skipping in the background worker. Checks both ingested
  documents and in-flight jobs via content_hash on the jobs table.
- Go client handles 409 with distinct messages for already-imported
  documents vs already-queued jobs.
- Sanitize FTS5 search queries by quoting each token to prevent syntax
  errors from special characters like ?, *, ", (), AND, OR, NOT.
- Add try/except safety net around FTS5 execute for edge cases.
- Add main branch guard to release.sh to prevent releasing from
  feature branches.
- Update specs and README to reflect new behaviour.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-26 23:05:07 +00:00
steve 63654a59b8 Fix tea asset upload syntax in release script
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-26 22:10:48 +00:00
steve 178a4a39fe Bump version to 2.0.4 v2.0.4 2026-03-26 21:57:04 +00:00
steve 9aab79d49b v2 restructure: Go client, Docker engine, release tooling
- Remove v1 Python CLI (src/kb_search/, tests/, root pyproject.toml, uv.lock, .venv)
- Add Go client with cross-platform build (client/)
- Add FastAPI engine with NVIDIA and multi-stage ROCm Dockerfiles (engine/)
- Add VERSION files for client and engine, wired into builds
- Add release.sh for automated build, tag, release, and Docker push
- Update README with build/release docs and ROCm migration note
- Clean up .gitignore for v2 project structure

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-26 21:52:25 +00:00
steve 2030976b85 Add GPU device control, Docker support, and v2 client-server design
- Add configurable device selection for embeddings (embedding.device) and
  Docling ingestion (ingestion.device) with env var overrides (KB_DEVICE,
  KB_INGEST_DEVICE) to control GPU/CPU usage per component
- Add `kb doctor` command for safe GPU diagnostics
- Add Dockerfile (NVIDIA CUDA) and compose.yaml for containerised GPU usage
- Add OpenSpec v2 change (kb-v2-client-server): proposal, design, specs, and
  tasks for client-server architecture with Go CLI, FastAPI engine, async
  ingestion queue, and GPU-vendor-agnostic Docker deployment
- Add uv.lock for reproducible installs
- Gitignore examples/ directory (test data only)

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-25 20:17:31 +00:00
steve f245c24928 Initial MVP 2026-03-23 20:38:42 +00:00