Commit Graph

8 Commits

Author SHA1 Message Date
steve 6dfc13be1d Add reranking, RRF fusion, bench harness, tag contexts, and data ingestion
Implements five of the six enhancements from docs/kb-enhancements-proposal.htm,
closing the retrieval-quality gap identified in the qmd review.

- Cross-encoder reranking: new kb/reranker.py loads an optional reranking
  model at startup (KB_RERANK_ENABLED, KB_RERANKER_MODEL,
  KB_RERANK_CANDIDATES). Search degrades gracefully to plain hybrid
  retrieval when the model is absent. Exposed via a "rerank" block in
  /status, a rerank flag on search, and --no-rerank in the CLI.
- RRF rank fusion: FTS and vector lists now merge by reciprocal rank
  fusion with a top-rank bonus, replacing the old score blend. Scores are
  comparable across queries.
- Bench harness and explain traces: kb bench runs a query fixture against
  each backend and reports precision@k, recall and MRR. --explain returns a
  per-result score breakdown.
- Tag context descriptions: tags carry an optional one-line description
  (kb tag-describe), returned as tag_contexts with search results. Adds a
  tags.description column migration.
- Structured data ingestion: .json/.yaml/.toml files ingest as text via the
  new "data" doc type, pretty-printing minified JSON before chunking.

Query expansion (proposal item 5) is deliberately left out pending bench
results. Requires engine v3.3.0.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-21 09:51:33 +01:00
steve e9a282ddb1 Document KB_BULK_SAFETY_PERCENT in README, DEVELOPER, MCP, and SKILL docs
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-04 22:43:42 +01:00
steve b5a203d2aa Add bulk operations and remove collections abstraction
- Add bulk delete, bulk tags, and bulk set-tags engine endpoints
  (POST /api/v1/bulk/delete, /bulk/tags, /bulk/set-tags)
- Filter-based selection: by tags, doc_type, ID list, ID range
- Safety threshold (KB_BULK_SAFETY_PERCENT, default 70%) prevents
  accidental mass operations unless force=true
- Synchronous execution with audit trail via jobs table
- Add kb_bulk_delete, kb_bulk_tags, kb_bulk_set_tags MCP tools
- Add kb bulk-remove, bulk-tag, bulk-set-tags CLI commands
- Remove collection abstraction from MCP server (use tags instead)
- Remove kb_set_collection MCP tool
- Update SKILL.md, MCP.md, README.md documentation

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-04 22:34:47 +01:00
steve e7136a4a20 Add MCP server, note mutation endpoint, and updated_at tracking (v3.0.0)
New MCP server (mcp/) exposes kb operations as native MCP tools over
Streamable HTTP with Bearer token auth. Supports collections via tag
conventions, chunked file uploads, and agent-side search patterns.

Engine gains PATCH /api/v1/notes/{id} for in-place note updates with
transactional re-chunk/re-embed, and updated_at column on documents.

Go client adds updatenote command and Patch HTTP method.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-02 21:34:55 +01:00
steve 2d179af557 Fix search human-mode output to match engine API response
The Go client struct expected a nested document object and top-level
page/section fields, but the engine returns flat results with metadata
in chunk_metadata. This caused empty display for title, type, tags,
page, and section in human output mode.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-02 16:17:35 +01:00
steve afbe270181 Replace implicit note shorthand with explicit addnote command and split README
Two changes:

1. structured-add-commands: The implicit note shorthand (kb "text") caused
   accidental note creation from mistyped commands. Replaced with explicit
   kb addnote <text> command. Root command reverts to standard Cobra
   behaviour. Updated examples, tests, SKILL.md, and specs.

2. split-readme-developer-docs: Moved build-from-source instructions, release
   process, API reference, and ROCm migration notes from README.md into a
   new DEVELOPER.md. README now links to DEVELOPER.md for dev workflows.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-31 20:48:22 +01:00
steve 7f4decee26 Reindex command, implicit note shorthand, add→addfile rename
- Add `kb reindex` command with confirmation prompt and --yes flag
- Add implicit note shorthand: `kb "my note"` submits a note directly
- Rename `add` to `addfile`, remove --note/--title/--type flags
- Add client-side file extension validation before upload
- Add `kb examples` command for common usage patterns
- Update README, SKILL.md, and main specs
- Archive completed changes and sync delta specs

BREAKING: `kb add` renamed to `kb addfile`, `kb add --note` replaced by `kb "text"`

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-29 13:58:04 +01:00
steve f245c24928 Initial MVP 2026-03-23 20:38:42 +00:00