Add reranking, RRF fusion, bench harness, tag contexts, and data ingestion

Implements five of the six enhancements from docs/kb-enhancements-proposal.htm,
closing the retrieval-quality gap identified in the qmd review.

- Cross-encoder reranking: new kb/reranker.py loads an optional reranking
  model at startup (KB_RERANK_ENABLED, KB_RERANKER_MODEL,
  KB_RERANK_CANDIDATES). Search degrades gracefully to plain hybrid
  retrieval when the model is absent. Exposed via a "rerank" block in
  /status, a rerank flag on search, and --no-rerank in the CLI.
- RRF rank fusion: FTS and vector lists now merge by reciprocal rank
  fusion with a top-rank bonus, replacing the old score blend. Scores are
  comparable across queries.
- Bench harness and explain traces: kb bench runs a query fixture against
  each backend and reports precision@k, recall and MRR. --explain returns a
  per-result score breakdown.
- Tag context descriptions: tags carry an optional one-line description
  (kb tag-describe), returned as tag_contexts with search results. Adds a
  tags.description column migration.
- Structured data ingestion: .json/.yaml/.toml files ingest as text via the
  new "data" doc type, pretty-printing minified JSON before chunking.

Query expansion (proposal item 5) is deliberately left out pending bench
results. Requires engine v3.3.0.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

SKILL.md

@@ -30,11 +30,13 @@ Returns JSON with ranked results combining full-text and semantic search.
 **Flags:**
 - `-n, --top N` — number of results (default: 10)
 - `--tags tag1,tag2` — filter by tags (AND logic)
-- `--type pdf|markdown|code|note` — filter by document type
+- `--type pdf|markdown|code|note|data` — filter by document type
 - `--format json|human` — output format (always use json for parsing)
 - `--fts-only` — keyword search only (skip semantic)
 - `--vec-only` — semantic search only (skip keyword)
 - `--threshold FLOAT` — minimum score cutoff
+- `--explain` — include a per-result score breakdown (FTS/vector scores and ranks, fusion contributions, rerank blend)
+- `--no-rerank` — skip server-side cross-encoder reranking for lower latency (when the engine has it enabled)
 
 ## Adding files
 
@@ -45,7 +47,7 @@ kb addfile ~/docs/ --recursive                  # directory (recursive)
 kb addfile ~/docs/ --recursive --tags reference # directory with tags
 ```
 
-Supported file types: `.pdf`, `.docx`, `.html`, `.md`, `.txt`, `.py`, `.sh`, `.go`. Unsupported extensions are rejected before upload.
+Supported file types: `.pdf`, `.docx`, `.html`, `.md`, `.txt`, `.py`, `.sh`, `.go`, `.json`, `.yaml`, `.yml`, `.toml`. Unsupported extensions are rejected before upload. Data files (`.json`/`.yaml`/`.yml`/`.toml`) are ingested as text with doc type `data`; minified JSON is pretty-printed before chunking.
 
 **Flags:**
 - `--tags tag1,tag2` — tags (comma-separated)
@@ -66,11 +68,17 @@ kb remove <doc_id> --yes                 # remove without confirmation
 ## Tag management
 
 ```bash
-kb tags --format json                    # list all tags with counts
+kb tags --format json                    # list all tags with counts and descriptions
 kb tag <doc_id> --add important,ops      # add tags to a document
 kb tag <doc_id> --remove draft           # remove tags from a document
+kb tag-describe ops "Lab operations runbooks"   # set a tag context description
+kb tag-describe ops                      # clear a tag's description
 ```
 
+Tag descriptions are returned as `tag_contexts` with every search result on a
+document carrying the tag — use them to judge which of several similar-scoring
+chunks actually answers the question.
+
 ## Bulk operations
 
 Operate on multiple documents at once using filter-based selection. Filters combine with AND logic.
@@ -138,6 +146,7 @@ All commands support:
   "results": [
     {
       "chunk_id": 1423,
+      "document_id": 87,
       "score": 0.031,
       "text": "To install the latest version of git from source...",
       "chunk_index": 3,
@@ -146,11 +155,13 @@ All commands support:
       "doc_type": "pdf",
       "source_path": "/home/user/docs/git-admin.pdf",
       "created_at": "2026-03-15T10:30:00",
-      "tags": ["git", "admin"]
+      "tags": ["git", "admin"],
+      "tag_contexts": {"admin": "System administration guides"}
     }
   ],
   "total_matches": 47,
-  "returned": 10
+  "returned": 10,
+  "reranked": true
 }
 ```
 
@@ -217,7 +228,8 @@ If the kb engine is already running via Docker Compose, add the MCP server by de
 ## Important notes
 
 - Always use `--format json` for machine parsing
-- The `score` field is relative, not absolute — compare scores within a result set
+- The `score` field is relative, not absolute — compare scores within a result set. Reranked hybrid scores (`"reranked": true`) are 0-1 blended values on a different scale from non-reranked RRF scores; don't compare across the two modes or apply `--threshold` expecting RRF-scale values on reranked output
+- When the engine reranker is enabled, results are already cross-encoder reranked server-side — no need to rerank them yourself
 - `chunk_metadata.page` is only present for PDF documents
 - `chunk_metadata.section_header` is only present for markdown documents with headers
 - Results are already ranked by relevance (hybrid FTS + vector search)