Add reranking, RRF fusion, bench harness, tag contexts, and data ingestion

Implements five of the six enhancements from docs/kb-enhancements-proposal.htm,
closing the retrieval-quality gap identified in the qmd review.

- Cross-encoder reranking: new kb/reranker.py loads an optional reranking
  model at startup (KB_RERANK_ENABLED, KB_RERANKER_MODEL,
  KB_RERANK_CANDIDATES). Search degrades gracefully to plain hybrid
  retrieval when the model is absent. Exposed via a "rerank" block in
  /status, a rerank flag on search, and --no-rerank in the CLI.
- RRF rank fusion: FTS and vector lists now merge by reciprocal rank
  fusion with a top-rank bonus, replacing the old score blend. Scores are
  comparable across queries.
- Bench harness and explain traces: kb bench runs a query fixture against
  each backend and reports precision@k, recall and MRR. --explain returns a
  per-result score breakdown.
- Tag context descriptions: tags carry an optional one-line description
  (kb tag-describe), returned as tag_contexts with search results. Adds a
  tags.description column migration.
- Structured data ingestion: .json/.yaml/.toml files ingest as text via the
  new "data" doc type, pretty-printing minified JSON before chunking.

Query expansion (proposal item 5) is deliberately left out pending bench
results. Requires engine v3.3.0.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

openspec/specs/agent-search-patterns/spec.md

@@ -2,7 +2,7 @@
 
 ## Purpose
 
-Documents recommended patterns for agent-side query expansion and reranking, which are caller responsibilities rather than engine features. These patterns are communicated via MCP tool descriptions.
+Documents recommended patterns for agent-side query expansion, plus how agent guidance interacts with the engine's optional server-side reranking. These patterns are communicated via MCP tool descriptions.
 
 ## Requirements
 
@@ -18,18 +18,26 @@ The `kb_search` MCP tool description SHALL include guidance on query expansion a
 
 ### Requirement: Reranking guidance in tool description
 
-The `kb_search` MCP tool description SHALL include guidance on agent-side reranking as a recommended pattern for improving precision.
+The `kb_search` MCP tool description SHALL describe the engine's server-side reranking behaviour and retain agent-side reranking as a fallback pattern.
 
-#### Scenario: Tool description includes reranking pattern
+#### Scenario: Tool description covers server-side reranking
 - **WHEN** an agent reads the `kb_search` tool description
-- **THEN** the description SHALL include guidance such as: "For precision, rerank the returned results using your own judgement based on relevance to the original question"
+- **THEN** the description SHALL state that results are reranked server-side by default when the engine has a reranker enabled, that `rerank=False` skips it for lower latency, and that `kb_status` reports whether reranking is active
+
+#### Scenario: Tool description retains agent-side fallback
+- **WHEN** an agent reads the `kb_search` tool description
+- **THEN** the description SHALL include guidance that, when the engine's reranker is disabled, the agent can rerank the returned results using its own judgement of relevance to the original question
 
 ---
 
-### Requirement: No engine-side LLM dependency
+### Requirement: No external LLM dependency
 
-The engine SHALL NOT require or use any external LLM API for search operations. Query expansion and reranking SHALL remain entirely agent-side concerns.
+The engine SHALL NOT require or use any external LLM API for search operations. Query expansion SHALL remain an agent-side concern. Reranking MAY be performed engine-side using a local, opt-in cross-encoder model; it SHALL never depend on an external API.
 
-#### Scenario: Engine has no LLM dependency
+#### Scenario: Engine has no external LLM dependency
 - **WHEN** the engine is deployed without any `ANTHROPIC_API_KEY` or similar LLM API configuration
 - **THEN** all search operations SHALL function fully, with no degraded results or missing features
+
+#### Scenario: Reranking is optional and degrades gracefully
+- **WHEN** the engine is deployed with `KB_RERANK_ENABLED` unset or false, or the reranker model fails to load
+- **THEN** all search operations SHALL function fully using hybrid retrieval alone, with responses reporting `"reranked": false`