steve
6dfc13be1d
Implements five of the six enhancements from docs/kb-enhancements-proposal.htm, closing the retrieval-quality gap identified in the qmd review. - Cross-encoder reranking: new kb/reranker.py loads an optional reranking model at startup (KB_RERANK_ENABLED, KB_RERANKER_MODEL, KB_RERANK_CANDIDATES). Search degrades gracefully to plain hybrid retrieval when the model is absent. Exposed via a "rerank" block in /status, a rerank flag on search, and --no-rerank in the CLI. - RRF rank fusion: FTS and vector lists now merge by reciprocal rank fusion with a top-rank bonus, replacing the old score blend. Scores are comparable across queries. - Bench harness and explain traces: kb bench runs a query fixture against each backend and reports precision@k, recall and MRR. --explain returns a per-result score breakdown. - Tag context descriptions: tags carry an optional one-line description (kb tag-describe), returned as tag_contexts with search results. Adds a tags.description column migration. - Structured data ingestion: .json/.yaml/.toml files ingest as text via the new "data" doc type, pretty-printing minified JSON before chunking. Query expansion (proposal item 5) is deliberately left out pending bench results. Requires engine v3.3.0. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
44 lines
2.5 KiB
Markdown
44 lines
2.5 KiB
Markdown
# Agent-Side Search Patterns
|
|
|
|
## Purpose
|
|
|
|
Documents recommended patterns for agent-side query expansion, plus how agent guidance interacts with the engine's optional server-side reranking. These patterns are communicated via MCP tool descriptions.
|
|
|
|
## Requirements
|
|
|
|
### Requirement: Query expansion guidance in tool description
|
|
|
|
The `kb_search` MCP tool description SHALL include guidance on query expansion as a recommended pattern for complex queries.
|
|
|
|
#### Scenario: Tool description includes expansion pattern
|
|
- **WHEN** an agent reads the `kb_search` tool description
|
|
- **THEN** the description SHALL include guidance such as: "For complex queries, consider expanding into 2-3 variant phrasings and calling this tool multiple times, then deduplicating results by chunk_id"
|
|
|
|
---
|
|
|
|
### Requirement: Reranking guidance in tool description
|
|
|
|
The `kb_search` MCP tool description SHALL describe the engine's server-side reranking behaviour and retain agent-side reranking as a fallback pattern.
|
|
|
|
#### Scenario: Tool description covers server-side reranking
|
|
- **WHEN** an agent reads the `kb_search` tool description
|
|
- **THEN** the description SHALL state that results are reranked server-side by default when the engine has a reranker enabled, that `rerank=False` skips it for lower latency, and that `kb_status` reports whether reranking is active
|
|
|
|
#### Scenario: Tool description retains agent-side fallback
|
|
- **WHEN** an agent reads the `kb_search` tool description
|
|
- **THEN** the description SHALL include guidance that, when the engine's reranker is disabled, the agent can rerank the returned results using its own judgement of relevance to the original question
|
|
|
|
---
|
|
|
|
### Requirement: No external LLM dependency
|
|
|
|
The engine SHALL NOT require or use any external LLM API for search operations. Query expansion SHALL remain an agent-side concern. Reranking MAY be performed engine-side using a local, opt-in cross-encoder model; it SHALL never depend on an external API.
|
|
|
|
#### Scenario: Engine has no external LLM dependency
|
|
- **WHEN** the engine is deployed without any `ANTHROPIC_API_KEY` or similar LLM API configuration
|
|
- **THEN** all search operations SHALL function fully, with no degraded results or missing features
|
|
|
|
#### Scenario: Reranking is optional and degrades gracefully
|
|
- **WHEN** the engine is deployed with `KB_RERANK_ENABLED` unset or false, or the reranker model fails to load
|
|
- **THEN** all search operations SHALL function fully using hybrid retrieval alone, with responses reporting `"reranked": false`
|