Complete outstanding ingestion and document UX work

This commit is contained in:
2026-08-22 18:54:07 +01:00
parent 932e889ee8
commit 5b5a9ecbc3
24 changed files with 696 additions and 41 deletions
+23 -2
View File
@@ -111,10 +111,12 @@ Override via environment variables (`KB_ENGINE_URL`, `KB_API_KEY`) or CLI flags
# Add notes
kb addnote "Always restart nginx after config changes"
kb addnote "Server room is building 3, floor 2" --tags ops
kb addnote "Deploy checklist" --wait
# Add files (async — uploads and exits immediately)
# Add files (async by default; --wait blocks until ingestion finishes)
kb addfile ~/docs/manual.pdf --tags admin
kb addfile ~/notes/ --recursive
kb addfile ~/docs/manual.pdf --wait
# Check ingestion progress
kb jobs
@@ -122,13 +124,16 @@ kb jobs
# Search
kb search "how to install git"
kb search "deploy process" --tags ops --type pdf
kb find "vehicle handbook" --type pdf
# Update a note in place
kb updatenote 42 "revised note content"
# Manage
kb list
kb info 1
kb list --title handbook
kb list --filename M38T_PHEV
kb info 1 --no-chunks
kb tags
kb tag 1 --add important
kb export 1 -o manual.pdf # download original file
@@ -160,6 +165,7 @@ The engine is configured via environment variables (set in the compose file or v
| `KB_INGEST_DEVICE` | `auto` | Docling layout detection device: `auto`, `cpu`, or `cuda` |
| `KB_API_KEY` | (none) | Optional Bearer token for API authentication |
| `KB_SEARCH_THRESHOLD` | `0.01` | Minimum score for search results (filters noise) |
| `KB_MIN_CHUNK_ALNUM` | `3` | Minimum alphanumeric characters retained in Docling PDF/DOCX/HTML chunks (`0` disables) |
| `KB_RERANK_ENABLED` | `false` (`true` in nvidia compose) | Load a cross-encoder and rerank hybrid results server-side |
| `KB_RERANKER_MODEL` | `BAAI/bge-reranker-v2-m3` | Cross-encoder model for reranking |
| `KB_RERANK_CANDIDATES` | `40` | Hybrid candidates scored by the reranker per query |
@@ -169,6 +175,21 @@ The engine is configured via environment variables (set in the compose file or v
| `HF_HUB_OFFLINE` | (none) | Set to `1` to prevent model downloads (use cached only) |
| `KB_DATA_PATH` | `./data` | Host path for bind mount (compose variable, not used by engine) |
### Repairing legacy note titles
Notes created by older clients may have a synthetic `<uuid>_note.note` title.
The maintenance command previews only unambiguous matches by default:
```bash
cd engine
python -m kb.maintenance.backfill_note_titles
python -m kb.maintenance.backfill_note_titles --apply
```
The apply mode reloads the configured embedding model and refreshes each
affected note's title, enriched full-text content, and vector embedding. Back
up the data directory before running it against production.
## Data portability
The data directory contains everything: SQLite database, model cache, and staging files. To migrate between hosts: