kb/DEVELOPER.md

Developer Guide

Instructions for building from source, releasing, and contributing to kb.

Building from source

Engine

cd engine

# NVIDIA GPU
KB_DATA_PATH=~/kb-data docker compose -f compose.nvidia.yaml up -d

Client

cd client
make build    # produces ./kb binary
make all      # or cross-compile: dist/kb-{os}-{arch}

Running tests

Engine

Engine tests run against SQLite (with sqlite-vec) and stub out the embedding model, so they only need lightweight dependencies — no torch/docling install:

uv venv /tmp/kb-test-venv
uv pip install --python /tmp/kb-test-venv/bin/python pytest pytest-asyncio fastapi httpx sqlite-vec
cd engine && /tmp/kb-test-venv/bin/python -m pytest

Client

cd client && go test ./...

Search-quality benchmarking

kb bench fixture.json runs a fixture of queries with known-relevant documents against each backend (fts, vec, hybrid, hybrid+rerank) and reports precision@k, recall, and MRR. See docs/bench-example.json for the fixture format.

Run a bench before and after any ranking change (RRF weights, reranker, model swap) and compare — keep a 20-30 query fixture against your real corpus outside the repo.

Building and releasing

Client and engine are versioned independently via client/VERSION and engine/VERSION. Each has its own release script and git tag prefix.

Release client

./release-client.sh --gitea              # patch bump, release via Gitea
./release-client.sh --github --minor     # minor bump, release via GitHub
./release-client.sh --gitea --no-increment  # release current version as-is
./release-client.sh --gitea --dry-run    # preview without doing anything

Creates tag client-vX.Y.Z, builds Go binaries for all platforms, and creates a Gitea/GitHub release with binaries attached.

The client embeds a MinEngineVersion (from client/MIN_ENGINE_VERSION) and will hard-fail if the connected engine is too old.

Release engine

./release-engine.sh --gitea              # patch bump, release via Gitea
./release-engine.sh --github --minor     # minor bump, release via GitHub
./release-engine.sh --gitea --no-increment  # release current version as-is
./release-engine.sh --gitea --dry-run    # preview without doing anything

Creates tag engine-vX.Y.Z, builds NVIDIA and CPU Docker images, creates a Gitea/GitHub release, and pushes images to the registry.

Checking versions

# Client
kb --version

# Engine
curl http://localhost:8000/api/v1/status | jq .version

Docker images

Images are pushed to docker.dcglab.co.uk/public/kb/engine with tags:

• engine-v2.0.6-nvidia / engine-v2.0.6-cpu — versioned
• latest-nvidia / latest-cpu — latest release

The release script authenticates to the registry using the DOCKER_DCGLAB_CI_USERNAME and DOCKER_DCGLAB_CI_PASSWORD environment variables.

Override the registry and org via environment variables:

REGISTRY=ghcr.io IMAGE_ORG=myorg ./release-engine.sh --github

Pushes are retried on transient registry failures. The engine images carry a ~5.6GB torch layer, and uploading it can fail with a 502 from the proxy in front of the registry (or a 500 on the manifest PUT that follows), which clears on a retry. Tune with:

PUSH_RETRIES=8 PUSH_RETRY_DELAY=20 ./release-engine.sh --gitea

API reference

All endpoints are under /api/v1/. Requires Authorization: Bearer <key> header when KB_API_KEY is set.

Method	Endpoint	Description
GET	/health	Health check (bypasses auth)
POST	/search	Hybrid search (JSON body)
POST	/jobs	Upload file/note for ingestion (multipart, returns 202 or 409 if duplicate)
GET	/jobs	List ingestion jobs
GET	/jobs/{id}	Job details
GET	/documents	List documents
GET	/documents/{id}	Document details with chunks
GET	/documents/{id}/file	Download original file
DELETE	/documents/{id}	Remove a document (and stored file)
PUT	/documents/{id}/tags	Add/remove tags
GET	/tags	List all tags (with descriptions)
PUT	/tags/{name}/description	Set/clear a tag context description
GET	/status	Engine status, GPU info, DB stats, rerank state
POST	/reindex	Re-embed all chunks
POST	/bulk/delete	Bulk delete documents by filter
POST	/bulk/tags	Bulk add/remove tags by filter
POST	/bulk/set-tags	Bulk replace tags by filter