2 Commits

Author SHA1 Message Date
steve 0c124c4ab7 Bump engine version to 3.0.1 2026-04-04 12:42:32 +01:00
steve da5b8435bc Add configurable allowed hosts for MCP remote access (KB_MCP_ALLOWED_HOSTS)
The MCP SDK's DNS rebinding protection rejects remote clients with 421
when the Host header isn't in the allowlist. Add KB_MCP_ALLOWED_HOSTS env
var (comma-separated IPs/FQDNs) to configure additional allowed hosts
while keeping localhost always permitted.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-04 12:39:43 +01:00
11 changed files with 204 additions and 1 deletions
+1 -1
View File
@@ -1 +1 @@
3.0.0
3.0.1
+2
View File
@@ -26,6 +26,8 @@ services:
- KB_ENGINE_URL=http://kb-engine:8000
- KB_API_KEY=${KB_API_KEY:-}
- KB_MCP_API_KEY=${KB_MCP_API_KEY:-}
# Comma-separated IPs/FQDNs allowed to connect remotely (e.g. 192.168.1.50,kb.example.com)
- KB_MCP_ALLOWED_HOSTS=${KB_MCP_ALLOWED_HOSTS:-}
depends_on:
- kb-engine
restart: unless-stopped
+2
View File
@@ -34,6 +34,8 @@ services:
- KB_ENGINE_URL=http://kb-engine:8000
- KB_API_KEY=${KB_API_KEY:-}
- KB_MCP_API_KEY=${KB_MCP_API_KEY:-}
# Comma-separated IPs/FQDNs allowed to connect remotely (e.g. 192.168.1.50,kb.example.com)
- KB_MCP_ALLOWED_HOSTS=${KB_MCP_ALLOWED_HOSTS:-}
depends_on:
- kb-engine
restart: unless-stopped
+2
View File
@@ -31,6 +31,8 @@ services:
- KB_ENGINE_URL=http://kb-engine:8000
- KB_API_KEY=${KB_API_KEY:-}
- KB_MCP_API_KEY=${KB_MCP_API_KEY:-}
# Comma-separated IPs/FQDNs allowed to connect remotely (e.g. 192.168.1.50,kb.example.com)
- KB_MCP_ALLOWED_HOSTS=${KB_MCP_ALLOWED_HOSTS:-}
depends_on:
- kb-engine
restart: unless-stopped
+8
View File
@@ -7,3 +7,11 @@ KB_ENGINE_URL = os.environ.get("KB_ENGINE_URL", "http://localhost:8000")
KB_API_KEY = os.environ.get("KB_API_KEY", "")
KB_MCP_API_KEY = os.environ.get("KB_MCP_API_KEY", "")
KB_MCP_PORT = int(os.environ.get("KB_MCP_PORT", "3000"))
KB_MCP_ALLOWED_HOSTS = os.environ.get("KB_MCP_ALLOWED_HOSTS", "")
def parse_allowed_hosts() -> list[str]:
"""Parse KB_MCP_ALLOWED_HOSTS into a list of host strings."""
if not KB_MCP_ALLOWED_HOSTS:
return []
return [h.strip() for h in KB_MCP_ALLOWED_HOSTS.split(",") if h.strip()]
+19
View File
@@ -5,6 +5,7 @@ import json
import logging
from mcp.server.fastmcp import FastMCP
from mcp.server.transport_security import TransportSecuritySettings
from starlette.applications import Starlette
from starlette.middleware import Middleware
from starlette.middleware.base import BaseHTTPMiddleware
@@ -81,6 +82,23 @@ async def _ensure_exclusive_collection(doc_id: int, collection: str) -> None:
engine.update_tags(doc_id, add=[new_tag])
# ---------------------------------------------------------------------------
# Transport security — DNS rebinding protection with configurable allowed hosts
# ---------------------------------------------------------------------------
_LOCALHOST_HOSTS = ["127.0.0.1:*", "localhost:*", "[::1]:*"]
_LOCALHOST_ORIGINS = ["http://127.0.0.1:*", "http://localhost:*", "http://[::1]:*"]
_extra_hosts = config.parse_allowed_hosts()
_allowed_hosts = _LOCALHOST_HOSTS + [f"{h}:*" for h in _extra_hosts]
_allowed_origins = _LOCALHOST_ORIGINS + [f"http://{h}:*" for h in _extra_hosts]
_transport_security = TransportSecuritySettings(
enable_dns_rebinding_protection=True,
allowed_hosts=_allowed_hosts,
allowed_origins=_allowed_origins,
)
# ---------------------------------------------------------------------------
# FastMCP server
# ---------------------------------------------------------------------------
@@ -93,6 +111,7 @@ mcp = FastMCP(
"authentication — all requests are authenticated via the Authorization "
"header at the HTTP transport layer."
),
transport_security=_transport_security,
)
@@ -0,0 +1,2 @@
schema: spec-driven
created: 2026-04-04
@@ -0,0 +1,59 @@
## Context
The MCP Python SDK includes DNS rebinding protection via `TransportSecuritySettings` in `mcp.server.transport_security`. When enabled, it validates the `Host` header against an allowlist and returns 421 for unrecognised hosts.
`FastMCP` auto-enables this protection when `host` is `127.0.0.1`, `localhost`, or `::1`, with a default allowlist of those three values (wildcard port). The kb MCP server does not pass a `host` to `FastMCP()` and does not pass `transport_security`, so the behaviour depends on the SDK's defaults — which have changed between versions and will likely keep changing.
Currently the server calls `mcp.streamable_http_app()` to get a Starlette sub-app and wraps it in its own Starlette app with `BearerAuthMiddleware`. The `transport_security` settings flow through `FastMCP()``Settings``StreamableHTTPSessionManager`, so they must be set at `FastMCP` construction time.
When a remote client connects (e.g. `Host: 192.168.1.50:3000`), the SDK rejects the request with 421 before our auth middleware even runs.
## Goals / Non-Goals
**Goals:**
- Allow operators to configure additional allowed hosts via environment variable so remote clients can connect.
- Support both IP addresses and FQDNs, with or without port.
- Preserve DNS rebinding protection (keep it enabled, just widen the allowlist).
- Maintain backward compatibility — unset variable means localhost-only, same as today.
**Non-Goals:**
- Disabling DNS rebinding protection entirely.
- Configuring allowed origins separately from allowed hosts (derive origins automatically from hosts).
- TLS termination or HTTPS — that belongs to a reverse proxy in front of the MCP container.
## Decisions
### 1. Environment variable format
`KB_MCP_ALLOWED_HOSTS` is a comma-separated list of hosts. Each entry is an IP address or FQDN without port and without scheme.
Examples: `192.168.1.50`, `kb.example.com`, `192.168.1.50,kb.example.com,10.0.0.1`
**Rationale:** Comma-separated is the simplest format that doesn't require quoting in Docker Compose YAML or shell environments. Ports are omitted because the wildcard-port pattern (`host:*`) covers all ports — operators shouldn't need to know the internal port.
**Alternative considered:** JSON array — rejected, awkward in env vars and Compose files.
### 2. Merge with localhost defaults
The parsed hosts are merged with the hardcoded localhost set (`127.0.0.1`, `localhost`, `[::1]`). Localhost is always allowed regardless of the env var value.
**Rationale:** Removing localhost would break local development and health checks. There's no reason to ever disallow it.
### 3. Auto-derive allowed origins from allowed hosts
For each allowed host, generate `http://<host>:*` as an allowed origin. No separate env var for origins.
**Rationale:** The MCP server doesn't serve HTTPS (TLS is terminated by a reverse proxy), so `http://` is always correct at the container level. If HTTPS origins are needed in future, a separate env var can be added then.
### 4. Pass TransportSecuritySettings explicitly to FastMCP
Always construct a `TransportSecuritySettings` with `enable_dns_rebinding_protection=True` and the merged allowlist, and pass it as `transport_security=` to `FastMCP()`. This makes the behaviour explicit rather than depending on SDK defaults.
**Rationale:** The SDK's auto-detection logic depends on the `host` parameter which we don't set, and the defaults may change between SDK versions. Being explicit removes the ambiguity.
## Risks / Trade-offs
- **Operator must know their Host header value** — If a reverse proxy rewrites the Host header, the operator needs to allowlist the rewritten value, not the original. → Mitigation: document this in Compose file comments.
- **No HTTPS origin support** — If a client sends `Origin: https://...`, it will be rejected. → Mitigation: acceptable for now; the MCP server sits behind a proxy that terminates TLS. Can add `KB_MCP_ALLOWED_ORIGINS` later if needed.
@@ -0,0 +1,26 @@
## Why
The MCP server uses the Python MCP SDK's built-in DNS rebinding protection, which validates the `Host` header on every request. By default it only allows `localhost`, `127.0.0.1`, and `[::1]`. When clients connect remotely — using an IP address or FQDN — the server returns 421 "Invalid Host header" and the connection fails. There is no way to configure allowed hosts without changing code.
## What Changes
- Add a new environment variable `KB_MCP_ALLOWED_HOSTS` that accepts a comma-separated list of additional allowed hosts (IPs and/or FQDNs).
- The MCP server passes these hosts (plus the existing localhost defaults) to the MCP SDK's `TransportSecuritySettings` when constructing the ASGI app.
- Both bare hosts and wildcard-port patterns are supported (e.g. `192.168.1.50` and `kb.example.com` both work, with any port).
- When `KB_MCP_ALLOWED_HOSTS` is empty or unset, behaviour is unchanged (localhost-only).
## Capabilities
### New Capabilities
_None — this is configuration of an existing component, not a new capability._
### Modified Capabilities
- `docker-deployment`: Add `KB_MCP_ALLOWED_HOSTS` to the MCP container's environment variables in Compose files and document its usage.
## Impact
- **mcp/config.py** — new `KB_MCP_ALLOWED_HOSTS` env var.
- **mcp/server.py** — construct `TransportSecuritySettings` with merged allowed hosts/origins and pass to the FastMCP app.
- **engine/compose.\*.yaml** — add `KB_MCP_ALLOWED_HOSTS` to the kb-mcp service environment block.
@@ -0,0 +1,65 @@
## ADDED Requirements
### Requirement: Configurable MCP allowed hosts
The MCP server SHALL accept a `KB_MCP_ALLOWED_HOSTS` environment variable containing a comma-separated list of additional hosts (IP addresses or FQDNs) that are permitted to connect. The server SHALL always allow `127.0.0.1`, `localhost`, and `[::1]` regardless of this setting. DNS rebinding protection SHALL always be enabled.
#### Scenario: Remote client connects with allowed host
- **WHEN** `KB_MCP_ALLOWED_HOSTS` is set to `192.168.1.50` and a client connects with `Host: 192.168.1.50:3000`
- **THEN** the server SHALL accept the request and process it normally
#### Scenario: Remote client connects with disallowed host
- **WHEN** `KB_MCP_ALLOWED_HOSTS` is set to `192.168.1.50` and a client connects with `Host: 10.0.0.99:3000`
- **THEN** the server SHALL return HTTP 421 "Invalid Host header"
#### Scenario: Multiple allowed hosts
- **WHEN** `KB_MCP_ALLOWED_HOSTS` is set to `192.168.1.50,kb.example.com`
- **THEN** the server SHALL accept requests with `Host` matching either `192.168.1.50` or `kb.example.com` on any port
#### Scenario: Variable unset or empty
- **WHEN** `KB_MCP_ALLOWED_HOSTS` is unset or empty
- **THEN** the server SHALL allow only localhost addresses (`127.0.0.1`, `localhost`, `[::1]`) with any port
#### Scenario: Localhost always allowed
- **WHEN** `KB_MCP_ALLOWED_HOSTS` is set to `192.168.1.50`
- **THEN** the server SHALL still accept requests with `Host: localhost:3000` or `Host: 127.0.0.1:3000`
#### Scenario: Allowed origins derived from allowed hosts
- **WHEN** `KB_MCP_ALLOWED_HOSTS` includes `192.168.1.50`
- **THEN** the server SHALL accept `Origin: http://192.168.1.50:3000` (and any port) in addition to localhost origins
## MODIFIED Requirements
### Requirement: Compose files for deployment
The project SHALL provide Docker Compose files for single-command deployment. Compose files SHALL use `build:` context for local development. Release notes SHALL document the versioned image tag for users pulling pre-built images.
#### Scenario: Start NVIDIA deployment
- **WHEN** an admin runs `docker compose -f compose.nvidia.yaml up -d`
- **THEN** the engine SHALL start with GPU access, bind-mount the data directory, and be reachable on the configured port
#### Scenario: Start ROCm deployment
- **WHEN** an admin runs `docker compose -f compose.rocm.yaml up -d`
- **THEN** the engine SHALL start with GPU access via ROCm device passthrough, bind-mount the data directory, and be reachable on the configured port
#### Scenario: Automatic restart
- **WHEN** the engine process crashes or the host reboots
- **THEN** Docker SHALL automatically restart the container (restart policy `unless-stopped`)
#### Scenario: Configure via environment
- **WHEN** an admin sets environment variables in the compose file (KB_MODEL, KB_API_KEY, KB_DEVICE, KB_MCP_ALLOWED_HOSTS, etc.)
- **THEN** the engine and MCP server SHALL use those values
#### Scenario: Pre-built image deployment
- **WHEN** an admin wants to use a pre-built engine image without building from source
- **THEN** the engine release notes SHALL include the exact `docker pull` command with the versioned tag (e.g. `docker.dcglab.co.uk/dcg/kb/engine:engine-v2.1.0-nvidia`)
#### Scenario: MCP allowed hosts in Compose
- **WHEN** the kb-mcp service is defined in a Compose file
- **THEN** the environment block SHALL include `KB_MCP_ALLOWED_HOSTS` with a comment explaining its format and purpose
@@ -0,0 +1,18 @@
## 1. Configuration
- [x] 1.1 Add `KB_MCP_ALLOWED_HOSTS` to `mcp/config.py` — read from env, default empty string
- [x] 1.2 Add host-parsing helper that splits the comma-separated value, strips whitespace, and filters empty entries
## 2. Transport security
- [x] 2.1 Build `TransportSecuritySettings` in `mcp/server.py` — merge localhost defaults with parsed `KB_MCP_ALLOWED_HOSTS`, derive allowed origins from allowed hosts
- [x] 2.2 Pass `transport_security=` to the `FastMCP()` constructor
## 3. Compose files
- [x] 3.1 Add `KB_MCP_ALLOWED_HOSTS=${KB_MCP_ALLOWED_HOSTS:-}` to the kb-mcp environment block in `compose.cpu.yaml`, `compose.nvidia.yaml`, and `compose.rocm.yaml` with a comment explaining the format
## 4. Verification
- [x] 4.1 Test: unset `KB_MCP_ALLOWED_HOSTS` — confirm localhost connects, remote host gets 421
- [x] 4.2 Test: set `KB_MCP_ALLOWED_HOSTS` to the server IP — confirm remote host connects successfully