Add configurable allowed hosts for MCP remote access (KB_MCP_ALLOWED_HOSTS)
The MCP SDK's DNS rebinding protection rejects remote clients with 421 when the Host header isn't in the allowlist. Add KB_MCP_ALLOWED_HOSTS env var (comma-separated IPs/FQDNs) to configure additional allowed hosts while keeping localhost always permitted. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
This commit is contained in:
@@ -26,6 +26,8 @@ services:
|
||||
- KB_ENGINE_URL=http://kb-engine:8000
|
||||
- KB_API_KEY=${KB_API_KEY:-}
|
||||
- KB_MCP_API_KEY=${KB_MCP_API_KEY:-}
|
||||
# Comma-separated IPs/FQDNs allowed to connect remotely (e.g. 192.168.1.50,kb.example.com)
|
||||
- KB_MCP_ALLOWED_HOSTS=${KB_MCP_ALLOWED_HOSTS:-}
|
||||
depends_on:
|
||||
- kb-engine
|
||||
restart: unless-stopped
|
||||
|
||||
@@ -34,6 +34,8 @@ services:
|
||||
- KB_ENGINE_URL=http://kb-engine:8000
|
||||
- KB_API_KEY=${KB_API_KEY:-}
|
||||
- KB_MCP_API_KEY=${KB_MCP_API_KEY:-}
|
||||
# Comma-separated IPs/FQDNs allowed to connect remotely (e.g. 192.168.1.50,kb.example.com)
|
||||
- KB_MCP_ALLOWED_HOSTS=${KB_MCP_ALLOWED_HOSTS:-}
|
||||
depends_on:
|
||||
- kb-engine
|
||||
restart: unless-stopped
|
||||
|
||||
@@ -31,6 +31,8 @@ services:
|
||||
- KB_ENGINE_URL=http://kb-engine:8000
|
||||
- KB_API_KEY=${KB_API_KEY:-}
|
||||
- KB_MCP_API_KEY=${KB_MCP_API_KEY:-}
|
||||
# Comma-separated IPs/FQDNs allowed to connect remotely (e.g. 192.168.1.50,kb.example.com)
|
||||
- KB_MCP_ALLOWED_HOSTS=${KB_MCP_ALLOWED_HOSTS:-}
|
||||
depends_on:
|
||||
- kb-engine
|
||||
restart: unless-stopped
|
||||
|
||||
@@ -7,3 +7,11 @@ KB_ENGINE_URL = os.environ.get("KB_ENGINE_URL", "http://localhost:8000")
|
||||
KB_API_KEY = os.environ.get("KB_API_KEY", "")
|
||||
KB_MCP_API_KEY = os.environ.get("KB_MCP_API_KEY", "")
|
||||
KB_MCP_PORT = int(os.environ.get("KB_MCP_PORT", "3000"))
|
||||
KB_MCP_ALLOWED_HOSTS = os.environ.get("KB_MCP_ALLOWED_HOSTS", "")
|
||||
|
||||
|
||||
def parse_allowed_hosts() -> list[str]:
|
||||
"""Parse KB_MCP_ALLOWED_HOSTS into a list of host strings."""
|
||||
if not KB_MCP_ALLOWED_HOSTS:
|
||||
return []
|
||||
return [h.strip() for h in KB_MCP_ALLOWED_HOSTS.split(",") if h.strip()]
|
||||
|
||||
@@ -5,6 +5,7 @@ import json
|
||||
import logging
|
||||
|
||||
from mcp.server.fastmcp import FastMCP
|
||||
from mcp.server.transport_security import TransportSecuritySettings
|
||||
from starlette.applications import Starlette
|
||||
from starlette.middleware import Middleware
|
||||
from starlette.middleware.base import BaseHTTPMiddleware
|
||||
@@ -81,6 +82,23 @@ async def _ensure_exclusive_collection(doc_id: int, collection: str) -> None:
|
||||
engine.update_tags(doc_id, add=[new_tag])
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Transport security — DNS rebinding protection with configurable allowed hosts
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
_LOCALHOST_HOSTS = ["127.0.0.1:*", "localhost:*", "[::1]:*"]
|
||||
_LOCALHOST_ORIGINS = ["http://127.0.0.1:*", "http://localhost:*", "http://[::1]:*"]
|
||||
|
||||
_extra_hosts = config.parse_allowed_hosts()
|
||||
_allowed_hosts = _LOCALHOST_HOSTS + [f"{h}:*" for h in _extra_hosts]
|
||||
_allowed_origins = _LOCALHOST_ORIGINS + [f"http://{h}:*" for h in _extra_hosts]
|
||||
|
||||
_transport_security = TransportSecuritySettings(
|
||||
enable_dns_rebinding_protection=True,
|
||||
allowed_hosts=_allowed_hosts,
|
||||
allowed_origins=_allowed_origins,
|
||||
)
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# FastMCP server
|
||||
# ---------------------------------------------------------------------------
|
||||
@@ -93,6 +111,7 @@ mcp = FastMCP(
|
||||
"authentication — all requests are authenticated via the Authorization "
|
||||
"header at the HTTP transport layer."
|
||||
),
|
||||
transport_security=_transport_security,
|
||||
)
|
||||
|
||||
|
||||
|
||||
@@ -0,0 +1,2 @@
|
||||
schema: spec-driven
|
||||
created: 2026-04-04
|
||||
@@ -0,0 +1,59 @@
|
||||
## Context
|
||||
|
||||
The MCP Python SDK includes DNS rebinding protection via `TransportSecuritySettings` in `mcp.server.transport_security`. When enabled, it validates the `Host` header against an allowlist and returns 421 for unrecognised hosts.
|
||||
|
||||
`FastMCP` auto-enables this protection when `host` is `127.0.0.1`, `localhost`, or `::1`, with a default allowlist of those three values (wildcard port). The kb MCP server does not pass a `host` to `FastMCP()` and does not pass `transport_security`, so the behaviour depends on the SDK's defaults — which have changed between versions and will likely keep changing.
|
||||
|
||||
Currently the server calls `mcp.streamable_http_app()` to get a Starlette sub-app and wraps it in its own Starlette app with `BearerAuthMiddleware`. The `transport_security` settings flow through `FastMCP()` → `Settings` → `StreamableHTTPSessionManager`, so they must be set at `FastMCP` construction time.
|
||||
|
||||
When a remote client connects (e.g. `Host: 192.168.1.50:3000`), the SDK rejects the request with 421 before our auth middleware even runs.
|
||||
|
||||
## Goals / Non-Goals
|
||||
|
||||
**Goals:**
|
||||
|
||||
- Allow operators to configure additional allowed hosts via environment variable so remote clients can connect.
|
||||
- Support both IP addresses and FQDNs, with or without port.
|
||||
- Preserve DNS rebinding protection (keep it enabled, just widen the allowlist).
|
||||
- Maintain backward compatibility — unset variable means localhost-only, same as today.
|
||||
|
||||
**Non-Goals:**
|
||||
|
||||
- Disabling DNS rebinding protection entirely.
|
||||
- Configuring allowed origins separately from allowed hosts (derive origins automatically from hosts).
|
||||
- TLS termination or HTTPS — that belongs to a reverse proxy in front of the MCP container.
|
||||
|
||||
## Decisions
|
||||
|
||||
### 1. Environment variable format
|
||||
|
||||
`KB_MCP_ALLOWED_HOSTS` is a comma-separated list of hosts. Each entry is an IP address or FQDN without port and without scheme.
|
||||
|
||||
Examples: `192.168.1.50`, `kb.example.com`, `192.168.1.50,kb.example.com,10.0.0.1`
|
||||
|
||||
**Rationale:** Comma-separated is the simplest format that doesn't require quoting in Docker Compose YAML or shell environments. Ports are omitted because the wildcard-port pattern (`host:*`) covers all ports — operators shouldn't need to know the internal port.
|
||||
|
||||
**Alternative considered:** JSON array — rejected, awkward in env vars and Compose files.
|
||||
|
||||
### 2. Merge with localhost defaults
|
||||
|
||||
The parsed hosts are merged with the hardcoded localhost set (`127.0.0.1`, `localhost`, `[::1]`). Localhost is always allowed regardless of the env var value.
|
||||
|
||||
**Rationale:** Removing localhost would break local development and health checks. There's no reason to ever disallow it.
|
||||
|
||||
### 3. Auto-derive allowed origins from allowed hosts
|
||||
|
||||
For each allowed host, generate `http://<host>:*` as an allowed origin. No separate env var for origins.
|
||||
|
||||
**Rationale:** The MCP server doesn't serve HTTPS (TLS is terminated by a reverse proxy), so `http://` is always correct at the container level. If HTTPS origins are needed in future, a separate env var can be added then.
|
||||
|
||||
### 4. Pass TransportSecuritySettings explicitly to FastMCP
|
||||
|
||||
Always construct a `TransportSecuritySettings` with `enable_dns_rebinding_protection=True` and the merged allowlist, and pass it as `transport_security=` to `FastMCP()`. This makes the behaviour explicit rather than depending on SDK defaults.
|
||||
|
||||
**Rationale:** The SDK's auto-detection logic depends on the `host` parameter which we don't set, and the defaults may change between SDK versions. Being explicit removes the ambiguity.
|
||||
|
||||
## Risks / Trade-offs
|
||||
|
||||
- **Operator must know their Host header value** — If a reverse proxy rewrites the Host header, the operator needs to allowlist the rewritten value, not the original. → Mitigation: document this in Compose file comments.
|
||||
- **No HTTPS origin support** — If a client sends `Origin: https://...`, it will be rejected. → Mitigation: acceptable for now; the MCP server sits behind a proxy that terminates TLS. Can add `KB_MCP_ALLOWED_ORIGINS` later if needed.
|
||||
@@ -0,0 +1,26 @@
|
||||
## Why
|
||||
|
||||
The MCP server uses the Python MCP SDK's built-in DNS rebinding protection, which validates the `Host` header on every request. By default it only allows `localhost`, `127.0.0.1`, and `[::1]`. When clients connect remotely — using an IP address or FQDN — the server returns 421 "Invalid Host header" and the connection fails. There is no way to configure allowed hosts without changing code.
|
||||
|
||||
## What Changes
|
||||
|
||||
- Add a new environment variable `KB_MCP_ALLOWED_HOSTS` that accepts a comma-separated list of additional allowed hosts (IPs and/or FQDNs).
|
||||
- The MCP server passes these hosts (plus the existing localhost defaults) to the MCP SDK's `TransportSecuritySettings` when constructing the ASGI app.
|
||||
- Both bare hosts and wildcard-port patterns are supported (e.g. `192.168.1.50` and `kb.example.com` both work, with any port).
|
||||
- When `KB_MCP_ALLOWED_HOSTS` is empty or unset, behaviour is unchanged (localhost-only).
|
||||
|
||||
## Capabilities
|
||||
|
||||
### New Capabilities
|
||||
|
||||
_None — this is configuration of an existing component, not a new capability._
|
||||
|
||||
### Modified Capabilities
|
||||
|
||||
- `docker-deployment`: Add `KB_MCP_ALLOWED_HOSTS` to the MCP container's environment variables in Compose files and document its usage.
|
||||
|
||||
## Impact
|
||||
|
||||
- **mcp/config.py** — new `KB_MCP_ALLOWED_HOSTS` env var.
|
||||
- **mcp/server.py** — construct `TransportSecuritySettings` with merged allowed hosts/origins and pass to the FastMCP app.
|
||||
- **engine/compose.\*.yaml** — add `KB_MCP_ALLOWED_HOSTS` to the kb-mcp service environment block.
|
||||
@@ -0,0 +1,65 @@
|
||||
## ADDED Requirements
|
||||
|
||||
### Requirement: Configurable MCP allowed hosts
|
||||
|
||||
The MCP server SHALL accept a `KB_MCP_ALLOWED_HOSTS` environment variable containing a comma-separated list of additional hosts (IP addresses or FQDNs) that are permitted to connect. The server SHALL always allow `127.0.0.1`, `localhost`, and `[::1]` regardless of this setting. DNS rebinding protection SHALL always be enabled.
|
||||
|
||||
#### Scenario: Remote client connects with allowed host
|
||||
|
||||
- **WHEN** `KB_MCP_ALLOWED_HOSTS` is set to `192.168.1.50` and a client connects with `Host: 192.168.1.50:3000`
|
||||
- **THEN** the server SHALL accept the request and process it normally
|
||||
|
||||
#### Scenario: Remote client connects with disallowed host
|
||||
|
||||
- **WHEN** `KB_MCP_ALLOWED_HOSTS` is set to `192.168.1.50` and a client connects with `Host: 10.0.0.99:3000`
|
||||
- **THEN** the server SHALL return HTTP 421 "Invalid Host header"
|
||||
|
||||
#### Scenario: Multiple allowed hosts
|
||||
|
||||
- **WHEN** `KB_MCP_ALLOWED_HOSTS` is set to `192.168.1.50,kb.example.com`
|
||||
- **THEN** the server SHALL accept requests with `Host` matching either `192.168.1.50` or `kb.example.com` on any port
|
||||
|
||||
#### Scenario: Variable unset or empty
|
||||
|
||||
- **WHEN** `KB_MCP_ALLOWED_HOSTS` is unset or empty
|
||||
- **THEN** the server SHALL allow only localhost addresses (`127.0.0.1`, `localhost`, `[::1]`) with any port
|
||||
|
||||
#### Scenario: Localhost always allowed
|
||||
|
||||
- **WHEN** `KB_MCP_ALLOWED_HOSTS` is set to `192.168.1.50`
|
||||
- **THEN** the server SHALL still accept requests with `Host: localhost:3000` or `Host: 127.0.0.1:3000`
|
||||
|
||||
#### Scenario: Allowed origins derived from allowed hosts
|
||||
|
||||
- **WHEN** `KB_MCP_ALLOWED_HOSTS` includes `192.168.1.50`
|
||||
- **THEN** the server SHALL accept `Origin: http://192.168.1.50:3000` (and any port) in addition to localhost origins
|
||||
|
||||
## MODIFIED Requirements
|
||||
|
||||
### Requirement: Compose files for deployment
|
||||
|
||||
The project SHALL provide Docker Compose files for single-command deployment. Compose files SHALL use `build:` context for local development. Release notes SHALL document the versioned image tag for users pulling pre-built images.
|
||||
|
||||
#### Scenario: Start NVIDIA deployment
|
||||
- **WHEN** an admin runs `docker compose -f compose.nvidia.yaml up -d`
|
||||
- **THEN** the engine SHALL start with GPU access, bind-mount the data directory, and be reachable on the configured port
|
||||
|
||||
#### Scenario: Start ROCm deployment
|
||||
- **WHEN** an admin runs `docker compose -f compose.rocm.yaml up -d`
|
||||
- **THEN** the engine SHALL start with GPU access via ROCm device passthrough, bind-mount the data directory, and be reachable on the configured port
|
||||
|
||||
#### Scenario: Automatic restart
|
||||
- **WHEN** the engine process crashes or the host reboots
|
||||
- **THEN** Docker SHALL automatically restart the container (restart policy `unless-stopped`)
|
||||
|
||||
#### Scenario: Configure via environment
|
||||
- **WHEN** an admin sets environment variables in the compose file (KB_MODEL, KB_API_KEY, KB_DEVICE, KB_MCP_ALLOWED_HOSTS, etc.)
|
||||
- **THEN** the engine and MCP server SHALL use those values
|
||||
|
||||
#### Scenario: Pre-built image deployment
|
||||
- **WHEN** an admin wants to use a pre-built engine image without building from source
|
||||
- **THEN** the engine release notes SHALL include the exact `docker pull` command with the versioned tag (e.g. `docker.dcglab.co.uk/dcg/kb/engine:engine-v2.1.0-nvidia`)
|
||||
|
||||
#### Scenario: MCP allowed hosts in Compose
|
||||
- **WHEN** the kb-mcp service is defined in a Compose file
|
||||
- **THEN** the environment block SHALL include `KB_MCP_ALLOWED_HOSTS` with a comment explaining its format and purpose
|
||||
@@ -0,0 +1,18 @@
|
||||
## 1. Configuration
|
||||
|
||||
- [x] 1.1 Add `KB_MCP_ALLOWED_HOSTS` to `mcp/config.py` — read from env, default empty string
|
||||
- [x] 1.2 Add host-parsing helper that splits the comma-separated value, strips whitespace, and filters empty entries
|
||||
|
||||
## 2. Transport security
|
||||
|
||||
- [x] 2.1 Build `TransportSecuritySettings` in `mcp/server.py` — merge localhost defaults with parsed `KB_MCP_ALLOWED_HOSTS`, derive allowed origins from allowed hosts
|
||||
- [x] 2.2 Pass `transport_security=` to the `FastMCP()` constructor
|
||||
|
||||
## 3. Compose files
|
||||
|
||||
- [x] 3.1 Add `KB_MCP_ALLOWED_HOSTS=${KB_MCP_ALLOWED_HOSTS:-}` to the kb-mcp environment block in `compose.cpu.yaml`, `compose.nvidia.yaml`, and `compose.rocm.yaml` with a comment explaining the format
|
||||
|
||||
## 4. Verification
|
||||
|
||||
- [x] 4.1 Test: unset `KB_MCP_ALLOWED_HOSTS` — confirm localhost connects, remote host gets 421
|
||||
- [x] 4.2 Test: set `KB_MCP_ALLOWED_HOSTS` to the server IP — confirm remote host connects successfully
|
||||
Reference in New Issue
Block a user