steve
f1ed5b6e23
The CPU image was 11.2GB — larger than the CUDA one — and its single 6.72GB torch layer could not be pushed at all: the registry drops any blob upload taking longer than 60s, and that layer needed ~61s. Both problems came from install ordering. Dockerfile.cpu installed the project and sentence-transformers first, which resolved the default CUDA torch and pulled ~2.7GB of nvidia-* wheels, then reinstalled torch from the CPU index. Reinstalling replaces torch but leaves its transitive CUDA dependencies behind, orphaned and unused (torch reports 2.13.0+cpu, cuda.is_available() False). Installing CPU torch first, from the CPU index, means nothing ever requests a CUDA build. Everything else then resolves against the torch already present. CPU image 11.2GB -> 3.43GB largest layer 6.72GB -> 896MB push failed 5/5 at 60s -> succeeds in 8.6s Dockerfile.nvidia gets the same split. Its torch layer is unavoidably large (~2.8GiB compressed, ~52s at current throughput) so it stays the tightest thing in the stack, but splitting the application install off keeps that layer cached and the app layer small. Both now install torch before the source COPYs, so editing application code no longer invalidates the multi-GB layer. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
45 lines
1.3 KiB
Docker
45 lines
1.3 KiB
Docker
FROM nvidia/cuda:13.0.1-runtime-ubuntu24.04
|
|
|
|
ENV DEBIAN_FRONTEND=noninteractive
|
|
|
|
RUN apt-get update && apt-get install -y --no-install-recommends \
|
|
python3.12 python3.12-venv python3.12-dev python3-pip \
|
|
libpoppler-cpp-dev poppler-utils \
|
|
libgl1 libglib2.0-0 \
|
|
build-essential curl \
|
|
&& rm -rf /var/lib/apt/lists/*
|
|
|
|
COPY --from=ghcr.io/astral-sh/uv:latest /uv /usr/local/bin/uv
|
|
|
|
WORKDIR /app
|
|
|
|
# Install CUDA torch on its own, before the source COPYs.
|
|
#
|
|
# This is the bulk of the image (~2.8GiB compressed). Splitting it from the
|
|
# application install keeps it cached when only code changes, and keeps the
|
|
# app layer small. The registry drops any blob upload that takes longer than
|
|
# 60s, so this layer is deliberately the only large one.
|
|
RUN uv venv .venv && \
|
|
. .venv/bin/activate && \
|
|
UV_HTTP_TIMEOUT=600 uv pip install torch torchvision \
|
|
--index-url https://download.pytorch.org/whl/cu130
|
|
|
|
COPY pyproject.toml ./
|
|
COPY kb/ kb/
|
|
COPY main.py ./
|
|
COPY VERSION ./
|
|
|
|
RUN . .venv/bin/activate && \
|
|
uv pip install -e .
|
|
|
|
ENV PATH="/app/.venv/bin:$PATH"
|
|
ENV VIRTUAL_ENV="/app/.venv"
|
|
ENV KB_DEVICE=auto
|
|
ENV KB_INGEST_DEVICE=auto
|
|
ENV KB_DATA_DIR=/data
|
|
|
|
EXPOSE 8000
|
|
VOLUME ["/data"]
|
|
|
|
CMD ["uvicorn", "main:app", "--host", "0.0.0.0", "--port", "8000"]
|