Lesson 7: Deployment

Container image and environment promotion

Module 4 covered the minimal Dockerfile. Production adds a multi-stage build and a non-root user.

FROM python:3.12-slim AS builder

COPY --from=ghcr.io/astral-sh/uv:latest /uv /usr/local/bin/uv
WORKDIR /app

COPY pyproject.toml uv.lock ./
RUN uv sync --locked --no-dev --no-install-project

COPY src/ ./src/
RUN uv sync --locked --no-dev


FROM python:3.12-slim AS runtime

RUN useradd --create-home --uid 1000 appuser
WORKDIR /app

COPY --from=builder --chown=appuser:appuser /app/.venv /app/.venv
COPY --from=builder --chown=appuser:appuser /app/src /app/src

ENV PATH="/app/.venv/bin:$PATH" \
    PYTHONUNBUFFERED=1 \
    PYTHONDONTWRITEBYTECODE=1

USER appuser
EXPOSE 8000

CMD ["uvicorn", "recipe_extractor.main:app", "--host", "0.0.0.0", "--port", "8000"]

What each production addition buys. The multi-stage build leaves the build toolchain behind, producing a smaller image with a smaller attack surface. The non-root user limits what a compromise achieves. PYTHONUNBUFFERED makes logs appear immediately rather than sitting in a buffer, which matters when a container is killed. The dependency layer still precedes the source layer, for the caching reason from Module 4.

On worker processes. Running Uvicorn directly with one worker per container replica is a common and defensible pattern when an orchestrator handles restarts and scaling. A process manager running multiple workers per container is the alternative on plainer infrastructure. The decision is about who manages processes, and running both means two things managing the same lifecycle.

Environment promotion means one artifact moves through tiers.

build once  →  staging  →  production

The image is identical at every tier and only configuration differs. Rebuilding per environment means the thing you tested is not the thing you shipped, which defeats the purpose of testing it.

Tag images immutably by commit hash rather than by a moving tag, so a deployment names an exact artifact and a rollback names a different exact artifact.

Rollback strategy

Every deployment needs a way back, decided before it is needed.

1. Deploy the new version alongside the old
2. Wait for readiness to pass on the new instances
3. Shift traffic gradually
4. Watch error rate, p95 latency, and spend rate
5. Roll back on any breach, or complete the shift

Rollback must be faster than diagnosis. The correct response to a spike in errors after a deploy is to roll back and then investigate, not to investigate while users are affected.

What makes rollback hard, and the answers.

Database migrations that are not backwards compatible, which Module 7 answered with expand and contract: the old code must run against the new schema during the overlap.

Cache entries written by the new version, which Module 11 answered with versioned cache keys, so a rollback does not read entries the old code would misinterpret.

Queued jobs enqueued by the new version and processed by the old, which needs the payload to be readable by both.

Practise it. A rollback path that has never been executed is a plan rather than a capability.

Feature flags for model and prompt changes

Model and prompt changes are the highest-risk changes in this system, and they are the ones a code review cannot evaluate. Flags let you change them without a deployment and revert without one.

@dataclass(frozen=True)
class ModelFlags:
    generation_model: str
    prompt_version: PromptVersion
    rerank_enabled: bool
    retrieval_top_k: int
    rollout_percentage: int = 100


async def flags_for(user_id: str, deps: Dependencies) -> ModelFlags:
    """Resolve flags at request time, with stable per-user assignment."""
    base = await deps.flags.current()
    if base.rollout_percentage >= 100:
        return base
    digest = hashlib.sha256(f"{base.prompt_version}:{user_id}".encode()).hexdigest()
    in_rollout = int(digest[:8], 16) % 100 < base.rollout_percentage
    return base if in_rollout else await deps.flags.previous()

Three properties. Resolved at request time from a store, so a change takes effect in seconds rather than a deployment. Stable per user, using Module 11's assignment rule, so nobody sees two versions. And percentage-based, so a change reaches one percent of traffic before all of it.

Record the flag values on every request, alongside the cost and quality data from Modules 10 and 11. Without that, you cannot attribute a change in quality or spend to the flag that caused it, which is the entire point of rolling out gradually.

Flags accumulate and must be removed. A codebase with forty flags has two to the fortieth possible configurations, only one of which is tested. Remove a flag once its rollout completes, and treat that removal as part of the work rather than as tidying.