Add Per-Plan Rate Limiting to a FastMCP Server on macOS closes its troubleshooting with “back it with Redis (shared, atomic counters) if you run several instances behind a load balancer”, and Add Observability to a FastMCP Server on macOS adds a /health endpoint “for load balancers”. Neither article puts a load balancer in front of anything. This one does, and the first thing that happens is that the server stops working.

The reason is the transport. Streamable HTTP hands the client an Mcp-Session-Id during initialize, and that id lives in a dictionary inside one Python process. Round-robin sends the next request to a different process, which looks the id up, does not find it, and returns 404. You will watch that happen, then fix it twice: once in the nginx configuration, and once by moving session state into Redis so any instance can serve any request.

What you will build

  • server.py: a FastMCP server that keeps per-session state and stamps its own instance name on every response, including error responses.
  • A docker-compose.yml running three instances of it, one nginx, and one Redis.
  • Three nginx configurations: plain round-robin, sticky routing, and the sticky attempt that looks right and is not.
  • scripts/probe.py: a client that speaks Streamable HTTP over httpx directly, so the session header and the status codes are visible on every hop.
  • A pytest suite covering the tools, the probe’s parsing, and the consistency of the nginx configs with the compose file.

Prerequisites

  • macOS 13+ with Docker Desktop running. Validated on Docker 29.6.2 and Docker Compose v5.3.1; check with docker info --format '{{.ServerVersion}}' and docker compose version. If docker info errors, the daemon is stopped: open -a Docker and re-run until it answers.
  • uv 0.5+ (install), used on the host for the probe and the tests. Validated on uv 0.11.26.
  • Xcode Command Line Tools (xcode-select --install) for make.
  • Host port 8880 free. Check with lsof -nP -iTCP:8880 -sTCP:LISTEN; empty output means it is available.
  • No local nginx or Redis install is needed. Both run as pinned containers, so nothing on your machine is reconfigured and nothing in an existing Redis is touched.

Everything below runs from one project directory.

Step 1: Scaffold and lock down hygiene

Create the workspace and its .gitignore before anything else, so no build artifact or virtualenv is ever a candidate for a commit.

Create the files

mkdir -p round-robin-mcp-nginx-redis-macos
cd round-robin-mcp-nginx-redis-macos
touch .gitignore

Add the code: .gitignore

__pycache__/
*.py[cod]
.venv/
.uv/
.pytest_cache/
.ruff_cache/
.mypy_cache/
logs/
run/
*.log
*.pid
.DS_Store

Detailed breakdown

  • Standard uv and macOS hygiene. The Redis data lives in a Docker named volume rather than in the working tree, so there is no data directory to ignore.

Step 2: Initialize the project with uv

The host virtualenv runs the probe and the tests. The containers install from the same lockfile, which is why the lock is generated now rather than later.

Create the files

uv init --name macmcp --no-workspace --python 3.12
rm -f main.py hello.py
uv add fastmcp redis
uv add --dev pytest pytest-asyncio

uv init writes description = "Add your description here". Replace that line with the one below; nothing else in the generated file needs editing.

Add the code: pyproject.toml

[project]
name = "macmcp"
version = "0.1.0"
description = "A FastMCP server run as a pool behind nginx"
readme = "README.md"
requires-python = ">=3.12"
dependencies = [
    "fastmcp>=3.4.7",
    "redis>=8.1.0",
]

[dependency-groups]
dev = [
    "pytest>=9.1.1",
    "pytest-asyncio>=1.4.0",
]

Detailed breakdown

  • fastmcp brings the server, the Context state API, and httpx (used by the probe). It also brings py-key-value-aio, whose RedisStore is the session state backend in Step 12.
  • redis is the async client RedisStore drives. Adding it explicitly keeps the dependency visible instead of relying on an extra.
  • --python 3.12 is not decoration. Without it, uv init writes whichever interpreter it defaults to into requires-python and .python-version. If that is newer than the container’s Python, the uv sync --locked in Step 4 fails inside an image that cannot satisfy the constraint. Pinning it here keeps the host virtualenv and the image on the same interpreter.

Step 3: The server

One file, run three times with a different INSTANCE_ID. Two environment variables select the behaviour under test, and nothing else differs between the broken configuration and the fixed one.

Create the file

touch server.py

Add the code: server.py

"""One FastMCP instance, built to be run several times behind a load balancer.

Every tool response names the instance that produced it, so a request landing on
the wrong peer is visible in the output instead of being a theory. Session state
goes through FastMCP's session state store, which is in-process by default and
shared when `MCP_STATE_BACKEND=redis`. Nothing else about the server changes
between the broken configuration and the fixed one.
"""

import os

from fastmcp import Context, FastMCP
from starlette.middleware import Middleware
from starlette.requests import Request
from starlette.responses import JSONResponse

INSTANCE = os.environ.get("INSTANCE_ID", "mcp?")
STATE_BACKEND = os.environ.get("MCP_STATE_BACKEND", "memory").lower()
STATELESS = os.environ.get("MCP_STATELESS", "0") == "1"
REDIS_URL = os.environ.get("REDIS_URL", "redis://redis:6379/0")
PORT = int(os.environ.get("MCP_PORT", "8000"))


def build_state_store():
    """Return the session state store, or None to use FastMCP's in-process one."""
    if STATE_BACKEND == "redis":
        from key_value.aio.stores.redis import RedisStore

        # FastMCP wraps whatever store it is given in its own adapter, which
        # pins the collection to `fastmcp_state`, so keys land under
        # `fastmcp_state::<session>:<key>` regardless of what is set here.
        return RedisStore(url=REDIS_URL)
    return None


class InstanceHeader:
    """Stamp `X-Instance` on every response, including the session-manager's 404.

    The tool results name their instance in the body, but a rejected request has
    no body worth reading. Adding the header in ASGI middleware means the
    identity survives on error responses too, which is precisely the case this
    article is about.
    """

    def __init__(self, app):
        self.app = app

    async def __call__(self, scope, receive, send):
        if scope["type"] != "http":
            await self.app(scope, receive, send)
            return

        async def send_with_header(message):
            if message["type"] == "http.response.start":
                headers = list(message.get("headers", []))
                headers.append((b"x-instance", INSTANCE.encode()))
                message = {**message, "headers": headers}
            await send(message)

        await self.app(scope, receive, send_with_header)


mcp = FastMCP("macmcp", session_state_store=build_state_store())


@mcp.tool
async def add_note(note: str, ctx: Context) -> dict:
    """Append a note to this session's list and return the list so far."""
    notes = list(await ctx.get_state("notes") or [])
    notes.append(note)
    await ctx.set_state("notes", notes)
    return {"instance": INSTANCE, "notes": notes}


@mcp.tool
async def list_notes(ctx: Context) -> dict:
    """Return the notes recorded for this session."""
    notes = list(await ctx.get_state("notes") or [])
    return {"instance": INSTANCE, "notes": notes}


@mcp.custom_route("/healthz", methods=["GET"])
async def healthz(request: Request) -> JSONResponse:
    """Report which instance answered, for nginx and for `make ps`."""
    return JSONResponse({"status": "ok", "instance": INSTANCE})


if __name__ == "__main__":
    mcp.run(
        transport="http",
        host="0.0.0.0",
        port=PORT,
        stateless_http=STATELESS,
        middleware=[Middleware(InstanceHeader)],
        # FastMCP's DNS-rebinding host check is OFF by default in 3.4.7
        # (`http_host_origin_protection` defaults to False), so `allowed_hosts`
        # alone would never be read. Turn the check on, then allow any Host:
        # the containers are addressed as `mcp1`/`mcp2`/`mcp3` on the compose
        # network and as `localhost:8880` through nginx. Narrow this list to
        # the real hostnames in production.
        host_origin_protection=True,
        allowed_hosts=["*"],
        show_banner=False,
    )

Detailed breakdown

  • add_note and list_notes are the observable state. ctx.set_state and ctx.get_state write through FastMCP’s session state store, keyed on ctx.session_id. A lost session is not an abstraction here: the notes list comes back empty, or the request never reaches a tool at all.
  • session_state_store is the whole of Fix B. Passing None leaves FastMCP on its default in-process MemoryStore, which is per-container and therefore unshareable. Passing a RedisStore puts the same state behind all three instances. The tools do not change.
  • InstanceHeader is ASGI middleware, not FastMCP middleware. The rejection this article is about is produced by the MCP session manager before any FastMCP hook runs, so a FastMCP middleware would never see it. Wrapping the ASGI app instead stamps X-Instance on every response, error responses included, which is what makes the failure legible.
  • stateless_http switches the transport between minting session ids and not tracking sessions at all. Step 12 explains why Fix B needs it.
  • host_origin_protection=True turns the host check on; allowed_hosts=["*"] then accepts any Host. Both are needed, and the order of that sentence matters: http_host_origin_protection defaults to False in FastMCP 3.4.7, so allowed_hosts on its own is never read and the guard middleware is never installed. Requests arrive here as mcp1:8000 on the compose network and as localhost:8880 through nginx, which is why the list is open. Narrow it to the real hostnames in production — with the check on, a request whose Host is not in the list is refused with 421 Misdirected Request.
  • /healthz reports the instance, which turns a plain curl loop into a read-out of the balancing policy.

Step 4: Containerize the instance

The image is pinned at every layer, including the uv binary, so a rebuild next month produces the same server.

Create the file

touch Dockerfile

Add the code: Dockerfile

# The uv binary is copied from Astral's published image rather than installed
# with pip, so the toolchain is pinned by image tag like everything else here.
FROM python:3.12-slim-bookworm

COPY --from=ghcr.io/astral-sh/uv:0.11 /uv /usr/local/bin/uv

WORKDIR /app

ENV UV_COMPILE_BYTECODE=1 \
    UV_LINK_MODE=copy \
    UV_PROJECT_ENVIRONMENT=/usr/local

# Dependencies are installed from the lockfile in their own layer, so editing
# server.py does not reinstall the world.
COPY pyproject.toml uv.lock ./
RUN uv sync --locked --no-dev --no-install-project

COPY server.py ./

EXPOSE 8000

CMD ["python", "server.py"]

Detailed breakdown

  • UV_PROJECT_ENVIRONMENT=/usr/local installs into the image’s system Python instead of a .venv, so CMD is a plain python server.py with no activation step and no uv run wrapper in the container.
  • uv sync --locked fails if uv.lock disagrees with pyproject.toml, which turns a forgotten uv add into a build error rather than a container that quietly resolves different versions than your host.
  • --no-install-project installs dependencies only. The application is a single module copied in the next layer, so the dependency layer stays cached across edits to server.py.
  • EXPOSE 8000 documents the port for the compose network. No instance publishes a host port, because reaching one directly would sidestep the load balancer this article is about.

Step 5: Compose the pool

Three instances, one nginx, one Redis. Three variables pick the configuration under test, and the Makefile in Step 8 sets them.

Create the file

touch docker-compose.yml

Add the code: docker-compose.yml

# Three FastMCP instances, one nginx in front of them, one Redis beside them.
#
# The three instances are identical apart from INSTANCE_ID, which every tool
# response echoes back. Only nginx publishes a host port: the pool is reachable
# solely through the load balancer, which is the whole point of the exercise.
#
# Three variables select the configuration under test, and the Makefile targets
# set them:
#   NGINX_CONF         roundrobin.conf (broken) | sticky-ip.conf (Fix A)
#   MCP_STATE_BACKEND  memory (per instance)    | redis (shared, Fix B)
#   MCP_STATELESS      0 (transport sessions)   | 1 (no session table, Fix B)
services:
  redis:
    image: redis:8
    command: ["redis-server", "--save", "60", "1"]
    volumes:
      - redis-data:/data
    healthcheck:
      test: ["CMD", "redis-cli", "ping"]
      interval: 5s
      timeout: 3s
      retries: 10

  mcp1: &mcp
    build: .
    environment:
      INSTANCE_ID: mcp1
      MCP_STATE_BACKEND: ${MCP_STATE_BACKEND:-memory}
      MCP_STATELESS: ${MCP_STATELESS:-0}
      REDIS_URL: redis://redis:6379/0
    depends_on:
      redis:
        condition: service_healthy

  mcp2:
    <<: *mcp
    environment:
      INSTANCE_ID: mcp2
      MCP_STATE_BACKEND: ${MCP_STATE_BACKEND:-memory}
      MCP_STATELESS: ${MCP_STATELESS:-0}
      REDIS_URL: redis://redis:6379/0

  mcp3:
    <<: *mcp
    environment:
      INSTANCE_ID: mcp3
      MCP_STATE_BACKEND: ${MCP_STATE_BACKEND:-memory}
      MCP_STATELESS: ${MCP_STATELESS:-0}
      REDIS_URL: redis://redis:6379/0

  nginx:
    image: nginx:1.31
    ports:
      - "8880:8880"
    volumes:
      - ./nginx/${NGINX_CONF:-roundrobin.conf}:/etc/nginx/conf.d/default.conf:ro
    depends_on:
      - mcp1
      - mcp2
      - mcp3

volumes:
  redis-data:

Detailed breakdown

  • Containerizing the instances rather than running them on the host is what makes the pool real. Three host processes on three ports would work, but then the compose file only describes half the system, and host.docker.internal hops in and out of the VM on every request. Here one listing is the entire topology.
  • The &mcp anchor shares build and depends_on across the three services. environment is repeated in full rather than merged, because a YAML merge key replaces a mapping wholesale: writing only INSTANCE_ID under <<: *mcp would drop the other three variables.
  • ${NGINX_CONF:-roundrobin.conf} selects the config by mounting it over nginx’s default.conf. Switching balancing policy is then a variable, not an edit, which keeps every configuration in the repository side by side.
  • Only nginx publishes 8880. The instances are reachable at mcp1:8000 on the compose network and nowhere else.
  • condition: service_healthy holds the instances until Redis answers PING, so the Redis-backed run in Step 12 does not race its own database.
  • --save 60 1 keeps Redis’s default periodic snapshot to the named volume. Session state here is disposable, and make reset deletes the volume outright.

Step 6: Plain round-robin

nginx’s default policy, and the one that breaks the transport.

Create the file

mkdir -p nginx
touch nginx/roundrobin.conf

Add the code: nginx/roundrobin.conf

# Plain round-robin: nginx's default policy. Each request goes to the next
# server in the list, which is exactly what breaks a Streamable HTTP MCP
# session — the instance that minted the `Mcp-Session-Id` is not the instance
# that receives the next request.
upstream mcp_pool {
    # Without a shared memory zone, each nginx worker keeps its own round-robin
    # counter, so with `worker_processes auto` the first request on every worker
    # goes to the first server and the rotation is invisible. The zone puts the
    # upstream's run-time state in shared memory, where all workers advance the
    # same counter.
    zone mcp_pool 64k;

    server mcp1:8000;
    server mcp2:8000;
    server mcp3:8000;
}

server {
    listen 8880;

    location / {
        proxy_pass http://mcp_pool;
        proxy_http_version 1.1;
        proxy_set_header Host $http_host;
        proxy_set_header Connection "";

        # Streamable HTTP replies are SSE. Buffering would hold the whole
        # stream until it closed, so responses have to pass through unbuffered.
        proxy_buffering off;
        proxy_read_timeout 3600s;

        # Name the upstream that handled the request, so routing is visible in
        # the response headers even when the body is an error.
        add_header X-Served-By $upstream_addr always;
    }
}

Detailed breakdown

  • zone mcp_pool 64k is not optional for this demonstration. nginx keeps upstream run-time state per worker process unless the group has a shared memory zone. The container defaults to one worker per CPU, so on an 18-core machine the first eighteen requests can all land on mcp1 and the rotation looks broken before the MCP layer gets a chance to break it.
  • proxy_set_header Host $http_host preserves the port. $host drops it, and FastMCP redirects /mcp/ to /mcp, so with $host the client is sent to http://localhost/mcp and the port is gone.
  • proxy_buffering off and a long proxy_read_timeout are what SSE needs. With buffering on, nginx accumulates the response and the client waits for a stream that only ends when the session does.
  • proxy_http_version 1.1 with an empty Connection header stops nginx from sending Connection: close upstream, which would tear down the stream after one message.
  • add_header ... always applies to error responses too. Without always, nginx omits the header on a 4xx, which is the case that matters most here.

Step 7: The probe

A client library would hide the session header, which is the thing worth watching. The probe speaks Streamable HTTP over httpx and prints one line per hop.

Create the file

mkdir -p scripts
touch scripts/__init__.py scripts/probe.py

Add the code: scripts/probe.py

"""Drive a Streamable HTTP MCP session by hand and print every hop.

The point of this article is what happens at the HTTP layer, so the probe speaks
Streamable HTTP directly with `httpx` instead of using a client library that
hides the session header. Each hop prints the status code, the instance that
answered, and the `Mcp-Session-Id` in play, which is what makes a request landing
on the wrong peer visible.

    MCP_URL=http://localhost:8880/mcp uv run python scripts/probe.py
"""

import asyncio
import json
import os
import sys
from uuid import uuid4

import httpx

URL = os.environ.get("MCP_URL", "http://localhost:8880/mcp")
PROTOCOL_VERSION = "2025-06-18"
# The client invents its own session id when the server does not mint one. A
# stateless server has no session table, so the identity of the conversation has
# to come from the caller.
CLIENT_SESSION_ID = os.environ.get("CLIENT_SESSION_ID", uuid4().hex)


def parse_body(response: httpx.Response) -> dict:
    """Return the JSON-RPC payload from either a JSON or an SSE response."""
    content_type = response.headers.get("content-type", "")
    if content_type.startswith("text/event-stream"):
        for line in response.text.splitlines():
            if line.startswith("data:"):
                return json.loads(line[len("data:") :].strip())
        return {}
    if not response.text:
        return {}
    return response.json()


def tool_payload(body: dict) -> dict | None:
    """Pull the structured tool result out of a tools/call reply."""
    result = body.get("result")
    if not isinstance(result, dict):
        return None
    if "structuredContent" in result:
        return result["structuredContent"]
    for block in result.get("content", []):
        if block.get("type") == "text":
            try:
                return json.loads(block["text"])
            except (ValueError, KeyError):
                return None
    return None


async def rpc(
    client: httpx.AsyncClient, label: str, payload: dict, session_id: str | None
) -> tuple[httpx.Response, dict]:
    """Send one JSON-RPC message and report the hop in one line."""
    headers = {
        "Content-Type": "application/json",
        "Accept": "application/json, text/event-stream",
        "MCP-Protocol-Version": PROTOCOL_VERSION,
    }
    if session_id:
        headers["Mcp-Session-Id"] = session_id
    response = await client.post(URL, json=payload, headers=headers)
    body = parse_body(response)

    served_by = response.headers.get("X-Instance", "-")
    detail = ""
    result = tool_payload(body)
    if result is not None:
        detail = f" instance={result.get('instance')} notes={result.get('notes')}"
    elif body.get("error"):
        detail = f" error={body['error'].get('message')!r}"
    print(f"{label:<25} HTTP {response.status_code}  upstream={served_by}{detail}")
    return response, body


async def main() -> int:
    async with httpx.AsyncClient(timeout=10.0) as client:
        response, _ = await rpc(
            client,
            "initialize",
            {
                "jsonrpc": "2.0",
                "id": 1,
                "method": "initialize",
                "params": {
                    "protocolVersion": PROTOCOL_VERSION,
                    "capabilities": {},
                    "clientInfo": {"name": "probe", "version": "1.0"},
                },
            },
            session_id=None,
        )
        if response.status_code != 200:
            print("initialize failed; nothing else can be tried")
            return 1

        server_session = response.headers.get("Mcp-Session-Id")
        if server_session:
            session_id = server_session
            print(f"  session minted by the server: {session_id}")
        else:
            session_id = CLIENT_SESSION_ID
            print(f"  server minted no session; client uses: {session_id}")

        await rpc(
            client,
            "notifications/initialized",
            {"jsonrpc": "2.0", "method": "notifications/initialized"},
            session_id,
        )

        response, _ = await rpc(
            client,
            "tools/call add_note",
            {
                "jsonrpc": "2.0",
                "id": 2,
                "method": "tools/call",
                "params": {"name": "add_note", "arguments": {"note": "first"}},
            },
            session_id,
        )
        if response.status_code != 200:
            print("\nThe session did not survive the hop to the next instance.")
            return 1

        response, _ = await rpc(
            client,
            "tools/call list_notes",
            {
                "jsonrpc": "2.0",
                "id": 3,
                "method": "tools/call",
                "params": {"name": "list_notes", "arguments": {}},
            },
            session_id,
        )
        if response.status_code != 200:
            print("\nThe session did not survive the hop to the next instance.")
            return 1

        print("\nEvery request was served and the session state came back.")
        return 0


if __name__ == "__main__":
    sys.exit(asyncio.run(main()))

Detailed breakdown

  • initialize is sent with no session header at all. Supplying one would be rejected outright: the session manager treats an unknown id on initialize as an expired session. The id in the response headers is what the rest of the conversation carries.
  • The else branch is not defensive coding. A stateless server returns no Mcp-Session-Id, and the conversation still needs an identity for the state store to key on. Step 12 turns that branch into the fix.
  • parse_body handles both reply shapes. Streamable HTTP answers a request with an SSE frame by default and plain JSON in some paths, so the probe reads the data: line when the content type says text/event-stream.
  • X-Instance comes from the ASGI middleware in Step 3, so the printed upstream is correct even for the 404 where there is no body to inspect.
  • tool_payload prefers structuredContent and falls back to parsing the text block, which keeps the probe working across FastMCP versions that differ in which of the two they populate.
  • The exit code is non-zero on the first failed hop, which is what makes make probe usable as a check rather than something you read by eye.

Step 8: The Makefile

Plain make prints the help screen. Each configuration under test is one target.

Create the file

touch Makefile

Add the code: Makefile

.DEFAULT_GOAL := help

MCP_URL ?= http://localhost:8880/mcp

.PHONY: help build broken sticky-naive sticky shared probe health ps logs redis-keys test down reset clean

help:  ## Show this help screen
	@grep -E '^[a-zA-Z_-]+:.*?## .*$$' $(MAKEFILE_LIST) \
		| awk 'BEGIN {FS = ":.*?## "}; {printf "  \033[36m%-12s\033[0m %s\n", $$1, $$2}'

build:  ## Build the FastMCP instance image
	docker compose build

broken:  ## Plain round-robin, per-instance state (the failure)
	NGINX_CONF=roundrobin.conf MCP_STATE_BACKEND=memory MCP_STATELESS=0 \
		docker compose up -d --build

sticky-naive:  ## Sticky on the session header — the attempt that fails
	NGINX_CONF=sticky-session-hash.conf MCP_STATE_BACKEND=memory MCP_STATELESS=0 \
		docker compose up -d --build

sticky:  ## Fix A: sticky routing on the client address
	NGINX_CONF=sticky-ip.conf MCP_STATE_BACKEND=memory MCP_STATELESS=0 \
		docker compose up -d --build

shared:  ## Fix B: plain round-robin, Redis-backed shared session state
	NGINX_CONF=roundrobin.conf MCP_STATE_BACKEND=redis MCP_STATELESS=1 \
		docker compose up -d --build

probe:  ## Drive one MCP session through the load balancer
	@MCP_URL=$(MCP_URL) uv run python scripts/probe.py

health:  ## Ask the pool which instance answers, five times
	@for i in 1 2 3 4 5; do curl -s http://localhost:8880/healthz; echo; done

ps:  ## Show the running containers
	docker compose ps

logs:  ## Tail the logs of the three instances
	docker compose logs -f mcp1 mcp2 mcp3

redis-keys:  ## List the session state keys Redis is holding
	@docker compose exec redis redis-cli --scan --pattern 'fastmcp_state*'

test:  ## Run the test suite
	uv run pytest -v

down:  ## Stop everything (keep the Redis volume)
	docker compose down

reset:  ## Stop everything and wipe the Redis volume
	docker compose down -v

clean:  ## Remove local caches
	rm -rf .pytest_cache __pycache__ tests/__pycache__ scripts/__pycache__

Detailed breakdown

  • broken, sticky-naive, sticky, and shared are the four configurations the rest of the article walks through. Each sets the three variables and calls the same docker compose up -d --build, so switching between them recreates only the containers whose environment changed.
  • probe is the measurement and health is the balancing read-out. The second is worth running first: if health does not rotate, the problem is nginx, not MCP.
  • down keeps the Redis volume, reset deletes it. Only reset guarantees a clean slate between runs of Fix B.
  • Plain make runs the help target, which greps the ## comments so a new target documents itself.

Confirm the default target before going further:

make

  help         Show this help screen
  build        Build the FastMCP instance image
  broken       Plain round-robin, per-instance state (the failure)
  sticky-naive Sticky on the session header — the attempt that fails
  sticky       Fix A: sticky routing on the client address
  shared       Fix B: plain round-robin, Redis-backed shared session state
  probe        Drive one MCP session through the load balancer
  health       Ask the pool which instance answers, five times
  ps           Show the running containers
  logs         Tail the logs of the three instances
  redis-keys   List the session state keys Redis is holding
  test         Run the test suite
  down         Stop everything (keep the Redis volume)
  reset        Stop everything and wipe the Redis volume
  clean        Remove local caches

Step 9: Watch it break

Bring up the pool with plain round-robin. The first build pulls four images (python, uv, redis, nginx) and compiles the dependency layer, so allow a couple of minutes.

make broken

Check that nginx is actually rotating before blaming MCP for anything:

make health

{"status":"ok","instance":"mcp1"}
{"status":"ok","instance":"mcp2"}
{"status":"ok","instance":"mcp3"}
{"status":"ok","instance":"mcp1"}
{"status":"ok","instance":"mcp2"}

Five requests, three instances, clean rotation. Now run one MCP session through the same load balancer:

make probe

initialize                HTTP 200  upstream=mcp3
  session minted by the server: 3204c5f6cd0c487da17705fa7f9c9b7d
notifications/initialized HTTP 404  upstream=mcp1 error='Session not found'
tools/call add_note       HTTP 404  upstream=mcp2 error='Session not found'

The session did not survive the hop to the next instance.
make: *** [probe] Error 1

initialize reached mcp3, which created a session and returned its id. The very next request rotated to mcp1, which looked the id up in its own dictionary of live sessions, did not find it, and answered 404 with Session not found. The third request rotated again to mcp2 and failed the same way. The session id is real, and it is meaningless anywhere except the process that minted it.

The exact instance names vary from run to run, and so does the session id. What is stable is the shape: one instance mints, every other instance rejects.

Nothing is wrong with the server, and nothing is wrong with nginx. Round-robin is doing exactly what it was configured to do. The mismatch is that Streamable HTTP sessions are process-local state and round-robin assumes requests are interchangeable.

Step 10: The sticky fix that does not work

The obvious repair is to route on the session header. nginx can hash any variable into a consistent upstream choice, and $http_mcp_session_id is right there.

Create the file

touch nginx/sticky-session-hash.conf

Add the code: nginx/sticky-session-hash.conf

# The obvious sticky-routing attempt — and it does not work.
#
# `hash $http_mcp_session_id consistent` does pin every request carrying the
# same session id to one instance. The problem is which one: `initialize`
# arrives with no session id at all, so it is routed before the id exists, and
# the id it comes back with is a random uuid4 from whichever instance happened
# to answer. Hashing that id afterwards picks an instance uniformly from the
# pool, which is the minting instance only about one time in three.
#
# The `map` keeps `initialize` from pinning every new session to one instance
# (an empty hash key is a constant), but it cannot fix the mismatch.
#
# Run it with `make sticky-naive` and watch the 404s continue. `sticky-ip.conf`
# is the configuration that actually holds a session together.
map $http_mcp_session_id $sticky_key {
    ""      $request_id;
    default $http_mcp_session_id;
}

upstream mcp_pool {
    zone mcp_pool 64k;
    hash $sticky_key consistent;

    server mcp1:8000;
    server mcp2:8000;
    server mcp3:8000;
}

server {
    listen 8880;

    location / {
        proxy_pass http://mcp_pool;
        proxy_http_version 1.1;
        proxy_set_header Host $http_host;
        proxy_set_header Connection "";

        proxy_buffering off;
        proxy_read_timeout 3600s;

        add_header X-Served-By $upstream_addr always;
    }
}

Detailed breakdown

  • hash <key> consistent builds a ketama ring over the upstream list, so a given key maps to a given server and adding or removing a server moves only a fraction of the keys.
  • The map handles the empty key. Every initialize arrives with no Mcp-Session-Id, and an empty string is a constant, so without the map every new session in the entire pool would be created on the same instance. $request_id is unique per request and spreads them.
  • Neither directive addresses the ordering problem, which is the reason this file exists as a worked negative result rather than a fix.

Run it:

make sticky-naive
make probe

initialize                HTTP 200  upstream=mcp3
  session minted by the server: fa24ffaa9ee543e5acdd4f2a32b48234
notifications/initialized HTTP 404  upstream=mcp1 error='Session not found'
tools/call add_note       HTTP 404  upstream=mcp1 error='Session not found'

The session did not survive the hop to the next instance.
make: *** [probe] Error 1

Look closely at what changed. The two follow-up requests both went to mcp1, where under plain round-robin they went to two different instances. The hash is working: it pins the session to one instance. That instance is not mcp3, which is where the session actually lives.

Run it again and it fails the same way, with a different session id hashing to the same wrong place:

initialize                HTTP 200  upstream=mcp3
  session minted by the server: 072856959681420dada33e1ba3614ff2
notifications/initialized HTTP 404  upstream=mcp1 error='Session not found'
tools/call add_note       HTTP 404  upstream=mcp1 error='Session not found'

The session did not survive the hop to the next instance.
make: *** [probe] Error 1

Run it enough times and one passes. This is a representative passing run out of the eighteen, not the literal third attempt:

initialize                HTTP 200  upstream=mcp3
  session minted by the server: c9b59faa5b764ec7aef6fd1f731f64c4
notifications/initialized HTTP 202  upstream=mcp3
tools/call add_note       HTTP 200  upstream=mcp3 instance=mcp3 notes=['first']
tools/call list_notes     HTTP 200  upstream=mcp3 instance=mcp3 notes=['first']

Every request was served and the session state came back.

That third run is the most dangerous result in this article. Nothing was fixed. The session id simply happened to hash to the instance that minted it, which with three instances is a coin flip weighted one in three. A configuration that works one time in three is worse than one that never works, because it reaches production.

Over eighteen runs of the probe against this config, the session id hashed back to the minting instance eight times. Eighteen runs is far too small a sample to pin the rate down — the theory is what carries the claim, and the theory says one in three, because a ketama hash of a fresh uuid4 is independent of which instance happened to mint it. The measurement is here to show the failure is intermittent rather than to establish its frequency. The map is doing its job either way: initialize landed on mcp1 five times, mcp2 seven times, and mcp3 six times, so new sessions are spread across the pool rather than piling onto one instance. The follow-up routing is simply independent of where the session lives.

The ordering is the problem. nginx has to choose an upstream for initialize before a session id exists, and the id it gets back is a random uuid4 from the instance that answered. Hashing that id afterwards selects an instance uniformly from the pool, unrelated to the one holding the session. The routing key has to be something the client already carries on its first request.

Step 11: Fix A — pin on the client address

The client’s address exists before initialize is sent, which makes it a usable routing key.

Create the file

touch nginx/sticky-ip.conf

Add the code: nginx/sticky-ip.conf

# Fix A: pin on something the client already has before `initialize` runs.
#
# The session id cannot be the routing key, because routing happens first. The
# client's address does exist on the very first request, so `ip_hash` sends
# `initialize` and every follow-up from one client to the same instance, and the
# session stays where it was minted.
#
# What this buys and what it does not:
#   * it is one directive, and it needs no cooperation from the client;
#   * every client behind one NAT or egress proxy collapses onto one instance,
#     so the balance is only as good as the spread of client addresses;
#   * the pin dies with the instance — there is no copy of the session anywhere
#     else, so a restart drops every conversation that was pinned to it;
#   * changing the server list redistributes clients and moves live sessions.
upstream mcp_pool {
    zone mcp_pool 64k;
    ip_hash;

    server mcp1:8000;
    server mcp2:8000;
    server mcp3:8000;
}

server {
    listen 8880;

    location / {
        proxy_pass http://mcp_pool;
        proxy_http_version 1.1;
        proxy_set_header Host $http_host;
        proxy_set_header Connection "";

        proxy_buffering off;
        proxy_read_timeout 3600s;

        add_header X-Served-By $upstream_addr always;
    }
}

Detailed breakdown

  • ip_hash derives the upstream from the client address, which is known at the moment initialize arrives. The whole conversation therefore stays on the instance that minted its session.
  • A client-supplied header would work equally well and balance better, if you control the client: have it generate a conversation id, send it on every request including initialize, and hash that instead. A stock MCP client does not do this, which is why the address is the practical key.
  • The caveats in the comment are the real cost. This routes a conversation to a single process and keeps no copy of it anywhere, so the failure modes are structural rather than configuration mistakes.

Run it:

make sticky
make probe

initialize                HTTP 200  upstream=mcp3
  session minted by the server: 8ec1f03e5c234499813b4be41a7ee40a
notifications/initialized HTTP 202  upstream=mcp3
tools/call add_note       HTTP 200  upstream=mcp3 instance=mcp3 notes=['first']
tools/call list_notes     HTTP 200  upstream=mcp3 instance=mcp3 notes=['first']

Every request was served and the session state came back.

Every hop stayed on mcp3. The note written by add_note is read back by list_notes, and notifications/initialized returns 202 Accepted, which is the correct answer for a JSON-RPC notification.

The cost is visible in the same terminal:

make health

{"status":"ok","instance":"mcp3"}
{"status":"ok","instance":"mcp3"}
{"status":"ok","instance":"mcp3"}
{"status":"ok","instance":"mcp3"}
{"status":"ok","instance":"mcp3"}

All five health checks went to mcp3, because they come from the same address as the probe. mcp1 and mcp2 are running and idle. On a laptop every request arrives from the Docker gateway, so the pool collapses to one instance; in production the same thing happens to every user behind one corporate NAT. Fix A holds sessions together and gives up on balance in exactly the cases where the clients are least evenly distributed.

The larger problem is that mcp3 is now a single point of failure for every session pinned to it. Stop that container and those conversations are gone, since the only copy of their state was in its memory.

Step 12: Fix B — shared session state in Redis

Move the state out of the process and any instance can serve any request. Two changes, both already written into server.py in Step 3, and both selected by environment variables:

  • MCP_STATE_BACKEND=redis passes a RedisStore as FastMCP’s session_state_store, so ctx.set_state and ctx.get_state read and write a database all three instances share.
  • MCP_STATELESS=1 turns off the transport’s session table. This is the part that is easy to miss: sharing application state does not help while the transport is still rejecting the request before a tool runs. The 404 in Step 9 comes from the MCP session manager, not from the tool, and no amount of shared state changes that. A stateless transport builds a fresh transport per request and validates no session id, so every instance accepts every request.

With the session table gone, the server no longer mints an Mcp-Session-Id, so the conversation’s identity has to come from the client. That is the else branch in the probe: it generates an id and sends it on every request, and ctx.session_id reads it back off the header to key the state store.

Run it against plain round-robin, the same configuration that failed in Step 9:

make shared
make probe

initialize                HTTP 200  upstream=mcp1
  server minted no session; client uses: 5c810ca8be8849db9e620c2250760e94
notifications/initialized HTTP 202  upstream=mcp2
tools/call add_note       HTTP 200  upstream=mcp3 instance=mcp3 notes=['first']
tools/call list_notes     HTTP 200  upstream=mcp1 instance=mcp1 notes=['first']

Every request was served and the session state came back.

Four requests, three different instances, no 404. initialize was handled by mcp1 and returned no session id. add_note ran on mcp3 and wrote a note. list_notes ran on mcp1, a different process that never saw the write, and read it back. That is the result the corpus has been pointing at.

The state is in Redis, under a key derived from the client’s session id:

make redis-keys

fastmcp_state::5c810ca8be8849db9e620c2250760e94:notes

The probe invents a new session id on every run, so each run gets its own key. Pin the id and the state outlives the client process as well as the instance:

export CLIENT_SESSION_ID=demo-session-1
make probe
make probe

initialize                HTTP 200  upstream=mcp2
  server minted no session; client uses: demo-session-1
notifications/initialized HTTP 202  upstream=mcp3
tools/call add_note       HTTP 200  upstream=mcp1 instance=mcp1 notes=['first']
tools/call list_notes     HTTP 200  upstream=mcp2 instance=mcp2 notes=['first']

Every request was served and the session state came back.

initialize                HTTP 200  upstream=mcp3
  server minted no session; client uses: demo-session-1
notifications/initialized HTTP 202  upstream=mcp1
tools/call add_note       HTTP 200  upstream=mcp2 instance=mcp2 notes=['first', 'first']
tools/call list_notes     HTTP 200  upstream=mcp3 instance=mcp3 notes=['first', 'first']

Every request was served and the session state came back.

The second run appended to a list written by a different instance in a different process. Eight requests across two client invocations touched all three instances, and every one of them saw the same conversation. Unset CLIENT_SESSION_ID before moving on, or later runs keep appending to this list.

FastMCP wraps whatever store you give it in an adapter that pins the collection name to fastmcp_state, so the key is fastmcp_state::<session-id>:<state-key> whatever default_collection you pass to RedisStore. Entries carry a 24-hour TTL, which FastMCP sets and does not currently expose as configuration, so a Redis without a persistent volume loses sessions on restart and a Redis with one expires them a day after their last write.

Step 13: What shared state does not fix

Fix B makes request/response work from any instance. It does not make the server-to-client stream work, and the difference is worth being precise about before you rely on it.

Streamable HTTP has a second channel: the client opens a GET on the same endpoint and holds it open as an SSE stream, which is how the server pushes notifications, progress, and log messages. Under Fix A that stream works, because the session exists in a process and the GET is routed to it. Go back to Fix A, open a session, and keep its id:

make sticky

SID=$(curl -s -D- -o /dev/null -X POST http://localhost:8880/mcp \
  -H 'Content-Type: application/json' \
  -H 'Accept: application/json, text/event-stream' \
  -H 'MCP-Protocol-Version: 2025-06-18' \
  -d '{"jsonrpc":"2.0","id":1,"method":"initialize","params":{"protocolVersion":"2025-06-18","capabilities":{},"clientInfo":{"name":"probe","version":"1.0"}}}' \
  | tr -d '\r' | awk '/^mcp-session-id:/ {print $2}')
echo "$SID"

151ae357ec9149fab9440755bd53d44d

Then ask for the stream, which comes back from the instance that holds it:

curl -s -D- -o /dev/null -X GET http://localhost:8880/mcp \
  -H 'Accept: text/event-stream' \
  -H 'MCP-Protocol-Version: 2025-06-18' \
  -H "Mcp-Session-Id: $SID" --max-time 4

HTTP/1.1 200 OK
Server: nginx/1.31.3
Content-Type: text/event-stream
Transfer-Encoding: chunked
Connection: keep-alive
cache-control: no-cache, no-transform
mcp-session-id: 151ae357ec9149fab9440755bd53d44d
x-instance: mcp3

--max-time 4 is what ends that command, so curl exits 28. An open stream that never closes on its own is the expected result.

Under Fix B the same request needs no session id, and is refused outright:

make shared

curl -s -D- -o /dev/null -X GET http://localhost:8880/mcp \
  -H 'Accept: text/event-stream' \
  -H 'MCP-Protocol-Version: 2025-06-18' --max-time 4

HTTP/1.1 405 Method Not Allowed
Server: nginx/1.31.3
Content-Type: text/plain; charset=utf-8
Content-Length: 18
allow: POST, DELETE
x-instance: mcp1

A stateless transport does not support the GET stream at all, so there is no server-initiated channel to route or to resume. Shared session state does not recover it, because the thing that is missing is not state: it is a live TCP connection to one process, and no key-value store makes a socket shareable. A message pushed to a client is pushed down a connection that terminates in exactly one instance.

That leaves a real trade-off rather than a strict upgrade:

  • Tools that answer requests are fully covered by Fix B. Most MCP servers are entirely this.
  • Server-initiated notifications, progress reporting, and sampling need the stream, and therefore need Fix A’s stickiness, or a fan-out layer that routes a notification to whichever instance currently holds that client’s stream.
  • Resumability (replaying missed events after a reconnect) needs an event store shared across instances and a way to route the reconnect. fastmcp.server.event_store.EventStore takes the same AsyncKeyValue backends as the state store, so EventStore(storage=RedisStore(url=...)) answers the storage half. The routing half is still stickiness.
  • Concurrent writes within one session can lose data. add_note reads the list, appends, and writes it back. Under Fix A those three steps happen inside one process; under Fix B two overlapping calls in the same session can land on two instances, both read the same list, and the second write silently discards the first note. Fix B is what makes the race reachable. If a session can issue concurrent writes, keep the accumulation on the Redis side — an RPUSH to a list is atomic where read-modify-write is not.
  • The session id becomes unauthenticated input. With stateless_http=1 the conversation is identified by whatever the client sends in Mcp-Session-Id; ctx.session_id reads the header directly. The CLIENT_SESSION_ID override in Step 12 is that property used deliberately, and it cuts both ways: any caller who guesses or replays another client’s id reads and writes that conversation’s state. The stateful path at least mints a server-side uuid4 the client cannot choose. Treat the id as a bearer token — make it unguessable, and authenticate the caller separately rather than trusting the header.

The honest summary is that Fix B is the right default for tool-serving MCP servers and is not a general replacement for sticky routing. A server that pushes to its clients needs both: shared state so any instance can answer a call, and stickiness so the stream has somewhere stable to live.

Step 14: Test what does not need containers

The routing failure needs the pool running. Everything around it can be checked in a fraction of a second, including whether the nginx configs still agree with the compose file.

Create the files

mkdir -p tests
touch tests/__init__.py tests/test_pool.py pytest.ini

Add the code: pytest.ini

[pytest]
asyncio_mode = auto
filterwarnings =
    ignore::DeprecationWarning

Add the code: tests/test_pool.py

"""Tests for the pool: the tools, the probe's parsing, and the nginx configs.

The interesting failure in this project is a routing failure, which needs the
containers running. What can be checked without Docker is everything around it:
that the tools accumulate state per session, that the probe reads a Streamable
HTTP reply correctly, and that the three nginx configurations stay in step with
the compose file. The last one is the check that catches a renamed service.
"""

import json
import re
from pathlib import Path

import pytest
from fastmcp import Client

import server
from scripts.probe import parse_body, tool_payload

PROJECT_ROOT = Path(__file__).resolve().parent.parent
NGINX_DIR = PROJECT_ROOT / "nginx"
INSTANCES = ("mcp1", "mcp2", "mcp3")


class FakeResponse:
    """The two response shapes `parse_body` has to handle."""

    def __init__(self, content_type: str, text: str):
        self.headers = {"content-type": content_type}
        self.text = text

    def json(self):
        return json.loads(self.text)


# --- the tools ---------------------------------------------------------------


async def test_add_note_accumulates_within_a_session():
    async with Client(server.mcp) as client:
        first = await client.call_tool("add_note", {"note": "alpha"})
        second = await client.call_tool("add_note", {"note": "beta"})
    assert first.structured_content["notes"] == ["alpha"]
    assert second.structured_content["notes"] == ["alpha", "beta"]


async def test_list_notes_reads_back_what_was_added():
    async with Client(server.mcp) as client:
        await client.call_tool("add_note", {"note": "gamma"})
        listed = await client.call_tool("list_notes", {})
    assert listed.structured_content["notes"] == ["gamma"]


async def test_each_session_gets_its_own_notes():
    async with Client(server.mcp) as client:
        await client.call_tool("add_note", {"note": "delta"})
    async with Client(server.mcp) as other:
        listed = await other.call_tool("list_notes", {})
    assert listed.structured_content["notes"] == []


async def test_every_response_names_its_instance():
    async with Client(server.mcp) as client:
        result = await client.call_tool("list_notes", {})
    assert result.structured_content["instance"] == server.INSTANCE


# --- the state store selection -----------------------------------------------


def test_memory_backend_uses_fastmcps_own_store(monkeypatch):
    monkeypatch.setattr(server, "STATE_BACKEND", "memory")
    assert server.build_state_store() is None


def test_redis_backend_builds_a_redis_store(monkeypatch):
    from key_value.aio.stores.redis import RedisStore

    monkeypatch.setattr(server, "STATE_BACKEND", "redis")
    assert isinstance(server.build_state_store(), RedisStore)


# --- the probe ---------------------------------------------------------------


def test_parse_body_reads_an_sse_reply():
    body = parse_body(
        FakeResponse("text/event-stream", 'event: message\ndata: {"result": {"ok": 1}}\n\n')
    )
    assert body == {"result": {"ok": 1}}


def test_parse_body_reads_a_plain_json_reply():
    body = parse_body(FakeResponse("application/json", '{"error": {"message": "nope"}}'))
    assert body == {"error": {"message": "nope"}}


def test_parse_body_tolerates_an_empty_reply():
    assert parse_body(FakeResponse("application/json", "")) == {}


def test_tool_payload_prefers_structured_content():
    body = {"result": {"structuredContent": {"instance": "mcp2", "notes": ["a"]}}}
    assert tool_payload(body) == {"instance": "mcp2", "notes": ["a"]}


def test_tool_payload_falls_back_to_the_text_block():
    body = {"result": {"content": [{"type": "text", "text": '{"instance": "mcp1"}'}]}}
    assert tool_payload(body) == {"instance": "mcp1"}


def test_tool_payload_returns_none_for_an_error_reply():
    assert tool_payload({"error": {"message": "Session not found"}}) is None


# --- the nginx configurations ------------------------------------------------


@pytest.mark.parametrize(
    "name", ["roundrobin.conf", "sticky-session-hash.conf", "sticky-ip.conf"]
)
def test_every_config_balances_over_all_three_instances(name):
    text = (NGINX_DIR / name).read_text()
    listed = re.findall(r"^\s*server\s+(mcp\d):8000;", text, re.MULTILINE)
    assert tuple(listed) == INSTANCES


@pytest.mark.parametrize(
    "name", ["roundrobin.conf", "sticky-session-hash.conf", "sticky-ip.conf"]
)
def test_every_config_shares_upstream_state_between_workers(name):
    # Without `zone` each nginx worker keeps a private round-robin counter and
    # the rotation is invisible, which is a confusing way to start the article.
    assert "zone mcp_pool" in (NGINX_DIR / name).read_text()


@pytest.mark.parametrize(
    "name", ["roundrobin.conf", "sticky-session-hash.conf", "sticky-ip.conf"]
)
def test_every_config_passes_sse_through_unbuffered(name):
    assert "proxy_buffering off" in (NGINX_DIR / name).read_text()


def test_round_robin_config_pins_nothing():
    text = (NGINX_DIR / "roundrobin.conf").read_text()
    assert "ip_hash" not in text and "hash " not in text


def test_fix_a_pins_on_the_client_address():
    assert "ip_hash;" in (NGINX_DIR / "sticky-ip.conf").read_text()


def test_configs_named_by_the_makefile_all_exist():
    makefile = (PROJECT_ROOT / "Makefile").read_text()
    for name in re.findall(r"NGINX_CONF=(\S+)", makefile):
        assert (NGINX_DIR / name).is_file(), name


def test_compose_defines_every_instance_the_configs_balance_over():
    compose = (PROJECT_ROOT / "docker-compose.yml").read_text()
    for instance in INSTANCES:
        assert re.search(rf"^  {instance}:", compose, re.MULTILINE), instance

Detailed breakdown

  • The tool tests use FastMCP’s in-memory client, which drives the real server object without HTTP. They pin the behaviour the routing tests depend on: notes accumulate within a session and do not leak between sessions.
  • The state-store tests cover the one branch that decides Fix B. monkeypatch.setattr on the module constant avoids re-importing the module to change an environment variable.
  • The probe tests exercise both reply shapes and the error path, so a change in how FastMCP frames results shows up as a test failure rather than as a probe that silently prints nothing.
  • The nginx tests are the ones that earn their keep over time. Renaming a compose service or adding a fourth instance without updating all three configs is a mistake that no unit test would normally catch, and it presents as a confusing routing failure rather than an error.
  • scripts/__init__.py and tests/__init__.py make both directories importable, which is what lets the test module import scripts.probe.
  • pytest.ini is what makes the async tests run at all. asyncio_mode = auto lets pytest-asyncio collect bare async def test_* functions without a decorator on each one; without it every async test in this file errors instead of running. filterwarnings quiets deprecations raised from inside the dependency tree, which are not actionable here.

Run them:

uv run pytest -q

.........................                                                [100%]
25 passed in 0.32s

The 25 is the number to check; the elapsed time varies from machine to machine and run to run. Nothing here starts a container, so the suite stays fast whether or not the pool is up.

Step 15: Tear down

make reset

This stops all five containers, removes the network, and deletes the Redis volume. Use make down instead to keep the volume between runs.

Troubleshooting

  • Every request goes to the same instance and you did not ask for that. The upstream is missing zone mcp_pool 64k;. nginx keeps round-robin state per worker, and with one worker per CPU the first request on each worker picks the first server. Confirm with make health, which should rotate.
  • initialize returns 307. The endpoint is /mcp, not /mcp/, and FastMCP redirects the trailing-slash form. If the redirect also lost the port, nginx is sending Host $host; use $http_host.
  • 400 Bad Request: Missing session ID rather than 404. The request carried no Mcp-Session-Id header at all, so the instance read it as the start of a new conversation, minted a fresh session, and then refused the message for not carrying that session’s id. Any instance in the pool answers this way, so it tells you nothing about routing — it is a client bug. Worth fixing promptly: every such request leaves a new server-side transport behind.
  • 404 Session not found under Fix B. The instances are still running with MCP_STATELESS=0. Confirm with docker compose exec mcp1 env | grep MCP_, and note that docker compose up -d only recreates a container when its environment changes, so switching targets is what applies the change.
  • make shared works but state does not carry between runs. That is the default: the probe generates a fresh session id per run, so a new key per run is expected. Set CLIENT_SESSION_ID to reuse one. If make redis-keys shows no keys at all, the instances are still on the memory backend.
  • The build fails pulling ghcr.io/astral-sh/uv. A transient registry timeout. Re-run make broken; the layer caches once it succeeds.
  • docker info errors. Docker Desktop is not running. open -a Docker, wait for it to answer, then re-run.

Recap

  • Round-robin breaks Streamable HTTP because the Mcp-Session-Id is created by one process and stored in its memory. Every other instance answers 404 Session not found.
  • Hashing on the session header does not fix it. Routing happens before the session id exists, and hashing the id afterwards picks an instance unrelated to the one that minted it. The negative result is worth keeping in the repo.
  • Fix A pins on the client address. One directive, no client changes, and two real costs: clients behind one NAT collapse onto one instance, and losing that instance loses every session on it.
  • Fix B moves session state to Redis and drops the transport’s session table. Plain round-robin then works, with four consecutive requests served by three different instances and the state intact.
  • Fix B does not restore the server-to-client SSE stream. A stateless transport answers 405 on the GET, and a shared store cannot share a socket. Servers that push to clients still need stickiness.

Next improvements