Hand-written JSON fixtures cover the two or three cases you thought of while writing them. A generator covers hundreds, and it keeps covering them after the schema changes. This article builds two generators for the same order-intake API: a naive one that invents every field from random primitives, and a second one that samples a JSON seed database so every request refers to a product and a customer the API actually knows about.

The naive generator is not a throwaway. It produces payloads that pass schema validation and fail everything downstream, which is exactly what you need to prove the API rejects garbage. The seeded generator produces payloads that should succeed, which is what you need to prove the happy path survives a refactor. Most useful test suites want both.

Everything runs locally with uv, FastAPI, and pytest. No external service, no LLM, no network.

Prerequisites

  • macOS (validated on macOS 26.5.2; any Unix-like shell works)
  • Python 3.12 or later
  • uv 0.11 or later (curl -LsSf https://astral.sh/uv/install.sh | sh)
  • curl (pre-installed on macOS)
  • make (pre-installed on macOS)
  • Familiarity with JSON and HTTP status codes

If you have not used FastAPI before, Expose an Existing FastAPI App as MCP on macOS covers the framework basics from the other direction.

Step 1: Scaffold the project

Create the file

uv init synth-api
cd synth-api
rm main.py README.md
uv add fastapi "uvicorn[standard]" httpx
uv add --dev pytest
mkdir -p api synth data tests tmp
touch api/__init__.py synth/__init__.py
touch .gitignore

Add the code: .gitignore

__pycache__/
*.pyc
.venv/
.pytest_cache/
*.egg-info/
dist/
build/
.DS_Store
*.log
tmp/

Detailed breakdown

  • uv init synth-api writes a pyproject.toml, a .python-version, and a placeholder main.py. The placeholder and the generated README.md are removed because this project uses its own package layout.
  • fastapi and uvicorn[standard] provide the API under test. httpx is the HTTP client the batch runner uses, and FastAPI’s TestClient is built on it, so one dependency covers both.
  • pytest goes in the dev group, so it is installed for local runs and CI but is not a runtime dependency.
  • api/ holds the service, synth/ holds the generators, data/ holds the seed database, and tmp/ is scratch space for payloads piped into curl.
  • tmp/ is ignored, but data/ deliberately is not: the seed database is source material that the API and the tests both read, so it belongs in version control.
  • __pycache__/, *.pyc, .venv/, and .pytest_cache/ cover the artifacts uv and pytest generate on every run.

Step 2: Create the JSON seed database

The seed database stands in for the tables a real service would query. Keeping it as plain JSON means it is readable in a diff, editable without a migration, and loadable from a test without a container.

It serves two roles in this project. The API treats it as reference data and rejects any request that contradicts it. The Example 2 generator samples from the same rows, so its output is consistent with the API’s view of the world by construction.

Create the file

touch data/catalog.json
touch data/customers.json

Add the code: data/catalog.json

[
  { "sku": "SKU-4021", "name": "Trail Runner 12", "category": "footwear", "price_cents": 12900, "popularity": 34, "in_stock": true },
  { "sku": "SKU-4022", "name": "Trail Runner 12 Wide", "category": "footwear", "price_cents": 12900, "popularity": 9, "in_stock": true },
  { "sku": "SKU-5510", "name": "Merino Crew Sock", "category": "apparel", "price_cents": 1850, "popularity": 61, "in_stock": true },
  { "sku": "SKU-5511", "name": "Merino Base Layer", "category": "apparel", "price_cents": 7400, "popularity": 22, "in_stock": true },
  { "sku": "SKU-6300", "name": "Alloy Water Bottle", "category": "gear", "price_cents": 2450, "popularity": 47, "in_stock": true },
  { "sku": "SKU-6301", "name": "Insulated Flask 1L", "category": "gear", "price_cents": 3900, "popularity": 15, "in_stock": true },
  { "sku": "SKU-7100", "name": "Headlamp 400", "category": "gear", "price_cents": 5600, "popularity": 28, "in_stock": true },
  { "sku": "SKU-7101", "name": "Headlamp 400 Rechargeable", "category": "gear", "price_cents": 8200, "popularity": 12, "in_stock": false }
]

Detailed breakdown

  • sku is the join key. Both the API and the seeded generator index the catalog by it.
  • price_cents is an integer, so every total the generator computes and every total the API recomputes is exact. Floating-point currency would introduce rounding disagreements that look like generator bugs.
  • popularity is a relative weight, not a count. The seeded generator uses it to pick common products more often than rare ones, which makes a batch of synthetic traffic resemble production traffic instead of a uniform sweep.
  • in_stock: false on SKU-7101 gives the generator a row it must skip and gives the API a rule to enforce. One deliberately unsellable product is enough to prove the filter works.
  • SKU-4021 and SKU-4022 share a price and a category on purpose. Near-duplicate rows catch code that assumes a price uniquely identifies a product.

Add the code: data/customers.json

[
  { "customer_id": "CUST-0001", "name": "Ada Rowan", "tier": "gold", "channels": ["web", "mobile"] },
  { "customer_id": "CUST-0002", "name": "Bo Whitfield", "tier": "standard", "channels": ["web"] },
  { "customer_id": "CUST-0003", "name": "Cleo Nakamura", "tier": "standard", "channels": ["mobile"] },
  { "customer_id": "CUST-0004", "name": "Dev Okonkwo", "tier": "gold", "channels": ["web", "partner"] },
  { "customer_id": "CUST-0005", "name": "Elin Vasquez", "tier": "partner", "channels": ["partner"] }
]

Detailed breakdown

  • customer_id uses a CUST-#### format. Remember that shape: Example 1 guesses a different one, and the guess is what makes its requests fail.
  • channels is a per-customer allowlist rather than a global enum. It encodes a cross-field constraint, so a request is only valid if its channel is legal for that customer. A generator that picks a channel at random violates the rule roughly half the time; one that reads this list never does.
  • tier is unused by the API here. Seed data usually carries fields the current endpoint ignores, and leaving one in place keeps the example honest.
  • Five customers is enough to exercise the constraint without making the file tedious to read.

Step 3: Build the API under test

Create the file

touch api/main.py

Add the code: api/main.py

"""Order intake API: the target under test for synthetic requests."""

import json
from pathlib import Path
from typing import Literal

from fastapi import FastAPI, HTTPException
from pydantic import BaseModel, Field

DATA_DIR = Path(__file__).resolve().parent.parent / "data"


def _load(name: str) -> list[dict]:
    return json.loads((DATA_DIR / name).read_text(encoding="utf-8"))


CATALOG = {row["sku"]: row for row in _load("catalog.json")}
CUSTOMERS = {row["customer_id"]: row for row in _load("customers.json")}


class OrderLine(BaseModel):
    sku: str
    quantity: int = Field(ge=1, le=20)
    unit_price_cents: int = Field(ge=1)


class OrderRequest(BaseModel):
    request_id: str
    customer_id: str
    channel: Literal["web", "mobile", "partner"]
    currency: Literal["USD"]
    lines: list[OrderLine] = Field(min_length=1, max_length=5)
    total_cents: int = Field(ge=1)


app = FastAPI(title="Order Intake")


@app.get("/health")
def health() -> dict:
    return {"status": "ok", "skus": len(CATALOG), "customers": len(CUSTOMERS)}


@app.post("/orders", status_code=201)
def create_order(order: OrderRequest) -> dict:
    errors: list[str] = []

    customer = CUSTOMERS.get(order.customer_id)
    if customer is None:
        errors.append(f"unknown customer {order.customer_id}")
    elif order.channel not in customer["channels"]:
        errors.append(f"channel {order.channel} not enabled for {order.customer_id}")

    expected_total = 0
    for line in order.lines:
        product = CATALOG.get(line.sku)
        if product is None:
            errors.append(f"unknown sku {line.sku}")
            continue
        if not product["in_stock"]:
            errors.append(f"sku {line.sku} is out of stock")
        if product["price_cents"] != line.unit_price_cents:
            errors.append(
                f"price mismatch for {line.sku}: "
                f"sent {line.unit_price_cents}, catalog {product['price_cents']}"
            )
        expected_total += product["price_cents"] * line.quantity

    if not errors and expected_total != order.total_cents:
        errors.append(f"total mismatch: sent {order.total_cents}, expected {expected_total}")

    if errors:
        raise HTTPException(status_code=422, detail=errors)

    return {
        "order_id": f"ORD-{order.request_id[:8]}",
        "customer_id": order.customer_id,
        "lines": len(order.lines),
        "total_cents": order.total_cents,
    }

Detailed breakdown

  • DATA_DIR is resolved from __file__, so the API finds data/ regardless of the directory the server or pytest was started from.
  • CATALOG and CUSTOMERS are dicts built once at import, which turns every lookup in the handler into a hash lookup. Loading at import also means a malformed seed file fails loudly at startup instead of on the first request.
  • OrderLine and OrderRequest are the schema layer. Field(ge=1, le=20), Literal[...], and Field(min_length=1, max_length=5) reject anything structurally wrong before the handler body runs, and FastAPI turns those failures into a 422 with a machine-readable error list.
  • create_order is the semantic layer, and this split is the point of the whole article. Types and ranges are cheap to satisfy at random; referential integrity (a real customer, a real SKU, the catalog price, a total that adds up) is not.
  • Errors accumulate into a list rather than raising on the first one, so a single response tells you everything wrong with a payload. When you are debugging a generator, one round trip beats four.
  • continue after an unknown SKU skips the price and total checks for that line. Without it, an unknown SKU would also report a bogus total mismatch, and the real problem would be buried.
  • The total check only runs when nothing else failed. A total computed from products that do not exist is not a meaningful comparison.
  • order_id slices the first 8 characters of request_id, which gives the response a value you can trace back to the request that produced it.

Start the server

uv run uvicorn api.main:app --reload

Leave it running and open a second terminal for everything below.

curl -s http://127.0.0.1:8000/health

{"status":"ok","skus":8,"customers":5}

The counts confirm both seed files loaded. Now send one hand-written order that satisfies every rule:

curl -s -X POST http://127.0.0.1:8000/orders \
  -H 'Content-Type: application/json' \
  -d '{"request_id":"11111111-2222-3333-4444-555555555555","customer_id":"CUST-0002","channel":"web","currency":"USD","lines":[{"sku":"SKU-5510","quantity":2,"unit_price_cents":1850}],"total_cents":3700}'

{"order_id":"ORD-11111111","customer_id":"CUST-0002","lines":1,"total_cents":3700}

That single fixture took a minute to write, and it covers one path. Writing two hundred by hand is how test suites end up with three fixtures and a comment apologizing for it.

The two rejection styles are worth seeing side by side. A schema violation is caught by Pydantic before the handler runs:

curl -s -X POST http://127.0.0.1:8000/orders \
  -H 'Content-Type: application/json' \
  -d '{"request_id":"abc","customer_id":"CUST-0001","channel":"fax","currency":"USD","lines":[],"total_cents":100}' \
  | python3 -m json.tool

{
    "detail": [
        {
            "type": "literal_error",
            "loc": [
                "body",
                "channel"
            ],
            "msg": "Input should be 'web', 'mobile' or 'partner'",
            "input": "fax",
            "ctx": {
                "expected": "'web', 'mobile' or 'partner'"
            }
        },
        {
            "type": "too_short",
            "loc": [
                "body",
                "lines"
            ],
            "msg": "List should have at least 1 item after validation, not 0",
            "input": [],
            "ctx": {
                "field_type": "List",
                "min_length": 1,
                "actual_length": 0
            }
        }
    ]
}

A semantic violation gets through the schema and is caught by the handler, which reports plain strings:

curl -s -X POST http://127.0.0.1:8000/orders \
  -H 'Content-Type: application/json' \
  -d '{"request_id":"11111111-2222-3333-4444-555555555555","customer_id":"CUST-0003","channel":"web","currency":"USD","lines":[{"sku":"SKU-7101","quantity":1,"unit_price_cents":8200}],"total_cents":8200}'

{"detail":["channel web not enabled for CUST-0003","sku SKU-7101 is out of stock"]}

Both return 422. Only the second one requires a generator that knows something about the data.

Step 4: Example 1 — generate from random primitives

The first generator knows the schema and nothing else. Every value is invented: random digits for a SKU, a random integer for a customer id, a random price.

Create the file

touch synth/simple.py

Add the code: synth/simple.py

"""Example 1: build request payloads from random primitives only."""

import random
import string
import uuid

CHANNELS = ("web", "mobile", "partner")


def request_id(rng: random.Random) -> str:
    """A seeded stand-in for uuid4() so runs are reproducible."""
    return str(uuid.UUID(int=rng.getrandbits(128), version=4))


def random_sku(rng: random.Random) -> str:
    return "SKU-" + "".join(rng.choices(string.digits, k=4))


def random_line(rng: random.Random) -> dict:
    return {
        "sku": random_sku(rng),
        "quantity": rng.randint(1, 5),
        "unit_price_cents": rng.randrange(500, 15000, 50),
    }


def random_order(rng: random.Random) -> dict:
    lines = [random_line(rng) for _ in range(rng.randint(1, 3))]
    return {
        "request_id": request_id(rng),
        "customer_id": f"C-{rng.randint(1, 9999):04d}",
        "channel": rng.choice(CHANNELS),
        "currency": "USD",
        "lines": lines,
        "total_cents": sum(line["quantity"] * line["unit_price_cents"] for line in lines),
    }

Detailed breakdown

  • Every function takes an rng instead of calling the module-level random functions. Passing the generator in is what makes a failing run reproducible: the same seed replays the same payloads, so a bug found in CI can be rerun locally.
  • request_id builds a UUID from rng.getrandbits(128) rather than calling uuid.uuid4(). uuid4() draws from the OS entropy pool and ignores your seed, which would leave one field different on every replay.
  • random_sku produces a syntactically plausible SKU. It matches the format the API expects and refers to nothing.
  • unit_price_cents uses randrange(500, 15000, 50), so prices land on 50-cent boundaries. Realistic-looking values in the wrong places are more useful for testing than obviously fake ones, because they fail for semantic reasons rather than being filtered out early.
  • quantity stays within the model’s ge=1, le=20, and the line count stays within min_length=1, max_length=5. The payload is schema-valid on purpose.
  • total_cents is computed from the lines rather than randomized, so the arithmetic is internally consistent. The generator is as correct as it can be without knowing the catalog.
  • customer_id is formatted as C-####, which is the kind of detail nobody double-checks. The real ids are CUST-####. Guessing an identifier format is one of the most common ways synthetic data quietly diverges from the system it is meant to exercise.

Try it

uv run python -c "
import json, random
from synth.simple import random_order
print(json.dumps(random_order(random.Random(1337)), indent=2))
"

{
  "request_id": "ccd1118f-bccf-4637-8d36-7ad167433a86",
  "customer_id": "C-5093",
  "channel": "partner",
  "currency": "USD",
  "lines": [
    {
      "sku": "SKU-9757",
      "quantity": 3,
      "unit_price_cents": 10300
    },
    {
      "sku": "SKU-6393",
      "quantity": 2,
      "unit_price_cents": 9800
    },
    {
      "sku": "SKU-1830",
      "quantity": 3,
      "unit_price_cents": 10800
    }
  ],
  "total_cents": 82900
}

Those values are what seed 1337 produces on CPython 3.12. A different seed, or a future change to Python’s random internals, would produce different values; nothing below depends on the exact ones.

The total checks out: 3 × 10300 + 2 × 9800 + 3 × 10800 = 30900 + 19600 + 32400 = 82900. Send it to the API:

uv run python -c "import json,random;from synth.simple import random_order;print(json.dumps(random_order(random.Random(1337))))" > tmp/simple-order.json
curl -s -X POST http://127.0.0.1:8000/orders \
  -H 'Content-Type: application/json' \
  -d @tmp/simple-order.json | python3 -m json.tool

{
    "detail": [
        "unknown customer C-5093",
        "unknown sku SKU-9757",
        "unknown sku SKU-6393",
        "unknown sku SKU-1830"
    ]
}

Four semantic failures on a structurally perfect payload. This generator is a good negative-path fuzzer and a useless happy-path fixture: it can prove the API rejects unknown references, and it can never get past them.

One caveat worth naming. random_sku draws from 10,000 possible values while the catalog holds 8, so a generated SKU collides with a real one about once every 1,250 draws. A test that asserts “these requests are always rejected” on SKUs alone would be flaky. The suite in Step 8 asserts on the customer id instead, where the format mismatch makes rejection certain.

Step 5: Load the seed database

Create the file

touch synth/seed.py

Add the code: synth/seed.py

"""Load the JSON seed database that both the API and the generator read."""

import json
from dataclasses import dataclass
from pathlib import Path

DATA_DIR = Path(__file__).resolve().parent.parent / "data"


@dataclass(frozen=True)
class SeedData:
    products: list[dict]
    customers: list[dict]

    @property
    def sellable(self) -> list[dict]:
        return [p for p in self.products if p["in_stock"]]


def load_seed(data_dir: Path = DATA_DIR) -> SeedData:
    products = json.loads((data_dir / "catalog.json").read_text(encoding="utf-8"))
    customers = json.loads((data_dir / "customers.json").read_text(encoding="utf-8"))
    if not products or not customers:
        raise ValueError(f"seed data under {data_dir} is empty")
    return SeedData(products=products, customers=customers)

Detailed breakdown

  • SeedData gives the generator one object to pass around instead of two loose lists, and frozen=True stops code from rebinding products or customers on a shared instance. The freeze is shallow: the lists and the row dicts inside them are still mutable, so treat “do not edit the rows” as a convention the type hint documents rather than a guarantee the runtime enforces.
  • sellable filters out-of-stock products at the point of use. Encoding the rule here rather than in the generator means every future generator inherits it.
  • load_seed takes a data_dir argument with a default, so a test can point it at a fixture directory with a deliberately broken catalog without touching the real files.
  • The empty check converts a silent failure into a loud one. An empty catalog would otherwise surface as IndexError from deep inside rng.choices, which is a confusing place to start debugging.
  • Reading with encoding="utf-8" keeps behavior identical across platforms whose default encoding differs, which matters as soon as a product name contains a non-ASCII character.

Step 6: Example 2 — generate from real rows

The second generator picks a customer, picks products, and copies their real prices. Nothing about the identity of an entity is invented; the randomness decides which real rows appear and how many of each.

Create the file

touch synth/seeded.py

Add the code: synth/seeded.py

"""Example 2: build request payloads by sampling the JSON seed database."""

import random

from synth.seed import SeedData
from synth.simple import request_id

MAX_LINES = 3
MAX_QUANTITY = 4


def weighted_sample(rng: random.Random, products: list[dict], k: int) -> list[dict]:
    """Pick k distinct products, favoring the ones with higher popularity."""
    pool = list(products)
    picked: list[dict] = []
    for _ in range(min(k, len(pool))):
        weights = [p["popularity"] for p in pool]
        choice = rng.choices(pool, weights=weights, k=1)[0]
        pool.remove(choice)
        picked.append(choice)
    return picked


def random_order(rng: random.Random, seed_data: SeedData) -> dict:
    customer = rng.choice(seed_data.customers)
    products = weighted_sample(rng, seed_data.sellable, rng.randint(1, MAX_LINES))

    lines = [
        {
            "sku": product["sku"],
            "quantity": rng.randint(1, MAX_QUANTITY),
            "unit_price_cents": product["price_cents"],
        }
        for product in products
    ]

    return {
        "request_id": request_id(rng),
        "customer_id": customer["customer_id"],
        "channel": rng.choice(customer["channels"]),
        "currency": "USD",
        "lines": lines,
        "total_cents": sum(line["quantity"] * line["unit_price_cents"] for line in lines),
    }

Detailed breakdown

  • request_id is imported from synth.simple rather than duplicated. Fields with no real-world referent (ids, timestamps, nonces) can stay fully synthetic; only the fields that join to something need real values.
  • weighted_sample draws one product at a time and removes it from the pool, so a request never lists the same SKU twice. rng.sample would also give distinct picks but ignores weights, and rng.choices respects weights but repeats. The loop is the small price of wanting both.
  • The weights come straight from the catalog’s popularity column, so the distribution of synthetic traffic follows the data rather than a constant the author picked. When the catalog changes, the traffic profile changes with it, and no generator code is touched.
  • seed_data.sellable is what gets sampled, so SKU-7101 never appears. The generator produces requests that should succeed; a request that trips the out-of-stock rule belongs in a negative-path test written on purpose, not in the happy-path batch.
  • channel is drawn from customer["channels"], which satisfies the cross-field constraint by construction. This is the pattern that generalizes: read the constraint from the data instead of restating it in generator code, where it would drift out of sync with the API.
  • unit_price_cents is copied from the catalog rather than randomized, so the price check passes and total_cents agrees with the server’s own arithmetic.
  • MAX_LINES and MAX_QUANTITY sit below the schema’s max_length=5 and le=20. The happy-path generator stays inside the limits; probing the boundaries is a separate job with its own fixtures.

Try it

uv run python -c "
import json, random
from synth.seed import load_seed
from synth.seeded import random_order
print(json.dumps(random_order(random.Random(1337), load_seed()), indent=2))
"

{
  "request_id": "643cb56d-4ec1-4fc6-bee2-9f53ebf644bb",
  "customer_id": "CUST-0005",
  "channel": "partner",
  "currency": "USD",
  "lines": [
    {
      "sku": "SKU-6300",
      "quantity": 3,
      "unit_price_cents": 2450
    },
    {
      "sku": "SKU-5510",
      "quantity": 4,
      "unit_price_cents": 1850
    },
    {
      "sku": "SKU-6301",
      "quantity": 3,
      "unit_price_cents": 3900
    }
  ],
  "total_cents": 26450
}

CUST-0005 is the partner-only customer, and the generator picked partner. The total is 3 × 2450 + 4 × 1850 + 3 × 3900 = 7350 + 7400 + 11700 = 26450, and every price matches the catalog. Send it:

uv run python -c "import json,random;from synth.seed import load_seed;from synth.seeded import random_order;print(json.dumps(random_order(random.Random(1337), load_seed())))" > tmp/seeded-order.json
curl -s -X POST http://127.0.0.1:8000/orders \
  -H 'Content-Type: application/json' \
  -d @tmp/seeded-order.json | python3 -m json.tool

{
    "order_id": "ORD-643cb56d",
    "customer_id": "CUST-0005",
    "lines": 3,
    "total_cents": 26450
}

Same seed, same schema, same code shape as Example 1. The only difference is where the values came from.

Step 7: Add a CLI to emit and post batches

One payload proves the idea. A batch runner is what you actually use, either to dump JSON lines for another tool or to drive traffic at a running server.

Create the file

touch synth/cli.py

Add the code: synth/cli.py

"""Emit synthetic order requests, and optionally POST them at an API."""

import argparse
import json
import random
import sys
from collections import Counter

import httpx

from synth import seeded, simple
from synth.seed import load_seed


def build_orders(mode: str, count: int, seed: int) -> list[dict]:
    rng = random.Random(seed)
    if mode == "simple":
        return [simple.random_order(rng) for _ in range(count)]
    seed_data = load_seed()
    return [seeded.random_order(rng, seed_data) for _ in range(count)]


def post_orders(orders: list[dict], base_url: str) -> Counter:
    tally: Counter = Counter()
    with httpx.Client(base_url=base_url, timeout=10.0) as client:
        for order in orders:
            response = client.post("/orders", json=order)
            tally[response.status_code] += 1
            if response.status_code != 201:
                print(f"{response.status_code} {response.json()['detail']}", file=sys.stderr)
    return tally


def main() -> int:
    parser = argparse.ArgumentParser(description="Generate synthetic order requests.")
    parser.add_argument("--mode", choices=("simple", "seeded"), default="seeded")
    parser.add_argument("--count", type=int, default=5)
    parser.add_argument("--seed", type=int, default=1337)
    parser.add_argument("--post", metavar="BASE_URL", help="POST each request to this API")
    args = parser.parse_args()

    orders = build_orders(args.mode, args.count, args.seed)

    if not args.post:
        for order in orders:
            print(json.dumps(order))
        return 0

    tally = post_orders(orders, args.post)
    print(f"posted {len(orders)} requests: " + ", ".join(f"{k}={v}" for k, v in sorted(tally.items())))
    return 0 if set(tally) == {201} else 1


if __name__ == "__main__":
    raise SystemExit(main())

Detailed breakdown

  • build_orders creates one random.Random(seed) for the whole batch, so --count 25 --seed 1337 reproduces the same 25 payloads every time. Seeding per-order would make every request in the batch identical.
  • load_seed() is called once per batch rather than once per order, which keeps a 10,000-request run from re-reading and re-parsing the JSON files 10,000 times.
  • Without --post, the CLI writes one JSON object per line. JSON Lines pipes cleanly into jq, into a load-testing tool, or into a file used as a fixture corpus.
  • httpx.Client is opened once with a context manager, so the batch reuses a connection instead of paying for a new one per request. The 10-second timeout keeps a hung server from stalling the run indefinitely.
  • Failures print to stderr while the summary goes to stdout, so ... --post URL > /dev/null shows only what went wrong.
  • The Counter of status codes is the actual result of a batch. Individual responses are noise; the distribution is the signal.
  • Returning 1 unless every response was 201 is what makes the post target usable as a CI smoke check. A shell && chain or a CI step will fail on a non-zero exit.

Run both modes against the API

With the server from Step 3 still running:

uv run python -m synth.cli --mode simple --count 3 --seed 1337 --post http://127.0.0.1:8000

422 ['unknown customer C-5093', 'unknown sku SKU-9757', 'unknown sku SKU-6393', 'unknown sku SKU-1830']
422 ['unknown customer C-8975', 'unknown sku SKU-8549']
422 ['unknown customer C-5035', 'unknown sku SKU-5650', 'unknown sku SKU-6612', 'unknown sku SKU-0935']
posted 3 requests: 422=3

uv run python -m synth.cli --mode seeded --count 25 --seed 1337 --post http://127.0.0.1:8000

posted 25 requests: 201=25

Twenty-five accepted orders, no hand-written fixtures, and rerunning the command sends the identical batch.

Step 8: Test the API with both generators

Create the file

pyproject.toml already exists from uv init and the two uv add commands. Append the [tool.pytest.ini_options] block, and replace the placeholder description that uv init wrote. The rest of the file should already match.

Add the code: pyproject.toml

[project]
name = "synth-api"
version = "0.1.0"
description = "Synthetic JSON request generation for API testing"
readme = "README.md"
requires-python = ">=3.12"
dependencies = [
    "fastapi>=0.141.1",
    "httpx>=0.28.1",
    "uvicorn[standard]>=0.52.1",
]

[dependency-groups]
dev = [
    "pytest>=9.1.1",
]

[tool.pytest.ini_options]
pythonpath = ["."]
testpaths = ["tests"]

Detailed breakdown

  • pythonpath = ["."] puts the project root on sys.path during collection, so from api.main import app and from synth.seed import load_seed resolve without installing the project or scattering conftest.py files. Without it, collection fails with ModuleNotFoundError: No module named 'api'.
  • testpaths = ["tests"] stops pytest from walking .venv/ and data/ looking for tests, which makes a bare uv run pytest noticeably faster.
  • The version floors are what uv add resolved at the time of writing. Leaving them as floors lets uv sync pick up patch releases; pin them exactly if you need byte-identical CI runs.

Create the file

touch tests/test_generators.py

Add the code: tests/test_generators.py

"""Tests for the two generators, independent of the API."""

import random

from api.main import OrderRequest
from synth import seeded, simple
from synth.seed import load_seed

SEED_DATA = load_seed()


def test_simple_order_matches_the_schema():
    order = simple.random_order(random.Random(1337))
    parsed = OrderRequest.model_validate(order)
    assert parsed.total_cents == sum(l.quantity * l.unit_price_cents for l in parsed.lines)


def test_same_seed_produces_the_same_order():
    first = seeded.random_order(random.Random(99), SEED_DATA)
    second = seeded.random_order(random.Random(99), SEED_DATA)
    assert first == second


def test_seeded_orders_reference_real_rows():
    rng = random.Random(2026)
    known_skus = {p["sku"] for p in SEED_DATA.sellable}
    known_customers = {c["customer_id"] for c in SEED_DATA.customers}

    for _ in range(200):
        order = seeded.random_order(rng, SEED_DATA)
        customer = next(c for c in SEED_DATA.customers if c["customer_id"] == order["customer_id"])
        assert order["customer_id"] in known_customers
        assert order["channel"] in customer["channels"]
        assert {line["sku"] for line in order["lines"]} <= known_skus
        assert len({line["sku"] for line in order["lines"]}) == len(order["lines"])

Detailed breakdown

  • Reusing OrderRequest from the API to validate generator output means the schema has exactly one definition. When a field is added to the model, this test fails until the generator produces it, which is the failure you want.
  • test_simple_order_matches_the_schema pins the claim that Example 1 is structurally valid. If it ever starts producing schema violations, the negative tests in the next file would pass for the wrong reason.
  • test_same_seed_produces_the_same_order is the guard on reproducibility. It catches an accidental uuid.uuid4(), a datetime.now(), or a set iteration sneaking into the generator, any of which would make a CI failure unreplayable.
  • test_seeded_orders_reference_real_rows runs 200 draws against the invariants that matter: real customer, legal channel for that customer, sellable SKUs, no duplicate lines. Two hundred draws exercise every sellable row dozens of times (28 to 112 line draws per SKU on the seeds used here) while keeping the suite fast.
  • The subset check <= on sets is what enforces “sampled, not invented”. The distinctness check pins weighted_sample’s pool removal.
  • SEED_DATA is loaded once at module level and shared, which is safe here because no test writes to it. A test that does need to alter the seed data should call load_seed() for its own copy rather than editing the shared rows, since the frozen dataclass does not block that.

Create the file

touch tests/test_api_with_synthetic_requests.py

Add the code: tests/test_api_with_synthetic_requests.py

"""Drive the API with both generators through FastAPI's in-process test client."""

import random

from fastapi.testclient import TestClient

from api.main import app
from synth import seeded, simple
from synth.seed import load_seed

client = TestClient(app)
SEED_DATA = load_seed()


def test_health_reports_the_seed_counts():
    body = client.get("/health").json()
    assert body == {"status": "ok", "skus": 8, "customers": 5}


def test_simple_requests_are_rejected_as_unknown_references():
    rng = random.Random(1337)
    for _ in range(50):
        response = client.post("/orders", json=simple.random_order(rng))
        assert response.status_code == 422
        assert any("unknown customer" in item for item in response.json()["detail"])


def test_seeded_requests_are_all_accepted():
    rng = random.Random(4242)
    for _ in range(200):
        order = seeded.random_order(rng, SEED_DATA)
        response = client.post("/orders", json=order)
        assert response.status_code == 201, response.json()
        assert response.json()["total_cents"] == order["total_cents"]


def test_a_mutated_price_is_caught():
    order = seeded.random_order(random.Random(7), SEED_DATA)
    order["lines"][0]["unit_price_cents"] += 1
    response = client.post("/orders", json=order)
    assert response.status_code == 422
    assert "price mismatch" in response.json()["detail"][0]

Detailed breakdown

  • TestClient drives the ASGI app in-process, so 250 requests run in a fraction of a second with no port to bind and no server to start. The same generators work against a live server through synth.cli --post.
  • test_health_reports_the_seed_counts fails if someone edits a seed file without updating the tests, which turns a silent change in test coverage into a visible one.
  • The negative test asserts on "unknown customer" rather than on the SKU errors, for the collision reason from Step 4. A random SKU-#### can match a real catalog entry by chance; C-#### can never match CUST-####, so this assertion is deterministic.
  • test_seeded_requests_are_all_accepted is the payoff. Two hundred distinct, data-consistent orders all return 201, and response.json() is attached to the assertion so a failure shows the server’s error list instead of just the code.
  • Comparing the echoed total_cents to the generated one confirms the server recomputed the same total from the catalog, rather than trusting the number in the request.
  • test_a_mutated_price_is_caught takes a known-good payload and breaks one field by one cent. Mutating valid data is often the cheapest way to write a negative test, and it stays correct as the schema evolves.

Run the suite

uv run pytest -v

============================= test session starts ==============================
platform darwin -- Python 3.12.9, pytest-9.1.1, pluggy-1.6.0 -- .venv/bin/python3
cachedir: .pytest_cache
configfile: pyproject.toml
testpaths: tests
plugins: anyio-4.14.2
collecting ... collected 7 items

tests/test_api_with_synthetic_requests.py::test_health_reports_the_seed_counts PASSED [ 14%]
tests/test_api_with_synthetic_requests.py::test_simple_requests_are_rejected_as_unknown_references PASSED [ 28%]
tests/test_api_with_synthetic_requests.py::test_seeded_requests_are_all_accepted PASSED [ 42%]
tests/test_api_with_synthetic_requests.py::test_a_mutated_price_is_caught PASSED [ 57%]
tests/test_generators.py::test_simple_order_matches_the_schema PASSED    [ 71%]
tests/test_generators.py::test_same_seed_produces_the_same_order PASSED  [ 85%]
tests/test_generators.py::test_seeded_orders_reference_real_rows PASSED  [100%]

========================= 7 passed, 1 warning in 0.29s =========================

Absolute paths are trimmed above, and the reported duration varies by machine (runs here landed between 0.26s and 0.68s). The warning is a Starlette deprecation notice about TestClient and httpx; see Troubleshooting.

Step 9: Wrap it in a Makefile

Create the file

touch Makefile

Add the code: Makefile

.DEFAULT_GOAL := help
API_URL ?= http://127.0.0.1:8000
COUNT ?= 25
SEED ?= 1337

.PHONY: help install run gen-simple gen-seeded post test clean

help: ## Show this help screen
	@echo "Synthetic API request generator"
	@echo ""
	@echo "Targets:"
	@grep -E '^[a-zA-Z_-]+:.*?## ' $(MAKEFILE_LIST) \
		| awk 'BEGIN {FS = ":.*?## "}; {printf "  %-12s %s\n", $$1, $$2}'
	@echo ""
	@echo "Variables: API_URL=$(API_URL) COUNT=$(COUNT) SEED=$(SEED)"

install: ## Sync dependencies with uv
	uv sync

run: ## Start the order intake API on $(API_URL)
	uv run uvicorn api.main:app --reload

gen-simple: ## Print $(COUNT) naively random requests as JSON lines
	uv run python -m synth.cli --mode simple --count $(COUNT) --seed $(SEED)

gen-seeded: ## Print $(COUNT) catalog-backed requests as JSON lines
	uv run python -m synth.cli --mode seeded --count $(COUNT) --seed $(SEED)

post: ## POST $(COUNT) catalog-backed requests at a running API
	uv run python -m synth.cli --mode seeded --count $(COUNT) --seed $(SEED) --post $(API_URL)

test: ## Run the pytest suite
	uv run pytest -v

clean: ## Remove caches and bytecode
	rm -rf .pytest_cache **/__pycache__

Detailed breakdown

  • .DEFAULT_GOAL := help makes a bare make print the target list instead of running the first target by accident.
  • The help target greps its own $(MAKEFILE_LIST) for lines with a ## comment, so adding a target with a comment adds a help entry. There is no second list to keep in sync.
  • API_URL, COUNT, and SEED use ?=, so they can be overridden per invocation: make post COUNT=500 SEED=7. Echoing their current values in the help screen documents the knobs where they are used.
  • Every Python target goes through uv run, which resolves the project environment without an activated virtualenv.
  • post is the target to wire into CI behind a started server. It exits non-zero unless every response is 201.
  • clean removes only generated caches. tmp/ is left alone because scratch payloads are sometimes worth keeping around while debugging.

Verify the default target

make

Synthetic API request generator

Targets:
  help         Show this help screen
  install      Sync dependencies with uv
  run          Start the order intake API on $(API_URL)
  gen-simple   Print $(COUNT) naively random requests as JSON lines
  gen-seeded   Print $(COUNT) catalog-backed requests as JSON lines
  post         POST $(COUNT) catalog-backed requests at a running API
  test         Run the pytest suite
  clean        Remove caches and bytecode

Variables: API_URL=http://127.0.0.1:8000 COUNT=25 SEED=1337

The $(COUNT) placeholders appear literally because ## comments are matched as text by grep, not expanded by make. The values are printed on the last line.

Step 10: Full validation run

From a clean checkout:

make install
make test

In one terminal:

make run

In another:

make post COUNT=100 SEED=42

posted 100 requests: 201=100

Then confirm the negative path still fails as expected:

uv run python -m synth.cli --mode simple --count 10 --seed 42 --post http://127.0.0.1:8000 2>/dev/null

posted 10 requests: 422=10

Two commands, one asserting the API accepts data-consistent traffic and one asserting it rejects invented references.

Troubleshooting

ModuleNotFoundError: No module named 'api' when running pytest. The [tool.pytest.ini_options] block is missing or pythonpath = ["."] was dropped. Confirm the header line of pytest’s output says configfile: pyproject.toml.

ModuleNotFoundError: No module named 'synth' from the CLI. Run it as a module from the project root (uv run python -m synth.cli), not as a file path.

FileNotFoundError for catalog.json. Both api/main.py and synth/seed.py resolve data/ relative to their own __file__, so this points at a missing or renamed file rather than a wrong working directory. Check that data/ was not caught by a broadened .gitignore entry.

StarletteDeprecationWarning: Using httpx with starlette.testclient is deprecated; install httpx2 instead. Emitted by Starlette on import with current FastAPI and httpx 0.28. The tests pass; the warning tracks an upcoming migration in Starlette. Leave it visible, or silence it in [tool.pytest.ini_options] with a filterwarnings entry once you have decided how to handle the migration.

httpx.ConnectError from make post. The server is not running, or it bound to a different port. Start it with make run and check curl -s $API_URL/health first.

Every seeded request returns 422 with “price mismatch”. The generator and the API are reading different catalogs. That happens after copying data/ somewhere and editing one copy, and it is the failure mode the shared seed file is designed to prevent.

Batches are not reproducible across machines. Confirm both machines run the same Python minor version and that no code path calls uuid.uuid4(), random.random() at module level, or datetime.now(). test_same_seed_produces_the_same_order catches most of these.

Recap

  • The API validates in two layers: Pydantic checks structure, and the handler checks referential integrity against a JSON seed database. Random data clears the first layer easily and the second one almost never.
  • Example 1 invents every field. It is a fast negative-path fuzzer that proves unknown references are rejected, and it cannot produce an acceptable request.
  • Example 2 samples the same JSON files the API validates against, copies real prices, and reads per-customer constraints out of the data. Its output is consistent by construction, so 200 generated orders were accepted without a single hand-written fixture.
  • Threading an explicit random.Random(seed) through every generator function makes any failure replayable. Seeded UUIDs keep even the id fields stable.
  • The catalog’s popularity column shapes the traffic distribution, so the synthetic load follows the data instead of a constant hard-coded in the generator.

Next improvements

  • Add an invalid-request generator that starts from a seeded order and mutates one field, giving each API rule a matching negative test.
  • Export a JSON Lines corpus with make gen-seeded COUNT=10000 > corpus.jsonl and replay it through a load-testing tool to separate correctness from throughput.
  • Derive the seed database from a production snapshot with identifiers and names replaced, keeping the shape and cardinality of real data without carrying real customer records.
  • Generate the seed files themselves for schema fuzzing, so the catalog size and price spread become test parameters.
  • Wire make post into CI behind a started server, and fail the build on any non-201 in the seeded batch.

For a broader view of where generated requests fit alongside unit and contract tests, see A Testing Strategy for MCP Servers on macOS.