scriptable.com

vLLM is the inference server many production LLM deployments sit behind, and until recently it had no answer on a Mac beyond a source build of its CPU backend. The vllm-metal plugin changed that: it keeps vLLM’s engine, scheduler and OpenAI-compatible API, and swaps the compute layer for MLX, Apple’s array framework, running on the GPU through Metal. By the end of this article you will have that server running on your Mac, answering on the same OpenAI-compatible endpoint you would deploy on a CUDA box, sized so it leaves the machine usable. You will then measure the number that decides whether vLLM is the right server for your workload at all: how much aggregate throughput continuous batching buys you, and how much per-request latency it costs. Continuous batching is vLLM’s core trick, running many requests through the model together and admitting new ones as others finish. ...

A language model is a function from text to text. You send it a list of messages, it returns one message, and nothing else happens. It cannot read your disk, check the clock, remember your previous question, or run the tool it just asked for. Every capability an AI product appears to have belongs to the program wrapped around the model. That program is the harness: the code that builds each request, executes the tools the model asks for, and decides what the model is shown in the first place. ...

An MCP server that adds two numbers has no opinion about who is calling it. One that posts to a social account has to answer a question before it can do anything at all: whose account? There are two workable answers. You can build a multi-tenant server that authenticates every caller and looks up the account they own, which is the shape Add GitHub OAuth to a FastMCP Server builds. Or you can build one process that acts as exactly one account, reads that account out of its own environment, and runs on the same machine as the client calling it. ...

If you already have Python code that talks to a local model through openai.OpenAI, there is a second way to run that model: import it directly. The llama-cpp-python package binds llama.cpp into your own process, so the weights load inside your program and generation is a function call rather than an HTTP request. By the end of this article you will have taken a working OpenAI-SDK client and rewritten it around the Llama constructor. No server to start, no base_url, no API key, and no port to keep free. The finished module is about forty lines and comes with a test suite that runs without loading a model at all. ...

An agent is a program that lets a language model decide which of your functions to run, runs them, and hands back the results until the model says it is done. There is no framework in this article and no API key. By the end you will have a coding agent on your own machine that lists, reads and writes files inside a directory you choose, asks permission before running a shell command, and stops instead of spinning when the model gets stuck. The core agent is a few hundred lines of Python, plus a model-free pytest suite, for a 795 MiB model. ...

By the end of this article, you will have Codex CLI installed on a Mac, authenticated with ChatGPT or an API key, and running a small task inside a Git repository with an approval policy and Seatbelt sandbox you understand. Prerequisites You need macOS with an interactive terminal, Git, and either a ChatGPT account that can use Codex or an OpenAI API key. You should know basic shell commands and have a disposable directory where an agent may create files. In this article, the workspace is that disposable project directory. A Git repository is a directory whose files and history are tracked by Git. Codex CLI is an agentic coding tool: it can inspect files, edit them, and run local commands. An approval policy controls when Codex asks before acting; a sandbox is an operating-system boundary that limits what those commands can access. Seatbelt is macOS’s native sandboxing mechanism. ...

Run the Buzz relay on an always-on Mac and the desktop app on the workstation you sit at, with Ansible driving both halves. The relay hosts a community — Buzz’s name for a shared workspace — and is the shared log every person and every agent in it writes to, so it wants a machine that does not close. Install and Use Buzz on macOS puts both programs on one laptop and binds that community to localhost:3000, which is a good way to see what Buzz does and a poor way to run it for real. ...

Hermes Agent installs from a shell script in a couple of minutes. Installing it the same way twice, a year apart, on a machine you have since forgotten the details of, is the harder problem — and it is the one Ansible solves. This article builds one role that installs Hermes on two Macs that differ in the ways that actually matter: devbot5 — the Mac you are typing on minime — a headless Mac mini Connection local, no SSH at all ssh Runs as your login account a dedicated service account launchd job LaunchAgent in your home LaunchDaemon in /Library API bound to 127.0.0.1 0.0.0.0 Those last two rows are what this article is about. A headless Mac has no one logged in, so there is no GUI session for a LaunchAgent to live in and the job has to be a LaunchDaemon that starts at boot and drops privileges. A laptop joins hotel and coffee-shop networks, so binding an agent’s API to every interface there would publish a shell to whoever else is on that LAN. ...

Nostr and IPFS get compared constantly, usually as rivals. They are not. They answer two different questions, and an application that needs both answers needs both protocols — or something that plays each part. Nostr answers “who said what, and when.” Its unit is a signed event. IPFS answers “what are these bytes.” Its unit is content-addressed data. That difference is worth getting straight, but it is not the interesting part. The interesting part is that both protocols are decentralized by design and neither is automatically decentralized by deployment, and the ways each one collapses back toward a single point of failure are mirror images of each other. IPFS falls over on the gateway and the pin; Nostr falls over on the relay. Both are deployment choices that inherit the protocol’s good name without inheriting the property that earned it. ...

Three systems name data three different ways, and the names get compared far more often than they get computed. This article builds a small Python tool that emits all three identifiers for one twelve-byte input, so you can see exactly where they agree and where they do not: a Nostr event id — the SHA-256 of a canonical serialization of a signed event an IPFS CIDv1 — a SHA-256 digest wrapped in a version, a codec, and base32 a Blossom hash — the same SHA-256, bare Two of those three turn out to be the same 32 bytes wearing different amounts of clothing, which is hard to believe from a table and obvious from a terminal. ...