Llama-Cpp

One command turns a downloaded GGUF file into an HTTP server that speaks the OpenAI API. Code written against openai.OpenAI runs against it with one line changed — the base_url — and nothing leaves your machine. This article starts that server, reads what its startup log is telling you, drives it with curl, and then builds a small Python client on the official OpenAI SDK. The interesting part is not the happy path, which takes about ninety seconds. It is the two places where “OpenAI-compatible” stops being the whole story: a reasoning model can hand your SDK an empty content field, and the -c you pass is not the context each request gets. ...

llama.cpp runs large language models directly on your machine, with no Python runtime, no server process you did not start, and no account. On Apple Silicon it uses Metal for GPU work and the unified memory architecture means a model does not have to be copied to a separate card before it can run. This article installs llama.cpp with Homebrew, pulls a model from Hugging Face, runs it two ways, and shows exactly where the weights land on disk. It closes with a small Python project that reads a GGUF file’s header with nothing but the standard library, so the model file stops being an opaque blob you downloaded and becomes something you can inspect and assert against. ...