What shipped

On 13 November 2026 the ggml-org maintainers tagged llama.cpp v1.0.0 and ggml v1.0.0 on the same day. llama.cpp is the C/C++ inference engine underneath a large share of local AI tooling, and ggml is the tensor library that does the maths on CPUs and GPUs. The Hugging Face announcement puts the project at about three and a half years old and says it supports more than 150 model architecture variants.

The version number matters more than the feature list. Until August, llama.cpp only had rolling bNUM build tags, so anyone linking against libllama had no reliable signal for when an upgrade would break their code. The 1.0 release turns that into a written promise.

The versioning contract

The announcement adopts semantic versioning (a scheme where the version number tells you what kind of change a release contains). The rules, as published by the maintainers:

  • Major: a breaking change to the public C API in ./include
  • Minor: backward-compatible features, backend updates, new model support or API additions
  • Patch: bug fixes with no API change

The maintainers' own wording is that "both llama.cpp and ggml will reflect API and ABI incompatibilities with major-version bumps." ABI (application binary interface) compatibility means a compiled program keeps working against a newer shared library without being rebuilt.

The plan behind the release goes back to a ggml discussion opened on 7 August 2026. The discussion sets a rule that an official llama.cpp vX.Y.Z tag is cut only when llama.cpp's bundled copy of ggml matches a released ggml version. Distribution packagers asked for that rule: Debian ships ggml as its own package, and Homebrew runs regression tests on dependent packages, so a mismatched pair would fail. The same discussion marks llama-ext.h as a non-public header whose changes don't trigger version bumps. Nightly bNUM builds carry on alongside the semver tags.

Breaking changes to plan for

Version 1.0 clears out old code before the promise starts. According to the release notes, the release:

  • removes 35 deprecated functions, plus llama_set_warmup and the defrag_thold parameter and its --defrag-thold flag
  • renames llama_set_embd_state_src to llama_set_ctx_state_src
  • removes clip.h, whose multimodal code now lives in the mtmd library
  • moves the NextN speculative decoding API (a method where a model predicts several tokens ahead to speed up generation) into the public llama.h header

On the ggml side, the ggml notes remove the legacy DIAG_MASK_INF and DIAG_MASK_ZERO operations and raise the RPC protocol to v9, so mixed-version peers in a distributed setup will now refuse to connect. If you run llama.cpp RPC workers across several machines, upgrade all of them together.

Bindings in Python, Rust, Go or Node that call the C API directly will need updating if they used any of the removed functions. Projects that only talk to llama-server over HTTP are mostly unaffected by the header changes. The server changes below still apply to them.

FP8 KV caches

The KV cache stores the attention keys and values for every token in the context window, and on long contexts the cache can use more memory than the model weights. Pull request #28392, merged on 10 November, adds caches in the OCP FP8 E4M3 format (an 8-bit floating-point type with four exponent bits and three mantissa bits). The server documentation now lists f8_e4m3 as a value for -ctk and -ctv, next to f16 (still the default), bf16, q8_0 and several 4- and 5-bit types.

An 8-bit cache takes half the bytes of the f16 default, which means roughly twice the context in the same memory. The PR supports optional static per-tensor scales, which matters for checkpoints such as NVIDIA's NVFP4 releases that ship with scaled FP8 caches. The PR thread gives no perplexity or speed comparison against the existing q8_0 cache, so we don't yet know which of the two is the better 8-bit choice on consumer GPUs. The PR also disables FP8 cache tests for multi-GPU and kv-offload=false setups and skips them on the OpenVINO backend. Test any of those configurations yourself before relying on them.

llama-server security changes

Anyone exposing llama-server beyond localhost should read the server section carefully. The release notes list fixes for DNS rebinding, server-side request forgery (SSRF, where an attacker tricks the server into fetching internal URLs) through the CORS proxy, and browser access to child instances. The release also restricts HF_TOKEN to huggingface.co hosts and cancels MCP tool calls when a client disconnects or times out.

Two new flags are opt-in, and both change previous behaviour. The server docs describe them:

  • --allow-unauthenticated-tools permits server tools, MCP servers or the MCP proxy on a non-loopback address without an API key. The flag is off by default, so a tools-enabled server bound to a LAN address now needs --api-key or --api-key-file.
  • --media-allow-private-urls lets image_url and similar fields point at private or local addresses. By default the server accepts only public addresses, which will break multimodal pipelines that serve images from an internal file server.

The docs also say that enabling --tools or --agent narrows the --cors-origins default from * to same-origin requests. The basic launch command in the docs stays the same:

./llama-server -m models/7B/ggml-model.gguf -c 2048

Other additions

The release adds a host-memory cache for mixture-of-experts (MoE) models with a configurable CPU share, so experts that don't fit in VRAM can be served from system RAM more efficiently. It also adds an OpenAI-compatible /v1/audio/speech endpoint, support for Kimi-K3.1-Pro and Ripple-65B-Preview, and a redesigned web UI. The /v1/audio/speech endpoint appears in the release notes but isn't yet documented in the server README, which still describes only POST /tts.

What this means for builders in New Zealand

Local inference is a common answer for NZ teams who need data to stay onshore or on-premises, and many of those teams build on llama.cpp directly or through wrappers. A stable C API makes pinning a dependency practical. You can now pin to 1.x, take minor releases for new models and backend speed-ups, and treat a 2.0 tag as the signal to schedule migration work. Packaging through Debian, Homebrew, conda-forge, Winget and Docker also makes llama.cpp easier to get past an internal software approval process than a self-compiled binary.

Before upgrading, we'd suggest three checks. Search your bindings for the removed functions. Audit any server bound to a non-loopback address for an API key. Benchmark f8_e4m3 against q8_0 on your own hardware and prompts, because nobody has published that comparison yet. If you run those benchmarks, the results are worth posting on the ASAI projects page, since FP8 cache numbers for consumer cards don't exist publicly yet.

Sources

Build with people working on the same things

Join the Auckland Society for AI: NZ$599 a year with a ForgeBox and the members' projects area. Our Auckland meetups are open to everyone.

Become a member