What Edge0 released
On 12 November 2026 Edge0 published Edge0-145B and a smaller sibling, Edge0-120B, along with its inference framework and a desktop app. The company calls Edge0-145B "the world's first open-source on-device AI model with 145 billion parameters." The weights are on Hugging Face in two builds. The project README dates the checkpoints to 13 November and says both models are derived from Alibaba's Qwen3.8-Flash-Next.
Edge0 aims the model at software development, terminal work and agent workflows. Plenty of open-weight models target local coding already. What's different here is the parameter count Edge0 claims to fit on one gaming-class GPU, plus an engine designed around that limit.
How three-tier expert streaming works
Edge0-145B is a mixture-of-experts (MoE) model. An MoE model contains many small sub-networks called experts, and a router picks a handful of them for each token. Only those experts do any work for that token. The model card lists 48 layers with 512 experts per layer, 10 routed experts plus one shared expert per token, and about 10.12B active parameters out of 145B in total. Edge0 calls the design a "latent" MoE. In the launch post, each group of 64 experts shares two large matrices and keeps a small set of its own parameters, which shrinks the total weight volume.
Edge0's engine, edge0-strata, keeps the full model out of VRAM. Frequently used "hot" experts live on the GPU. An expert library of about 40GB sits in system RAM and moves to the GPU when the router asks for it. About 48GB of vocabulary data stays on the SSD and is read on demand. The engine overlaps those reads with computation, splits work between CPU and GPU, and uses a built-in 1.26B draft head for speculative decoding. Speculative decoding has a small model guess several tokens ahead so the large model can check all of the guesses in one pass. The README says edge0-strata is a fork of an MIT-licensed project called Strata.
Edge0 trained the weights at 4-bit precision from the start, so the published builds aren't a squeeze of a larger 16-bit checkpoint. The model card gives 47.79 GiB for the NVFP4 build (NVIDIA's 4-bit floating point format) and 40.81 GiB for the Q3_K build, a 3-bit format from the GGML family. The context window is 256K tokens. An int8 KV cache keeps 32K tokens of that context on the GPU, according to an eesel AI write-up.
Hardware and speed
The figures below are Edge0's single-run measurements of the Q3_K build, taken from the README:
- RTX 5090 D 24 GB: 121.2 tokens per second agent decode, 3,554.9 tokens per second prefill at 16K
- RTX 4090 24 GB: 91.7 tokens per second agent decode
- Apple M2 Ultra (Metal): 57.0 tokens per second agent decode
- RTX 5090 D capped at 16 GiB VRAM and 20 GiB RAM: 44.9 tokens per second agent decode
Edge0 calls the 16 GiB plus 20 GiB setup "a test limit, not a minimum system requirement", although eesel AI reports that the v0.2.0 Windows release notes list those numbers as the minimum. At full speed on the RTX 5090 D, the model card shows the NVFP4 build using 23.25 GiB of VRAM and 38.27 GiB of host RAM. A 24GB card with 64GB of system RAM and a fast NVMe drive looks like the realistic target. A 16GB card with 32GB of RAM will run the model, but slowly.
Apple support is unclear. The launch post promises day-one macOS support and quotes a 29GB peak on a MacBook Pro. The README limits macOS to the Q3_K build in the desktop app, and the model card says NVFP4 runs only on CUDA. eesel AI notes that the release files contradict the launch post on both memory and macOS. Before buying hardware for this model, test it on a Linux or Windows machine with an NVIDIA GPU and CUDA 13, because those are the platforms the docs fully support.
Benchmarks and what isn't verified
Edge0 reports 86.5 on Terminal-Bench 2.1 and 64.6 on DeepSWE 1.1, compared with 83.3 and 58.7 for the Qwen3.8-Flash-Next base, according to eesel AI's table. On GPQA-Diamond, IFBench and HLE the fine-tune scores slightly below its base. Most of the gain is in agentic coding, and general reasoning stays roughly where the base model was.
Nobody outside Edge0 has reproduced these numbers yet. The launch post says evaluation setups will arrive with a technical report. Edge0's own figures for the 120B model also differ between the launch post and the README. The model card mentions two practical snags. Image input needs a separate strata-vision helper that isn't in the repository. Replies can come back empty when reasoning uses up the whole token budget, and setting "reasoning_effort": "none" avoids the empty replies.
The licence
The framework code is Apache-2.0. The weights come under two licences: the Edge0 Community Model License 1.0 and the Qwen Community License 1.0. The README allows personal, research, educational and internal use, along with commercial use of generated outputs. Selling the model, running a commercial inference or fine-tuning service, or shipping an independent commercial coding or office assistant built on the model requires written permission from Edge0. Calling the release "open source" is a stretch. "Open weights with commercial limits" describes it better.
What builders could do with it
The licence allows an internal coding agent on a workstation, so a team can point its tools at Edge0-145B without code leaving the building. The model card says the server exposes OpenAI- and Anthropic-compatible endpoints on http://127.0.0.1:8000 under the model name edge0_145b, which means most agent harnesses will need only a base URL change. The documented start command is:
build/cpu/bin/edge0-infer serve --model /path/to/Edge0-145B-NVFP4Build edge0-strata and edge0-infer first, following the guides linked from the README. The model card also shows auto-generated llama.cpp, Ollama and vLLM snippets. Those snippets don't match Edge0's own instructions, so leave them alone.
For New Zealand teams that handle client code, health data or government work, a 145B-class model on one desk machine means sensitive repositories can stay onshore without renting overseas GPU capacity. Consultancies and product companies should check the licence before going further. A paid coding assistant or hosted API built on Edge0-145B needs a separate deal with Edge0. We'd treat the model as a strong internal tool and run our own evaluations on our own repositories before relying on the vendor's scores. Members who benchmark Edge0-145B on local hardware can share their results through ASAI projects.
Sources
- Edge0-145B: Open-Source On-Device AI with 145B Parameters (Edge0 blog)
- Edge0-AI/Edge0 README (GitHub)
- Edge0/Edge0-145B-NVFP4 model card (Hugging Face)
- Edge0-145B (2026): specs, hardware needs and how to run it (eesel AI)
- Edge0 releases Edge0-145B model running on 16 GiB consumer GPUs (AF.net)
Build with people working on the same things
Join the Auckland Society for AI: NZ$599 a year with a ForgeBox and the members' projects area. Our Auckland meetups are open to everyone.