What Tencent released

Tencent's Hunyuan team has released Hy4 flash as open weights in three variants. One is the instruction-tuned chat model, one is an FP8 build that uses less GPU memory, and one is a pretrained base model for further training. All three are on Hugging Face, ModelScope and GitCode, according to the model card. TokenPost reported the release on 18 November, and AIBase says the model first went live through Tencent's API on 16 November, with the weights following two days later.

The size depends on who's counting. The model card lists 220B total parameters, 64B of them in a component called Engram. Tencent's announcement, as AIBase reports it, gives 224B. Both sources agree that about 12B parameters are active for each token. The model reads text and images, writes text, and supports a context window of 1,048,576 tokens.

How the architecture keeps compute low

Hy4 flash is a mixture-of-experts (MoE) model: each layer holds many smaller sub-networks, called experts, and a router sends each token to only a few of them. The model card describes 42 layers. One is dense and 41 are MoE layers, each with 512 routed experts, of which 16 run per token alongside one shared expert. That's why a 220B model costs about as much compute per token as a 12B dense model, although every expert still has to be held in memory somewhere.

Engram is the less familiar part. It's a large table of learned embeddings for bigrams and trigrams (two- and three-token sequences), which the model looks up by hashing instead of computing. The GitHub repository puts the table at about 63.4B parameters, inserted at four points in the stack. A lookup table needs memory but almost no arithmetic, and the vLLM recipe takes advantage of that by keeping Engram tables in host RAM by default instead of on the GPU.

Attention mixes sparse and sliding-window layers at a 1:3 ratio using Multi-head Latent Attention, which compresses the key-value cache. AIBase reports that the cache is one sixth the size of Hy4's. Tencent also ships DFly2, a 0.85B drafter for speculative decoding. In speculative decoding a small model proposes several tokens and the large model checks them in one pass, which speeds up generation without changing the output. The card claims decoding above 150 tokens per second with megakernels, 4-bit weights and DFly2, but doesn't say which hardware produced that figure.

Running it on a server

The official serving path assumes a data centre. The vLLM recipe supports Hopper and Blackwell GPUs from H100 to GB300, uses eight H200s by default, and gives a conservative memory estimate of 345GB for FP8. Installation only works through a dedicated Docker image because no released vLLM wheel supports the model yet. AMD Instinct cards aren't supported, and neither is LoRA adapter serving. The recipe's default context is 8,192 tokens, so reaching anything close to 1M tokens means raising limits within whatever memory you have.

Running it locally

The local option comes from AngelSlim, Tencent's quantisation project, which publishes GGUF builds. GGUF is the single-file format llama.cpp loads. The recommended Q4_K_M file is 156.79 GiB, and the smallest, IQ2-AH, is 125.73 GiB. Support for Hy4 flash sits in an unmerged llama.cpp pull request, #32517, so you have to build that branch yourself. The GGUF page gives these commands:

git fetch origin pull/32517/head:hy4-flash
git checkout hy4-flash
cmake -B build
cmake --build build --config Release -j
./build/bin/llama-server -m Hy4-flash-Q4_K_M-00001-of-00002.gguf -c 8192 -ngl 99 --jinja --port 8080

On an M5 Max MacBook with 128GB, the page reports Q4_K_M decoding at 48.13 tokens per second with a 4K context and 44.15 at 32K, with prompt processing at 629.8 tokens per second at 4K. On a 128GB RTX Spark, the SPARK-4Bit-Fast build decodes at about 32 to 33 tokens per second. Mac users also need to check out a specific commit and apply a patch. Image input needs a separate projector file, which the file table doesn't list.

A 157 GiB file running on a 128GB machine needs an explanation, and the GGUF page doesn't give one. The Engram table is the obvious candidate for living outside fast memory, but we haven't found documentation confirming how llama.cpp handles it. Test your own context lengths and workloads on borrowed or rented hardware before buying a machine for this model.

What the benchmarks show, and what they leave out

The published scores are mostly for the base model: 71.1 on MMLU-Pro and 73.7 on MBPP-plus, among others. The model card itself warns that the base model saw synthetic instruction data, which can inflate few-shot scores. For the instruct model, Tencent reports 42.1 on WorkIndex, its internal agent benchmark, against 51.4 for the full Hy4. In a blind test with 85 users and 259 tasks, Hy4 flash scored 2.79 out of 4 for satisfaction, close to Claude Haiku 5.5's 2.81, at a median cost of about RMB 0.26 per task through Tencent's API. Scores on public agent benchmarks appear only as images in the repository. Every number here comes from Tencent, and we haven't seen independent replication yet.

The licence

The weights use the Tencent Hy License 1.0, which is based on Apache 2.0 with extra terms added. The grant is worldwide with no territorial exclusions, so New Zealand teams can use the model on the same terms as anyone else. Section 10 requires a separate agreement for model-as-a-service businesses with more than US$50 million in annual revenue, or with a product above 100 million monthly active users. Purely internal use is exempt. The licence counts quantised, fine-tuned and distilled versions as derivatives, so its conditions follow any modified weights you distribute. It doesn't address using the model's outputs to train other models.

What builders here could do with it

Hy4 flash gives teams that handle privacy-sensitive material, such as health records, legal files or customer transcripts, a capable multimodal model that runs on a single workstation in their own office. With the tool-call and reasoning parsers in vLLM and an OpenAI-compatible llama.cpp server, it can sit behind an existing agent loop with only a base URL change. For production, a pre-release llama.cpp branch and a Docker-only vLLM build are both reasons to pin exact versions and plan for changes when support reaches the main releases. A fine-tuning guide ships with the weights for teams that want to adapt the base model to domain data. If you get Hy4 flash running on local hardware, the society would like to see your configuration and numbers on the projects page.

Sources

Build with people working on the same things

Join the Auckland Society for AI: NZ$599 a year with a ForgeBox and the members' projects area. Our Auckland meetups are open to everyone.

Become a member