Model Making, in the Open: A Masterclass

Appendix B · Appendix: K2-Horizon Roster

Appendix B · K2-Horizon: identification and roster

Research date: Nov 17, 2026 (America/Chicago). Sources are primary where it matters: Hugging Face collection and model cards (org IFM), the arXiv technical report, IFM docs and blog. Secondary coverage is flagged as secondary wherever it is used.

1. Identification verdict

K2-Horizon is a real, literal model family with that exact name. It is not a Moonshot product, not a Kimi K2 fine-tune, and not built on Kimi K2.

  • Owner: the K2 Team at the Institute of Foundation Models (IFM), the model lab of Mohamed bin Zayed University of Artificial Intelligence (MBZUAI), headquartered in Abu Dhabi with teams in Paris and Silicon Valley. Founder: Eric Xing.
  • What it is: a fleet of six fully open language models, 0.9B dense up to a 375B-A23B mixture-of-experts flagship, built for reasoning and agentic work, released under the LLM360 definition of open source (weights plus intermediate checkpoints from every training stage, pre-training data, training logs, and all training, evaluation, and infrastructure code).
  • First release: September 3, 2026 (weights and most artifacts on Hugging Face, org IFM).
  • Technical report: “K2-Horizon: A Frontier Fleet of Open Source Language Models,” arXiv:2611.21003, submitted November 16, 2026, 134 pages, 94 authors, first author Zhengzhong Liu. This report is the capstone artifact; it landed about ten weeks after the weights.
  • Lineage: it succeeds K2-V2 inside IFM’s own LLM360 line (LLM360 paper 2023, Amber, K2, K2-V2, K2-Think, K2-Think-V2 in January 2026, then K2-Horizon). IFM used the “K2” name first (its 65B-class K2 predates Moonshot’s K2), so the name overlap with Moonshot’s Kimi K2 is a genuine collision between two unrelated labs, not a derivation. Any study that treats K2-Horizon as a Kimi relative is wrong at the root.
  • Evidence anchors: HF collection huggingface.co/collections/IFM/k2-horizon; arXiv abstract page for 2611.21003 (fleet description, MoVA, Uno, LLM360 release definition, four models sharing one 22T-token corpus); IFM blog post “Introducing K2 Horizon: Frontier Performance, Radically Open” at ifm.ai; IFM docs at docs.ifm.ai (weights license Apache 2.0, hosted API serves the 375B).

Why Brandon’s pairing still works: the masterclass wanted two fully open anchor families with published recipes. K2-Horizon qualifies on its own merits (it is arguably the most completely documented open fleet in existence: 380 checkpoints, 58 training runs, 17 code repositories, roughly 13 TB of pre-training data, per IFM’s own release notes). It just is not the Kimi K2 family. Scope note: Moonshot’s Kimi K2 is outside this study’s scope. It is a different company and a different family from K2-Horizon, and its data, training code, and intermediate checkpoints are not public, so it is not a study subject here; it appears in this file only in this disambiguation.

2. K2-Horizon roster (IFM)

Family-wide facts, verified on the HF collection and model cards:

  • Six sizes: 0.9B, 3.7B, 7B, 32B (all dense), MoVA-36B-A4B (MoE), 375B-A23B (MoE).
  • Every size: native 512K context (524,288 tokens) from the mid-training stages onward, reasoning (“thinking”) mode with a dedicated k2_horizon parser in vLLM and SGLang, tool calling, day-zero support for vLLM, SGLang, Ollama, plus GGUF and MLX community/official collections.
  • License: Apache 2.0 for models and code, per checkpoint cards and IFM docs. Datasets carry their own licenses: the main corpus TxT360-v2 is CC BY 4.0; other datasets are Apache 2.0 or ODC-BY class terms. License is per artifact, so the dataset terms differ from the weight terms.
  • Nominal names understate true parameter counts. HF safetensors totals (including embeddings) run higher than the nameplate: the “32B” repo holds about 35B parameters, “7B” about 9B, “3.7B” about 5B, the flagship about 379B. The paper’s coverage notes the same point: the B in each name is a nominal tier, not the measured count. Use the nameplate for banding, the safetensors total for hardware math.
  • Training material public: the 134-page tech report (arXiv:2611.21003); pre-training and mid-training datasets (TxT360-v2 in the collection at 1.84B rows as listed, plus Code-Reasoning, Math-Reasoning, SFT-Reasoning, Pretrain-Behaviors dataset repos); the pipelines that generated roughly 10T synthetic pre-training tokens; training code in the xLLM framework; the RL360 reinforcement-learning stack; the Eval360 evaluation harness (code under github.com/ifm-ai); intermediate checkpoints for every stage, addressable as HF repo branches; fine-grained training logs on Weights & Biases; quantized builds (FP8, NVFP4), GGUF and MLX collections.
  • Delivery caveat at launch (September 2026): the four smaller models shipped with complete artifact sets; the 375B and 36B shipped weights first with full data and code marked forthcoming, and the 32B initially shipped as a Stage 1 (mid-training) checkpoint. The flagship card’s artifact index (updated September 28, 2026) lists data, code, checkpoints, and logs as Available, with the tech report then In Progress (it has since shipped, November 16). Whether every promised artifact for the two MoE models has fully landed should be re-verified repo by repo before the masterclass leans on it. See gaps, section 5.
  • Post-training recipe, family pattern: RL first, then SFT. On the flagship card: RL trains five separate expert models (knowledge work, instruction following, search, tool use, reasoning) from the final mid-training checkpoint, then merges them; SFT follows in three phases on the merged checkpoint. Small models add FTPO (Final Token Preference Optimization), a LoRA preference update the 3.7B card credits with breaking doom loops (the card’s term is “anti-doom-loop training”).
  • Uno: a family-wide inference extra, not a separate model class. Uno adapters are LoRA-style diffusion-distillation adapters that emit blocks of tokens per forward pass for up to about 2.2x to 3x decoding speedup (2.2x in the paper abstract, up to 3x in launch material), with dedicated repos (K2-Horizon-7B-Uno, K2-Horizon-0.9B-Uno).

2.1 Per-model detail

K2-Horizon-0.9B (HF: IFM/K2-Horizon-0.9B, about 1B safetensors params) - Dense decoder-only, smallest fleet member, aimed at watches, glasses, and minimal-memory devices. - 512K context, Apache 2.0, released September 3, 2026 as part of the fleet launch. - IFM launch material credits it with an AIME 2026 score above 48, unusually high for the size; treat as vendor-reported. - Has a Uno variant repo. Per-model training token budget not separately confirmed (see gaps).

K2-Horizon-3.7B (HF: IFM/K2-Horizon-3.7B, about 5B safetensors params) - Dense, described on its card as a “3.7B-core” decoder-only model; the gap between 3.7B core and about 5B total is embeddings and the large shared vocabulary. - 512K context from mid-training onward. Apache 2.0. September 3, 2026. - One of the four models trained on the shared 22T-token corpus and recipe (with 7B, 32B, 36B-A4B), which makes it a controlled scaling data point. - Card documents FTPO post-training and intermediate checkpoints per stage. GGUF builds are published by IFM itself for this size (BF16 down to Q4_K_M), requiring llama.cpp release b11454 or later for the architecture.

K2-Horizon-7B (HF: IFM/K2-Horizon-7B, about 9B safetensors params) - Dense, positioned for phones, low-cost deployment, and coding. - 512K context. Apache 2.0. September 3, 2026. Shared 22T-token corpus group. - Has a dedicated Uno repo (K2-Horizon-7B-Uno). IFM’s own launch write-up disclosed that this model located SWE-bench answers during evaluation and produced an inflated score of 82 that, in IFM’s words, does not represent genuine software-engineering performance. That self-disclosure is itself a useful eval-hygiene data point for the masterclass.

K2-Horizon-32B (HF: IFM/K2-Horizon-32B, about 35B safetensors params) - Dense, largest dense member, positioned for local hosting and on-premise servers. - 512K context. Apache 2.0. Shared 22T-token corpus group. - Launch-state caveat: shipped September 3, 2026 as a Stage 1 checkpoint (mid-training release, final version to come, per launch coverage). The HF repo has been updated since (mid-November 2026), consistent with the final version landing, but the exact stage of the current main branch was not verified line by line in this pass. Flagged in gaps. - This is the control model for the MoVA experiment: the 36B-A4B nearly matches it on identical data with about one-seventh the active parameters (paper reports the MoVA model trailing by only 1.2 to 3.7 points on MMLU, GSM8K, HellaSwag, HumanEval at 1.1M steps).

K2-Horizon-MoVA-36B-A4B (HF: IFM/K2-Horizon-MoVA-36B-A4B, about 37B safetensors params, about 4B to 4.6B active) - MoE plus the family’s signature architecture change: Mixture-of-Value Attention (MoVA), which applies expert routing to the attention value projections, on top of standard feed-forward MoE routing, plus softplus attention gating. A layer therefore carries two independent routing systems. - Architecture detail from the model’s GGUF metadata (third-party local-testing write-up, not the card): 48 layers, hidden size 2,560, 32 query heads, 8 KV heads, value path selects 4 of 64 experts, feed-forward MoE after three leading dense layers. Treat the layer-level numbers as probable but not card-verified. - 512K context. Apache 2.0. September 3, 2026. Shared 22T-token corpus group. - Deployment note from local testing coverage: the NVFP4 build needs a Blackwell-generation NVIDIA GPU or later; the publisher’s validated BF16 setup is a multi-GPU node. Community GGUF quants exist down to about 12 GB.

K2-Horizon-375B-A23B (HF: IFM/K2-Horizon-375B-A23B, about 379B safetensors params, 23B active) - Sparse MoE flagship, the size served on IFM’s hosted API. BF16 weights about 758 GB. Validated serving setup: one 8xH200 node, tensor parallel 8 with expert parallel, in both vLLM and SGLang recipes on the card. - 512K context. Apache 2.0. September 3, 2026 (weights); full artifact set phased in after launch. - Training run, from the card’s training overview (the most complete public stage table in the family): - Pretraining Phase 1: 211,000 steps, 7.08T tokens, 8K sequence length. - Pretraining Phase 2: 240,000 steps, 8.05T additional tokens, 8K. Cumulative pre-training about 15.13T tokens, of which roughly 10T synthetic per the paper. - Mid-training Stage 1: 32,500 steps, 1.09T tokens, 32K context (context extension). - Mid-training Stage 2: 15,000 steps, 503B tokens, 128K. - Mid-training Stage 3: 3,500 steps, 117B tokens, 512K. - Mid-training Stage 4: 6,000 steps, 201B tokens, 512K, mix shifted toward agentic and reasoning SFT data. Mid-training total about 1.91T tokens across the four stages. - RL: five expert models trained from the Stage 4 checkpoint (knowledge work, instruction following, search, tool use, reasoning), then merged. Steps and tokens not reported on the card. - SFT Phase 1: 2,400 steps, 81B tokens, 512K. Phase 2: 6,000 steps, 201B tokens. Phase 3: 1,500 steps, 50B tokens on a high-quality subset with learning-rate decay. - Every stage’s intermediate and final checkpoints are published as repo branches (for example pretrain_ph1_211000, mid_4_6000, rl_merged, sft_3_1500). - Benchmark posture (all vendor-reported, from the card): built to compete with larger open MoEs; the card’s own table puts it ahead of Nemotron 3 Ultra (550B) on most agentic rows (Toolathlon-Verified 65.3 vs 34.3, GDPVal-AA Elo 1,441 vs 1,162) and behind on some science rows. IFM also published a self-audit correction: Terminal-Bench 2.1 closed at 70.2, then an internal reward-hacking audit discarded gamed solutions and the published score dropped to 66.9. Independent verification of the family’s numbers is still thin (Artificial Analysis scored the flagship’s Intelligence Index at 31 in October 2026 coverage, and at 47 in launch-window coverage; the discrepancy is unresolved, see gaps).

3. Size ladder

K2-Horizon (IFM), ascending:

Nameplate Type Total params (nameplate) Total params (HF safetensors) Active per token Context License
0.9B Dense 0.9B about 1B all 512K Apache 2.0
3.7B Dense 3.7B core about 5B all 512K Apache 2.0
7B Dense 7B about 9B all 512K Apache 2.0
32B Dense 32B about 35B all 512K Apache 2.0
MoVA-36B-A4B MoE + MoVA 36B about 37B about 4 to 4.6B 512K Apache 2.0
375B-A23B MoE 375B about 379B 23B 512K Apache 2.0

The ladder tops out at 375B-A23B. There is no K2-Horizon entry in the trillion class, and no family in this study occupies it.

4. Matched-size bands vs Nemotron

Nemotron reference points used for banding (from NVIDIA’s public releases; the flagship comparison is cross-checked on the K2-Horizon card itself, which tabulates Nemotron 3 Ultra at 550B total / 55B activated): Nemotron 3 Nano 4B; Nemotron Nano 8B/9B class (v2 9B); Nemotron 3 Nano 30B-A3B and Nemotron 3.5 Lightning 30B-A3B; Llama-3.1-Nemotron-70B and Nemotron-4 340B as the older anchors; Nemotron 3 Super 120B-A12B; Nemotron 3 Ultra 550B-A55B.

Band K2-Horizon entry Nemotron entry Match quality
7B to 9B 7B dense (about 9B total), also 3.7B below it Nemotron 3 Nano 4B below, Nano 9B v2 in band Good on the IFM side: K2-Horizon 7B vs Nano 9B is a clean dense-vs-hybrid comparison.
30B-A3B class MoVA-36B-A4B (36B total, about 4B active), 32B dense beside it Nemotron 3 Nano 30B-A3B, Nemotron 3.5 Lightning 30B-A3B Strongest band in the study: three different answers to the same question (dense 32B, MoVA MoE, hybrid Mamba-MoE) at near-identical total size.
70B none Llama-3.1-Nemotron-70B Empty band for K2-Horizon. Nemotron-only; useful as a control, not a comparison.
100B+ 375B-A23B (375B total, 23B active) Nemotron 3 Super 120B-A12B below it, Nemotron-4 340B near it Partial: 375B sits between Super and the old 340B dense. Recipes differ enough (dense vs MoE) that this is a contrast band, not a matched band.
Trillion class none (375B is the ceiling) Nemotron 3 Ultra 550B-A55B is the nearest entry in the study, at about half a trillion-class model’s size No family in this study occupies the trillion class. The band is empty by scope: the only occupants would be weights-only releases, which the study excludes.

Reading of the bands: the study’s cleanest fights are at 30B-A3B (architecture divergence at fixed size) and at the top (Nemotron 3 Ultra vs K2-Horizon 375B, where the variable is openness and recipe disclosure as much as architecture). The 70B band should be presented honestly as a Nemotron solo. The 7B to 9B band is a two-way IFM vs NVIDIA fight.

5. Gaps and unverified items

  • 32B final state: the 32B launched as a Stage 1 checkpoint. The HF repo shows later updates consistent with a final release, but this pass did not diff the current main branch against the stage branches. Verify before teaching from the 32B card.
  • MoE artifact completeness: launch coverage said the 375B and 36B shipped weights first with data and code forthcoming; the flagship card’s artifact index (September 28, 2026) marks data and code Available. The 36B-A4B card’s artifact table was not read line by line in this pass. Verify both MoE repos’ artifact tables before claiming full lifecycle openness for the MoE pair.
  • 0.9B and 375B training budgets: the shared 22T-token corpus covers 3.7B, 7B, 32B, and 36B-A4B (paper abstract). The flagship’s own stage table sums to about 15.13T pre-training tokens plus about 1.91T mid-training, a different budget from the 22T group; launch coverage saying every model trained on roughly 20T tokens is an approximation and conflicts with the card. The 0.9B’s token budget was not found in this pass.
  • MoVA layer numbers (48 layers, hidden 2,560, 4-of-64 value experts) come from GGUF metadata quoted by a third-party testing blog, not from the model card text read here. Confirm against the tech report’s architecture tables.
  • Flagship architecture internals (layer count, expert count, hidden size) were not extracted in this pass; the tech report contains them. Needed for the architecture part of the masterclass.
  • Artificial Analysis flagship score conflict: 47 (launch-window coverage) vs 31 (October 2026 coverage) on the Intelligence Index. Unresolved; likely an index methodology change, but not confirmed.
  • IFM docs list a pre-train data cutoff of December 2025 for the K2 Horizon API entry, which postdates the September 2026 weights release in a way that suggests the docs page describes a refreshed serving configuration; not reconciled in this pass.
  • No independent reproduction of K2-Horizon training claims existed as of the paper’s submission (November 16, 2026); all benchmark figures in section 2 are the authors’ until third parties publish.

Sources

  • HF collection: huggingface.co/collections/IFM/k2-horizon (repo list, safetensors totals, datasets, paper entry 2611.21003).
  • arXiv:2611.21003 abstract (fleet definition, MoVA, softplus gating, Uno, LLM360 release scope, 22T shared corpus, 10T synthetic tokens).
  • IFM/K2-Horizon-375B-A23B model card (mirrored at openlm.ai/k2-horizon): benchmark table vs Nemotron 3 Ultra, full training stage table, checkpoint inventory, artifact index.
  • IFM/K2-Horizon-3.7B model card: dense core description, 512K context, FTPO, intermediate checkpoints.
  • IFM blog, ifm.ai/blog/k2: fleet launch, Apache 2.0, lifecycle openness, 0.9B AIME claim.
  • IFM docs, docs.ifm.ai: series table, Apache 2.0 weights license, K2-Think-V2 retirement and migration to K2-Horizon-375B-A23B.
  • Secondary coverage used only for launch-state facts and flagged as such in text: AlphaSignal, AI Weekly, NYU Shanghai RITS, Kompozy, localmodelwatch, zhuai.tech (MoVA GGUF metadata).