Part 6 · Part 6: Midtraining and Annealing, the Hidden Stage
Model-Making Masterclass, Part 6. Written Nov 17, 2026. Engineer voice: plain language, terms defined when they first matter, every load-bearing number tied to a tech report, model card, or flagged as unverified.
The stage nobody puts on the slide
Ask a lab how their model was made and you get two answers: pretraining (trillions of tokens, general data) and post-training (instruction tuning, RL, the works). Between those two sits a third stage that rarely gets a slide and often decides what the model is actually good at. It is where the learning rate comes down for the last time, where the data mix quietly swaps from “everything we could crawl” to “the best material we have, repeated,” where context windows get stretched from thousands of tokens to hundreds of thousands, and where a general base model gets converted into a domain model. The open families in this study publish this stage. That is a large part of why they are in the study.
The stage has three problems for a reader. It has no agreed name. It has no agreed boundary. And its budget, typically 2 to 10 percent of the run, is small enough that labs treat it as a detail, even though the return on that small slice is routinely the largest capability jump per token in the whole pipeline. This part names the pieces, prices them per family, and ends where this masterclass always ends: what the stage means for building a cyber model.
Names and boundaries: classify by what changed, not by the label
Three terms circulate, and families use them inconsistently.
- Annealing is the oldest term. It names a learning-rate event: the final decay of the rate toward zero at the end of training. In a classic cosine run the anneal is just the tail of the schedule and the data does not change. In modern open recipes the decay window doubles as a data event, because the last tokens a model sees at a vanishing learning rate are the ones it settles on. When a report says “annealing” today, it usually means decay plus a better mix.
- Midtraining is Ai2’s term, now spreading. It names a distinct stage between pretraining and post-training, with its own curated mix, its own (restarted, then decayed) learning-rate schedule, and a stated purpose: patch the base model’s measured weaknesses before post-training begins. OLMo 2’s report uses “mid-training” and “annealing” for the same stage in adjacent paragraphs, which tells you the vocabulary is still settling.
- Continued pretraining names open-ended further training of an existing model on new material: a new language, a new modality, a new domain. The learning rate is rewarmed rather than merely decayed, and the run has no fixed relationship to the original schedule. Apertus 1.5 (about 4T extra tokens for the 8B, 2T for the 70B, adding multimodal data and extending context to 262,144) is continued pretraining. So is the Indic stage of Nemotron 3.5 Super VL.
Because the labels drift, classify any given stage by three questions instead. Did the learning rate restart or rewarm, or only decay? Did the data mix change, and by how much? Did the context length or the modality change? The answers place the stage precisely, whatever the slide called it.
| Family | What they call it | LR behavior | Mix change | Context or modality change |
|---|---|---|---|---|
| OLMo 2 | Mid-training (Stage 2) | Restart, linear decay to zero | Total: Dolmino Mix replaces the pretraining mix | None (4K throughout) |
| OLMo 3 | Midtraining, then long-context extension | Stage-wise schedules | Dolmino, then Longmino | 8K to 65K in the third stage |
| SmolLM3 | Stage 3 decay; long-context extension; reasoning midtraining | Decay, then extension runs | Decay mix rebalanced; extension keeps the mix | 4K to 64K trained, 128K via YaRN at inference |
| K2-Horizon 375B | Mid-training, Stages 1 to 4 | Not fully specified in the card summary (flagged below) | Final stage shifts toward agentic and reasoning SFT data | 8K to 32K to 128K to 512K, staged |
| Nemotron 3 Super | Pretraining Phase 2 | One schedule, two data phases | 20T diversity phase, then 5T high-quality phase | None in pretraining (1M context is a config and serving claim) |
| Nemotron 3.5 Super VL | Mid-training, continued pretraining, extension (card’s own words) | Per-stage | SWE mid-training (67B), Indic continued pretraining (100B) | 35B-token extension to 1M context; vision grafted |
| Granite 4.0 | Four-stage curriculum | Two decay stages with different LR schedules | Code and math upweighted, then high-quality decay stages | None disclosed as a separate stage |
| Apertus 1.5 | Continued pretraining | Rewarmed | Multimodal data added | Context extended to 262,144 |
Two lessons fall out of the table. First, NVIDIA mostly refuses the word: the quality shift that Ai2 calls midtraining is, in Nemotron 3 Super, simply Phase 2 of pretraining, a fifth of the run spent on high-quality data. The function is identical; only the bookkeeping differs. Second, K2-Horizon’s mid-training is mostly not about data quality at all. Three of its four stages are context extension, and only the fourth changes the mix, blurring into SFT data. Same label, different job. Read the stage, not the name.
The schedule that makes the stage possible: WSD
One enabling technology sits under most of these recipes: the warmup-stable-decay (WSD) learning-rate schedule, named by the MiniCPM team (Hu et al., 2024). A cosine schedule commits you to a total token count on day one, and its checkpoints are unfinished by construction, since the rate is still high mid-run. WSD instead warms up, holds the rate constant for an unbounded “stable” phase, and decays only when you decide to stop. Any stable-phase checkpoint can be branched into a short decay to produce a finished model, and Hägele et al. (2024) showed a cooldown of roughly 20 percent of the run or less matches a cosine tuned to that length.
That one property reorganizes the economics of this whole part. Anneal experiments stop being separate full runs and become cheap branches off one trunk. Ai2’s micro-annealing (below) is a direct application. So is Marin’s mid-run recovery: the Marin 8B retrospective records a discarded cooldown and a decision to rewarm and switch the mixture to Nemotron-CC mid-run, which is only possible because the trunk checkpoint had not been decayed into a dead end. A decayed checkpoint is a finished artifact; a stable-phase checkpoint is an asset. Secondary summaries of the Nemotron 3 Ultra report describe NVIDIA using WSD across a 20T-token horizon, which would make the pattern industrial as well as academic; treat that specific claim as reported, not card-verified (see flagged items).
The anneal mix: half replay, half patch list
The best-documented anneal mix in the field is Ai2’s Dolmino Mix 1124, built for OLMo 2 and published down to per-source token counts. Its design is worth internalizing because every other family’s mix is a variation on it. For the 50B-token version used by the 7B:
| Source | Share of the 50B mix | Sampling of the source pool |
|---|---|---|
| High-quality web (DCLM-Baseline fastText top 7 percent, plus FineWeb quality 2 and up) | 47.2% | 3.23% of the 752B pool |
| Dolmino Math Mix (mostly synthetic: TinyGSM, MathCoder2, TuluMath, plus GSM8K train) | 20.8% | 100% (the whole 10.7B pool) |
| FLAN instruction data (decontaminated) | 16.6% | 50% of the 17B pool |
| Wikipedia and Wikibooks | 7.1% | 100% |
| peS2o academic papers | 5.9% | 5.15% of the 58.6B pool |
| Stack Exchange Q&A | 2.5% | 100% |
Four design rules are visible in that table.
- Keep a general anchor near half the mix. The single largest component is still web text, just the most aggressively filtered slice of it. The anneal sharpens the model; it does not narrow it to a textbook. This is the replay principle, and it recurs everywhere in this part (OLMo 3’s long-context mix is two-thirds short data; see below).
- Treat the mix as a patch list for measured weaknesses. OLMo 2’s pretraining left math weak, so math is a fifth of the anneal, against a low single-digit share in pretraining. Ai2’s method for choosing patches is itself published: micro-annealing. Candidate sources are tested independently in small anneal runs branched off the same trunk checkpoint (OLMo 3 used 10B-token micro-anneals), and the report’s claim is that a source’s effect in isolation roughly persists when it is mixed with the others, which makes mix design composable instead of alchemical.
- Repeat the elite, ration the bulk. Small, excellent sources are consumed whole or repeatedly: in the 300B version of Dolmino, FLAN is sampled at 200 percent (two epochs) and Stack Exchange at 400 percent, while the giant web pool never exceeds about a fifth consumed. Repetition of good data beats unique mediocre data at this stage, because the learning rate is dying and every token gets a vote on the final weights.
- Prime the next stage. FLAN in a base model’s anneal is instruction data arriving early, on purpose. OLMo 3’s Dolmino goes further: its 100B tokens, drawn from a public 2.2T-token pool, fold in instruction data and reasoning traces explicitly to prepare the base for post-training. The boundary between midtraining and SFT is porous, and the open families exploit that instead of pretending otherwise.
The variations, briefly. SmolLM3’s decay stage (the last 1.1T of its 11.2T pretraining) runs 63 percent web, 24 percent code, 13 percent math, with instruction and reasoning data folded in, against 85/12/3 in its first stage: the same rebalance-toward-specialized pattern, executed inside pretraining rather than as a separate stage. SmolLM2’s four-stage curriculum ends the same way, a final decay phase upsampling its best math and code. Granite 4.0’s card describes two high-quality decay stages with different learning-rate schedules closing its four-stage curriculum, but IBM does not publish the shares, so Granite’s anneal can be admired and not copied (flagged below). Nemotron 3 Super’s 5T-token high-quality phase is the pattern at industrial scale and industrial opacity: the phase exists, its size is stated, its composition is described only at the corpus-family level.
And K2-Horizon’s flagship makes the opposite allocation: its mid-training spends almost nothing on mix redesign until Stage 4 (201B tokens at 512K context, mix shifted toward agentic and reasoning SFT data), because its mid-training problem was context, not quality. Which problem your base model has decides which of these templates you are copying.
Model souping: the anneal, averaged
OLMo 2’s signature move is to run the anneal more than once and average the results. For the 7B, Ai2 ran the 50B-token Dolmino anneal three times from the same Stage 1 checkpoint, changing only the data order, then averaged the three final weight sets element by element. The 13B souped four runs (three at 100B, one at 300B); the 32B followed the same pattern. OLMo 3’s 32B repeated the trick at its midtraining stage: two 100B-token Dolmino runs with different data orders, weight-averaged before the long-context stage.
The mechanism comes from Wortsman et al. (2022), “Model soups” (arXiv:2203.05482). Models fine-tuned from a shared initialization tend to land in a single low-error basin, so the straight-line average of their weights is itself a good model, often better than every ingredient, at zero extra inference cost. The paper’s two recipes matter here. A uniform soup averages everything, and fails when one ingredient wandered out of the basin (in their CLIP sweep, a uniform soup of 72 runs scored below the best single run). A greedy soup ranks ingredients by validation score and adds each only if the average improves, which guarantees the soup is never worse than its best ingredient on that validation set. OLMo’s setting is the friendly case for uniform souping: the ingredients share an initialization and a mix, and differ only in data order, so basin membership is nearly assured.
The published gains are real and modest, which is exactly what a variance-reduction technique should look like. In OLMo 2’s report tables, the 7B’s three-run soup beats its best single anneal run on the team’s aggregate measures (their GSM-style math aggregate moves from 75.6 for the best single run to 77.0 for the soup). More persuasive is the independent replication: the LLM-jp group applied Dolmino mixes to their own unrelated pretrained bases and reproduced the pattern end to end. Their mid-training alone took a 7.7B base from 14.86 to over 51 on GSM8K, and souping the runs then beat the best single seed on 10 of 12 benchmarks, GSM8K included (53.5 best seed, 59.1 soup). An anneal mix, it turns out, is a portable artifact: it transfers across model families. Remember that in the cyber section.
When souping beats a single anneal: when run-to-run variance is dominated by data-order noise rather than by mix quality, when the anneal runs are cheap enough to parallelize (they are independent, so wall-clock cost can be flat), and when you want insurance against one unlucky ordering. When it does not: when ingredients start from different initializations, when one ingredient is trained on a materially different mix and has left the shared basin, or when a single ingredient is simply bad and you average uniformly anyway. Note what OLMo does not do: it does not soup across different mixes. The soup controls for order, not for recipe.
The cost math is friendly. OLMo 2’s Stage 2 is 5 to 10 percent of training FLOPs by the report’s own accounting. Souping the 7B tripled a 50B-token anneal into 150B tokens against roughly 4T tokens of Stage 1 exposure: about 2.5 percent extra total tokens for a consistent, replicated gain and a free variance estimate. A 2025 follow-up formalizes why this works so well: the WSM paper (arXiv:2507.17634) proves that weighted checkpoint averaging can emulate learning-rate decay schedules, which recasts souping and annealing as two views of one operation. Averaging is decay, performed in weight space after the fact.
Context extension: a separate stage because it is a separate skill
A model trained at 8K context does not merely get worse at 512K; it has never seen positions that far out, and its positional encoding is extrapolating into unmeasured territory. Extension is therefore run as its own stage, on its own data, with its own checks. Three published ladders show the design space.
| Family | Ladder | Positional method | Data design |
|---|---|---|---|
| K2-Horizon 375B | 8K pretrain, then 1.09T at 32K, 503B at 128K, 117B at 512K, 201B at 512K with mix shift | Not specified in the card summary (flagged) | Extension dominates the mid-training budget: 1.91T tokens total |
| SmolLM3 | 100B tokens in two 50B steps: 4K to 32K, then 32K to 64K; 128K at inference only | RoPE theta raised to 1.5M, then 5M; NoPE on three of four layers (Part 4); YaRN for the inference extrapolation | Same general mix; the team reports upsampling specialized long-context data did not beat the positional approach |
| OLMo 3 | 50B (7B) or 100B (32B) tokens at 65K after midtraining | YaRN, factor 8 over 8,192 base positions, applied only on the full-attention layers (three of four layers are sliding-window and need no scaling) | Longmino mix: 34 percent long documents (olmOCR science PDF pool), 66 percent Dolmino short data, plus synthetic aggregation tasks inside long documents |
Three engineering points. First, token budgets shrink as context grows (K2-Horizon: 1.09T, then 503B, then 117B), because long sequences are expensive per token and because the marginal skill being taught narrows at each rung. Second, the positional fix differs by architecture: SmolLM3 raises the RoPE base frequency (theta) in steps and leans on its NoPE layers, OLMo 3 applies YaRN surgically to only the layers that attend globally, and both report the surgical version working better than brute-force data upsampling. Third, the replay anchor from the anneal section returns in force: OLMo 3’s extension mix is two-thirds short, high-quality data. Extension stages trained on long documents alone reliably damage short-context performance; the short data is the insurance premium.
What breaks, and how families test for it. The failure modes are short-context regression (the model gets subtly worse at ordinary lengths), retrieval dilution (needle-in-haystack accuracy decays well inside the advertised window), and positional artifacts (repetition, lost beginnings). The open families test with the same base suite rerun after extension, plus dedicated long-context suites: OLMo 3 reports RULER scores at 65K, and Nemotron 3 Nano’s card leads with RULER at 256K (92.9) and 1M (86.3). The discipline to copy is the pairing: no extension claim without the short-context suite rerun beside it.
Two more data points bound the space. Nemotron 3.5 Super VL bought its 1M context with a dedicated 35B-token extension stage, cheap because the hybrid Mamba architecture carries most sequence state without attention (Part 4). The contrast worth naming: weights-only releases outside this study also extend context in refresh releases, with the method undisclosed. They extend context too; they just do not teach you how, which is the whole reason the study’s extension evidence comes only from the families above.
One caution that belongs to Part 8 but must be planted here: extension gains are fragile downstream. SmolLM3’s team found their preference-optimization stage had damaged the long-context behavior the extension bought, and repaired it in weight space, merging 0.9 of the aligned checkpoint with 0.1 of the midtraining checkpoint. If your pipeline extends context in midtraining, budget a regression check after every later stage, not just after extension itself.
Domain continued pretraining: conversion under forgetting pressure
The fourth job of this stage is conversion: take a finished general base and make it a domain model. The roster’s cleanest published example is linguistic. Nemotron 3.5 Super VL’s backbone (Nemotron 3 Super Base) received, in sequence: an MTP head trained on 100B tokens, 1T tokens of distillation from Nemotron 3 Ultra Base (distillation mechanics are Part 8’s subject; here it counts as a conversion stage), 100B tokens of Indic continued pretraining on a synthetic corpus spanning 22 languages (the full corpus, about 2.83T tokens co-created with IIT Madras, was released with the model), the 35B-token context extension above, and 67B tokens of software-engineering mid-training. Read that card and you are reading this entire part in miniature: five conversion stages, each with a named budget, stacked on one backbone.
Modality grafting is the same operation with new organs. Apertus 1.5’s continued pretraining added multimodal data across trillions of tokens. Nemotron 3 Nano Omni extends the Nano backbone with vision and audio towers. The roster does not publish freeze-versus-joint-training details for these grafts at the level this part would need, so the honest statement is architectural: grafting stages exist, their budgets are published, their internal training discipline mostly is not (flagged below).
The governing risk of every conversion is catastrophic forgetting: gradient steps on narrow new data erase capabilities the base paid trillions of tokens to learn. The mitigations visible in this roster are the ones already introduced, now doing double duty:
- Replay. Mix general high-quality data into the domain stage (the 47 percent web anchor in Dolmino, the 66 percent short data in Longmino). Nobody in this roster runs a pure-domain stage at base-model scale.
- Learning-rate discipline. Anneal-style stages decay to zero, which limits drift but also ends the conversation: a decayed model cannot be cheaply continued. WSD-style stable checkpoints keep the option open, and Marin’s rewarm-and-switch-mix recovery shows the option being exercised in production.
- Weight-space repair. Souping and merging (SmolLM3’s 0.9/0.1 merge) can restore a damaged capability from an earlier checkpoint without retraining, provided the checkpoints share a basin.
- Measurement. Every conversion stage in the open families is bracketed by the same eval suite run before and after. Forgetting you do not measure is forgetting you ship.
What the stage costs
| Family and model | Main run | Midtraining, annealing, extension total | Share |
|---|---|---|---|
| OLMo 2 7B | About 4T tokens Stage 1 exposure | 3 anneal runs of 50B (souped) | 5 to 10% of FLOPs (report’s figure); about 4% of tokens counting all soup ingredients |
| OLMo 2 13B | About 5T tokens | 3 runs of 100B plus 1 run of 300B (souped) | About 12% of tokens counting all ingredients |
| OLMo 3 7B | 5.93T tokens | 100B Dolmino plus 50B Longmino | About 2.5% of tokens |
| OLMo 3 32B | 5.5T tokens | Two 100B Dolmino runs (souped) plus 100B Longmino | About 5% of tokens |
| SmolLM3 3B | 11.2T tokens | 100B extension plus 140B reasoning midtraining | About 2% of tokens (the stage 3 decay, 1.1T, is counted inside pretraining here) |
| SmolLM2 1.7B | 11T tokens | Final decay stage about 1T | About 9% of tokens, inside the curriculum |
| K2-Horizon 375B | 15.13T tokens | 1.91T across four mid-training stages | About 11% of pre-SFT tokens |
| Nemotron 3 Super | 25T tokens | 5T high-quality Phase 2 | 20% of tokens, booked as pretraining, not midtraining |
| Granite 4.0 Micro | 14.5T tokens staged total | Two decay stages of 2T and 0.5T (card stage table) | About 17% of tokens |
The spread is the finding. Ai2 runs the leanest version, 2.5 to 5 percent, and gets documented, replicated gains. K2-Horizon spends 11 percent because its mid-training is mostly context extension for a 512K flagship. NVIDIA books 20 percent and calls it pretraining. There is no consensus budget because there is no single job: quality annealing is cheap, context extension is priced by the target length, and domain conversion is priced by the domain. Budget the job you actually have.
The cyber read: this is where a general base becomes a cyber base
Everything above converges on one operational fact for LAB2: the cheapest place to turn a general open base into a cyber base is this stage, not pretraining and not SFT. Pretraining is too expensive to redo for a domain, and SFT teaches format and behavior on top of whatever knowledge the base already has. The anneal is where knowledge-level diet changes land with outsized effect, because the decaying learning rate makes the final mix the model’s settled memory. The anchors’ published recipes translate into a concrete design:
- Build the cyber anneal mix like Dolmino, not like a data dump. Roughly half high-quality general replay (elite-filtered web, documentation, Wikipedia-class reference) as the forgetting anchor. Then patch slices, upsampled hard: vulnerability write-ups and advisories, CVE records with their references, exploit and patch pairs, secure-code corpora, protocol and RFC text, vendor documentation, postmortems. Dolmino put its patch domain (math) at about a fifth of the mix and repeated its small elite sources two to four times over; a cyber mix should expect the same shape, with the scarcest, highest-signal sources (verified patch diffs, quality exploit write-ups) repeated rather than rationed. Licensing discipline from Part 2 applies unchanged: every slice needs a provenance and terms check before it enters the mix, and teacher-generated synthetic slices carry the teacher’s terms with them.
- Micro-anneal the sources before committing. Ai2’s method is directly portable: branch short anneals off one stable checkpoint, one per candidate cyber source, and measure each source’s effect on both cyber evals and the general suite. The LLM-jp replication matters here: it proved a published anneal mix transfers to an unrelated base, which means LAB2 can also benchmark candidate mixes against public ones (a Dolmino-style general anneal is the control every cyber mix must beat).
- Soup the anneal. Run the cyber anneal three times with different data orders and average. At Step A sizes this costs a few percent of the total budget, buys a consistent gain, and insures against one ordering over-fitting a fashionable vulnerability class. It also produces, as a byproduct, a variance estimate across runs, which Proof of Defense reporting can use.
- Keep an undecayed trunk. Hold the pre-anneal checkpoint at a stable learning rate (the WSD pattern) so LAB2 can branch multiple domain anneals from one base: a general cyber anneal, an OT-focused anneal, a cloud-focused anneal, without re-paying pretraining or inheriting a previous anneal’s dead end. Marin’s discarded cooldown is the cautionary tale; do not decay the only copy of your trunk.
- Plan context extension as its own stage, with replay. Cyber work is long-document work: full log windows, memory dumps summarized to text, whole repositories, multi-hour traces. A cyber base needs its context bought deliberately, staged like K2-Horizon’s ladder, with two-thirds general short data in the extension mix and the short-context suite rerun after, per OLMo 3’s discipline. An 8B Step A model does not need 512K; it needs a verified, regression-tested 128K more than it needs an advertised 1M.
- Measure forgetting in both directions. A cyber anneal can erode general instruction-readiness (test the general suite) and can shift refusal behavior before post-training ever runs (test the refusal probes at base level too). Part 8 handles refusal as a post-training decision, but the anneal sets the prior it starts from, and Part 9’s execution-verified evals are the instrument. Temporal discipline from Part 2 also binds here: anneal data and eval data must be split by time, or the cyber evals measure memorized CVEs instead of capability.
Cost realism, in agentic terms: at the 7B to 9B Step A band, an Ai2-style program (a 100B-token-class anneal, souped three ways, plus a 50B-token-class extension) is a few hundred billion tokens of training on hardware LAB2 already targets, and the binding constraint is corpus curation and licensing review, not compute. That is the stage’s final lesson: midtraining is where curation effort converts to capability at the best exchange rate in the pipeline, and it is the stage the closed labs are least able to keep you from copying, because the open families published the recipes.
Flagged and unverified items
- K2-Horizon mid-training internals. The stage table (token counts, contexts, the Stage 4 mix shift) comes from the flagship’s model card as compiled in the roster. The tech report (arXiv:2611.21003) presumably specifies learning-rate schedules and per-stage mixes; those details were not extracted for this part, and the positional-encoding method used for its extension ladder is not stated in the card summary. Do not cite K2-Horizon’s extension method until the report is read directly.
- Granite 4.0 anneal shares. The four-stage curriculum and its two decay stages are described on the model cards; mixture proportions for the decay stages are not published, and the staged token totals used in the cost table come from the 4.0 Micro card’s stage table as compiled in Part 2 of this masterclass.
- Nemotron 3 Ultra WSD claim. The statement that Ultra trains with a WSD schedule over a 20T horizon rests on secondary summaries of the report, not a direct read for this part.
- Nemotron 3 Super Phase 2 composition. The 20T/5T phase split is from the roster’s card and report summaries; the high-quality phase’s internal mix is described at corpus-family level only.
- Modality grafting discipline. Freeze schedules and joint-training details for the Nano Omni towers and Apertus 1.5’s multimodal stages were not available at the level this part requires; budgets only.
- OLMo 3 32B final merges. Secondary coverage variously describes the 32B’s post-midtraining merges; this part relies on the report-based account (two Dolmino runs averaged). A claim in some coverage of additional final-checkpoint merges was not verified against the report and is not used.
Sources for this piece
Ai2, OLMo 2 tech report (arXiv:2501.00656), including the mid-training deep dive, Dolmino Mix 1124 composition tables, micro-annealing, and checkpoint soup results; Ai2, OLMo 3 tech report (arXiv:2512.13961), Dolmino and Longmino stage budgets and mix proportions. LLM-jp, “Mid-Training LLM-jp on OLMo2 Data” (2025), independent replication including soup results. Wortsman et al., “Model soups: averaging weights of multiple fine-tuned models improves accuracy without increasing inference time” (arXiv:2203.05482). Hu et al., MiniCPM (2024), WSD schedule; Hägele et al. (2024), cooldown equivalence; WSM, “Decay-free learning rate schedule via checkpoint merging” (arXiv:2507.17634). Hugging Face, SmolLM3 release post and model cards (stage mixes, extension recipe, APO merge); SmolLM2 paper (arXiv:2502.02737). IFM K2 Team, K2-Horizon model card stage table and tech report (arXiv:2611.21003). NVIDIA Nemotron 3 Super and 3.5 Super VL model cards and roster compilations (phase splits, Indic continued pretraining, SWE mid-training, extension budget). IBM Granite 4.0 model cards (four-stage curriculum). Marin 8B retrospective (cooldown discard and rewarm). Roster files: research/nemotron-roster.md, research/k2-horizon-roster.md, research/open-families-roster.md.