Part 7 · Part 7: Post-Training I, SFT and the Shape of Behavior
Model-Making Masterclass, Part 7. Written Nov 17, 2026. Engineer voice: plain language, terms defined when they first matter, every load-bearing number tied to a tech report, model card, or dataset card, or flagged as unverified.
What SFT is, mechanically and honestly
Supervised fine-tuning is the simplest stage in this entire study. Take the base model, show it rendered conversations (system prompt, user turn, assistant turn, in the chat template the model will see at deployment), and run the same next-token cross-entropy loss used in pretraining, almost always with the prompt tokens masked out so the loss lands only on the assistant’s words. Nothing about the machinery is new. Everything about the effect is disproportionate: a stage that consumes well under one percent of the total training tokens is the stage that decides whether the model acts like an assistant at all.
That disproportion is the first thing to understand, because it explains both SFT’s power and its limits. Pretraining and midtraining (Parts 5 and 6) built a distribution over text. SFT does not rebuild that distribution. It selects from it: it raises the probability of the behaviors the examples demonstrate (answer the question, use the tool format, reason before answering, refuse this category, keep it short) until those behaviors become the default sample. The open families in this study publish enough to test that claim from several directions, and this part reads SFT the way an engineer should: as a data problem with a thin training wrapper, where the mix is the design and the hyperparameters are plumbing.
What SFT teaches, and what it cannot
What it teaches reliably: format, persona, and task shape. The canonical evidence is old and still load-bearing. The LIMA result (Zhou et al., 2023) fine-tuned a 65B base on 1,000 carefully curated examples and produced a model competitive with far more heavily aligned systems on human preference, which is the origin of the superficial alignment hypothesis: almost everything the model knows comes from pretraining, and alignment mostly teaches style and format. Every family in this study behaves consistently with the weak version of that claim. A few tens of thousands of good examples are enough to install a chat persona, a tool-calling syntax, an answer structure. SmolLM2’s small models reached usable instruction following on a reduced variant of SmolTalk. Apertus shipped a compliant, multilingual assistant whose SFT stage is a small fraction of its 15T-token life. Format is cheap. That is a measured fact across at least four independent pipelines, not a slogan.
What it teaches partially: elicited capability. The strong version of superficial alignment (SFT can only restyle what pretraining already built) does not survive the open record. A 2025 re-examination (“Revisiting the Superficial Alignment Hypothesis,” arXiv:2505.06084 in this part’s sources as the OpenReview version) found post-training performance scaling as a power law in the number of fine-tuning examples across math, coding, and multihop reasoning: a handful of examples aligns style, but performance keeps climbing with data volume, and post-training measurably improves the model’s ability to integrate knowledge it already has. The roster’s cleanest exhibit is NVIDIA’s Qwen specialist pair at 7B: AceReason-Nemotron (SFT on DeepSeek-R1 traces, then GRPO) against OpenReasoning-Nemotron (SFT only, on a large curated reasoning set, no RL at all). The SFT-only model is genuinely strong, which is the elicitation point: curated teacher traces at scale moved real benchmark capability, not just formatting. Tulu 3 makes the same point from the other direction: Ai2 built its SFT mixture skill by skill (math, code, instruction following, chat, knowledge) and iterated the mix against a development suite, and the published stage checkpoints let anyone verify that the SFT stage, before any preference training or RL, already delivers most of the final model’s benchmark profile.
What it cannot teach: a new ceiling. Three boundaries recur. First, the teacher boundary: SFT on distilled traces caps the student near the teacher’s demonstrated behavior on that data. Llama-Nemotron’s report is explicit that its large-scale RL stage exists because SFT alone leaves the student at or below the teacher (DeepSeek-R1), and the RL stage is what let the flagship surpass it. SFT transfers reasoning behavior; it does not originate it. Second, the base boundary: SFT cannot install knowledge or skills the base never formed. Nemotron-Terminal is the narrow, honest version of this: SFT on synthetic terminal-task data produced large Terminal-Bench gains at 8B, because the base already had the underlying code and shell competence and the data taught the task shape. The same recipe on a base without that competence would teach the shape of an answer the model cannot fill. Third, the distribution boundary: SFT teaches the behaviors present in the mix, at the mix’s proportions, including the ones you did not intend. Every failure mode in this part’s last technical section is a mix proportion expressing itself.
The working summary for a builder: SFT is a behavior selector and a capability elicitor, priced in data curation, and its ceiling is set jointly by the base model below it and the teachers above it. Budget accordingly: most of the money in this stage is data money.
Data construction: four supply chains
Open SFT sets are built from four supply chains, and every family’s mixture is a blend of them. The families differ in the blend, the scale, and the quality gates, and those differences are the design.
Supply chain 1: human-written and human-collected. Real user prompts (WildChat is the standard source), human annotation projects (OpenAssistant), and legacy task collections (FLAN). This is the smallest and most expensive chain, and it survives mainly as prompt supply: the prompts are real even when the responses are regenerated. Tulu 3’s mixture is about 57 percent drawn from public datasets of this kind by prompt count. Its value is distribution honesty: real users ask messier questions than any generator invents.
Supply chain 2: teacher distillation. Prompts from anywhere, responses generated by a stronger model, filtered, and trained on. This is the volume chain of the current era. NVIDIA’s Llama-Nemotron post-training dataset is the industrial exhibit: roughly 30M samples in v1.1, with math at about 22.1M samples generated by sampling many responses per prompt from teacher models (about 1M unique math prompts underneath, so the design is depth per prompt, not breadth), code at about 10.1M, science at about 709K, and then the long tail that reveals intent: instruction following 56K, chat 40K, safety 31K. The successor set for Nemotron 3 (Nemotron-Post-Training-Dataset-v1, about 25.7M samples) rebalances toward STEM at about 20.7M with tool calling as a named category at about 310K. Read those proportions as a statement about what SFT is for in NVIDIA’s recipe: it is primarily a reasoning-transfer mechanism, and chat is a rounding error. AceReason and OpenReasoning are this chain pointed at one skill: R1 traces, curated, at 7B to 32B scale.
Supply chain 3: persona-driven synthesis. Instead of collecting prompts, generate them from a space of invented users. Tulu 3’s synthetic share (about 43 percent of its 939,344 prompts) was generated against roughly 250,000 personas, so the prompt distribution covers occupations, expertise levels, and intents that real-user logs underrepresent. SmolTalk’s task-specific sets use the same trick at smaller scale: Smol-Summarize and Smol-Rewrite source texts were generated by prompting a teacher with system prompts and personas from PersonaHub, then the same teacher produced the summaries and rewrites. Persona synthesis is how you buy coverage of the tail: the tasks nobody happened to type into a chatbot last year.
Supply chain 4: self-synthesis. The teacher generates both sides with minimal seeding. The Magpie method prompts an aligned model with only the template tokens that precede a user turn and lets it hallucinate the instruction, then generate the response. SmolTalk’s core, smol-magpie-ultra (400K conversations in the shipped mix, filtered down from about 1M generated three-turn conversations from Llama-3.1-405B-Instruct), is this chain, and Hugging Face’s ablation result is the reason it matters: models trained on smol-magpie-ultra alone beat models trained on the popular public sets of the day (OpenHermes 2.5, Magpie-Pro) on IFEval and MT-Bench. Self-synthesis is the cheapest chain per sample and the one most sensitive to filtering, which is why SmolTalk’s gates are as instructive as its data: reward-model scoring (ArmoRM) for quality, embedding-based semantic dedup, and n-gram decontamination of the constraint subset against IFEval.
The mixtures, side by side
| Family / set | Scale | Dominant chains | Notes |
|---|---|---|---|
| Tulu 3 SFT mixture | 939,344 prompts | Public sets (57%), persona synthesis (43%) | Skill-targeted subsets, iterated against a dev suite; 8-gram decontamination, whole datasets dropped on failure |
| Dolci (OLMo 3) Think SFT | About 2.25M rows | Distillation (OpenThoughts3-style reasoning corpora) | Later cleaning passes removed about 203K rows (9 percent) including 148K self-contradictory math problems found by a model screen |
| SmolTalk (SmolLM2) | About 1.1M samples | Self-synthesis (Magpie-Ultra), persona task sets, public subsets | smol-magpie-ultra 400K, smol-summarize 100K, smol-rewrite 50K, smol-constraints 36K, OpenHermes 100K, plus everyday conversations |
| SmolTalk2 (SmolLM3) | 23 weighted subsets | Distillation plus self-synthesis, split by mode | Separate think and no-think subsets; the split is the design (see template section) |
| Llama-Nemotron PTD v1.1 | About 30M samples | Teacher distillation, rejection sampling | Math about 22.1M from about 1M unique prompts; chat 40K; safety 31K |
| Nemotron PTD v1 (Nemotron 3) | About 25.7M samples | Teacher distillation | STEM about 20.7M; tool calling a named 310K category |
| Nemotron 3 Super SFT (recipe docs) | About 7M samples, 15+ domains | Distillation plus targeted synthesis | Safety slice uses deliberative alignment reasoning traces; low-effort reasoning slice at 2 percent |
| K2-Horizon flagship SFT | 332B tokens across three phases | Distillation plus agentic synthesis (report level) | Phases priced in the next section; per-phase mixes not extracted in this pass |
| Apertus post-training | Not extracted in this pass (flagged) | Public sets, license-filtered, decontaminated | Alignment data split into standard topics and controversial topics as separate artifacts |
Two quality-gate patterns deserve names because LAB2 will reuse both. Verify, then train: wherever a checker exists, the open recipes use it before SFT, not after. NVIDIA’s math depth comes from rejection sampling, which is verification by regeneration: keep the responses that reach the known answer. The agentic version of the same gate is trajectory verification: generate tool-use trajectories, execute them, and keep only the ones the environment confirms; Nemotron-Terminal’s task corpus and K2-Horizon’s agentic SFT data are the roster’s published exhibits of the pattern. Dolci’s cleaning screens delete problems a strong model judges self-contradictory, and Ai2 published the controlled experiment on that deletion (a pooled MATH gain of 1.69 points, confidence interval excluding zero but below their pre-registered margin, verdict honestly recorded as inconclusive). Decontaminate or drop: Tulu 3 ran 8-gram overlap checks against its eval suite and dropped or filtered whole datasets that failed; the worked example, 11.3 percent of NuminaMath-TIR overlapping MATH test items, is in Part 2. A gate you cannot see is a gate that does not exist; the recipes above teach because their gates are published artifacts, not report adjectives.
Multi-phase SFT: why one pass is not the shape anymore
The textbook pipeline runs SFT once, on one mixture. Three of this study’s families no longer do that, and their reasons are distinct enough to enumerate.
K2-Horizon: SFT after RL, in three phases, at pretraining-like scale. The flagship’s order is inverted relative to the textbook: RL first (five expert models trained from the final mid-training checkpoint, then merged), SFT second, on the merged checkpoint. The SFT then runs in three phases: Phase 1 at 2,400 steps and 81B tokens, Phase 2 at 6,000 steps and 201B tokens, Phase 3 at 1,500 steps and 50B tokens on a high-quality subset with learning-rate decay. Total, about 332B tokens of SFT, which is larger than many families’ entire midtraining. The phase logic, at the level the card and launch coverage support, mirrors the anneal logic of Part 6: broad coverage first, then a final short phase on the best material at a dying learning rate, so the last thing the model settles on is the highest-quality behavior. The report also tracks a failure metric across the phases that most labs never publish: reasoning repetition (the doom-loop failure, where the model repeats an action or a command) falls to about 3.8 percent by the end of SFT Phase 1, and at the released checkpoint an environment returning unchanged terminal output no longer triggers the measured repeat failure. SFT phases, in this design, are partly behavior repair stages with their own acceptance metrics. Per-phase mixture compositions were not extracted for this part and are flagged below.
Llama-Nemotron: SFT as one stage in a longer argument. NVIDIA’s sequence for the reasoning line is architecture search, recovery training (distillation plus continued pretraining), then SFT, then large-scale RL. The SFT stage’s distinctive job is installing the reasoning toggle: the dataset pairs reasoning responses (system instruction “detailed thinking on”) with direct responses (“detailed thinking off”) so one checkpoint carries both behaviors, selected at inference by a lightweight system prompt rather than by a separate model. The staging lesson is about division of labor: recovery training restores what the architecture surgery cost, SFT installs the behavior contract, RL pushes past the teacher. Each stage has one job and the report can therefore attribute gains stage by stage, the same discipline Ai2’s per-stage checkpoints enforce mechanically.
Nemotron 3 Super: SFT, then a semi-on-policy polish for a control surface. NVIDIA’s published SFT recipe documentation adds a stage most recipes do not have: after the main SFT (about 7M samples across 15+ domains, assistant-only loss masking, packed sequences), a short semi-on-policy SFT stage of 350 steps collects rollouts from the current checkpoint and fine-tunes reasoning-budget control, truncating about 12 percent of reasoning traces to random budgets so the model learns to reason to a specified length. The generalizable point: once the base behavior is installed, small on-policy SFT passes are how you calibrate a control (here, effort) without reopening the whole recipe. SmolLM3’s ending rhymes: its post-training closes with a weight-space merge (0.9 of the preference-trained checkpoint, 0.1 of the midtraining checkpoint) to repair long-context behavior the alignment stage damaged, covered in Part 6. Phases exist because each stage breaks something the next stage is designed to fix, and the open recipes have started publishing the breakage instead of hiding it.
The contrast case: Tulu 3’s single mixture. Ai2 runs one SFT stage on one 939K-prompt mixture, two epochs, and gets a fully competitive instruct model. The difference is intent: Tulu’s mixture was iterated as a whole against the development suite before training, so the phasing happened in data space (mix, measure, remix) rather than in training space. Both patterns work. Phased training buys you stage-level attribution and repair points; a single iterated mixture buys you simplicity and a cleaner ablation story. Pick based on which kind of debugging your team is better at.
The chat template is a behavior surface
Part 3 covered tokenizers and vocabulary; this section covers only what SFT adds: the template and its control tokens are trained behavior, not formatting trivia.
Three designs from the roster span the space. Plain-text control: Llama-Nemotron’s toggle is an ordinary system-prompt sentence (“detailed thinking on”), no special token, which makes the behavior portable across tokenizers and legible to users, at the cost of spending prompt tokens and relying on instruction following to hold the mode. Template-level modes: SmolLM3’s chat template carries /think and /no_think switches, and SmolTalk2 is physically split into think and no-think subsets so the two modes are trained as two data distributions sharing one checkpoint. Token-level control: Nemotron 3 renders conversations through a Jinja template with explicit special tokens (the im_start and im_end family) and role-based structure, and its recipe handles a subtlety most templates ignore: in multi-turn conversations, prior turns’ reasoning content is dropped when a new user message arrives, matching what the user actually sees at inference, so the model is never trained on context it will not have. Apertus publishes its chatbot system prompt in the tech report as part of the artifact, which is the right instinct: the default system prompt is part of the trained behavior contract and should be versioned like code.
The rule that falls out: whatever control surface you choose, it must be trained, at scale, in both positions. A toggle that appears in 2 percent of SFT data is a suggestion; Nemotron 3’s low-effort reasoning slice sits at exactly that share and is backed by the dedicated budget-control stage described above, which is the honest way to make a small share hold.
Mechanics: packing, masking, epochs, rates
The plumbing is boring and consequential, and the open recipes publish enough of it to copy.
Masking. The near-universal choice is completion-only loss: system and user tokens get mask 0, assistant tokens get mask 1. NVIDIA’s Nemotron 3 recipe documentation states it in a table (system ignored, user ignored, assistant learned) and gives the reason plainly: the model should learn to respond, not to predict prompts. Training on the full sequence spends capacity modeling the user distribution and, worse, teaches the model to generate user turns. Where full-sequence loss still appears is in continued-pretraining-flavored SFT and in some small-model recipes; treat it as a deliberate exception, not a default.
Loss normalization is a real design decision. Two subtleties hide inside “average the loss.” Tulu 3’s open-instruct sums token losses (per the model cards’ “Loss Accumulation: Sum” note), which weights long responses more. NVIDIA’s Nemotron 3 Super recipe makes the choice explicit and two-stage: Stage 1 averages over all output tokens in the packed global batch (token-level), Stage 2 switches to a per-conversation normalized loss averaged equally across conversations (sample-level), specifically to reduce the dominance of long outputs. If your SFT data mixes short chat with very long reasoning traces, this choice quietly decides which behavior wins. It is the kind of detail that never appears in a launch post and absolutely appears in outcomes.
Packing. Production SFT packs multiple conversations into one sequence up to the context limit for throughput, with masks preventing cross-conversation attention leakage in correct implementations. Nemotron 3’s recipe packs; open-instruct historically trains at a fixed 4,096 maximum length. The interaction to watch: packing plus token-level loss averaging is what makes long-trace dominance a problem in the first place.
Epochs and rates. Tulu 3’s published SFT hyperparameters are the reference point: 2 epochs, effective batch size 128, maximum sequence length 4,096, linear schedule with 3 percent warmup, learning rate 5e-6 at 8B and 2e-6 at 70B and 405B. Note how low those rates are relative to community fine-tuning defaults: SFT at production scale is a nudge, not a rewrite, and the rate falls as the model grows. K2-Horizon’s phases run at 512K context, an order of magnitude beyond Tulu’s 4K, because its SFT data is agentic trajectories rather than chat; sequence length in SFT is a data property before it is a training property.
Published failure modes
The open families document SFT going wrong in four recurring ways. Each is a mix or mechanics decision expressing itself, which means each is fixable at this stage.
Format overfitting and the alignment tax on diversity. Train hard on one response shape and the model applies it everywhere: reasoning traces on questions that wanted one line, headers and bullet lists in casual chat, refusal phrasing leaking into benign answers. The LIMA-era finding cuts both ways: if a thousand examples can install a persona, a skewed million can install a tic. The mitigation visible in the mixtures is deliberate breadth in the tail categories (chat, everyday conversation, rewriting) even when they contribute nothing to benchmark scores. NVIDIA’s 40K chat samples inside a 30M set look decorative; they are ballast.
Verbosity drift. Teacher traces are long, rejection sampling prefers complete answers, and reward-flavored filtering prefers thorough ones, so SFT data drifts long and the model follows. The symptoms are measured in the open record as response-length growth across post-training stages, and two of this study’s recipes engineer against it directly: Nemotron 3’s sample-level loss stage exists to stop long outputs dominating the gradient, and its budget-control SFT teaches length as a controllable variable rather than a habit. Tulu’s answer comes one stage later (length-normalized DPO, Part 8), which is itself an admission: verbosity installed at SFT is expensive to remove downstream. Control length in the data (cap trace lengths, include concise exemplars for concise tasks) or pay for it in every later stage.
Repetition and doom loops. Agentic SFT data carries a specific failure: trajectories where the agent repeats an action, a command, or a reasoning step, either present in the teacher data or induced by training on long self-similar traces. K2-Horizon is the family that publishes the metric: repetition tracked across SFT phases (down to about 3.8 percent after Phase 1), and on the small models a dedicated preference method (FTPO, Final Token Preference Optimization) that the 3.7B card credits with breaking doom loops. The lesson generalizes beyond agents: any behavior your data demonstrates repeatedly, including its failure shapes, is a behavior you are teaching. Filter trajectories for loops before training, not after deployment.
Refusal overreach from safety shares. Safety SFT data is a tiny share by count (31K of about 30M in NVIDIA’s v1.1 set, roughly a tenth of a percent) with an outsized behavioral footprint, because refusal is a high-salience, low-diversity behavior: a few thousand near-identical examples teach a trigger, and the trigger generalizes past its intent. The failure signature is over-refusal on benign adjacent prompts, the class of error over-safety suites (XSTest and successors) exist to measure. Apertus handles the boundary explicitly by splitting alignment data into standard topics and controversial topics as separate artifacts with their own construction, including a published ideological sensitivity classifier in the appendix. Nemotron 3’s safety slice uses deliberative alignment traces (the model reasons about the policy before answering) rather than bare refusals, which teaches a decision procedure instead of a reflex. Both are attempts at the same goal: make the safety behavior discriminative, not volumetric. The share is a dial, and the open record says to set it by measuring over-refusal, not by vibes.
The cyber read: SFT is where a cyber model gets its working habits
For LAB2, SFT is the stage where the cyber base built in Part 6’s anneal becomes a cyber worker. The anchors’ lessons translate directly.
Build the taxonomy before the data. Tulu’s method, skill-targeted subsets iterated against a development suite, is the template. For a cyber model the skills are the gym and DeepThreat task families: triage (is this finding real), static analysis explanation, patch drafting, patch verification support, log and alert summarization, detection-rule authoring, threat-intel extraction, and tool-driven investigation. Each gets its own subset, its own dev evals, and its own share decision, so a weak skill is a mix bug you can fix rather than a model flaw you lament. The taxonomy is also where Proof of Defense plugs in: the task families the fleet is scored on should be the task families SFT is built from, with the eval instances held out by construction.
Worked examples must carry tool traces, not just answers. The pattern to copy is the verified agentic pipeline the open recipes demonstrate (trajectory verification from the quality-gate section above, with Nemotron-Terminal’s task corpus as the roster’s closest cyber-shaped exhibit): define the tool specs (scanner outputs, sandbox execution, code search, patch application), generate tasks against them, execute trajectories in the real environment (the cyber range is exactly this environment), and keep only trajectories that verify. A cyber SFT example should read the way the work actually happens: alert or code in, tool calls with real outputs interleaved, a conclusion the environment confirmed. Final-answer-only examples teach the model to sound like an analyst. Trace examples teach it to work like one, and they double as the substrate Part 8’s RL will improve.
Verify, then train, with the verifiers you already own. The cyber domain is unusually rich in checkers: the exploit either runs or it does not, the patch either closes the hole under re-test or it does not, the detection rule either fires on the malicious sample and stays silent on the benign corpus or it does not. Rejection sampling against those checkers is the highest-leverage data construction available to LAB2, the direct analogue of NVIDIA sampling twenty responses per math prompt and keeping the correct ones. Temporal discipline from Part 2 binds throughout: SFT examples built from CVEs must be split from eval CVEs by time, or the scores measure memorization.
Refusal calibration by vulnerability class is an SFT data decision. This is the cyber-specific version of the safety-share problem, and it is sharper here because the classes are concrete. The model must comply with defensive work on every class, comply with dual-use analysis under the fleet’s operating rules, and the calibration target is a per-class compliance profile, not a global refusal rate. That profile is set in this stage, by the composition of the safety and boundary subsets: paired examples per vulnerability class (the same technique at defensive, analytical, and offensive framings, with the intended behavior demonstrated for each), deliberative traces that reason about context and authorization before answering in Nemotron 3’s style, and an over-refusal eval per class run against every SFT candidate. Part 1 set the release target (a clean license chain, stage checkpoints so refusal behavior is diffable); this stage is where the behavior being diffed is actually written. Get the mix wrong here and Part 8’s RL will spend itself fighting SFT’s reflexes.
Keep the phases. At Step A scale, LAB2 does not need K2-Horizon’s 332B tokens of SFT. It needs the shape: a broad first pass over the full task taxonomy, a second pass weighted toward verified agentic traces, and a short final pass on the highest-quality subset at a decaying rate, with per-stage checkpoints and the refusal and repetition metrics reported per phase, K2-Horizon style. The stage is cheap. The curation is the cost, exactly as Part 6 said about the anneal.
Flagged and unverified items
- K2-Horizon SFT phase purposes. Token budgets and step counts per phase are from the flagship’s model card as compiled in the roster. The characterization of Phase 3 as a high-quality polish phase follows the card’s own description (high-quality subset, learning-rate decay). Per-phase mixture compositions were not extracted from the tech report (arXiv:2611.21003) for this part; the repetition figure (about 3.8 percent after Phase 1) rests on launch coverage of the report, not a direct read.
- Apertus SFT mixture size. The tech report’s post-training chapter structure (data overview, decontamination, SFT data, alignment data, format and chat template) is confirmed from the report itself; mixture sizes and SFT hyperparameters were not extracted in this pass.
- Dolci composition. The Think SFT row count (about 2.25M) and the cleaning-pass figures come from Ai2’s Dolci dataset cards, including the Clean-v2 card’s account of the Kimi K3 contradiction screen. The full Dolci mixture proportions by source were not extracted line by line.
- Nemotron 3 Super SFT details. The two-stage loss, the 7M-sample figure, and the 350-step budget-control stage come from NVIDIA’s published recipe documentation for the open reproduction recipe, which NVIDIA states uses a subset of the full training data; the released model’s exact SFT configuration may differ in detail.
- Llama-Nemotron toggle data shares. The dataset’s category totals are from the HF card and NVIDIA’s developer blog (v1.1 figures). The reasoning-on versus reasoning-off split per category is described qualitatively in the paper; exact per-category splits were not tabulated for this part.
- OpenReasoning-Nemotron dataset size. Described here as a large curated reasoning set without a hard count; published totals for the underlying GenSelect-style corpus were not verified in this pass.
Sources for this piece
Ai2, Tulu 3 tech report (arXiv:2411.15124) and Tulu 3 SFT model cards (hyperparameters, mixture, decontamination); Ai2, Dolci-Think-SFT and Dolci-Think-SFT-Clean-v2 dataset cards (row counts, cleaning screens, controlled deletion experiment); Hugging Face, SmolLM2 paper (arXiv:2502.02737, SmolTalk construction and ablations) and SmolTalk dataset materials (smol-magpie-ultra, smol-constraints, smol-rewrite, smol-summarize composition); Hugging Face, SmolLM3 release materials and SmolTalk2 (think and no-think subsets); NVIDIA, Llama-Nemotron paper (arXiv:2505.00949, SFT stage, reasoning toggle, stage attribution) and Llama-Nemotron-Post-Training-Dataset card plus developer blog (category totals, teacher generation, rejection sampling depth); NVIDIA, Nemotron-Post-Training-Dataset-v1 card (25.7M distribution); NVIDIA Nemotron 3 recipe documentation (assistant-only masking, two-stage SFT loss, 7M-sample SFT, semi-on-policy budget-control stage, multi-turn reasoning handling); IFM K2 Team, K2-Horizon flagship model card stage table (RL-then-SFT order, three SFT phases) and launch coverage of the tech report (arXiv:2611.21003) for repetition tracking and FTPO; Swiss AI Initiative, Apertus tech report (arXiv:2509.14233, post-training chapter structure, standard versus controversial alignment split); Zhou et al., LIMA (arXiv:2305.11206); Revisiting the Superficial Alignment Hypothesis (OpenReview, 2025). Roster files: research/nemotron-roster.md, research/k2-horizon-roster.md, research/open-families-roster.md.