Amazon Introduces Self-Distilled Reasoning to Enhance AI Model Fine-Tuning and Combat Catastrophic Forgetting

In a significant advancement for artificial intelligence model customization, Amazon has unveiled Self-Distilled Reasoning (SDR), a novel technique designed to address critical challenges in fine-tuning large language models (LLMs) such as the Amazon Nova 2 family. This innovative approach aims to make high-quality Chain-of-Thought (CoT) reasoning traces more accessible and cost-effective for Supervised Fine-Tuning (SFT), while simultaneously mitigating the pervasive problem of catastrophic forgetting, where models lose previously acquired general capabilities during specialized training.

The Intricacies of Fine-Tuning and the "Reasoning Suppression Problem"

Supervised Fine-Tuning (SFT) is a common method for adapting pre-trained LLMs to specific tasks or domains. It involves training the model on a dataset of input-output pairs. A key capability that has been shown to significantly enhance prediction performance in frontier models like Amazon Nova 2 is "reasoning," often manifested through Chain-of-Thought (CoT) traces. CoT involves the model generating intermediate reasoning steps before arriving at a final answer, mimicking human problem-solving. This process has demonstrated substantial performance gains, particularly on complex problems in areas like coding and mathematics.

However, a major hurdle for developers has been the practical impossibility and prohibitive cost of creating extensive, high-quality CoT reasoning traces for large SFT datasets. Consequently, many developers opt to bypass reasoning during SFT, training models solely on input-output pairs. This expediency, while initially saving resources, leads to a critical issue dubbed the "reasoning suppression problem." When SFT is conducted on datasets lacking reasoning traces, and especially when reasoning mode is active during training, the model can lose its inherent ability to reason during inference, even when explicitly prompted to do so. Researchers hypothesize this occurs because the loss function, which calculates errors on both reasoning and output tokens, penalizes the generation of coherent reasoning steps when only direct outputs are provided. This effectively trains the model to take "shortcuts," bypassing its robust reasoning mechanisms and favoring superficial correlations over deeper problem-solving patterns.

Catastrophic Forgetting: A Pervasive Challenge

Beyond reasoning suppression, fine-tuning without careful consideration of general capabilities often leads to "catastrophic forgetting." This phenomenon describes the tendency of neural networks to forget previously learned information upon learning new information. In the context of LLMs, a model fine-tuned on a specific domain might excel at its new task but drastically lose proficiency in foundational skills like mathematics, coding, or general knowledge that it possessed prior to fine-tuning. For instance, vanilla SFT has been observed to cause a model’s Math performance to plummet from an impressive 70 percent (base model) to a mere 6 percent on average, highlighting the severity of this issue.

Model Merging: A Traditional, Yet Imperfect, Solution

Historically, model merging has been a common strategy to counteract catastrophic forgetting. This technique involves creating a weighted interpolation between the fine-tuned model checkpoint and the original base model. By blending the two, developers aim to recover general skills that might have been lost during domain-specific fine-tuning. While model merging can indeed restore capabilities such as mathematical or coding proficiency, it often presents a trade-off: preserving general performance can come at the expense of forfeiting some of the gains achieved on the specific SFT target task. Furthermore, determining the optimal merging weight requires careful tuning and adds a post-hoc interpolation step to the training pipeline, complicating the overall workflow.

Amazon Nova 2 and the Need for Refined Customization

The Amazon Nova 2 family of models, known for its advanced reasoning capabilities, underscores the importance of effective customization. These models are designed to excel in complex cognitive tasks, and their full potential on specific domains can only be unlocked through sophisticated fine-tuning techniques like SFT and Reinforcement Fine-Tuning (RFT). For SFT, the availability of "golden CoT traces"—validated and cleaned thinking traces from a strong teacher model—has been a key requirement to realize performance gains. The absence of such traces has presented a significant bottleneck for developers aiming to leverage Nova 2’s power on their unique datasets.

Self-Distilled Reasoning (SDR): A Paradigm Shift

Amazon’s Self-Distilled Reasoning (SDR) emerges as a practical and highly effective solution to these intertwined problems. SDR re-uses the Chain-of-Thought from the base Amazon Nova 2 Lite model itself as a substitute for explicitly annotated reasoning datasets. This innovative approach integrates self-distillation principles, where a model learns from its own predictions, particularly its problem-solving process, rather than just final outputs.

The SDR process unfolds in three key stages:

1. Trace Generation: The base reasoning model (e.g., Amazon Nova 2 Lite) is used to generate reasoning traces for each example in the target SFT dataset. This can be done using either "basic reasoning" (model naturally generates reasoning from the question) or "guided reasoning" (model reconstructs plausible reasoning given both the question and the ground-truth answer).
2. Data Augmentation: The generated reasoning traces are then prepended to the original outputs in the training data. This effectively creates a new, augmented dataset that includes intermediate thought processes.
3. Fine-Tuning: The model is then fine-tuned with reasoning mode explicitly enabled, providing supervision on both the reasoning and output tokens.

This methodology offers several compelling advantages: it incurs zero annotation cost, as no human effort is required for trace creation; the generated reasoning traces are inherently consistent with the base model’s own problem-solving approach; the technique is scalable and applicable to existing SFT datasets across diverse domains; and it provides flexibility to experiment with different teacher models for trace generation.

The Foundational Role of Reasoning in SFT

The efficacy of SDR underscores the multifaceted role of reasoning traces in SFT, connecting it to broader paradigms in AI training:

• Implicit Policy Regularization: Reasoning traces act as an implicit form of regularization, compelling the fine-tuned model to maintain proximity to the base model’s underlying policy. This mechanism helps prevent catastrophic forgetting and capability degradation, akin to KL-regularization in Reinforcement Learning from Human Feedback (RLHF).
• Process Supervision: Unlike traditional outcome supervision (where only the final answer is judged), reasoning provides process supervision. By observing step-by-step thinking, the model gains a deeper understanding of problem-solving, leading to more robust generalization, similar to reward shaping in RFT.
• Self-Distillation: As a form of self-distillation, SDR leverages the model’s own internal representations and problem-solving processes as a rich source of self-supervision. This aligns with recent research advocating self-distillation as a powerful mechanism for continual learning and mitigating forgetting.

By applying the base model’s own reasoning traces as training targets, SDR creates a learning signal that is consistent with the model’s internal representations while providing dense, token-level supervision throughout the entire reasoning process.

Experimental Validation and Key Findings

Amazon validated SDR across a suite of benchmarks, including MedMCQA (medical question answering), CoCoHD (long-context structured extraction), and Invoice-OCR (multimodal document understanding), none of which originally contained reasoning traces. An additional ablation study was conducted on the LLaVA CoT benchmark to investigate scenarios with partial reasoning data.

The experimental results consistently demonstrated the superior performance of SDR:

• Combatting Catastrophic Forgetting: SDR proved highly effective at retaining general capabilities. For instance, on the Math (control) benchmark, vanilla SFT caused Math accuracy to drop from 70 percent (base model) to an average of 6 percent. With SDR, this performance was almost entirely recovered, maintaining Math accuracy at or above 65-70 percent across various scenarios. This stark contrast highlights SDR’s strength in preserving foundational skills without the complexities of model merging.
• Improving Target Performance: Beyond retention, SDR often led to improved target performance. In several cases, SDR achieved similar or even better target performance than pure SFT (without merging) or model merging. For example, on MedMCQA, SDR lifted target accuracy by approximately three percentage points (from 63.8 percent to 66.6 percent) while recovering Math from 0 percent to 67.9 percent. Across various benchmarks, SDR delivered an average relative gain of over 6.5 percent in target performance compared to the best model-merging checkpoints.
• Enhanced Reasoning Behavior: The ablation study on partial reasoning datasets provided further insights. Without SDR prefill, reducing reasoning data below 75 percent caused catastrophic forgetting on the Math control benchmark, with accuracy plummeting from 66.6 percent to as low as 2.5 percent at 0 percent reasoning data. Crucially, models trained with little to no reasoning data and no SDR produced zero reasoning tokens at inference, indicating a complete loss of step-by-step reasoning ability. In contrast, SDR prefill ensured that Math control remained stable (64-72 percent) and models consistently produced a significant number of reasoning tokens (e.g., 859 median tokens even with 0 percent original reasoning data), demonstrating that SDR effectively teaches and preserves reasoning behavior.
• Impact on Inference Performance: Enabling reasoning at inference time consistently delivered significant performance boosts, often doubling accuracy on benchmarks like MathVista (from 14-22 percent to 35-47 percent), regardless of training data composition. Interestingly, training with SDR prefill also improved evaluation performance even when reasoning was disabled during inference, suggesting that SDR instills a more robust problem-solving approach within the model.

Practical Implications and Recommendations for Practitioners

These findings carry significant implications for developers working with customized AI models. SDR offers a zero-cost approach to simultaneously improve target performance and preserve general capabilities, eliminating the need for complex, post-hoc model merging and its associated trade-offs.

Amazon provides a clear decision guide for practitioners:

• Default SFT Jobs: For new SFT jobs without reasoning data where general performance regression is observed, SDR with Nova 2 Lite-generated traces is recommended for the best balance of target performance and general capability preservation, especially if latency constraints accommodate reasoning during inference. If latency is critical, model merging remains an option for recovery.
• Weak Base Model: When the base model struggles on the target domain, guided reasoning during trace generation can provide additional signal.
• Partial Reasoning Datasets: For datasets with less than 50 percent reasoning traces, pre-filling missing traces with Nova 2 Lite reasoning (a one-time, offline process) is crucial to prevent general performance collapse. For datasets with 50 percent or more reasoning, training with reasoning turned on is generally sufficient.
• No Need for General Capabilities: If preserving general capabilities is not a priority, standard SFT without reasoning data can be acceptable, as reasoning data has minimal impact on target performance in such cases.
• Continual Learning and Multi-Skill Preservation: SDR is particularly valuable for continual learning settings, where new skills are acquired over time, and previously learned skills must be preserved. Combining SDR with data mixing and a small (or zero) merge weight is a promising avenue for optimal retention.
• Using Model Merging with SDR: If model merging is still desired, a smaller merge weight is recommended when SDR is applied, as SDR already provides the regularization that merging typically compensates for.

Considerations for Teacher Model Selection

An important caveat emerged regarding the choice of teacher model for generating traces: a stronger teacher isn’t always better. Experiments showed that Amazon Nova 2 Pro traces, while improving target accuracy, could cause a proportional drop in general performance. This aligns with prior research indicating that a large student-teacher gap can negatively impact distillation. Consequently, Lite-generated traces (e.g., Amazon Nova 2 Lite for Nova 2 Lite customers) offer the most balanced outcome across target and general performance and are recommended as the default. Stronger-teacher or guided-reasoning variants are best reserved for specific cases where the base model is weak on the target domain and the potential cost to general capabilities is acceptable.

Conclusion: A Leap Forward for Accessible and Robust AI Customization

The introduction of Self-Distilled Reasoning by Amazon marks a significant step forward in making advanced AI model customization more accessible, cost-effective, and robust. By offering a zero-cost method to embed critical reasoning capabilities and prevent catastrophic forgetting, SDR empowers developers to fully harness the power of models like Amazon Nova 2 on their specific domains. This innovation not only streamlines the fine-tuning workflow but also contributes to the broader field of continual learning, ensuring that AI models can adapt and grow without sacrificing the valuable knowledge they’ve already acquired. As AI applications become increasingly specialized, techniques like SDR will be instrumental in bridging the gap between general intelligence and domain-specific expertise, fostering a new era of adaptable and resilient AI systems.