Magistral-Small-2506 Overview
Description
Magistral-Small-2506 is a lightweight, general-purpose language model that generates and understands natural language for tasks like Q&A, summarization, and instruction following. Designed for efficiency, it balances performance with low computational overhead, making it suitable for real-world applications. Building upon Mistral Small 3.1 (2503), with added reasoning capabilities, undergoing SFT from Magistral Medium traces and RL on top, it's a small, efficient reasoning model with 24B parameters.
Magistral Small can be deployed locally, fitting within a single RTX 4090 or a 32GB RAM MacBook once quantized.
This model is for research and development only.
Learn more about Magistral in our blog post.
The model was presented in the paper Magistral.
- Supported Languages: Supports dozens of languages, including English, French, German, Greek, Hindi, Indonesian, Italian, Japanese, Korean, Malay, Nepali, Polish, Portuguese, Romanian, Russian, Serbian, Spanish, Turkish, Ukrainian, Vietnamese, Arabic, Bengali, Chinese, and Farsi
Key Features
-
Reasoning: Capable of long chains of reasoning traces before providing an answer.
-
Multilingual: Supports dozens of languages, including English, French, German, Greek, Hindi, Indonesian, Italian, Japanese, Korean, Malay, Nepali, Polish, Portuguese, Romanian, Russian, Serbian, Spanish, Turkish, Ukrainian, Vietnamese, Arabic, Bengali, Chinese, and Farsi.
-
Context Window: A 128k context window, but performance might degrade past 40k. Hence we recommend setting the maximum model length to 40k.
Third-Party Community Consideration
This model is not owned or developed by NVIDIA. This model has been developed and built to a third-party's requirements for this application and use case; see link to Non-NVIDIA MistralAi/Magistral-Small-2506.
License/Terms of Use
GOVERNING TERMS: This trial service is governed by the NVIDIA API Trial Terms of Service. Use of this model is governed by the NVIDIA Community Model License. Additional Information: Apache 2.0.
Deployment Geography
Global
Use Case
Magistral Small is a reasoning model with multilingual capabilities that can be deployed locally, fitting within a single RTX 4090.
Release Date
Build.NVIDIA.com: 2024-07-09 via link
Hugging Face: 2024-06-25 link
Reference(s)
Technical Paper: Magistral
Model Architecture
Architecture Type: Decoder-only (Transformer-based)
Network Architecture: Transformer-based, decoder-only architecture with rotary positional embeddings, multi-head self-attention, and long-context support (up to 128k tokens). Based on Mistral open-weight foundation models.
Model Architecture:
- Architecture Type: Transformer decoder
- Network Architecture: Transformer with FlashAttention
- Parameter Count: 24B
- Base Model: Mistral-Small-3.1-24B-Base-2503
Input
Input Type(s): Text
Input Format(s): String
Input Parameters: One-Dimensional (1D)
Other Properties Related to Input: Context Length: 128k
Output
Output Type(s): Text
Output Format(s): String
Output Parameters: One-Dimensional (1D)
Other Properties Related to Output: Max theoretical context length: 128,000 tokens, Practical limit: ≈ 40k tokens (40960)
Our AI models are designed and/or optimized to run on NVIDIA GPU-accelerated systems. By leveraging NVIDIA’s hardware (e.g. GPU cores) and software frameworks (e.g., CUDA libraries), the model achieves faster training and inference times compared to CPU-only solutions.**
Software Integration
Runtime Engine: vLLM
Supported Hardware Microarchitecture Compatibility
- NVIDIA Ampere
- NVIDIA Blackwell
- NVIDIA Hopper
- NVIDIA Lovelace
Preferred Operating System(s)
Linux
Model Version(s)
Magistral-Small-2506 1.0
Training, Testing, and Evaluation Datasets
Training Dataset
- Data Collection Method by dataset: Hybrid: Automated, Human
- Labeling Method by dataset: Hybrid: Automated, Human
- Properties: Trained on ~2 trillion tokens from web, books, code, and conversations; no sensors used (not applicable for text-based data).
Testing Dataset
- Data Collection Method by dataset: Hybrid: Automated, Human
- Labeling Method by dataset: Hybrid: Automated, Human
- Properties: Undisclosed
Evaluation Dataset
-
Data Collection Method by dataset: Hybrid: Automated, Human
-
Labeling Method by dataset: Hybrid: Automated, Human
-
Properties: The benchmarks noted in the following session were used for evaluation.
Benchmark Results
Model | AIME24 pass@1 | AIME25 pass@1 | GPQA Diamond | Livecodebench (v5) |
---|---|---|---|---|
Magistral Medium | 73.59% | 64.95% | 70.83% | 59.36% |
Magistral Small | 70.68% | 62.76% | 68.18% | 55.84% |
Inference
Acceleration Engine: vLLM
Test Hardware:
- L40s x2
Ethical Considerations
NVIDIA believes Trustworthy AI is a shared responsibility and we have established policies and practices to enable development for a wide array of AI applications. When downloaded or used in accordance with our terms of service, developers should work with their internal model team to ensure this model meets requirements for the relevant industry and use case and addresses unforeseen product misuse.
Please report security vulnerabilities or NVIDIA AI Concerns here.