Mistral Small 3: Complete Benchmark Performance Guide

mistral small-3 2025-01-29 Apache 2.0

In-depth analysis of Mistral Small 3 performance on MMLU, HumanEval, GSM8K, MATH, BBH benchmarks, including pricing and use cases.

Key Specifications

Vendormistral
Versionsmall-3
Release Date2025-01-29
Context Window32768 tokens
Input Modalitiestext
Output Modalitiestext
LicenseApache 2.0
Documentationhttps://docs.mistral.ai/

Benchmark Performance

BenchmarkScoreUnitEvaluated AtNotesSource
MMLU76.9%2025-01-295-shotview
HUMANEVAL74.4pass@12025-01-29—view
GSM8K79.6%2025-01-290-shot CoTview
MATH39.2%2025-01-290-shot CoTview
BBH71.3%2025-01-293-shot CoTview
GPQA30.1%2025-01-290-shotview
IFEVAL76.9%2025-01-29prompt_strictview
ARC91.8%2025-01-29challengeview
MUSR46%2025-01-290-shotview
WINOGRANDE78.8%2025-01-290-shotview

Pricing

TierPriceCurrency
Input$0.2 / MtokUSD
Output$0.6 / MtokUSD
Cache Read$0 / MtokUSD
Cache Write$0 / MtokUSD

Source: https://mistral.ai/technology/ · as of 2025-01-29

Compliance

  • Data Residency: EU
  • SOC2: ✓
  • HIPAA: ✗
  • GDPR: ✓
  • ISO 27001: ✓

Mistral Small 3

Descripción del modelo

Mistral Small 3 开源模型, 24B 参数, 32K 上下文, 性能接近 Mistral Large 2, Apache 2.0 可商用。

Especificaciones principales

ProveedorVersiónFecha de lanzamientoVentana de contextoModalidades de entradaModalidades de salidaLicencia
Mistralsmall-32025-01-2932KtexttextApache 2.0

Rendimiento en benchmarks

BenchmarkPuntuaciónUnidadNotas
MMLU (Massive Multitask Language Understanding)76.9%5-shot
HumanEval74.4pass@1—
GSM8K (Grade School Math 8K)79.6%0-shot CoT
MATH39.2%0-shot CoT
BBH (BIG-Bench Hard)71.3%3-shot CoT
GPQA30.1%0-shot
IFEval76.9%prompt_strict
ARC91.8%challenge
MUSR46.0%0-shot
WinoGrande78.8%0-shot

Precios

EntradaSalidaLectura cachéEscritura caché
————

por millón de tokens

Fortalezas

  • 可靠的通用模型。

Debilidades

  • 闭源专有模型，不支持自托管。

Casos de uso

  • 代码生成与调试

Referencias