Key Specifications

Benchmark Performance

Benchmark	Score	Unit	Evaluated At	Notes	Source
MMLU	76.5	%	2024-09-25	5-shot	view
HUMANEVAL	73.1	pass@1	2024-09-25	—	view
GSM8K	82.6	%	2024-09-25	0-shot CoT	view
MATH	52.5	%	2024-09-25	0-shot CoT	view
BBH	76.1	%	2024-09-25	3-shot CoT	view
GPQA	42.2	%	2024-09-25	0-shot	view
IFEVAL	74.5	%	2024-09-25	prompt_strict	view
ARC	94.6	%	2024-09-25	challenge	view
MUSR	53.5	%	2024-09-25	0-shot	view
WINOGRANDE	79.3	%	2024-09-25	0-shot	view

Source: https://ai.meta.com/blog/ · as of 2024-09-25

Name: Llama 3.2 90B Vision
Brand: meta

Meta Llama 3.2 90B Vision 多模态开源模型, 128K 上下文, 支持图像理解, 性能接近 GPT-4V。

Fornecedor	Versão	Data de lançamento	Janela de contexto	Modalidades de entrada	Modalidades de saída	Licença
Meta	3.2-90b-vision	2024-09-25	128K	text, image	text	Llama 3.2 Community License

Benchmark	Pontuação	Unidade	Notas
MMLU (Massive Multitask Language Understanding)	76.5	%	5-shot
HumanEval	73.1	pass@1	—
GSM8K (Grade School Math 8K)	82.6	%	0-shot CoT
MATH	52.5	%	0-shot CoT
BBH (BIG-Bench Hard)	76.1	%	3-shot CoT
GPQA	42.2	%	0-shot
IFEval	74.5	%	prompt_strict
ARC	94.6	%	challenge
MUSR	53.5	%	0-shot
WinoGrande	79.3	%	0-shot

Entrada	Saída	Leitura de cache	Escrita de cache
—	—	—	—

por milhão de tokens