Key Specifications

Vendor	google
Version	1.5-pro-002
Release Date	2024-09-24
Context Window	2e+06 tokens
Input Modalities	text, image, audio, video
Output Modalities	text
License	Proprietary
Documentation	https://ai.google.dev/gemini-api/docs

Benchmark Performance

Benchmark	Score	Unit	Evaluated At	Notes	Source
MMLU	87	%	2024-09-24	5-shot	view
HUMANEVAL	90.3	pass@1	2024-09-24	—	view
GSM8K	95	%	2024-09-24	0-shot CoT	view
MATH	55.9	%	2024-09-24	0-shot CoT	view
BBH	87.2	%	2024-09-24	3-shot CoT	view
GPQA	56.5	%	2024-09-24	0-shot	view
IFEVAL	87.9	%	2024-09-24	prompt_strict	view
ARC	95.9	%	2024-09-24	challenge	view
MUSR	71.6	%	2024-09-24	0-shot	view
WINOGRANDE	85.8	%	2024-09-24	0-shot	view

Name: Gemini 1.5 Pro 002
Brand: google

Google Gemini 1.5 Pro 002 生产版本, 2M 上下文, 改进指令遵循与数学推理能力。

Proveedor	Versión	Fecha de lanzamiento	Ventana de contexto	Modalidades de entrada	Modalidades de salida	Licencia
Google	1.5-pro-002	2024-09-24	2000K	text, image, audio, video	text	Proprietary

Benchmark	Puntuación	Unidad	Notas
MMLU (Massive Multitask Language Understanding)	87.0	%	5-shot
HumanEval	90.3	pass@1	—
GSM8K (Grade School Math 8K)	95.0	%	0-shot CoT
MATH	55.9	%	0-shot CoT
BBH (BIG-Bench Hard)	87.2	%	3-shot CoT
GPQA	56.5	%	0-shot
IFEval	87.9	%	prompt_strict
ARC	95.9	%	challenge
MUSR	71.6	%	0-shot
WinoGrande	85.8	%	0-shot

Entrada	Salida	Lectura caché	Escritura caché
—	—	—	—

por millón de tokens