High Throughput Inference: AI Model Selection Guide

High Throughput Inference

Descripción del caso de uso

This page focuses on the High Throughput Inference use case, covering core requirements, recommended models, and evaluation criteria.

Criterios de selección

• 开源权重许可

Criterios de evaluación

• 开源权重许可

Mejores prácticas

• 提供系统提示以建立上下文
• 监控 token 用量与成本

Modelos recomendados

See the recommended models list for this use case (path: /en/use-cases/high-throughput/).