High Throughput Inference: AI Model Selection Guide

High Throughput Inference

Visão geral do caso de uso

This page focuses on the High Throughput Inference use case, covering core requirements, recommended models, and evaluation criteria.

Critérios de seleção

• 开源权重许可

Critérios de avaliação

• 开源权重许可

Melhores práticas

• 提供系统提示以建立上下文
• 监控 token 用量与成本

Modelos recomendados

See the recommended models list for this use case (path: /en/use-cases/high-throughput/).