vLLM

概述

vLLM is an open-source inference and serving engine for large language models, originally developed at UC Berkeley. Its PagedAttention memory management delivers high throughput and efficient GPU utilization, and it exposes an OpenAI-compatible API server. vLLM is free under Apache-2.0 and has become a de facto standard for self-hosted LLM serving.

主要功能

• PagedAttention for efficient KV-cache memory management
• High-throughput continuous batching of requests
• OpenAI-compatible API server out of the box
• Broad support for open model architectures and quantization

适用场景

• Serving open LLMs in production at high throughput
• Self-hosting an OpenAI-compatible inference endpoint
• Benchmarking and research on efficient LLM inference

定价

vLLM is free and open source under Apache-2.0; costs are limited to the GPU infrastructure it runs on.

常见问题

What is vLLM used for?

vLLM serves large language models with high throughput, exposing an OpenAI-compatible API for self-hosted deployments.

Is vLLM free?

Yes. vLLM is open source under Apache-2.0; you only pay for the hardware you run it on.

What is PagedAttention?

PagedAttention is vLLM's memory-management technique that pages the KV cache, raising GPU utilization and throughput.

同类产品

• Amazon Bedrock — AWS 的托管服务，可通过单一 API 使用来自多家提供商的基础模型构建应用。
• Amazon Nova — AWS 推出的亚马逊自研基础模型与服务组合，覆盖文本、多模态、语音、定制模型构建以及可自动化界面操作的智能体。
• Azure AI Foundry — 微软的平台，可在 Azure 上构建、评估并部署 AI 应用与智能体。
• Baseten — 将机器学习模型部署并作为可扩展 API 提供服务的平台。
• Cerebras — AI 算力公司，其推理云基于晶圆级硬件托管开放语言模型。
• Civitai — 用于分享与发现开放图像生成模型、LoRA 与生成作品的社区平台。

查看vLLM的全部替代品 → · 查看全部平台与基础设施产品 →

来源

• Documentation ↗
• Source code (GitHub) ↗

本记录最近复核于 2026-07-30。

机器可读记录：/api/products/vllm.json · 发现错误？提交纠错