vllm-plugin-FL Overview

vllm-plugin-FL is a plugin for the vLLM inference/serving framework, built on FlagOS’s unified multi-chip backend — including the unified operator library FlagGems and the unified communication library FlagCX. It extends vLLM’s capabilities and performance across diverse hardware environments. Without changing vLLM’s original interfaces or usage patterns, the same command can run model inference/serving on different chips.

• Features
• Operator dispatch mechanism
  • Directory structure
  • Core concepts
  • Architecture overview