sglang-plugin-FL Overview

sglang-plugin-FL is an out-of-tree (OOT) plugin for SGLang, built on FlagOS’s unified multi-chip backend — including the unified operator library FlagGems and the unified communication library FlagCX. It extends SGLang’s inference capabilities across diverse hardware platforms. Without changing SGLang’s original interfaces or usage patterns, the same command can run model inference on different chips.

• Features
  • Layer 1 — ATen Operators
  • Layer 2 — SGLang Fused Kernels
  • Layer 3 — Distributed Communication
• How plugin works ?
  • Load plugin
  • Dispatch hook
  • Dispatch Architecture (shared with vllm-plugin-FL)
  • ATen replacement
• Project structure