Training Models with Paddle and FlagCX

FlagCX is now fully integrated into Paddle as an optional high-performance communication backend. This integration enables efficient distributed training on multiple hardware platforms, including support for heterogeneous training on Nvidia and Iluvatar GPUs.

Use the guides below to quickly get started with training models using Paddle + FlagCX.

Homogeneous training

Train on a single type of hardware platform:

Hardware	User Guide
Nvidia GPU	Guide for using paddle with FlagCX on nvidia GPU environment
KLX XPU	Guide for using paddle with FlagCX on XPU machines
Iluvatar GPU	Guide for using paddle with FlagCX on iluvatar machines

Heterogeneous training

Train across different hardware platforms simultaneously:

Hardware Combination	User Guide
Nvidia GPU + Iluvatar GPU	Guide for training on nvidia and iluvatar GPUs

• Guide for using paddle with FlagCX on nvidia GPU environment
  • Environment setup
  • Compile paddle with FlagCX
  • Train model using paddle + FlagCX
• Guide for using paddle with FlagCX on iluvatar machines
  • Environment setup
  • Compile paddle with FlagCX
  • Train model using paddle + FlagCX
• Guide for using paddle with FlagCX on XPU machines
  • Environment setup
  • Compile paddle with FlagCX
  • Train model using paddle + FlagCX
• Guide for training on nvidia and iluvatar GPUs
  • Environment setup
  • Training on heterogeneous ai accelerators (nvidia GPU + iluvatar GPU)