Backend support#
The following table summarizes the currently supported communication backends and their corresponding capabilities.
Backend |
NCCL |
IXCCL |
CNCL |
MCCL |
XCCL |
DUCCL |
HCCL |
MUSACCL |
RCCL |
TCCL |
ECCL |
|---|---|---|---|---|---|---|---|---|---|---|---|
Mode |
Homo/Hetero |
Homo/Hetero |
Homo/Hetero |
Homo/Hetero |
Homo/Hetero |
Homo/Hetero |
Homo/Hetero |
Homo/Hetero |
Homo/Hetero |
Homo/Hetero |
Homo/Hetero |
send |
✓/✓ |
✓/✓ |
✓/✓ |
✓/✓ |
✓/✓ |
✓/✓ |
✓/☓ |
✓/✓ |
✓/✓ |
✓/✓ |
✓/✓ |
recv |
✓/✓ |
✓/✓ |
✓/✓ |
✓/✓ |
✓/✓ |
✓/✓ |
✓/☓ |
✓/✓ |
✓/✓ |
✓/✓ |
✓/✓ |
broadcast |
✓/✓ |
✓/✓ |
✓/✓ |
✓/✓ |
✓/✓ |
✓/✓ |
✓/☓ |
✓/✓ |
✓/✓ |
✓/✓ |
✓/✓ |
gather |
✓/✓ |
✓/✓ |
✓/✓ |
✓/✓ |
☓/☓ |
✓/✓ |
✓/☓ |
✓/✓ |
✓/✓ |
✓/✓ |
✓/✓ |
scatter |
✓/✓ |
✓/✓ |
✓/✓ |
✓/✓ |
✓/☓ |
✓/✓ |
✓/☓ |
✓/✓ |
✓/✓ |
✓/✓ |
✓/✓ |
reduce |
✓/✓ |
✓/✓ |
✓/✓ |
✓/✓ |
✓/✓ |
✓/✓ |
✓/☓ |
✓/✓ |
✓/✓ |
✓/✓ |
✓/✓ |
allreduce |
✓/✓ |
✓/✓ |
✓/✓ |
✓/✓ |
✓/✓ |
✓/✓ |
✓/☓ |
✓/✓ |
✓/✓ |
✓/✓ |
✓/✓ |
allgather |
✓/✓ |
✓/✓ |
✓/✓ |
✓/✓ |
✓/✓ |
✓/✓ |
✓/☓ |
✓/✓ |
✓/✓ |
✓/✓ |
✓/✓ |
reducescatter |
✓/✓ |
✓/✓ |
✓/✓ |
✓/✓ |
✓/✓ |
✓/✓ |
✓/☓ |
✓/✓ |
✓/✓ |
✓/✓ |
✓/✓ |
alltoall |
✓/✓ |
✓/✓ |
✓/✓ |
✓/✓ |
✓/✓ |
✓/✓ |
✓/☓ |
✓/✓ |
✓/✓ |
✓/✓ |
✓/✓ |
alltoallv |
✓/✓ |
✓/✓ |
✓/✓ |
✓/✓ |
✓/☓ |
✓/✓ |
✓/☓ |
✓/✓ |
✓/✓ |
✓/✓ |
✓/✓ |
group ops |
✓/✓ |
✓/✓ |
✓/✓ |
✓/✓ |
✓/✓ |
✓/✓ |
✓/☓ |
✓/✓ |
✓/✓ |
✓/✓ |
✓/✓ |
Note that Homo and Hetero modes refer to communications among homogeneous and heterogeneous clusters. All native collective communications libraries can be referenced through the links below (in alphabetic order):
CNCL, Cambricon Communications Library.
DUCCL, DU Collective Communications Library.
ECCL, Enflame Collective Communications Library.
HCCL, Ascend Communications Library.
IXCCL, Iluvatar Corex Collective Communications Library.
MCCL, Metax Collective Communications Library.
MUSACCL, Musa Collective Communications Library.
NCCL, NVIDIA Collective Communications Library.
RCCL, ROCm Communication Collectives Library.
TCCL, TsingMicro Communication Collectives Library.
XCCL, Kunlunxin XPU Collective Communications Library.
Additionally, FlagCX supports three collective communication libraries for host-side communication: