DeepSpeed
Training Setup
Inference Setup
Training API
Inference API
Model Checkpointing
Activation Checkpointing
ZeRO
AutoEP (Automatic Expert Parallelism)
Mixture of Experts (DeepSpeed MoE)
Transformer Kernels
Pipeline Parallelism
Model Specification
Training
Extending Pipeline Parallelism
Optimizers
Learning Rate Schedulers
Flops Profiler
Autotuning
Memory Requirements
Host Memory Pinning
Monitoring
DeepSpeed
Pipeline Parallelism
Edit on GitHub
Pipeline Parallelism
Model Specification
Training
Extending Pipeline Parallelism