DeepSpeed
  • Training Setup
  • Inference Setup
  • Training API
  • Inference API
  • Model Checkpointing
  • Activation Checkpointing
  • ZeRO
  • AutoEP (Automatic Expert Parallelism)
  • Mixture of Experts (DeepSpeed MoE)
  • Transformer Kernels
  • Pipeline Parallelism
  • Optimizers
  • Learning Rate Schedulers
  • Flops Profiler
  • Autotuning
  • Memory Requirements
  • Monitoring
DeepSpeed
  • Overview: module code

All modules for which code is available

  • deepspeed.checkpoint.utils
  • deepspeed.inference.config
  • deepspeed.moe.layer
  • deepspeed.monitor.config
  • deepspeed.ops.adam.cpu_adam
  • deepspeed.ops.adam.fused_adam
  • deepspeed.ops.lamb.fused_lamb
  • deepspeed.profiling.flops_profiler.profiler
  • deepspeed.runtime.lr_schedules
  • deepspeed.runtime.pipe.module
  • deepspeed.runtime.pipe.schedule
  • deepspeed.runtime.pipe.topology
  • deepspeed.runtime.torch_autocast
  • deepspeed.runtime.zero.config
  • deepspeed.runtime.zero.offload_config
  • deepspeed.utils.tensor_fragment
  • deepspeed.utils.zero_to_fp32

© Copyright 2020, Microsoft.

Built with Sphinx using a theme provided by Read the Docs.