DeepSpeed
stable
Training Setup
Inference Setup
Training API
Inference API
Model Checkpointing
Activation Checkpointing
ZeRO
Mixture of Experts (MoE)
Transformer Kernels
Pipeline Parallelism
Optimizers
Learning Rate Schedulers
Flops Profiler
Autotuning
Memory Requirements
Monitoring
DeepSpeed
Index
Edit on GitHub
Index
A
|
B
|
C
|
D
|
E
|
F
|
G
|
I
|
J
|
L
|
M
|
N
|
O
|
P
|
Q
|
R
|
S
|
T
|
U
|
W
|
Z
A
activation (deepspeed.inference.config.QuantizationConfig attribute)
add_config_arguments() (in module deepspeed)
allgather_bucket_size (deepspeed.runtime.zero.config.DeepSpeedZeroConfig attribute)
allgather_partitions (deepspeed.runtime.zero.config.DeepSpeedZeroConfig attribute)
allreduce_tied_weight_gradients() (deepspeed.pipe.PipelineModule method)
autotuner (in module deepspeed.autotuning)
B
backward() (deepspeed.runtime.pipe.engine.PipelineEngine method)
(in module deepspeed.DeepSpeedEngine)
BackwardPass (class in deepspeed.runtime.pipe.schedule)
base_dir (deepspeed.inference.config.DeepSpeedInferenceConfig attribute)
(deepspeed.inference.config.InferenceCheckpointConfig attribute)
buffer_count (deepspeed.runtime.zero.config.DeepSpeedZeroOffloadOptimizerConfig attribute)
(deepspeed.runtime.zero.config.DeepSpeedZeroOffloadParamConfig attribute)
buffer_size (deepspeed.runtime.zero.config.DeepSpeedZeroOffloadParamConfig attribute)
BufferOpInstruction (class in deepspeed.runtime.pipe.schedule)
build() (deepspeed.pipe.LayerSpec method)
C
checkpoint (deepspeed.inference.config.DeepSpeedInferenceConfig attribute)
checkpoint() (in module deepspeed.checkpointing)
checkpoint_config (deepspeed.inference.config.DeepSpeedInferenceConfig attribute)
checkpoint_dir (deepspeed.inference.config.InferenceCheckpointConfig attribute)
CheckpointFunction (class in deepspeed.checkpointing)
ckpt_layer_path() (deepspeed.pipe.PipelineModule method)
ckpt_layer_path_list() (deepspeed.pipe.PipelineModule method)
ckpt_prefix() (deepspeed.pipe.PipelineModule method)
clone_tensors_for_torch_save() (in module deepspeed.checkpoint.utils)
config (deepspeed.inference.config.DeepSpeedInferenceConfig attribute)
configure() (in module deepspeed.checkpointing)
contiguous_gradients (deepspeed.runtime.zero.config.DeepSpeedZeroConfig attribute)
convert_zero_checkpoint_to_fp32_state_dict() (in module deepspeed.utils.zero_to_fp32)
copy_params_from() (deepspeed.zero.TiledLinear method)
cpu_offload (deepspeed.runtime.zero.config.DeepSpeedZeroConfig attribute)
cpu_offload_param (deepspeed.runtime.zero.config.DeepSpeedZeroConfig attribute)
cpu_offload_use_pin_memory (deepspeed.runtime.zero.config.DeepSpeedZeroConfig attribute)
CudaRNGStatesTracker (class in deepspeed.checkpointing)
D
DataParallelSchedule (class in deepspeed.runtime.pipe.schedule)
deepspeed.profiling.flops_profiler.profiler
module
deepspeed.runtime.pipe.engine
module
deepspeed.runtime.pipe.schedule
module
DeepSpeedCPUAdam (class in deepspeed.ops.adam)
DeepSpeedTransformerConfig (class in deepspeed)
DeepSpeedTransformerLayer (class in deepspeed)
device (deepspeed.runtime.zero.config.DeepSpeedZeroOffloadOptimizerConfig attribute)
(deepspeed.runtime.zero.config.DeepSpeedZeroOffloadParamConfig attribute)
dtype (deepspeed.inference.config.DeepSpeedInferenceConfig attribute)
E
elastic_checkpoint (deepspeed.runtime.zero.config.DeepSpeedZeroConfig attribute)
enable_cuda_graph (deepspeed.inference.config.DeepSpeedInferenceConfig attribute)
enabled (deepspeed.inference.config.DeepSpeedMoEConfig attribute)
(deepspeed.inference.config.DeepSpeedTPConfig attribute)
(deepspeed.inference.config.QuantizationConfig attribute)
(deepspeed.monitor.config.CSVConfig attribute)
(deepspeed.monitor.config.TensorBoardConfig attribute)
(deepspeed.monitor.config.WandbConfig attribute)
end_profile() (deepspeed.profiling.flops_profiler.profiler.FlopsProfiler method)
ep_group (deepspeed.inference.config.DeepSpeedInferenceConfig attribute)
(deepspeed.inference.config.DeepSpeedMoEConfig attribute)
ep_mp_group (deepspeed.inference.config.DeepSpeedInferenceConfig attribute)
(deepspeed.inference.config.DeepSpeedMoEConfig attribute)
ep_size (deepspeed.inference.config.DeepSpeedInferenceConfig attribute)
(deepspeed.inference.config.DeepSpeedMoEConfig attribute)
estimate_zero2_model_states_mem_needs_all_cold() (in module deepspeed.runtime.zero.stage_1_and_2)
estimate_zero2_model_states_mem_needs_all_live() (in module deepspeed.runtime.zero.stage_1_and_2)
estimate_zero3_model_states_mem_needs_all_cold() (in module deepspeed.runtime.zero.stage3)
estimate_zero3_model_states_mem_needs_all_live() (in module deepspeed.runtime.zero.stage3)
eval_batch() (deepspeed.runtime.pipe.engine.PipelineEngine method)
F
fast_init (deepspeed.runtime.zero.config.DeepSpeedZeroOffloadOptimizerConfig attribute)
filter_match() (deepspeed.runtime.pipe.ProcessTopology method)
FlopsProfiler (class in deepspeed.profiling.flops_profiler.profiler)
forward() (deepspeed.moe.layer.MoE method)
(deepspeed.pipe.PipelineModule method)
(deepspeed.runtime.pipe.engine.PipelineEngine method)
(deepspeed.zero.TiledLinear method)
(in module deepspeed.DeepSpeedEngine)
(in module deepspeed.InferenceEngine)
ForwardPass (class in deepspeed.runtime.pipe.schedule)
FusedAdam (class in deepspeed.ops.adam)
FusedLamb (class in deepspeed.ops.lamb)
G
gather_16bit_weights_on_model_save (deepspeed.runtime.zero.config.DeepSpeedZeroConfig attribute)
GatheredParameters (class in deepspeed.zero)
get_additional_losses() (deepspeed.pipe.PipelineModule method)
get_axis_comm_lists() (deepspeed.runtime.pipe.ProcessTopology method)
get_axis_list() (deepspeed.runtime.pipe.ProcessTopology method)
get_axis_names() (deepspeed.runtime.pipe.ProcessTopology method)
get_coord() (deepspeed.runtime.pipe.ProcessTopology method)
get_cuda_rng_tracker() (in module deepspeed.checkpointing)
get_dim() (deepspeed.runtime.pipe.ProcessTopology method)
get_dp_process_group() (deepspeed.zero.Init method)
get_fp32_state_dict_from_zero_checkpoint() (in module deepspeed.utils.zero_to_fp32)
get_model_profile() (in module deepspeed.profiling.flops_profiler.profiler)
get_partition_rank() (deepspeed.zero.Init method)
get_rank() (deepspeed.runtime.pipe.ProcessTopology method)
get_rank_repr() (deepspeed.runtime.pipe.ProcessTopology method)
get_total_duration() (deepspeed.profiling.flops_profiler.profiler.FlopsProfiler method)
get_total_flops() (deepspeed.profiling.flops_profiler.profiler.FlopsProfiler method)
get_total_macs() (deepspeed.profiling.flops_profiler.profiler.FlopsProfiler method)
get_total_params() (deepspeed.profiling.flops_profiler.profiler.FlopsProfiler method)
group (deepspeed.monitor.config.WandbConfig attribute)
I
ignore_unused_parameters (deepspeed.runtime.zero.config.DeepSpeedZeroConfig attribute)
InferenceSchedule (class in deepspeed.runtime.pipe.schedule)
Init (class in deepspeed.zero)
init_distributed() (in module deepspeed)
init_inference() (in module deepspeed)
initialize() (in module deepspeed)
injection_policy (deepspeed.inference.config.DeepSpeedInferenceConfig attribute)
injection_policy_tuple (deepspeed.inference.config.DeepSpeedInferenceConfig attribute)
is_configured() (in module deepspeed.checkpointing)
is_first_stage (deepspeed.runtime.pipe.schedule.PipeSchedule property)
is_first_stage() (deepspeed.runtime.pipe.engine.PipelineEngine method)
is_gradient_accumulation_boundary() (deepspeed.runtime.pipe.engine.PipelineEngine method)
(in module deepspeed.DeepSpeedEngine)
is_last_stage (deepspeed.runtime.pipe.schedule.PipeSchedule property)
is_last_stage() (deepspeed.runtime.pipe.engine.PipelineEngine method)
J
job_name (deepspeed.monitor.config.CSVConfig attribute)
(deepspeed.monitor.config.TensorBoardConfig attribute)
L
LayerSpec (class in deepspeed.pipe)
legacy_stage1 (deepspeed.runtime.zero.config.DeepSpeedZeroConfig attribute)
load_checkpoint() (in module deepspeed.DeepSpeedEngine)
load_from_fp32_weights (deepspeed.runtime.zero.config.DeepSpeedZeroConfig attribute)
load_module_state_dict() (deepspeed.runtime.pipe.engine.PipelineEngine method)
load_state_dict_from_zero_checkpoint() (in module deepspeed.utils.zero_to_fp32)
LoadMicroBatch (class in deepspeed.runtime.pipe.schedule)
LRRangeTest (class in deepspeed.runtime.lr_schedules)
M
max_in_cpu (deepspeed.runtime.zero.config.DeepSpeedZeroOffloadParamConfig attribute)
max_live_parameters (deepspeed.runtime.zero.config.DeepSpeedZeroConfig attribute)
max_out_tokens (deepspeed.inference.config.DeepSpeedInferenceConfig attribute)
max_reuse_distance (deepspeed.runtime.zero.config.DeepSpeedZeroConfig attribute)
memory_efficient_linear (deepspeed.runtime.zero.config.DeepSpeedZeroConfig attribute)
mics_hierarchical_params_gather (deepspeed.runtime.zero.config.DeepSpeedZeroConfig attribute)
mics_shard_size (deepspeed.runtime.zero.config.DeepSpeedZeroConfig attribute)
min_out_tokens (deepspeed.inference.config.DeepSpeedInferenceConfig attribute)
model_parallel_cuda_manual_seed() (in module deepspeed.checkpointing)
model_persistence_threshold (deepspeed.runtime.zero.config.DeepSpeedZeroConfig attribute)
module
deepspeed.profiling.flops_profiler.profiler
deepspeed.runtime.pipe.engine
deepspeed.runtime.pipe.schedule
module_state_dict() (deepspeed.runtime.pipe.engine.PipelineEngine method)
MoE (class in deepspeed.moe.layer)
moe (deepspeed.inference.config.DeepSpeedInferenceConfig attribute)
moe_experts (deepspeed.inference.config.DeepSpeedInferenceConfig attribute)
(deepspeed.inference.config.DeepSpeedMoEConfig attribute)
moe_type (deepspeed.inference.config.DeepSpeedInferenceConfig attribute)
mp_size (deepspeed.inference.config.DeepSpeedInferenceConfig attribute)
mpu (deepspeed.inference.config.DeepSpeedInferenceConfig attribute)
(deepspeed.inference.config.DeepSpeedTPConfig attribute)
N
num_micro_batches (deepspeed.runtime.pipe.schedule.PipeSchedule property)
num_pipe_buffers() (deepspeed.runtime.pipe.schedule.DataParallelSchedule method)
(deepspeed.runtime.pipe.schedule.InferenceSchedule method)
(deepspeed.runtime.pipe.schedule.PipeSchedule method)
(deepspeed.runtime.pipe.schedule.TrainSchedule method)
num_stages (deepspeed.runtime.pipe.schedule.PipeSchedule property)
nvme_path (deepspeed.runtime.zero.config.DeepSpeedZeroOffloadOptimizerConfig attribute)
(deepspeed.runtime.zero.config.DeepSpeedZeroOffloadParamConfig attribute)
O
offload_optimizer (deepspeed.runtime.zero.config.DeepSpeedZeroConfig attribute)
offload_param (deepspeed.runtime.zero.config.DeepSpeedZeroConfig attribute)
OnebitAdam (class in deepspeed.runtime.fp16.onebit.adam)
OnebitLamb (class in deepspeed.runtime.fp16.onebit.lamb)
OneCycle (class in deepspeed.runtime.lr_schedules)
OptimizerStep (class in deepspeed.runtime.pipe.schedule)
output_path (deepspeed.monitor.config.CSVConfig attribute)
(deepspeed.monitor.config.TensorBoardConfig attribute)
overlap_comm (deepspeed.runtime.zero.config.DeepSpeedZeroConfig attribute)
override_module_apply (deepspeed.runtime.zero.config.DeepSpeedZeroConfig attribute)
P
param_persistence_threshold (deepspeed.runtime.zero.config.DeepSpeedZeroConfig attribute)
pin_memory (deepspeed.runtime.zero.config.DeepSpeedZeroOffloadOptimizerConfig attribute)
(deepspeed.runtime.zero.config.DeepSpeedZeroOffloadParamConfig attribute)
PipeInstruction (class in deepspeed.runtime.pipe.schedule)
pipeline_loading_checkpoint (deepspeed.runtime.zero.config.DeepSpeedZeroConfig attribute)
pipeline_read (deepspeed.runtime.zero.config.DeepSpeedZeroOffloadOptimizerConfig attribute)
pipeline_write (deepspeed.runtime.zero.config.DeepSpeedZeroOffloadOptimizerConfig attribute)
PipelineEngine (class in deepspeed.runtime.pipe.engine)
PipelineModule (class in deepspeed.pipe)
PipeSchedule (class in deepspeed.runtime.pipe.schedule)
prefetch_bucket_size (deepspeed.runtime.zero.config.DeepSpeedZeroConfig attribute)
print_model_aggregated_profile() (deepspeed.profiling.flops_profiler.profiler.FlopsProfiler method)
print_model_profile() (deepspeed.profiling.flops_profiler.profiler.FlopsProfiler method)
ProcessTopology (class in deepspeed.runtime.pipe)
project (deepspeed.monitor.config.WandbConfig attribute)
Q
qkv (deepspeed.inference.config.QuantizationConfig attribute)
quant (deepspeed.inference.config.DeepSpeedInferenceConfig attribute)
R
ratio (deepspeed.runtime.zero.config.DeepSpeedZeroOffloadOptimizerConfig attribute)
RecvActivation (class in deepspeed.runtime.pipe.schedule)
RecvGrad (class in deepspeed.runtime.pipe.schedule)
reduce_bucket_size (deepspeed.runtime.zero.config.DeepSpeedZeroConfig attribute)
reduce_scatter (deepspeed.runtime.zero.config.DeepSpeedZeroConfig attribute)
ReduceGrads (class in deepspeed.runtime.pipe.schedule)
ReduceTiedGrads (class in deepspeed.runtime.pipe.schedule)
register_external_parameter() (in module deepspeed.zero)
replace_method (deepspeed.inference.config.DeepSpeedInferenceConfig attribute)
replace_with_kernel_inject (deepspeed.inference.config.DeepSpeedInferenceConfig attribute)
reset() (in module deepspeed.checkpointing)
reset_activation_shape() (deepspeed.runtime.pipe.engine.PipelineEngine method)
reset_profile() (deepspeed.profiling.flops_profiler.profiler.FlopsProfiler method)
return_tuple (deepspeed.inference.config.DeepSpeedInferenceConfig attribute)
round_robin_gradients (deepspeed.runtime.zero.config.DeepSpeedZeroConfig attribute)
S
safe_get_full_fp32_param() (in module deepspeed.utils)
safe_get_full_grad() (in module deepspeed.utils)
safe_get_full_optimizer_state() (in module deepspeed.utils)
safe_get_local_fp32_param() (in module deepspeed.utils)
safe_get_local_grad() (in module deepspeed.utils)
safe_get_local_optimizer_state() (in module deepspeed.utils)
safe_set_full_fp32_param() (in module deepspeed.utils)
safe_set_full_optimizer_state() (in module deepspeed.utils)
safe_set_local_fp32_param() (in module deepspeed.utils)
safe_set_local_optimizer_state() (in module deepspeed.utils)
save_16bit_model() (in module deepspeed.DeepSpeedEngine)
save_checkpoint() (in module deepspeed.DeepSpeedEngine)
save_mp_checkpoint_path (deepspeed.inference.config.DeepSpeedInferenceConfig attribute)
(deepspeed.inference.config.InferenceCheckpointConfig attribute)
SendActivation (class in deepspeed.runtime.pipe.schedule)
SendGrad (class in deepspeed.runtime.pipe.schedule)
set_batch_fn() (deepspeed.runtime.pipe.engine.PipelineEngine method)
set_dataiterator() (deepspeed.runtime.pipe.engine.PipelineEngine method)
set_empty_params (deepspeed.inference.config.DeepSpeedInferenceConfig attribute)
set_train_batch_size() (deepspeed.runtime.pipe.engine.PipelineEngine method)
stage (deepspeed.runtime.pipe.schedule.PipeSchedule property)
(deepspeed.runtime.zero.config.DeepSpeedZeroConfig attribute)
stage3_gather_fp16_weights_on_model_save (deepspeed.runtime.zero.config.DeepSpeedZeroConfig attribute)
start_profile() (deepspeed.profiling.flops_profiler.profiler.FlopsProfiler method)
step() (deepspeed.runtime.pipe.engine.PipelineEngine method)
(in module deepspeed.DeepSpeedEngine)
steps() (deepspeed.runtime.pipe.schedule.PipeSchedule method)
stop_profile() (deepspeed.profiling.flops_profiler.profiler.FlopsProfiler method)
sub_group_size (deepspeed.runtime.zero.config.DeepSpeedZeroConfig attribute)
T
team (deepspeed.monitor.config.WandbConfig attribute)
tensor_parallel (deepspeed.inference.config.DeepSpeedInferenceConfig attribute)
TiedLayerSpec (class in deepspeed.pipe)
TiledLinear (class in deepspeed.zero)
topology() (deepspeed.pipe.PipelineModule method)
tp_group (deepspeed.inference.config.DeepSpeedTPConfig attribute)
tp_size (deepspeed.inference.config.DeepSpeedTPConfig attribute)
train_batch() (deepspeed.runtime.pipe.engine.PipelineEngine method)
training_mp_size (deepspeed.inference.config.DeepSpeedInferenceConfig attribute)
TrainSchedule (class in deepspeed.runtime.pipe.schedule)
transposed_mode (deepspeed.inference.config.DeepSpeedInferenceConfig attribute)
triangular_masking (deepspeed.inference.config.DeepSpeedInferenceConfig attribute)
triton_autotune (deepspeed.inference.config.DeepSpeedInferenceConfig attribute)
type (deepspeed.inference.config.DeepSpeedMoEConfig attribute)
U
use_multi_rank_bucket_allreduce (deepspeed.runtime.zero.config.DeepSpeedZeroConfig attribute)
use_triton (deepspeed.inference.config.DeepSpeedInferenceConfig attribute)
W
WarmupCosineLR (class in deepspeed.runtime.lr_schedules)
WarmupDecayLR (class in deepspeed.runtime.lr_schedules)
WarmupLR (class in deepspeed.runtime.lr_schedules)
weight (deepspeed.inference.config.QuantizationConfig attribute)
Z
zero (deepspeed.inference.config.DeepSpeedInferenceConfig attribute)
zero_hpz_partition_size (deepspeed.runtime.zero.config.DeepSpeedZeroConfig attribute)
zero_quantized_gradients (deepspeed.runtime.zero.config.DeepSpeedZeroConfig attribute)
zero_quantized_nontrainable_weights (deepspeed.runtime.zero.config.DeepSpeedZeroConfig attribute)
zero_quantized_weights (deepspeed.runtime.zero.config.DeepSpeedZeroConfig attribute)
ZeroOneAdam (class in deepspeed.runtime.fp16.onebit.zoadam)
Read the Docs
v: stable
Versions
latest
stable
rtd-staging
Downloads
pdf
On Read the Docs
Project Home
Builds