Inference API¶
deepspeed.init_inference()
returns an inference engine
of type InferenceEngine
.
for step, batch in enumerate(data_loader):
#forward() method
loss = engine(batch)
Forward Propagation¶
- deepspeed.InferenceEngine.forward(self, *inputs, **kwargs)¶
Execute forward propagation
- Parameters
*inputs – Variable length input list
**kwargs – variable length keyword arguments