Inference API
deepspeed.init_inference()
returns an inference engine
of type InferenceEngine
.
for step, batch in enumerate(data_loader):
#forward() method
loss = engine(batch)
Forward Propagation
- deepspeed.InferenceEngine.forward(self, *inputs, **kwargs)
Execute forward propagation
- Parameters
*inputs – Variable length input list
**kwargs – variable length keyword arguments