Skip to content

Infernet Services

Inference services are a collection of Infernet-compatible containers that can be used to perform a variety of inference requests with the Infernet Node.

Each service is a docker containerized python quart app that utilizes infernet-ml's workflows to perform different flavors of ML inference. The source and destination of the requests to these services can be either offchain or onchain, and some also support streaming for offchain destinations. These services are designed to be scalable, efficient, and easy to integrate with various machine learning models and data sources.

The following workflows are currently supported as Infernet services:

Service Description Supports streaming
CSS Closed-Source Software, models like OpenAI Yes
HF Client HuggingFace Inference Client No
ONNX Open Neural Network Exchange No
TGI Client Text Generation Inference Client Yes
Torch PyTorch Inference No