Infernet Services

Inference services are a collection of Infernet-compatible containers that can be used to perform a variety of inference requests with the Infernet Node.

Each service is a docker containerized python quart app that utilizes infernet-ml's workflows to perform different flavors of ML inference. The source and destination of the requests to these services can be either offchain or onchain, and some also support streaming for offchain destinations. These services are designed to be scalable, efficient, and easy to integrate with various machine learning models and data sources.

The following workflows are currently supported as Infernet services:

Service	Description	Supports streaming
CSS	Closed-Source Software, models like OpenAI	Yes
HF Client	HuggingFace Inference Client	No
ONNX	Open Neural Network Exchange	No
TGI Client	Text Generation Inference Client	Yes
Torch	PyTorch Inference	No