Infernet Services
Inference services are a collection of Infernet-compatible containers that can be used to perform a variety of inference requests with the Infernet Node.
Each service is a docker containerized python quart app that
utilizes infernet-ml
's
workflows to perform different flavors of ML inference. The source and destination of
the requests to these services can be either offchain
or onchain
, and some also
support streaming for offchain destinations. These services are designed to be scalable,
efficient, and easy to integrate with various machine learning models and data sources.
The following workflows are currently supported as Infernet services:
Service | Description | Supports streaming |
---|---|---|
CSS | Closed-Source Software, models like OpenAI | Yes |
HF Client | HuggingFace Inference Client | No |
ONNX | Open Neural Network Exchange | No |
TGI Client | Text Generation Inference Client | Yes |
Torch | PyTorch Inference | No |
EZKL | EZKL Proof Generation | No |