Models
Pipelogic relies on Nvidia Triton and Torchserve to run inference with ML models. Only exported frozen models such as Tensorflow, PyTorch, Onnx can be used. Each Machine learning framework provides their own tool for exporting models.
Conversion tools of the most commonly used libraries:
Download the Model
git clone https://huggingface.co/hustvl/yolos-tiny
Install Conversion Tool
pip install optimum[exporters] polygraphy
Freeze Model
optimum-cli export onnx --model hustvl/yolos-tiny .
Rename exported model to model.onnx.
Inspect Model
Inspect Model to find input and output layers
polygraphy inspect model model.onnx --mode=onnx
Prompt
[I] ==== ONNX Model ====
Name: main_graph | ONNX Opset: 12
---- 1 Graph Input(s) ----
{pixel_values [dtype=float32, shape=('batch_size', 'num_channels', 'height', 'width')]}
---- 2 Graph Output(s) ----
{logits [dtype=float32, shape=('batch_size', 'num_queries', 92)],
pred_boxes [dtype=float32, shape=('batch_size', 'num_queries', 4)]}
---- 214 Initializer(s) ----
---- 1306 Node(s) ----
Create Model Configuration
Triton needs Model Configuration config.pbtxt file to run the model. Check triton documentation on how to prepare it.
Enter input and output model layers from previous step into config.pbtxt file.
config.pbtxt
input: [
{
name: "pixel_values",
data_type: TYPE_FP32,
dims: [
1,
3,
512,
1333
]
}
],
output: [
{
name: "logits",
data_type: TYPE_FP32,
dims: [
-1,
-1,
92
]
},
{
name: "pred_boxes",
data_type: TYPE_FP32,
dims: [
-1,
-1,
4
]
}
],
Prepare Model Structure
Models should follow Triton Inference Server file structure.
Model Structures
+-- model_name
|
+-- config.pbtxt
+-- 1
|
+-- model.onnx
Next Steps: