Welcome to SiMa.ai's ML Model webpage!

A large set of models are supported¹ on the SiMa.ai platform as part of Palette.

This Model List:

• Covers multiple frameworks such as PyTorch and ONNX.
• Draws from various repositories including Torchvision, ONNX Model Zoo, and Open Model Zoo for OpenVINO.

For all supported models, links/instructions are provided for the pre-trained FP32 models along with compilation scripts and PyTorch to ONNX conversion script.

To run the models on this repository using the Palette ModelSDK, make sure you have successfully installed Palette from the SiMa.ai Developer Portal.

SiMa.ai's subset of compatible models references repositories like Torchvison, ONNX Model Zoo, and Open Model Zoo for OpenVINO. These repositories offer pretrained models in floating-point 32-bit (FP32) format that need to be quantized and compiled for SiMa.ai's MLSoC using the Palette's ModelSDK. To this end, certain helper scripts are provided as part of this repository that fetch the models from original pretrained model repositories. Here we describe instructions for getting the models from these repositories. To review model details, refer to the original papers, datasets, etc. mentioned in the corresponding source links provided. Bring your data and get started on running models of interest on SiMa.ai's MLSoC.

Torchvision's torchvision.modelssubpackage offers ML model architectures along with pretrained weights. SiMa.ai's ModelSDK can consume models from PyTorch that include the model topology and weights: either using TorchScript, or exporting the models to ONNX. Given developers' familiarity with ONNX, this repository provides a helper script (torchvision_to_onnx.py) to download the Torchvision model(s) and convert them to ONNX automatically.

• To use the script, either clone the repository or download and copy to the Palette docker image.
• From inside the Palette container, the following command can be used to download and convert models:

user123@9bb247385914:/home/docker/sima-cli/models$ python torchvision_to_onnx.py --model_name densenet121
Downloading: "https://github.com/pytorch/vision/zipball/v0.16.0" to /root/.cache/torch/hub/v0.16.0.zip
/usr/local/lib/python3.10/site-packages/torchvision/models/_utils.py:208: UserWarning: The parameter 'pretrained' is deprecated since 0.13 and may be removed in the future, please use 'weights' instead.
  warnings.warn(
/usr/local/lib/python3.10/site-packages/torchvision/models/_utils.py:223: UserWarning: Arguments other than a weight enum or `None` for 'weights' are deprecated since 0.13 and may be removed in the future. The current behavior is equivalent to passing `weights=DenseNet121_Weights.IMAGENET1K_V1`. You can also use `weights=DenseNet121_Weights.DEFAULT` to get the most up-to-date weights.
  warnings.warn(msg)
Downloading: "https://download.pytorch.org/models/densenet121-a639ec97.pth" to /root/.cache/torch/hub/checkpoints/densenet121-a639ec97.pth
100%|��������������������������������������������������������������������������������������������������������| 30.8M/30.8M [00:00<00:00, 52.6MB/s]
Before torch.onnx.export tensor([[[[ 1.7745,  0.7670, -0.2136,  ..., -1.5743, -0.4873,  1.0913],
          [ 0.0137, -0.9518,  0.8827,  ..., -0.1733, -0.1817,  2.1811],
          [ 0.6135, -0.9099, -2.0007,  ...,  0.3961, -0.4789, -1.5344],
          ...,
          [-1.1500, -0.1356,  0.5894,  ..., -1.2137,  0.8792,  0.6761],
          [-0.3458, -0.6029,  0.9585,  ...,  0.0141, -1.8495, -0.9339],
          [-0.4006, -1.1134, -0.3972,  ..., -0.5107, -0.8084, -1.4360]]]])
============== Diagnostic Run torch.onnx.export version 2.0.1+cpu ==============
verbose: False, log level: Level.ERROR
======================= 0 NONE 0 NOTE 0 WARNING 0 ERROR ========================

After torch.onnx.export

• The downloaded and converted model can be viewed as below.

user123@9bb247385914:/home/docker/sima-cli/models$ ls
densenet121.onnx  LICENSE.txt  README.md	scripts  torchvision_to_onnx.py  

• The model is now successully downloaded from Torchvision repository and ready for usage with Palette tools.

ONNX Model Zoo is a repository of pretrained ML models for various tasks including computer vision. In order to download a Palette-supported pretrained model from ONNX model zoo, the link provided in the Model List section above can be used. Palette has been verified against the model version (indicated by the model name suffix) and it is recommended to download the correct version from the ONNX model zoo. For example, to download mobilenetv2-7 model from ONNX model zoo, the correct version can be located at the link provided in table above and the ONNX model can either be manually downloaded or using the wget as shown below.

user123@9bb247385914:/home/docker/sima-cli/models$ wget https://github.com/onnx/models/blob/main/archive/vision/classification/mobilenet/model/mobilenetv2-7.onnx
--2023-12-22 01:55:36--  https://github.com/onnx/models/blob/main/archive/vision/classification/mobilenet/model/mobilenetv2-7.onnx
Resolving github.com (github.com)... 140.82.112.4
Connecting to github.com (github.com)|140.82.112.4|:443... connected.
HTTP request sent, awaiting response... 200 OK
Length: 10145 (9.9K) [text/plain]
Saving to: ‘mobilenetv2-7.onnx’

mobilenetv2-7.onnx               100%[=========================================================>]   9.91K  --.-KB/s    in 0s      

2023-12-22 01:55:37 (75.9 MB/s) - ‘mobilenetv2-7.onnx’ saved [10145/10145]

Intel's OpenVINO model zoo offers a helper tool omz_downloader to download the pretrained models to local system. This comes as part of openvino-dev package installable via pip command. Follow instructions in OpenVINO installation guide to install omz_downloader and download pretrained OpenVINO models. In cases where the original pretrained model is in *.pth format, it is essential to convert to *.onnx format using omz_converter tool or other PyTorch to ONNX converter tools.

Helper scripts to compile each model are provided through this repository. The source code for these helper scripts can be reviewed using links in the Model List section. These compiler scripts come with multiple preconfigured settings for input resolutions, calibration scheme, quantization method etc. These can be adjusted per needs and full details on how to exercise various compile options are provided in Palette User Guide, available through SiMa.ai developer zone. After cloning this repository, the user should download the model of interest, and access the corresponding script for that model. It is important to ensure the path of model file in the helper script, referenced through model_path variable, is correct.

• The model can be compiled from the Palette docker using this helper script with the command:python [HELPER_SCRIPT]

user123@9bb247385914:/home/docker/sima-cli/models$ python torchvision_to_onnx.py --model_name densenet121
user123@9bb247385914:/home/docker/sima-cli/models$ mkdir models && mv densenet121.onnx models/
user123@9bb247385914:/home/docker/sima-cli/models$ ls ./models
densenet121.onnx
user123@9bb247385914:/home/docker/sima-cli/models$ python scripts/densenet121/densenet121.py
Model SDK version: 1.3.0

Running calibration ...DONE
...
Running quantization ...DONE

• After successful compilation, the resulting files are generated in result/[MODEL_NAME_CALIBRATION_OPTIONS]/mpk folder which now has *.yaml, *.json, *.lm generated as outputs of compilation. These files together can be used for performance estimation as described in the Palette User Guide available as part of Palette.

user123@9bb247385914:/home/docker/sima-cli/models$ ls
LICENSE.txt  models  README.md	result	scripts  torchvision_to_onnx.py
user123@9bb247385914:/home/docker/sima-cli/models$ ls result/
densenet121_asym_True_per_ch_True
user123@9bb247385914:/home/docker/sima-cli/models$ ls result/densenet121_asym_True_per_ch_True/mpk/
densenet121_mpk.tar.gz	densenet121_stage1_mla_compressed.mlc  densenet121_stage1_mla.ifm.mlc  densenet121_stage1_mla.mlc  densenet121_stage1_mla.ofm_chk.mlc

The primary license for the models in the SiMa.ai Model List is the BSD 3-Clause License, see LICENSE. However:

Certain models may be subject to additional restrictions and/or terms. To the extent a LICENSE.txt file is provided for a particular model, please review the LICENSE.txt file carefully as you are responsible for your compliance with such restrictions and/or terms.