A collection of pre-trained, state-of-the-art models in the ONNX format

repo name	onnx/models
repo link	https://github.com/onnx/models
homepage
language	Jupyter Notebook
size (curr.)	165588 kB
stars (curr.)	2030
created	2017-10-06
license	MIT License

ONNX Model Zoo

Open Neural Network Exchange (ONNX) is an open standard format for representing machine learning models. ONNX is supported by a community of partners who have implemented it in many frameworks and tools.

The ONNX Model Zoo is a collection of pre-trained, state-of-the-art models in the ONNX format contributed by community members like you. Accompanying each model are Jupyter notebooks for model training and running inference with the trained model. The notebooks are written in Python and include links to the training dataset as well as references to the original paper that describes the model architecture.

We are standardizing on Git LFS (Large File Storage) to store our ONNX model files. To download an ONNX model, navigate to the appropriate Github page and click the Download button on the top right.

Models

Read the Usage section below for more details on the file formats in the ONNX Model Zoo (.onnx, .pb, .npz), downloading multiple ONNX models through Git LFS command line, and starter Python code for validating your ONNX model using test data.

Vision

• Image Classification
• Object Detection & Image Segmentation
• Body, Face & Gesture Analysis
• Image Manipulation

Language

• Machine Comprehension
• Machine Translation
• Language Modelling

Other

• Visual Question Answering & Dialog
• Speech & Audio Processing
• Other interesting models

Image Classification

This collection of models take images as input, then classifies the major objects in the images into 1000 object categories such as keyboard, mouse, pencil, and many animals.

Model Class	Reference	Description
MobileNet	Sandler et al.	Light-weight deep neural network best suited for mobile and embedded vision applications. Top-5 error from paper - ~10%
ResNet	He et al.	A CNN model (up to 152 layers). Uses shortcut connections to achieve higher accuracy when classifying images. Top-5 error from paper - ~3.6%
SqueezeNet	Iandola et al.	A light-weight CNN model providing AlexNet level accuracy with 50x fewer parameters. Top-5 error from paper - ~20%
VGG	Simonyan et al.	Deep CNN model(up to 19 layers). Similar to AlexNet but uses multiple smaller kernel-sized filters that provides more accuracy when classifying images. Top-5 error from paper - ~8%
AlexNet	Krizhevsky et al.	A Deep CNN model (up to 8 layers) where the input is an image and the output is a vector of 1000 numbers. Top-5 error from paper - ~15%
GoogleNet	Szegedy et al.	Deep CNN model(up to 22 layers). Comparatively smaller and faster than VGG and more accurate in detailing than AlexNet. Top-5 error from paper - ~6.7%
CaffeNet	Krizhevsky et al.	Deep CNN variation of AlexNet for Image Classification in Caffe where the max pooling precedes the local response normalization (LRN) so that the LRN takes less compute and memory.
RCNN_ILSVRC13	Girshick et al.	Pure Caffe implementation of R-CNN for image classification. This model uses localization of regions to classify and extract features from images.
DenseNet-121	Huang et al.	Model that has every layer connected to every other layer and passes on its own feature providing strong gradient flow and more diversified features.
Inception_V1	Szegedy et al.	This model is same as GoogLeNet, implemented through Caffe2 that has improved utilization of the computing resources inside the network and helps with the vanishing gradient problem. Top-5 error from paper - ~6.7%
Inception_V2	Szegedy et al.	Deep CNN model for Image Classification as an adaptation to Inception v1 with batch normalization. This model has reduced computational cost and improved image resolution compared to Inception v1. Top-5 error from paper ~4.82%
ShuffleNet_V1	Zhang et al.	Extremely computation efficient CNN model that is designed specifically for mobile devices. This model greatly reduces the computational cost and provides a ~13x speedup over AlexNet on ARM-based mobile devices. Compared to MobileNet, ShuffleNet achieves superior performance by a significant margin due to it’s efficient structure. Top-1 error from paper - ~32.6%
ShuffleNet_V2	Zhang et al.	Extremely computation efficient CNN model that is designed specifically for mobile devices. This network architecture design considers direct metric such as speed, instead of indirect metric like FLOP. Top-1 error from paper - ~30.6%
ZFNet-512	Zeiler et al.	Deep CNN model (up to 8 layers) that increased the number of features that the network is capable of detecting that helps to pick image features at a finer level of resolution. Top-5 error from paper - ~14.3%

Domain-based Image Classification

This subset of models classify images for specific domains and datasets.

Model Class	Reference	Description
MNIST-Handwritten Digit Recognition	Convolutional Neural Network with MNIST	Deep CNN model for handwritten digit identification

Object Detection & Image Segmentation

Object detection models detect the presence of multiple objects in an image and segment out areas of the image where the objects are detected. Semantic segmentation models partition an input image by labeling each pixel into a set of pre-defined categories.

Model Class	Reference	Description
Tiny YOLOv2	Redmon et al.	A real-time CNN for object detection that detects 20 different classes. A smaller version of the more complex full YOLOv2 network.
SSD	Liu et al.	Single Stage Detector: real-time CNN for object detection that detects 80 different classes.
Faster-RCNN	Ren et al.	Increases efficiency from R-CNN by connecting a RPN with a CNN to create a single, unified network for object detection that detects 80 different classes.
Mask-RCNN	He et al.	A real-time neural network for object instance segmentation that detects 80 different classes. Extends Faster R-CNN as each of the 300 elected ROIs go through 3 parallel branches of the network: label prediction, bounding box prediction and mask prediction.
YOLO v2	Redmon et al.	A CNN model for real-time object detection system that can detect over 9000 object categories. It uses a single network evaluation, enabling it to be more than 1000x faster than R-CNN and 100x faster than Faster R-CNN.
YOLO v2-coco	Redmon et al.	A CNN model for real-time object detection system that can detect over 9000 object categories. It uses a single network evaluation, enabling it to be more than 1000x faster than R-CNN and 100x faster than Faster R-CNN. This model is trained with COCO dataset and contains 80 classes.
YOLO v3	Redmon et al.	A deep CNN model for real-time object detection that detects 80 different classes. A little bigger than YOLOv2 but still very fast. As accurate as SSD but 3 times faster.
Tiny YOLOv3	Redmon et al.	A smaller version of YOLOv3 model.
DUC	Wang et al.	Deep CNN based pixel-wise semantic segmentation model with >80% mIOU (mean Intersection Over Union). Trained on cityscapes dataset, which can be effectively implemented in self driving vehicle systems.
FCN	Long et al.	Deep CNN based segmentation model trained end-to-end, pixel-to-pixel that produces efficient inference and learning. Built off of AlexNet, VGG net, GoogLeNet classification methods. contribute

Body, Face & Gesture Analysis

Face detection models identify and/or recognize human faces and emotions in given images. Body and Gesture Analysis models identify gender and age in given image.

Model Class	Reference	Description
ArcFace	Deng et al.	A CNN based model for face recognition which learns discriminative features of faces and produces embeddings for input face images.
CNN Cascade	Li et al.	The model operates at multiple resolutions, quickly rejecting the background regions in the fast low resolution stages in an image and carefully evaluates a small number of challenging candidates in the last high resolution stage. contribute
Emotion FerPlus	Barsoum et al.	Deep CNN for emotion recognition trained on images of faces.
Age and Gender Classification using Convolutional Neural Networks	Levi et al.	This model accurately classifies gender and age even the amount of learning data is limited.contribute

Image Manipulation

Image manipulation models use neural networks to transform input images to modified output images. Some popular models in this category involve style transfer or enhancing images by increasing resolution.

Model Class	Reference	Description
Unpaired Image to Image Translation using Cycle consistent Adversarial Network	Zhu et al.	The model uses learning to translate an image from a source domain X to a target domain Y in the absence of paired examples. contribute
Super Resolution with sub-pixel CNN	Shi et al.	A deep CNN that uses sub-pixel convolution layers to upscale the input image.
Fast Neural Style Transfer	Johnson et al.	This method uses a loss network pretrained for image classification to define perceptual loss functions that measure perceptual differences in content and style between images. The loss network remains fixed during the training process.

Speech & Audio Processing

This class of models uses audio data to train models that can identify voice, generate music, or even read text out loud.

Model Class	Reference	Description
Speech recognition with deep recurrent neural networks	Graves et al.	A RNN model for sequential data for speech recognition. Labels problems where the input-output alignment is unknowncontribute
Deep voice: Real time neural text to speech	Arik et al.	A DNN model that performs end-to-end neural speech synthesis. Requires fewer parameters and it is faster than other systems. contribute
Sound Generative models	WaveNet: A Generative Model for Raw Audio	A CNN model that generates raw audio waveforms. Has predictive distribution for each audio sample. Generates realistic music fragments. contribute

Machine Comprehension

This subset of natural language processing models that answer questions about a given context paragraph.

Model Class	Reference	Description
Bidirectional Attention Flow	Seo et al.	A model that answers a query about a given context paragraph.
BERT-Squad	Devlin et al.	This model answers questions based on the context of the given input paragraph.
GPT-2	Radford et al.	A large transformer-based language model that given a sequence of words within some text, predicts the next word.

Machine Translation

This class of natural language processing models learns how to translate input text to another language.

Model Class	Reference	Description
Neural Machine Translation by jointly learning to align and translate	Bahdanau et al.	Aims to build a single neural network that can be jointly tuned to maximize the translation performance. contribute
Google’s Neural Machine Translation System	Wu et al.	This model helps to improve issues faced by the Neural Machine Translation (NMT) systems like parallelism that helps accelerate the final translation speed.contribute

Language Modelling

This subset of natural language processing models learns representations of language from large corpuses of text.

Model Class	Reference	Description
Deep Neural Network Language Models	Arisoy et al.	A DNN acoustic model. Used in many natural language technologies. Represents a probability distribution over all possible word strings in a language. contribute

Visual Question Answering & Dialog

This subset of natural language processing models uses input images to answer questions about those images.

Model Class	Reference	Description
VQA: Visual Question Answering	Agrawal et al.	A model that takes an image and a free-form, open-ended natural language question about the image and outputs a natural-language answer. contribute
Yin and Yang: Balancing and Answering Binary Visual Questions	Zhang et al.	Addresses VQA by converting the question to a tuple that concisely summarizes the visual concept to be detected in the image. Next, if the concept can be found in the image, it provides a “yes” or “no” answer. Its performance matches the traditional VQA approach on unbalanced dataset, and outperforms it on the balanced dataset. contribute
Making the V in VQA Matter	Goyal et al.	Balances the VQA dataset by collecting complementary images such that every question is associated with a pair of similar images that result in two different answers to the question, providing a unique interpretable model that provides a counter-example based explanation. contribute
Visual Dialog	Das et al.	An AI agent that holds a meaningful dialog with humans in natural, conversational language about visual content. Curates a large-scale Visual Dialog dataset (VisDial). contribute

Other interesting models

There are many interesting deep learning models that do not fit into the categories described above. The ONNX team would like to highly encourage users and researchers to contribute their models to the growing model zoo.

Model Class	Reference	Description
Text to Image	Generative Adversarial Text to image Synthesis	Effectively bridges the advances in text and image modeling, translating visual concepts from characters to pixels. Generates plausible images of birds and flowers from detailed text descriptions. contribute
Time Series Forecasting	Modeling Long- and Short-Term Temporal Patterns with Deep Neural Networks	The model extracts short-term local dependency patterns among variables and to discover long-term patterns for time series trends. It helps to predict solar plant energy output, electricity consumption, and traffic jam situations. contribute
Recommender systems	DropoutNet: Addressing Cold Start in Recommender Systems	A collaborative filtering method that makes predictions about an individual’s preference based on preference information from other users.contribute
Collaborative filtering	Neural Collaborative Filtering	A DNN model based on the interaction between user and item features using matrix factorization. contribute
Autoencoders	A Hierarchical Neural Autoencoder for Paragraphs and Documents	An LSTM (long-short term memory) auto-encoder to preserve and reconstruct multi-sentence paragraphs.contribute

Usage

Every ONNX backend should support running the models out of the box. After downloading and extracting the tarball of each model, you will find:

• A protobuf file model.onnx that represents the serialized ONNX model.
• Test data (in the form of serialized protobuf TensorProto files or serialized NumPy archives).

Usage - Test data starter code

The test data files can be used to validate ONNX models from the Model Zoo. We have provided the following interface examples for you to get started. Please replace onnx_backend in your code with the appropriate framework of your choice that provides ONNX inferencing support, and likewise replace backend.run_model with the framework’s model evaluation logic.

There are two different formats for the test data files:

• Serialized protobuf TensorProtos (.pb), stored in folders with the naming convention test_data_set_*.

import numpy as np
import onnx
import os
import glob
import onnx_backend as backend

from onnx import numpy_helper

model = onnx.load('model.onnx')
test_data_dir = 'test_data_set_0'

# Load inputs
inputs = []
inputs_num = len(glob.glob(os.path.join(test_data_dir, 'input_*.pb')))
for i in range(inputs_num):
    input_file = os.path.join(test_data_dir, 'input_{}.pb'.format(i))
    tensor = onnx.TensorProto()
    with open(input_file, 'rb') as f:
        tensor.ParseFromString(f.read())
    inputs.append(numpy_helper.to_array(tensor))

# Load reference outputs
ref_outputs = []
ref_outputs_num = len(glob.glob(os.path.join(test_data_dir, 'output_*.pb')))
for i in range(ref_outputs_num):
    output_file = os.path.join(test_data_dir, 'output_{}.pb'.format(i))
    tensor = onnx.TensorProto()
    with open(output_file, 'rb') as f:
        tensor.ParseFromString(f.read())
    ref_outputs.append(numpy_helper.to_array(tensor))

# Run the model on the backend
outputs = list(backend.run_model(model, inputs))

# Compare the results with reference outputs.
for ref_o, o in zip(ref_outputs, outputs):
    np.testing.assert_almost_equal(ref_o, o)

• Serialized Numpy archives, stored in files with the naming convention test_data_*.npz. Each file contains one set of test inputs and outputs.

import numpy as np
import onnx
import onnx_backend as backend

# Load the model and sample inputs and outputs
model = onnx.load(model_pb_path)
sample = np.load(npz_path, encoding='bytes')
inputs = list(sample['inputs'])
outputs = list(sample['outputs'])

# Run the model with an onnx backend and verify the results
np.testing.assert_almost_equal(outputs, backend.run_model(model, inputs))

Usage - Git LFS

On default, cloning this repository will not download any ONNX models. Install Git LFS with pip install git lfs.

To download a specific model: git lfs pull --include="[MODELNAME].onnx"

To download all models: git lfs pull --include="*" --exclude=""

Usage - Model visualization

You can see visualizations of each model’s network architecture by using Netron.

Contributions

Do you want to contribute a model? To get started, pick any model presented above with the contribute link under the Description column. The links point to a page containing guidelines for making a contribution.

License

MIT License