A curated list of awesome edge machine learning resources, including research papers, inference engines, challenges, books, meetups and others.

Awesome Edge Machine Learning

A curated list of awesome edge machine learning resources, including research papers, inference engines, challenges, books, meetups and others.

Table of Contents

• Papers
  • Applications
  • AutoML
  • Efficient Architectures
  • Federated Learning
  • ML Algorithms For Edge
  • Network Pruning
  • Others
  • Quantization
• Datasets
• Inference Engines
• Books
• Challenges
• Other Resources
• Contribute
• License

Papers

Applications

There is a countless number of possible edge machine learning applications. Here, we collect papers that describe specific solutions.

AutoML

Automated machine learning (AutoML) is the process of automating the end-to-end process of applying machine learning to real-world problems.Wikipedia AutoML is for example used to design new efficient neural architectures with a constraint on a computational budget (defined either as a number of FLOPS or as an inference time measured on real device) or a size of the architecture.

Efficient Architectures

Efficient architectures represent neural networks with small memory footprint and fast inference time when measured on edge devices.

Federated Learning

Federated Learning enables mobile phones to collaboratively learn a shared prediction model while keeping all the training data on device, decoupling the ability to do machine learning from the need to store the data in the cloud.Google AI blog: Federated Learning

ML Algorithms For Edge

Standard machine learning algorithms are not always able to run on edge devices due to large computational requirements and space complexity. This section introduces optimized machine learning algorithms.

Network Pruning

Pruning is a common method to derive a compact network – after training, some structural portion of the parameters is removed, along with its associated computations.Importance Estimation for Neural Network Pruning

Others

This section contains papers that are related to edge machine learning but are not part of any major group. These papers often deal with deployment issues (i.e. optimizing inference on target platform).

Quantization

Quantization is the process of reducing a precision (from 32 bit floating point into lower bit depth representations) of weights and/or activations in a neural network. The advantages of this method are reduced model size and faster model inference on hardware that support arithmetic operations in lower precision.

Datasets

Visual Wake Words Dataset

Visual Wake Words represents a common microcontroller vision use-case of identifying whether a person is present in the image or not, and provides a realistic benchmark for tiny vision models. Within a limited memory footprint of 250 KB, several state-of-the-art mobile models achieve accuracy of 85-90% on the Visual Wake Words dataset.

Inference Engines

List of machine learning inference engines and APIs that are optimized for execution and/or training on edge devices.

Arm Compute Library

• Source code: https://github.com/ARM-software/ComputeLibrary
• Arm

Bender

• Source code: https://github.com/xmartlabs/Bender
• Documentation: https://xmartlabs.github.io/Bender/
• Xmartlabs

Caffe 2

• Source code: https://github.com/pytorch/pytorch/tree/master/caffe2
• Documentation: https://caffe2.ai/
• Facebook

CoreML

• Documentation: https://developer.apple.com/documentation/coreml
• Apple

Deeplearning4j

• Documentation: https://deeplearning4j.org/docs/latest/deeplearning4j-android
• Skymind

Embedded Learning Library

• Source code: https://github.com/Microsoft/ELL
• Documentation: https://microsoft.github.io/ELL
• Microsoft

Feather CNN

• Source code: https://github.com/Tencent/FeatherCNN
• Tencent

MACE

• Source code: https://github.com/XiaoMi/mace
• Documentation: https://mace.readthedocs.io/
• XiaoMi

MNN

• Source code: https://github.com/alibaba/MNN
• Alibaba

MXNet

• Documentation: https://mxnet.incubator.apache.org/versions/master/faq/smart_device.html
• Amazon

NCNN

• Source code: https://github.com/tencent/ncnn
• Tencent

Neural Networks API

• Documentation: https://developer.android.com/ndk/guides/neuralnetworks/
• Google

Paddle Mobile

• Source code: https://github.com/PaddlePaddle/paddle-mobile
• Baidu

Qualcomm Neural Processing SDK for AI

• Source code: https://developer.qualcomm.com/software/qualcomm-neural-processing-sdk
• Qualcomm

Tengine

• Source code: https://github.com/OAID/Tengine
• OAID

TensorFlow Lite

• Source code: https://github.com/tensorflow/tensorflow/tree/master/tensorflow/lite
• Documentation: https://www.tensorflow.org/lite/
• Google

dabnn

• Source code: https://github.com/JDAI-CV/dabnn
• JDAI Computer Vision

Books

List of books with focus on on-device (e.g., edge or mobile) machine learning.

TinyML: Machine Learning with TensorFlow on Arduino, and Ultra-Low Power Micro-Controllers

• Authors: Pete Warden, Daniel Situnayake
• Published: 2020

Machine Learning by Tutorials: Beginning machine learning for Apple and iOS

• Author: Matthijs Hollemans
• Published: 2019

Core ML Survival Guide

• Author: Matthijs Hollemans
• Published: 2018

Building Mobile Applications with TensorFlow

• Author: Pete Warden
• Published: 2017

Challenges

Low Power Recognition Challenge (LPIRC)

Competition with focus on the best vision solutions that can simultaneously achieve high accuracy in computer vision and energy efficiency. LPIRC is regularly held during computer vision conferences (CVPR, ICCV and others) since 2015 and the winners’ solutions have already improved 24 times in the ratio of accuracy divided by energy.

• Online Track

• Onsite Track

Other Resources

Awesome EMDL

Embedded and mobile deep learning research resources

Awesome Mobile Machine Learning

A curated list of awesome mobile machine learning resources for iOS, Android, and edge devices

Awesome Pruning

A curated list of neural network pruning resources

Efficient DNNs

Collection of recent methods on DNN compression and acceleration

Machine Think

Machine learning tutorials targeted for iOS devices

Pete Warden’s blog

Contribute

Unlike other awesome list, we are storing data in YAML format and markdown files are generated with awesome.py script.

Every directory contains data.yaml which stores data we want to display and config.yaml which stores its metadata (e.g. way of sorting data). The way how data will be presented is defined in renderer.py.

License

To the extent possible under law, Bisonai has waived all copyright and related or neighboring rights to this work.