uTensor/uTensor
TinyML AI inference library
| repo name | uTensor/uTensor |
| repo link | https://github.com/uTensor/uTensor |
| homepage | |
| language | C++ |
| size (curr.) | 8223 kB |
| stars (curr.) | 1089 |
| created | 2017-09-21 |
| license | Apache License 2.0 |
uTensor - Test Release
Note: If you are looking for stable releases, checkout master.
Release Note
- Updated uTensor ReadMe
- Updated uTensor-CLI ReadMe
- ROM Tensor support
Introduction
What is it?
uTensor is an extremely light-weight machine learning inference framework built on Mbed and Tensorflow. It consists of a runtime library and an offline tool. The total size of graph definition and algorithm implementation of a 3-layer MLP produced by uTensor is less than 32kB in the resulting binary (excluding the weights).
How does it work?
A model is constructed and trained in Tensorflow. uTensor takes the model and produces a .cpp and .hpp file. These files contains the generated C++11 code needed for inferencing. Working with uTensor on the embedded side is as easy as copy-and-paste. The function interface looks like this:
#include "models/deep_mlp.hpp"
...
Context ctx; //creating a context
...
//preparing for the input tensor
...
get_deep_mlp_ctx(Context& ctx, Tensor* input_0); //perform inference
ctx.eval();
S_TENSOR prediction = ctx.get({"y_pred:0"}); //getting the result
The .hpp and .cpp files can be generated given a model (protocal buffer) file, for example:
$ utensor-cli deep_mlp.pb --output-nodes=y_pred
...
... Generate weight file: models/deep_mlp_weight.hpp
... Generate header file: models/deep_mlp.hpp
... Generate source file: models/deep_mlp.cpp
What’s supported?
The project is work-in-progress. Here are the operators, of their quantized versions, that are currently avaliable:
- Add
- ArgMax
- Dropout
- MatMul
- Max
- Min
- Placeholder
- Quantization Ops
- ReLu
- Reshape
Quick Start
Hardware
uTensor should support any Mbed enabled board that has sufficient memory (128+ kB RAM and 512kB+ flash recommended). However, these two boards are popular among the core developers:
- FRDM-K66F: reference development environment
- DISCO-F413ZH: a good demo/application prototyping platform, wi-fi
- Any Mbed board with sufficient memory
The Environment
Getting Started
Creating a New Project
An end-to-end tutorial going from training a neural network to deployment on a device. You will need a K66F or a smiliar board for this tutorial.
MNIST Touch Screen
The example uses a 3-layer MLP trained on the MNIST dataset. The touch screen input is fed into the neural network for processing and the result is printed on the screen.
The Activity of Daily Living (ADL)
This example shows how to buffer time-series data into batches of snapshots. These snapshots are then fed into the neural network for inferencing. The model a small multi-layer MLP trained on the ADL dataset.
Development
uTensor is young and under going rapid development. Many exciting features are on the way:
- Convolution
- Pooling
- CMSIS-NN integration
- Smaller binary
- More efficient Run-time
You can also check the project page for the latest progress. If you’d like to take part in this project, please have a look at our contributor guide and feel free to reach out to us.
