Official Python toolkit for generic object tracking benchmark GOT-10k and beyond

GOT-10k Python Toolkit

UPDATE: All common tracking datasets (GOT-10k, OTB, VOT, UAV, TColor, DTB, NfS, LaSOT and TrackingNet) are supported. Support VOT2019 (ST/LT/RGBD/RGBT) downloading. Fix the randomness in ImageNet-VID (issue #13).

Run experimenets over common tracking benchmarks (code from siamfc):

This repository contains the official python toolkit for running experiments and evaluate performance on GOT-10k benchmark. The code is written in pure python and is compile-free. Although we support both python2 and python3, we recommend python3 for better performance.

For convenience, the toolkit also provides unofficial implementation of dataset interfaces and tracking pipelines for OTB (2013/2015), VOT (2013~2018), DTB70, TColor128, NfS (30/240 fps), UAV (123/20L), LaSOT and TrackingNet benchmarks. It also offers interfaces for ILSVRC VID and YouTube-BoundingBox (comming soon!) datasets.

GOT-10k is a large, high-diversity and one-shot database for training and evaluating generic purposed visual trackers. If you use the GOT-10k database or toolkits for a research publication, please consider citing:

"GOT-10k: A Large High-Diversity Benchmark for Generic Object Tracking in the Wild."
L. Huang, X. Zhao and K. Huang,
arXiv:1810.11981, 2018.

 [Project][PDF][Bibtex]

Table of Contents

• Installation
• Quick Start: A Concise Example
• Quick Start: Jupyter Notebook for Off-the-Shelf Usage
• How to Define a Tracker?
• How to Run Experiments on GOT-10k?
• How to Evaluate Performance?
• How to Plot Success Curves?
• How to Loop Over GOT-10k Dataset?
• Issues
• Contributors

Installation

Install the toolkit using pip (recommended):

pip install --upgrade got10k

Stay up-to-date:

pip install --upgrade git+https://github.com/got-10k/toolkit.git@master

Or, alternatively, clone the repository and install dependencies:

git clone https://github.com/got-10k/toolkit.git
cd toolkit
pip install -r requirements.txt

Then directly copy the got10k folder to your workspace to use it.

Quick Start: A Concise Example

Here is a simple example on how to use the toolkit to define a tracker, run experiments on GOT-10k and evaluate performance.

from got10k.trackers import Tracker
from got10k.experiments import ExperimentGOT10k

class IdentityTracker(Tracker):
    def __init__(self):
        super(IdentityTracker, self).__init__(name='IdentityTracker')
    
    def init(self, image, box):
        self.box = box

    def update(self, image):
        return self.box

if __name__ == '__main__':
    # setup tracker
    tracker = IdentityTracker()

    # run experiments on GOT-10k (validation subset)
    experiment = ExperimentGOT10k('data/GOT-10k', subset='val')
    experiment.run(tracker, visualize=True)

    # report performance
    experiment.report([tracker.name])

To run experiments on OTB, VOT or other benchmarks, simply change ExperimentGOT10k, e.g., to ExperimentOTB or ExperimentVOT, and root_dir to their corresponding paths for this purpose.

Quick Start: Jupyter Notebook for Off-the-Shelf Usage

Open quick_examples.ipynb in Jupyter Notebook to see more examples on toolkit usage.

How to Define a Tracker?

To define a tracker using the toolkit, simply inherit and override init and update methods from the Tracker class. Here is a simple example:

from got10k.trackers import Tracker

class IdentityTracker(Tracker):
    def __init__(self):
        super(IdentityTracker, self).__init__(
            name='IdentityTracker',  # tracker name
            is_deterministic=True    # stochastic (False) or deterministic (True)
        )
    
    def init(self, image, box):
        self.box = box

    def update(self, image):
        return self.box

How to Run Experiments on GOT-10k?

Instantiate an ExperimentGOT10k object, and leave all experiment pipelines to its run method:

from got10k.experiments import ExperimentGOT10k

# ... tracker definition ...

# instantiate a tracker
tracker = IdentityTracker()

# setup experiment (validation subset)
experiment = ExperimentGOT10k(
    root_dir='data/GOT-10k',    # GOT-10k's root directory
    subset='val',               # 'train' | 'val' | 'test'
    result_dir='results',       # where to store tracking results
    report_dir='reports'        # where to store evaluation reports
)
experiment.run(tracker, visualize=True)

The tracking results will be stored in result_dir.

How to Evaluate Performance?

Use the report method of ExperimentGOT10k for this purpose:

# ... run experiments on GOT-10k ...

# report tracking performance
experiment.report([tracker.name])

When evaluated on the validation subset, the scores and curves will be directly generated in report_dir.

However, when evaluated on the test subset, since all groundtruths are withholded, you will have to submit your results to the evaluation server for evaluation. The report function will generate a .zip file which can be directly uploaded for submission. For more instructions, see submission instruction.

See public evaluation results on GOT-10k’s leaderboard.

How to Plot Success Curves?

Assume that a list of all performance files (JSON files) are stored in report_files, here is an example showing how to plot success curves:

from got10k.experiments import ExperimentGOT10k

report_files = ['reports/GOT-10k/performance_25_entries.json']
tracker_names = ['SiamFCv2', 'GOTURN', 'CCOT', 'MDNet']

# setup experiment and plot curves
experiment = ExperimentGOT10k('data/GOT-10k', subset='test')
experiment.plot_curves(report_files, tracker_names)

The report file of 25 baseline entries can be downloaded from the Downloads page. You can also download single report file for each entry from the Leaderboard page.

How to Loop Over GOT-10k Dataset?

The got10k.datasets.GOT10k provides an iterable and indexable interface for GOT-10k’s sequences. Here is an example:

from PIL import Image
from got10k.datasets import GOT10k
from got10k.utils.viz import show_frame

dataset = GOT10k(root_dir='data/GOT-10k', subset='train')

# indexing
img_file, anno = dataset[10]

# for-loop
for s, (img_files, anno) in enumerate(dataset):
    seq_name = dataset.seq_names[s]
    print('Sequence:', seq_name)

    # show all frames
    for f, img_file in enumerate(img_files):
        image = Image.open(img_file)
        show_frame(image, anno[f, :])

To loop over OTB or VOT datasets, simply change GOT10k to OTB or VOT for this purpose.

Issues

Please report any problems or suggessions in the Issues page.

Contributors

• Lianghua Huang