February 29, 2020

311 words 2 mins read

Tencent/DVQA

Tencent/DVQA

Deep learning-based Video Quality Assessment

repo name Tencent/DVQA
repo link https://github.com/Tencent/DVQA
homepage
language Python
size (curr.) 2553 kB
stars (curr.) 127
created 2020-03-05
license

DVQA - Deep learning-based Video Quality Assessment

News

  • 12/17/2019 add pretrained model on PGC videos

Installation

We recommend to run the code with virtualenv. The code is developed with Python3.

Please install other prerequisites with the following command after invoking a virtual env.

pip install -r requirements.txt

All packages are required to run the code.

Dataset

Please prepare a dataset if you want to evaluate in batch or train the code from scratch on your own GPUs. The dataset should be in json format, e.g. your_dataset.json

{
    "test": {
        "dis": ["dis_1.yuv", "dis_2.yuv"],
        "ref": ["ref_1.yuv", "ref_2.yuv"],
        "fps": [30, 24],
        "mos": [94.2, 55.8],
        "height": [1080, 720],
        "width": [1920, 1280]
    },
    "train": {
        "dis": ["dis_3.yuv", "dis_4.yuv"],
        "ref": ["ref_3.yuv", "ref_4.yuv"],
        "fps": [50, 24],
        "mos": [85.2, 51.8],
        "height": [320, 720],
        "width": [640, 1280]
    }
}

For the time being, only YUV is supported. We will update modules to read bitstream.

Eval a dataset

Put all YUV files (both dis and ref) in a folder and prepare your_dataset.json accordingly. Invoke virtualenv and run:

python eval.py --multi_gpu --video_dir /dir/to/yuv --score_file_path /path/to/your_dataset.json --load_model ./save/model_pgc.pt

Train from scratch

Prepare dataset as above and simply run:

python train.py --multi_gpu --video_dir /dir/to/yuv --score_file_path /path/to/your_dataset.json --save_model ./save/your_new_trained.pt

Please check train.sh and opts.py if you would like to tweak other hyper-parameters.

Known issues

The pretrained model was trained on 720P PGC videos compressed with H.264/AVC. It runs well with video of a resolution 1920x1080 and below.

We are not sure about the performance when the code is run with the following scenario,

  1. PGC with other distortion types, especially time-related distortions.
  2. PGC with post-processing filters, like de-nosing, super-resolution, artifacts reduction, etc.
  3. UGC videos with pre-processing filter.
  4. UGC videos compressed with common codecs.

We will try to answer above questions. Stay tuned.

comments powered by Disqus