April 6, 2020

363 words 2 mins read



Code for our CVPR 2020 work.

repo name XiaohangZhan/deocclusion
repo link https://github.com/XiaohangZhan/deocclusion
language Python
size (curr.) 30205 kB
stars (curr.) 334
created 2020-03-10
license Apache License 2.0


Xiaohang Zhan, Xingang Pan, Bo Dai, Ziwei Liu, Dahua Lin, Chen Change Loy, “Self-Supervised Scene De-occlusion”, accepted to CVPR 2020 as an Oral Paper. [Project page].

For further information, please contact Xiaohang Zhan.

Demo Video

  • Watch the full demo video in YouTube or bilibili. The demo video contains vivid explanations of the idea, and interesting applications.

  • Below is an application of scene de-occlusion: image manipulation.


  • pytorch>=0.4.1

    pip install -r requirements.txt

Data Preparation

COCOA dataset proposed in Semantic Amodal Segmentation.

  1. Download COCO2014 train and val images from here and unzip.

  2. Download COCOA annotations from here and untar.

  3. Ensure the COCOA folder looks like:

      |-- train2014/
      |-- val2014/
      |-- annotations/
        |-- COCO_amodal_train2014.json
        |-- COCO_amodal_val2014.json
        |-- COCO_amodal_test2014.json
        |-- ...
  4. Create symbolic link:

    cd deocclusion
    mkdir data
    cd data
    ln -s /path/to/COCOA

KINS dataset proposed in Amodal Instance Segmentation with KINS Dataset.

  1. Download left color images of object data in KITTI dataset from here and unzip.

  2. Download KINS annotations from here corresponding to this commit.

  3. Ensure the KINS folder looks like:

      |-- training/image_2/
      |-- testing/image_2/
      |-- instances_train.json
      |-- instances_val.json
  4. Create symbolic link:

    cd deocclusion/data
    ln -s /path/to/KINS

LVIS dataset

  1. Download training and validation sets from here

Run Demos

  1. Download released models here and put the folder released under deocclusion.

  2. Run demos/demo_cocoa.ipynb or demos/demo_kins.ipynb.


train PCNet-M

  1. Train (taking COCOA for example).

    sh experiments/COCOA/pcnet_m/train.sh
  2. Monitoring status and visual results using tensorboard.

    sh tensorboard.sh $PORT

train PCNet-C

  1. Download the pre-trained image inpainting model using partial convolution here to pretrains/partialconv.pth

  2. Convert the model to accept 4 channel inputs.

    python tools/convert_pcnetc_pretrain.py
  3. Train (taking COCOA for example).

    sh experiments/COCOA/pcnet_c/train.sh
  4. Monitoring status and visual results using tensorboard.


  • Execute:

    sh tools/test_cocoa.sh


 author = {Zhan, Xiaohang and Pan, Xingang and Dai, Bo and Liu, Ziwei and Lin, Dahua and Loy, Chen Change},
 title = {Self-Supervised Scene De-occlusion},
 booktitle = {Proceedings of the IEEE conference on computer vision and pattern recognition (CVPR)},
 month = {June},
 year = {2020}


We used the code and models of GCA-Matting in our demo.

comments powered by Disqus