November 18, 2020

679 words 4 mins read



The code for our newly accepted paper in Pattern Recognition 2020: "U^2-Net: Going Deeper with Nested U-Structure for Salient Object Detection."

repo name NathanUA/U-2-Net
repo link
language Python
size (curr.) 20818 kB
stars (curr.) 2798
created 2020-05-04
license Apache License 2.0

U^2-Net (U square net)

The code for our paper U^2-Net (U square net) published in Pattern Recognition 2020:

U^2-Net: Going Deeper with Nested U-Structure for Salient Object Detection

Xuebin Qin, Zichen Zhang, Chenyang Huang, Masood Dehghan, Osmar R. Zaiane and Martin Jagersand.

Contact: xuebin[at]ualberta[dot]ca

Updates !!!

(2020-Nov-21) Recently, we found an interesting application of U^2-Net for human portrait drawing. Therefore, we trained another model for this task based on the APDrawingGAN dataset.

Sample Results: Kids

Sample Results: Ladies

Sample Results: Men

Usage for portrait generation

  1. Clone this repo to local
git clone
  1. Download the u2net_portrait.pth model and put it into the directory: ./saved_models/u2net_portrait/.

  2. Run on the testing set. (1) Download the train and test set from APDrawingGAN. These images and their ground truth are stitched side-by-side (512x1024). You need to split each of these images into two 512x512 images and put them into ./test_data/test_portrait_images/portrait_im/. You can also download the split testing set on GoogleDrive. (2) Running the inference with command python will ouptut the results into ./test_data/test_portrait_images/portrait_results.

  3. Run on your own dataset. (1) Prepare your images and put them into ./test_data/test_portrait_images/your_portrait_im/. To obtain enough details of the protrait, human head region in the input image should be close to or larger than 512x512. The head background should be relatively clear. (2) Run the prediction by command python will outputs the results to ./test_data/test_portrait_images/your_portrait_results/. (3) The difference between python and python is that we added a simple face detection step before the portrait generation in Because the testing set of APDrawingGAN are normalized and cropped to 512x512 for including only heads of humans, while our own dataset may varies with different resolutions and contents. Therefore, the code python will detect the biggest face from the given image and then crop, pad and resize the ROI to 512x512 for feeding to the network. The following figure shows how to take your own photos for generating high quality portraits.

Photo layout

(2020-Sep-13) Our U^2-Net based model is the 6th in MICCAI 2020 Thyroid Nodule Segmentation Challenge.

(2020-May-18) The official paper of our U^2-Net (U square net) (PDF in elsevier(free until July 5 2020), PDF in arxiv) is now available. If you are not able to access that, please feel free to drop me an email.

(2020-May-16) We fixed the upsampling issue of the network. Now, the model should be able to handle arbitrary input size. (Tips: This modification is to facilitate the retraining of U^2-Net on your own datasets. When using our pre-trained model on SOD datasets, please keep the input size as 320x320 to guarantee the performance.)

(2020-May-16) We highly appreciate Cyril Diagne for building this fantastic AR project: AR Copy and Paste using our U^2-Net (Qin et al, PR 2020) and BASNet(Qin et al, CVPR 2019). The demo video in twitter has achieved over 5M views, which is phenomenal and shows us more application possibilities of SOD.

U^2-Net Results (176.3 MB)

U^2-Net Results

Our previous work: BASNet (CVPR 2019)

Required libraries

Python 3.6
numpy 1.15.2
scikit-image 0.14.0
python-opencv PIL 5.2.0
PyTorch 0.4.0
torchvision 0.2.1

Usage for salient object detection

  1. Clone this repo
git clone
  1. Download the pre-trained model u2net.pth (176.3 MB) or u2netp.pth (4.7 MB) and put it into the dirctory ‘./saved_models/u2net/’ and ‘./saved_models/u2netp/’

  2. Cd to the directory ‘U-2-Net’, run the train or inference process by command: python or python respectively. The ‘model_name’ in both files can be changed to ‘u2net’ or ‘u2netp’ for using different models.

We also provide the predicted saliency maps (u2net results,u2netp results) for datasets SOD, ECSSD, DUT-OMRON, PASCAL-S, HKU-IS and DUTS-TE.

U^2-Net Architecture

U^2-Net architecture

Quantitative Comparison

Quantitative Comparison

Quantitative Comparison

Qualitative Comparison

Qualitative Comparison


title = {U2-Net: Going Deeper with Nested U-Structure for Salient Object Detection},
author = {Qin, Xuebin and Zhang, Zichen and Huang, Chenyang and Dehghan, Masood and Zaiane, Osmar and Jagersand, Martin},
journal = {Pattern Recognition},
volume = {106},
pages = {107404},
year = {2020}
comments powered by Disqus