November 24, 2019

544 words 3 mins read



Code for “Fast and Robust Multi-Person 3D Pose Estimation from Multiple Views” in CVPR'19

repo name zju3dv/mvpose
repo link
language Jupyter Notebook
size (curr.) 52199 kB
stars (curr.) 237
created 2019-01-13
license Other

Fast and Robust Multi-Person 3D Pose Estimation from Multiple Views

Fast and Robust Multi-Person 3D Pose Estimation from Multiple Views
Junting Dong, Wen Jiang, Qixing Huang, Hujun Bao, Xiaowei Zhou
CVPR 2019 Project Page

Any questions or discussions are welcomed!


  • Set up python environment
pip install -r requirements.txt
  • Compile the backend/tf_cpn/lib and backend/light_head_rcnn/lib
cd mvpose/backend/tf_cpn/lib/
cd ./lib_kernel/lib_nms
cd mvpose/backend/light_head_rcnn/lib/

Since they use py-faster-rcnn as backbone. Many people using faster-rcnn meet with some problems when compiling those components. Suggestions on google can be helpful.

  • Compile the pictorial function for acceleration
cd mvpose/src/m_lib/
python build_ext --inplace

Prepare models and datasets

  • Prepare models: Please put light-head-rcnn models to backend/light_head_rcnn/output/model_dump, backend/tf_cpn/log/model_dump to backend/tf_cpn/log/model_dump, and CamStyle model trained by myself to backend/CamStyle/logs

  • Prepare the datasets: Put datasets such as Shelf and CampusSeq1 to ./datasets/ Download Campus and Shelf datasets. Then, put datasets such as Shelf and CampusSeq1 to datasets/

  • Generate the camera parameters: Since each dataset uses different way to obtain the camera parameters, we show an example to deal with the Campus dataset:

    • Add following code to .datasets/CampusSeq1/Calibration/producePmat.m
    K = cell(1,3);
    K{1} = K1; K{2} = K2; K{3} = K3;
    m_RT = cell(1,3);
    m_RT{1} = RT1; m_RT{2} = RT2; m_RT{3} = RT3;
    save('m_RT.mat', 'm_RT');
    save('P.mat', 'P');
    • generate the camera_parameter.pickle
    python ./src/tools/ /parameter/dir ./datasets/CampusSeq1

    Here, we also provide the camera_parameter.pickle of Campus and Shelf. You can generate the .pickle file for your datasets using the same way.

Demo and Evaluate

Run the demo

python ./src/m_utils/ -d Campus
python ./src/m_utils/ -d Shelf

If all the configuration is OK, you may see the visualization of following items.


Evaluate on the Campus/Shelf datasets

python ./src/m_utils/ -d Campus
python ./src/m_utils/ -d Shelf

As long as the progress bar finished, you may see a beautified table of evaluation result and a csv file for the evaluation result will be save in ./result directory.

Accelerate the evaluation

Since the 2D pose estimator (CPN) is a little slow, we can save the predicted 2D poses and heatmaps and then start with these saved files.

  1. produce the files
python src/tools/ -d Campus -dump_dir ./datasets/Campus_processed
python src/tools/ -d Shelf -dump_dir ./datasets/Shelf_processed
  1. evaluate with saved 2D poses and heatmaps
python ./src/m_utils/ -d Campus -dumped ./datasets/Campus_processed
python ./src/m_utils/ -d Shelf -dumped ./datasets/Shelf_processed

Note: for the sake of convenience, we do not optimize on the size of dumped file. Therefore, the size of Campus_processed is around 4.0G and the size of Shelf_processed is around 234G. Please make sure your disk have 200+G free space. Any pull request to solve this issues will be welcomed.


If you find this code useful for your research, please use the following BibTeX entry.

  title={Fast and Robust Multi-Person 3D Pose Estimation from Multiple Views},
  author={Dong, Junting and Jiang, Wen and Huang, Qixing and Bao, Hujun and Zhou, Xiaowei},


This code uses these code (Light head rcnn, Cascaded Pyramid Network, CamStyle) as backbone. We gratefully appreciate the impact it had on our work. If you use our code, please consider citing the original paper as well.

comments powered by Disqus