📖 For more visual results, go checkout our project page
This repository will contain the official implementation of SIFU.
- [2024/6/18] Due to visa check problem, the author can not come to the conference center in person. We are sorry about this [sad][cry].
- [2024/4/5] Our paper has been accepted as Highlight (Top 11.9% of accepted papers)!
- [2024/2/28] We release the code of geometry reconstruction, including test and inference.
- [2024/2/27] SIFU has been accepted by CVPR 2024! See you in Seattle!
- [2023/12/13] We release the paper on arXiv.
- [2023/12/10] We build the Project Page.
- Ubuntu 20 / 18
- CUDA=11.6 or 11.7 or 11.8, GPU Memory > 16GB
- Python = 3.8
- PyTorch = 1.13.0 (official Get Started)
We thank @levnikolaevich and @GuangtaoLyu for provide valuable advice on the installation steps.
If you don't have conda or miniconda, please install that first:
sudo apt-get update && \
sudo apt-get upgrade -y && \
sudo apt-get install unzip libeigen3-dev ffmpeg build-essential nvidia-cuda-toolkit
mkdir -p ~/miniconda3 && \
wget https://repo.anaconda.com/miniconda/Miniconda3-latest-Linux-x86_64.sh -O ~/miniconda3/miniconda.sh && \
bash ~/miniconda3/miniconda.sh -b -u -p ~/miniconda3 && \
rm -rf ~/miniconda3/miniconda.sh && \
~/miniconda3/bin/conda init bash && \
~/miniconda3/bin/conda init zsh
# close and reopen the shell
git clone https://github.com/River-Zhang/SIFU.git
sudo apt-get install libeigen3-dev ffmpeg
cd SIFU
conda env create -f environment.yaml
conda activate sifu
pip install -r requirements.txt
Please download the checkpoint (google drive) and place them in ./data/ckpt
Please follow ICON to download the extra data, such as HPS and SMPL (using fetch_hps.sh
and fetch_data.sh
). There may be missing files about SMPL, and you can download from here and put them in /data/smpl_related/smpl_data/.
python -m apps.infer -cfg ./configs/sifu.yaml -gpu 0 -in_dir ./examples -out_dir ./results -loop_smpl 100 -loop_cloth 200 -hps_type pixie
# 1. Register at http://icon.is.tue.mpg.de/ or https://cape.is.tue.mpg.de/
# 2. Download CAPE testset
bash fetch_cape.sh
# evaluation
python -m apps.train -cfg ./configs/train/sifu.yaml -test
# TIP: the default "mcube_res" is 256 in apps/train.
If this work is helpful for your research, please consider citing the following BibTeX entry.
@InProceedings{Zhang_2024_CVPR,
author = {Zhang, Zechuan and Yang, Zongxin and Yang, Yi},
title = {SIFU: Side-view Conditioned Implicit Function for Real-world Usable Clothed Human Reconstruction},
booktitle = {Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)},
month = {June},
year = {2024},
pages = {9936-9947}
}