VIEW_SYNTHESIS.md

ShapeNet Novel View Synthesis

Introduction

We discuss the details about the novel view synthesis here. View synthesis requires generating novel views of objects or scenes based on arbitrary input views.

Form Left to Right: Input image, Results of Appearance Flow, Results of Ours, Ground-truth images.

Dataset Preparation

We use the chair and car categories of the ShapeNet dataset. Please download the preprocessed HDF5 files provided by Multi-view to Novel view:

• Car (150G)

• Chair (14G)

And Put the files into ./dataset/ShapeNet.

We split the dataset using the same method as that of Multi-view to Novel-View. Pleast download the txt files from here and put these files into ./dataset/ShapeNet.

Training and Testing

In order to test the model, you can download the trained weights from:

• ShapeNet Car
• ShapeNet Chair

Put them into ./result/shape_net_car_checkpoints, ./result/shape_net_chair_checkpoints respectively.

Then you can run the following example code to obtain the generated results.

python test.py \
--name=shape_net_car_checkpoints \
--model=shapenet \
--attn_layer=2,3 \
--kernel_size=2=5,3=3 \
--gpu_id=0 \
--dataset_mode=shapenet \
--sub_dataset=car \
--dataroot=./dataset/ShapeNet \
--results_dir=./eval_results/shape_net_car \
--checkpoints_dir=result 

If you want to train our model, you can run the following example code

# First, pre-train the Flow Field Estimator.
python train.py \
--name=shape_net_car \
--model=shapenetflow \
--attn_layer=2,3 \
--kernel_size=2=5,3=3 \
--gpu_id=0 \
--dataset_mode=shapenet \
--sub_dataset=car \
--dataroot=./dataset/ShapeNet

# Then, train the whole model in an end-to-end manner.
python train.py \
--name=shape_net_car \
--model=shapenet \
--attn_layer=2,3 \
--kernel_size=2=5,3=3 \
--gpu_id=0 \
--dataset_mode=shapenet \
--sub_dataset=car \
--dataroot=./dataset/ShapeNet \
--continue_train

The visdom is required to show the temporary results. You can access these results with:

http://localhost:8096