PeRFception-CO3D v2
In the first version of CO3D, some sequences 1) consist of too blurry images to render, 2) the same images repeatedly appear, 3) composed of forward facing views only, or 4) have wrong camera parameters. In such cases, Plenoxels suffer from floaters that could harm geometries. According to the official announcement in Co3D repository, the second version of CO3D is recently released with ameliorated quality: (link: https://github.com/facebookresearch/co3d)
- 2X larger number of sequences, and 4X larger number of frames.
- Improved image quality - less blocky artifacts due to better video decoding.
- All images within a sequence are cropped to the same height and width.
Comparison between PeRFception-CO3D v1 and v2
We experimentally demonstrate that rendering on the second version of CO3D would reduce information loss since dataset quality is much better. We render the official toy validation dataset provided by the second version of CO3D and compare reconstruction scores and required memory. For a fair comparison, we skipped 4 scenes that are not included in the first version. Accordingly, we have evaluated on 36 scenes. We provide the checkpoints in the link on the table below.
| Version | Memory | Compression Rate | PSNR | SSIM | LPIPS |
|---|---|---|---|---|---|
| v1 | 2.51GB | -8.7% | 28.82 | 0.8508 | 0.3534 |
| v2 | 2.58GB | -51.9% | 29.86 | 0.8563 | 0.3179 |
The models trained on the second version of CO3D shows much more accurate rendering. Especially, the LPIPS score, which represents perceptual similarity, is remarkably improved. Moreover, as the number of images in each sequence is much larger in CO3D-v2, our data format shows better compression rate on CO3D-v2. We visualize several examples of rendered scenes in the figure below.
