Self-Supervised Multi-View Stereo Reconstruction

Project Snapshot

Signal	Details
Timeline	May 2022 to Jun 2022
Organization	XPENG Robotics
Focus	Self supervised MVS 3D Reconstruction
Stack	Custom implementation and project-specific tooling

Pipeline

1. Using Photometric Consistency, Image Reconstruction loss as self-supervised loss
2. Use Self-Supervised trained model to predict depth maps
3. Using Aleatoric and Epistemic Uncertainty to Filter uncertainty depth values and generate Pseudo-labels
4. For Aleatoric Uncertainty use variance learning by adding small variance network
5. For Epistemic Uncertainty used MC-Dropout sampling method
6. Trained model again with Pseudo-labels to improve models performance since Supervised training setting is more effective than self-supervised training settings
7. Apply this method to generate Ground Truth values for our custom dataset to yields better MVS Reconstruction results

Outcome

The resulting pseudo ground-truth depth improves custom-dataset MVS reconstruction quality.