UDFNet: Unsupervised Disparity Fusion with Adversarial Networks

Fusing disparity maps from different methods is an useful technique to get a refined disparity map by leveraging the complimentary advantage. We present a model for disparity fusion that uses an adversarial network, which can be trained without using ground truth disparity data. We input two initial disparity maps (from the left view) along with auxiliary information (gradient, left & right intensity image) into the generator and train the generator to output a refined disparity map registered on the left view. The refined left disparity map and left intensity image are used to reconstruct a fake right intensity image. Finally, the fake and real right intensity images (from the right stereo vision camera) are fed into a discriminator. The trained network’s architecture is effective for the fusion task (90 fps on Kitti2015). The accuracy is on par or even better than the state-of-art supervised methods. A demo video is available https://youtu.be/XTHOF3kZGsU.