Single-Image Mesh Reconstruction and Pose Estimation via Generative Normal Map

Nan Xiang; Li Wang; Tao Jiang; Yanran Li; Xiaosong Yang; Jianjun Zhang

doi:10.1145/3328756.3328766

Single-Image Mesh Reconstruction and Pose Estimation via Generative Normal Map

Nan Xiang, Li Wang, Tao Jiang, Yanran Li, Xiaosong Yang, Jianjun Zhang

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution

Abstract

We present a unified learning framework for recovering both 3D mesh and camera pose of the object from a single image. Our approach learns to recover outer shape and surface geometric details of the mesh without relying on 3D supervision. We adopt multi-view normal maps as the 2D supervision so that the silhouette and geometric details information can be transferred to neural network. A normal mismatch based objective function is introduced to train the network, and the camera pose is parameterized into the objective, it integrates pose estimation with the mesh reconstruction in a same optimization procedure. We demonstrate the abilities of the proposed approach in generating 3D mesh and estimating camera pose with qualitative and quantitative experiments.

Original language	English
Title of host publication	Proceedings of the 32nd International Conference on Computer Animation and Social Agents
Place of Publication	New York, NY, USA
Publisher	Association for Computing Machinery, Inc
Pages	79–84
Number of pages	6
ISBN (Print)	9781450371599
DOIs	https://doi.org/10.1145/3328756.3328766
Publication status	Published - 1 Jul 2019
Event	The 32nd International Conference on Computer Animation and Social Agents 2019 - Paris, France Duration: 1 Jul 2019 → 3 Jul 2019 Conference number: 32 https://casa2019.sciencesconf.org/

Publication series

Name	CASA '19
Publisher	Association for Computing Machinery

Conference

Conference	The 32nd International Conference on Computer Animation and Social Agents 2019
Abbreviated title	CASA 2019
Country/Territory	France
City	Paris
Period	1/07/19 → 3/07/19
Internet address	https://casa2019.sciencesconf.org/

Keywords / Materials (for Non-textual outputs)

deep learning
pose estimation
mesh reconstruction

Access to Document

10.1145/3328756.3328766Licence: All Rights Reserved

https://dl.acm.org/doi/abs/10.1145/3328756.3328766Licence: All Rights Reserved

Cite this

@inproceedings{141d4736524c451f8d767d60b14020fe,

title = "Single-Image Mesh Reconstruction and Pose Estimation via Generative Normal Map",

abstract = "We present a unified learning framework for recovering both 3D mesh and camera pose of the object from a single image. Our approach learns to recover outer shape and surface geometric details of the mesh without relying on 3D supervision. We adopt multi-view normal maps as the 2D supervision so that the silhouette and geometric details information can be transferred to neural network. A normal mismatch based objective function is introduced to train the network, and the camera pose is parameterized into the objective, it integrates pose estimation with the mesh reconstruction in a same optimization procedure. We demonstrate the abilities of the proposed approach in generating 3D mesh and estimating camera pose with qualitative and quantitative experiments.",

keywords = "deep learning, pose estimation, mesh reconstruction",

author = "Nan Xiang and Li Wang and Tao Jiang and Yanran Li and Xiaosong Yang and Jianjun Zhang",

year = "2019",

month = jul,

day = "1",

doi = "10.1145/3328756.3328766",

language = "English",

isbn = "9781450371599",

series = "CASA '19",

publisher = "Association for Computing Machinery, Inc",

pages = "79–84",

booktitle = "Proceedings of the 32nd International Conference on Computer Animation and Social Agents",

note = "The 32nd International Conference on Computer Animation and Social Agents 2019, CASA 2019 ; Conference date: 01-07-2019 Through 03-07-2019",

url = "https://casa2019.sciencesconf.org/",

}

Xiang, N, Wang, L, Jiang, T, Li, Y, Yang, X & Zhang, J 2019, Single-Image Mesh Reconstruction and Pose Estimation via Generative Normal Map. in Proceedings of the 32nd International Conference on Computer Animation and Social Agents. CASA '19, Association for Computing Machinery, Inc, New York, NY, USA, pp. 79–84, The 32nd International Conference on Computer Animation and Social Agents 2019, Paris, France, 1/07/19. https://doi.org/10.1145/3328756.3328766

Single-Image Mesh Reconstruction and Pose Estimation via Generative Normal Map. / Xiang, Nan; Wang, Li; Jiang, Tao et al.
Proceedings of the 32nd International Conference on Computer Animation and Social Agents. New York, NY, USA: Association for Computing Machinery, Inc, 2019. p. 79–84 (CASA '19).

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution

TY - GEN

T1 - Single-Image Mesh Reconstruction and Pose Estimation via Generative Normal Map

AU - Xiang, Nan

AU - Wang, Li

AU - Jiang, Tao

AU - Li, Yanran

AU - Yang, Xiaosong

AU - Zhang, Jianjun

N1 - Conference code: 32

PY - 2019/7/1

Y1 - 2019/7/1

N2 - We present a unified learning framework for recovering both 3D mesh and camera pose of the object from a single image. Our approach learns to recover outer shape and surface geometric details of the mesh without relying on 3D supervision. We adopt multi-view normal maps as the 2D supervision so that the silhouette and geometric details information can be transferred to neural network. A normal mismatch based objective function is introduced to train the network, and the camera pose is parameterized into the objective, it integrates pose estimation with the mesh reconstruction in a same optimization procedure. We demonstrate the abilities of the proposed approach in generating 3D mesh and estimating camera pose with qualitative and quantitative experiments.

AB - We present a unified learning framework for recovering both 3D mesh and camera pose of the object from a single image. Our approach learns to recover outer shape and surface geometric details of the mesh without relying on 3D supervision. We adopt multi-view normal maps as the 2D supervision so that the silhouette and geometric details information can be transferred to neural network. A normal mismatch based objective function is introduced to train the network, and the camera pose is parameterized into the objective, it integrates pose estimation with the mesh reconstruction in a same optimization procedure. We demonstrate the abilities of the proposed approach in generating 3D mesh and estimating camera pose with qualitative and quantitative experiments.

KW - deep learning

KW - pose estimation

KW - mesh reconstruction

U2 - 10.1145/3328756.3328766

DO - 10.1145/3328756.3328766

M3 - Conference contribution

SN - 9781450371599

T3 - CASA '19

SP - 79

EP - 84

BT - Proceedings of the 32nd International Conference on Computer Animation and Social Agents

PB - Association for Computing Machinery, Inc

CY - New York, NY, USA

T2 - The 32nd International Conference on Computer Animation and Social Agents 2019

Y2 - 1 July 2019 through 3 July 2019

ER -

Single-Image Mesh Reconstruction and Pose Estimation via Generative Normal Map

Abstract

Publication series

Conference

Keywords / Materials (for Non-textual outputs)

Access to Document

Fingerprint

Cite this