This website requires JavaScript.

ECON: Explicit Clothed humans Optimized via Normal integration

Yuliang XiuJinlong YangXu CaoDimitrios TzionasMichael J. Black
Dec 2022
摘要
The combination of deep learning, artist-curated scans, and ImplicitFunctions (IF), is enabling the creation of detailed, clothed, 3D humans fromimages. However, existing methods are far from perfect. IF-based methodsrecover free-form geometry, but produce disembodied limbs or degenerate shapesfor novel poses or clothes. To increase robustness for these cases, existingwork uses an explicit parametric body model to constrain surfacereconstruction, but this limits the recovery of free-form surfaces such asloose clothing that deviates from the body. What we want is a method thatcombines the best properties of implicit representation and explicit bodyregularization. To this end, we make two key observations: (1) current networksare better at inferring detailed 2D maps than full-3D surfaces, and (2) aparametric model can be seen as a "canvas" for stitching together detailedsurface patches. Based on these, our method, ECON, has three main steps: (1) Itinfers detailed 2D normal maps for the front and back side of a clothed person.(2) From these, it recovers 2.5D front and back surfaces, called d-BiNI, thatare equally detailed, yet incomplete, and registers these w.r.t. each otherwith the help of a SMPL-X body mesh recovered from the image. (3) It "inpaints"the missing geometry between d-BiNI surfaces. If the face and hands are noisy,they can optionally be replaced with the ones of SMPL-X. As a result, ECONinfers high-fidelity 3D humans even in loose clothes and challenging poses.This goes beyond previous methods, according to the quantitative evaluation onthe CAPE and Renderpeople datasets. Perceptual studies also show that ECON'sperceived realism is better by a large margin. Code and models are availablefor research purposes at econ.is.tue.mpg.de
展开全部
图表提取

论文十问由沈向洋博士提出，鼓励大家带着这十个问题去阅读论文，用有用的信息构建认知模型。写出自己的十问回答，还有机会在当前页面展示哦。

  1. Q1
    论文试图解决什么问题？
    修宇亮 论文作者 2023/04/04

    ICON (CVPR22) 从单张图片中重建的人物几何，其mesh的质量，相比于其预测的normal map，有明显的差距，对于宽松衣服，这种落差尤其明显。ECON重新思考人物单目重建这个问题，既然normal map包含丰富的几何表面细节，而SMPL-X本身又提供了足够的人体结构姿态先验，那么，有没有可能从normal map出发，结合已有的SMPL-X，直接“优化”出拥有更好细节及灵活拓扑的三维人物？

  2. Q2
    这是否是一个新的问题？
    修宇亮 论文作者 2023/04/04

    单目人物重建这个问题，不新；三明治结构做重建这个思路，不新；用IF-Nets做补全，不新；Normal Integration，不新；但把以上几点有机地融合到一块，最终实现ECON这样的效果，还是蛮新的。

  3. Q3
    这篇文章要验证一个什么科学假设？
    修宇亮 论文作者 2023/04/04

    单目人物重建三大要素: details (normal), pose (SMPL-X), continuity (poisson)。两味原料，一勺汤，三者完备，直接上优化，不需要data-driven，就可以解决这个问题。

  4. Q4
    有哪些相关研究？如何归类？谁是这一课题在领域内值得关注的研究员？
  5. Q5
    论文中提到的解决方案之关键是什么？
  6. Q6
    论文中的实验是如何设计的？
  7. Q7
    用于定量评估的数据集是什么？代码有没有开源？
  8. Q8
    论文中的实验及结果有没有很好地支持需要验证的科学假设？
  9. Q9
    这篇论文到底有什么贡献？
  10. Q10
    下一步呢？有什么工作可以继续深入？
0
被引用
28
笔记
0
问答
领域
· Computer Science· CV