Research on development trend of key technologies for constructing metaverse scenes based on real environment

doi:10.12267/j.issn.2096-5931.2023.06.003

Abstract

Abstract:

The metaverse scene constructed based on the real physical environment is an important bridge connecting the physical world and the digital world, and an important way to achieve the transformation from digital twinning to virtual and real coexistence. First, this paper focuses on the key links of constructing metaverse scenes such as 3D reconstruction and rendering, and summarizes the current mainstream technology paths and innovation trends. Then, it discusses the optimization of artificial intelligence technology in related fields, as well as the research and theoretical optimization of neural rendering technology models. Finally, it analyzes the application pattern and innovation direction of key software and hardware products, and proposes suggestions for the current technological innovation in China.

Key words: metaverse, 3D reconstruction, rendering, neural rendering, Neural Radiance Fields (NeRF)

CLC Number:

TN929.11

SHUO Tianluan, DONG Yimin. Research on development trend of key technologies for constructing metaverse scenes based on real environment[J]. Information and Communications Technology and Policy, 2023, 49(6): 17-23.

Add to citation manager EndNote|Ris|BibTeX

URL:

http://ictp.caict.ac.cn/EN/10.12267/j.issn.2096-5931.2023.06.003

http://ictp.caict.ac.cn/EN/Y2023/V49/I6/17

Figures/Tables 3

References 9

[1]	MILDENHALL B, SRINIVASAN P P, TANCIK M, et al. Nerf: representing scenes as neuraradiance fields for view synthesis[C]//European Conference on Computer Vision. Springer Cham, 2020: 405-421.
[2]	REISER C, PENG S, LIAO Y, et al. KiloNeRF:speeding up neural radiance fields with thousands of tiny MLPs[C]//Proceedings of the IEEE/CVF International Conference on Computer Vision. IEEE, 2021:14315-14325.
[3]	NIEMEYER M, GEIGER A. GIRAFFE:representing scenes as compositional generative neural feature fields[C]//2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition(CVPR). IEEE, 2021: 11448-11459.
[4]	TANCIK M, CASSER V, YAN X, et al. Block-NeRF:scalable large scene neural view synthesis[C]//2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). IEEE, 2022.
[5]	QI C R, SU H, Mo K, et al. PointNet:deep learning on point sets for 3D classification and segmentation[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR). IEEE, 2017(7):652-660.
[6]	KARRAS T, LAINE S, AILA T. A style-based generator architecture for generative adversarial networks[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). IEEE, 2019:4401-4410.
[7]	MÜLLER T, EVANS A, SCHIED C, et al. Instant neural graphics primitives with a multiresolution hash encoding[J]. ACM Transactions on Graphics, 2022, 41(4):1-15.
[8]	RAO C, YU H, WAN H, et al. ICARUS:a specialized architecture for neural radiance fields rendering[J]. ACM Transactions on Graphics, 2022, 41(6):1-14.
[9]	HAN D, RYU J, KIM S, et al. 2.7 MetaVRain:a 133mW real-time hyper-realistic 3D-NeRF processor with 1D-2D hybrid-neural engines for metaverse on mobile devices[C]//2023 IEEE International Solid-State Circuits Conference (ISSCC). IEEE, 2023.

方式		优势	劣势
主动	结构光	三维结构、颜色、纹理复原精度高	易受环境光干扰
主动	飞行时间法	受环境光干扰小、成像速度快	价格较高、精度低
被动	单目视觉	算法成熟、计算量小、易部署	三维结构获取精度低,误差较大
被动	多目视觉	成本较低,三维结构获取精度高	计算量大

方式		优势	劣势
主动	结构光	三维结构、颜色、纹理复原精度高	易受环境光干扰
主动	飞行时间法	受环境光干扰小、成像速度快	价格较高、精度低
被动	单目视觉	算法成熟、计算量小、易部署	三维结构获取精度低,误差较大
被动	多目视觉	成本较低,三维结构获取精度高	计算量大

分类(名称)	三维重建		渲染		开源	商业
分类(名称)	基于被动式	基于主动式	实时	离线
Agisoft Metashape	√					√
Autodesk ReCap Photo	√					√
FARO Technologies Scene		√				√
Leica Geosystems Cyclone		√				√
Trimble RealWorks	√	√				√
Unreal Engine			√			√
Unity			√			√
Autodesk Arnold				√		√
Chaos Group V-Ray				√		√
Blender	√	√	√	√	√
Meshroom	√				√
OpenMVS	√				√
COLMAP	√				√
Open3D	√				√
Autodesk 3ds Max	√	√	√	√		√
Autodesk Maya	√	√	√	√		√
Cinema 4D	√	√	√	√		√
ZBrush				√		√
Adobe Substance			√	√		√
Pix4D	√					√

分类(名称)	三维重建		渲染		开源	商业
分类(名称)	基于被动式	基于主动式	实时	离线
Agisoft Metashape	√					√
Autodesk ReCap Photo	√					√
FARO Technologies Scene		√				√
Leica Geosystems Cyclone		√				√
Trimble RealWorks	√	√				√
Unreal Engine			√			√
Unity			√			√
Autodesk Arnold				√		√
Chaos Group V-Ray				√		√
Blender	√	√	√	√	√
Meshroom	√				√
OpenMVS	√				√
COLMAP	√				√
Open3D	√				√
Autodesk 3ds Max	√	√	√	√		√
Autodesk Maya	√	√	√	√		√
Cinema 4D	√	√	√	√		√
ZBrush				√		√
Adobe Substance			√	√		√
Pix4D	√					√

[1]	ZHOU Yijing, CUI Zhuo, HUANG Cuiting, LIN Wenzhen, CHEN Tao. An innovative system of data element circulation based on knowledge federation [J]. Information and Communications Technology and Policy, 2023, 49(6): 61-70.
[2]	WANG Yu. Research progress and application discussion on intelligence grading evaluation of transport network operation and management [J]. Information and Communications Technology and Policy, 2023, 49(6): 91-96.
[3]	ZHANG Kang, SANG Jieli, WU Meixi. Research on heterogeneous computing power network scheduling [J]. Information and Communications Technology and Policy, 2023, 49(5): 42-47.
[4]	ZHONG Yangfan, LIU Dan, WEN Fangzhi, GUO Rui, LIU Xiangdong. Research on key technology of large-scale application of single-phase immersion cooling in data center [J]. Information and Communications Technology and Policy, 2023, 49(5): 65-72.
[5]	ZHOU Caihong, WANG Yue, XU Kexin. Research on the development of data center storage capacity for the integration of computing, storage and transportation [J]. Information and Communications Technology and Policy, 2023, 49(5): 37-41.
[6]	ZHANG Dong, SU Yonglei, HAN Dong. Research on digital transformation of SMEs based on industrial PON [J]. Information and Communications Technology and Policy, 2023, 49(4): 86-90.
[7]	TIAN Wei, LIU Zhencheng. Practice and thinking of commercialization of scientific and technological achievements in state-owned enterprises [J]. Information and Communications Technology and Policy, 2023, 49(3): 30-35.
[8]	ZOU Feifei, YAN Kun, LI Wenyu. Research on evidence collection and identification of copyright cases in digital economy [J]. Information and Communications Technology and Policy, 2023, 49(3): 36-40.
[9]	ZHANG Xiye, WANG Sha. A probe into difficult issues of personal information protection compliance [J]. Information and Communications Technology and Policy, 2023, 49(3): 74-79.
[10]	TU Xiaojie, ZHANG Li, XIE Songyan. ICT industry development trend [J]. Information and Communications Technology and Policy, 2023, 49(3): 80-83.
[11]	ZHANG Jian, CAO Heng, WANG Zhe. Research on the security of software defined network architecture [J]. Information and Communications Technology and Policy, 2023, 49(2): 35-42.
[12]	ZHANG Pei, LIU Yuanyuan. Analysis of the domestic and international development and status of AI governance [J]. Information and Communications Technology and Policy, 2023, 49(1): 46-51.
[13]	ZHANG Hongke, ZHANG Chengliang, SUN Qiong. Considerations on cloud-network convergence [J]. Information and Communications Technology and Policy, 2022, 48(11): 1-6.
[14]	HU Jie, YANG Kun. Towards green integration of communications, sensing and computation in future 6G [J]. Information and Communications Technology and Policy, 2022, 48(11): 30-38.
[15]	LIANG Fang, TONG Tian, MA Herong, CAO Chang. Analysis of the development of the computing power network under east-data-west-computing project [J]. Information and Communications Technology and Policy, 2022, 48(11): 79-83.