混合现实视频透视技术关键挑战分析

doi:10.12267/j.issn.2096-5931.2024.06.004

信息通信技术与政策 ›› 2024, Vol. 50 ›› Issue (6): 23-29.doi: 10.12267/j.issn.2096-5931.2024.06.004

混合现实视频透视技术关键挑战分析

The analysis of video see-through challenges in mixed reality

向潇¹, 赵蕾²

1.中国信息通信研究院信息化与工业化融合研究所,北京 100083
2.甬江实验室新型显示与感知研究中心,宁波 315202

收稿日期:2024-05-20 出版日期:2024-06-25 发布日期:2024-07-30
作者简介:
向潇, 中国信息通信研究院信息化与工业化融合研究所工程师,主要从事虚拟现实、新型显示、智能终端等方面的研究工作;
赵蕾, 甬江实验室新型显示与感知研究中心研究员,主要从事显示光学、虚拟现实、增强现实等方面的技术研究工作

XIANG Xiao¹, ZHAO Lei²

1. Informatization and Industrialization Integration Research Institute, China Academy of Information and Communications Technology, Beijing 100083, China
2. Novel Display and Sensing Research Center, Yongjiang Laboratory, Ningbo 315202, China

Received:2024-05-20 Online:2024-06-25 Published:2024-07-30

摘要/Abstract

摘要：

随着用户对于虚拟现实体验沉浸性要求的不断提高,视频透视凭借对环境亮度适应性、视场角大小和虚实遮挡关系等方面的优势,成为当前相比于光学透视更具前景的混合现实技术路线。然而,当前视频透视技术的产品化工作仍处于起步阶段,已推出的混合现实终端在实际应用中存在若干问题。从时间、空间、图像质量三大维度切入,重点围绕图像延迟、视点匹配、视觉分辨率、透视畸变和动态范围5个问题展开分析,旨在为视频透视技术发展提供阶段性总结,理清思路以支持其未来发展。

关键词: 混合现实, 视频透视, 延时分析, 视点匹配, 图像质量

Abstract:

As the demand for the level of immersion in extended reality (which includes virtual reality and augmented reality) continues to grow, video see-through (VST) has become a more promising mixed-reality technique than optical see-through (OST) due to its advantages in environment brightness adaptability, field of view and occlusion implementation. However, a few products equipped with VST have showcased multiple problems as the technique is still in an early phase. Therefore, the analysis has been conducted in the aspects of temporal, spacial and image quality, focusing on problems of image latency, point of view accommodation, oculus resolution, perspective distortion and dynamic range. Hopefully, this paper can provide an insight into VST status quo and facilitate its future development.

Key words: mixed reality, video see-through, latency analysis, point of view accommodation, image quality

中图分类号:

R318
TP391

向潇, 赵蕾. 混合现实视频透视技术关键挑战分析[J]. 信息通信技术与政策, 2024, 50(6): 23-29.

XIANG Xiao, ZHAO Lei. The analysis of video see-through challenges in mixed reality[J]. Information and Communications Technology and Policy, 2024, 50(6): 23-29.

导出引用管理器 EndNote|Ris|BibTeX

链接本文:

http://ictp.caict.ac.cn/CN/10.12267/j.issn.2096-5931.2024.06.004

http://ictp.caict.ac.cn/CN/Y2024/V50/I6/23

图/表 4

参考文献 23

[1]	MILGRAM P, KISHINO F. A taxonomy of mixed reality visual displays[J]. IEICE Transactions on Information and Systems, 1994, 77(12): 1321-1329.
[2]	Work Portfolio Blog. What’s the difference between AR,VR and MR?[EB/OL]. (2024-02-14)[2024-05-20]. https://forwork.meta.com/blog/difference-between-vr-ar-and-mr/.
[3]	JOHN C. Latency mitigation strategies[EB/OL]. (2013-02-22)[2024-05-20]. https://web.archive.org/web/20130225013015/ http://www.altdevblogaday.com/2013/02/22/latency-mitigation-strategies/.
[4]	ADELSTEIN B D, LEE T G, ELLIS S R. Head tracking latency in virtual environments: psychophysics and a model[C]// Proceedings of the Human Factors and Ergonomics Society Annual Meeting. Los Angeles: Sage Publications, 2003, 47(20): 2083-2087.
[5]	MANIA K, ADELSTEIN B D, ELLIS S R, et al. Perceptual sensitivity to head tracking latency in virtual environments with varying degrees of scene complexity[C]// Proceedings of the 1st Symposium on Applied Perception in Graphics and Visualization. Los Angeles: ACM, 2004: 39-47.
[6]	ISHIHARA A, AGA H, ISHIHARA Y, et al. Integrating both parallax and latency compensation into video see-through head-mounted display[J]. IEEE Transactions on Visualization and Computer Graphics, 2023, 29(5): 2826-2836.
[7]	VAN WAVEREN J M P. The asynchronous time warp for virtual reality on consumer hardware[C]// Proceedings of the 22nd ACM Conference on Virtual Reality Software and Technology. New York: ACM, 2016: 37-46.
[8]	Nvidia Corporation. Nvidia CloudXR Suite[EB/OL]. 2024[2024-05-20]. https://www.nvidia.com/en-us/design-visualization/solutions/cloud-xr/.
[9]	中国信息通信研究院, 华为技术有限公司, 京东方科技集团股份有限公司. 虚拟(增强)现实白皮书[R/OL]. (2021-03-30)[2024-05-20]. http://www.caict.ac.cn/kxyj/qwfb/bps/202103/P020210331549691466973.pdf.
[10]	Qualcomm Inc. Snapdragon XR2 Gen 2 platform[EB/OL]. 2024 [2024-05-20]. https://www.qualcomm.com/products/mobile/snapdragon/xr-vr-ar/snapdragon-xr2-gen-2-platform#Overview.
[11]	Qualcomm Inc. Snapdragon XR2+Gen 2 platform[EB/OL]. 2024[2024-05-20]. https://www.qualcomm.com/products/mobile/snapdragon/xr-vr-ar/snapdragon-xr2-plus-gen-2-platform.
[12]	BJÖRK T. Apple vision pro benchmark test 1: see-through latency, photon-to-photon[EB/OL]. (2024-02-14)[2024-05-20]. https://www.optofidelity.com/insights/blogs/apple-vision-pro-benchmark-test-1-see-through-latency-photon-to-photon.
[13]	CHAURASIA G, NIEUWOUDT A, ICHIM A E, et al. Passthrough+ real-time stereoscopic view synthesis for mobile mixed reality[J]. Proceedings of the ACM on Computer Graphics and Interactive Techniques, 2020, 3(1): 1-17.
[14]	XIAO L, NOURI S, HEGLAND J, et al. Neuralpassthrough: learned real-time view synthesis for vr[C]// ACM Siggraph 2022 Conference Proceedings. Vancouver: ACM, 2022: 1-9.
[15]	ROLLAND J P, FUCHS H. Optical versus video see-through head-mounted displays in medical visualization[J]. Presence, 2000, 9(3): 287-309.
[16]	NDJIKI-NYA P, KOPPEL M, DOSHKOV D, et al. Depth image-based rendering with advanced texture synthesis for 3-D video[J]. IEEE Transactions on Multimedia, 2011, 13(3): 453-465.
[17]	JANTET V, GUILLEMOT C, MORIN L. Object-based layered depth images for improved virtual view synthesis in rate-constrained context[C]// 2011 18th IEEE International Conference on Image Processing. Brussels: IEEE Press, 2011: 125-128.
[18]	HEDMAN P, ALSISAN S, SZELISKI R, et al. Casual 3D photography[J]. ACM Transactions on Graphics (TOG), 2017, 36(6): 1-15.
[19]	FLYNN J, BROXTON M, DEBEVEC P, et al. Deepview: view synthesis with learned gradient descent[C]// Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. Long Beach: IEEE Press, 2019: 2367-2376.
[20]	GAO C, SARAF A, KOPF J, et al. Dynamic view synthesis from dynamic monocular video[C]// Proceedings of the IEEE/CVF International Conference on Computer Vision. Montreal: IEEE Press, 2021: 5712-5721.
[21]	GUTTAG K. Meta quest pro (part1) - - unbelievably bad AR passthrough[EB/OL]. (2023-01-03)[2024-05-20]. https://kguttag.com/2023/01/03/meta-quest-pro-part-1-unbelievably-bad-ar-passthrough/.
[22]	BEZMALINOVIC T. Passthrough on quest 3 will continue to improve, says meta CTO[EB/OL]. (2023-10-24)[2024-05-20]. https://mixed-news.com/en/meta-quest-3-passthrough-quality-will-improve/.
[23]	PATEL N. Apple vision pro review: magic, until it’s not[EB/OL]. (2024-01-30)[2024-05-20]. https://www.theverge.com/24054862/apple-vision-pro-review-vr-ar-headset-features-price.

混合现实视频透视技术关键挑战分析

The analysis of video see-through challenges in mixed reality

RichHTML

PDF

可视化

摘要/Abstract

引用本文

使用本文

图/表 4

参考文献 23

相关文章 5

编辑推荐

Metrics

本文评价

[1]	白睿钰, 王炫棋, 张梓墨, 吉博文. 皮层微创脑机接口传感器发展现状与趋势^*[J]. 信息通信技术与政策, 2024, 50(5): 18-25.
[2]	王坤, 黄依宁, 许敏鹏, 顾晓松, 明东. 用于运动准备诱发脑电响应识别的新型复合空间滤波算法研究^*[J]. 信息通信技术与政策, 2024, 50(5): 2-11.
[3]	王战阳, 张洪欣, 杨晨. 基于子空间方法的任务依赖脑电实时压缩算法研究^*[J]. 信息通信技术与政策, 2024, 50(5): 26-33.
[4]	查玮, 杨雨潇, 胡胜蓉. 浙江省脑机接口产业发展趋势研究^*[J]. 信息通信技术与政策, 2024, 50(5): 47-53.
[5]	徐明. 脑机接口技术的伦理挑战及其理性规制研究[J]. 信息通信技术与政策, 2024, 50(5): 54-59.