关键词: 大模型推理, AI存储, 键值缓存, 算存协同

Abstract:

With the continuous enhancement of large model capabilities and the deepening of inference applications, the scale of data processing has expanded drastically, and data processing requirements have become increasingly diversified, this has imposed higher demands on the collaborative between storage and computing power. In response to the new demands on storage systems by larger data volumes, larger model sizes, and longer context windows in current large model inference scenarios, this study first conducts an in-depth analysis of the implementation mechanisms, key technologies, and practical applications of both “computing-in-place-of-storage” and “storage-in-place-of-computing”, Subsequently, by integrating the current technological and industrial foundation as well as application scenario requirements, this paper proposes that based on access latency and bandwidth demands,a hierarchical and systematic collaborative storage model for the future development of computing-storage synergy is important. This paper aims to explore the specific implementation mechanisms and evolutionary pathways of compute-storage collaboration, providing valuable references for promoting the improvement of intelligent computing cluster utilization efficiency and better supporting the development of large model inference.

Key words: large model inference, AI storage, KV Cache, computing-storage collaboration