LI Lun, GUO Yingnan, WANG Zixuan. Large model intelligent computing infrastructure reinforces the enabling foundation for new industrialization[J]. Information and Communications Technology and Policy, 2024, 50(12): 2-6.
SEVILLA J, ROLDÁN E. Training compute of frontier AI models grows by 4-5x per year[J/OL]. Epoch AI, (2024-05-28)[2024-10-20]. https://epoch.ai/blog/training-compute-of-frontier-ai-models-grows-by-4-5x-per-year.
[2]
OpenAI R. GPT-4 technical report[J]. arXiv Preprint, arXiv: 2303.08774, 2023.
[3]
DUBEY A, JAUHRI A, PANDEY A, et al. The Llama 3 herd of models[J]. arXiv Preprint, arXiv: 2407.21783, 2024.
[4]
AMIN V, MARK L. Announcing cloud TPU v5e GA for cost-efficient AI model training and inference[EB/OL]. (2023-11-09)[2024-10-20]. https://cloud.google.com/blog/products/compute/announcing-cloud-tpu-v5e-in-ga.
[5]
JIANG Z, LIN H, ZHONG Y, et al. MegaScale: scaling large language model training to more than 10 000 GPUs[J]. arXiv Preprint, arXiv: 2402.15627, 2024.
[6]
QIAN K, XI Y, CAO J, et al. Alibaba HPN: a data center network for large language model training[C]// Proceedings of the ACM SIGCOMM2024 Conference, 2024.
