AI era, CPU is still strong!

The adoption of AI is on the rise, with a global survey showing that 58% of enterprises plan to incorporate generative AI technology into their business models in the near future. By 2026, more than $300 billion in investment is expected to flow into generative AI, including hardware, software, and solutions. In addition, more than half of edge applications will adopt AI technology, and by 2028, more than 80% of personal computers will evolve into AI-powered PCS to increase user productivity. At the same time, more than 80% of companies plan to introduce generative AI to enhance enterprise productivity by 2026.

The development of processor technology is the cornerstone of progress in the AI era. Today, major chip manufacturers (cpus, Gpus, ASics, etc.) are increasingly competing in the field of AI. While Gpus excel in some AI applications due to their parallel processing capabilities, cpus still have a place in handling complex data processing tasks, supporting multi-tasking operations, and optimizing AI model reasoning. In particular, Intel's fifth-generation Xeon CPU, through innovative design and technical optimization, has demonstrated strong strength in the field of reasoning and training of large AI models.

"CPU is more like a warrior, proficient in all 18 kinds of martial arts, a person can deal with many people, and the ability to fight alone is very strong." The GPU is more like the army, each person has no characteristics, but there are many people, the task is simple and the concurrency is high, because the GPU business logic is very simple, but the number of audits is large." Intel technology experts recently described the image at a media communication conference.

AI era: fight performance, but also fight cost performance

In the era of AI, performance and cost performance are two key factors. On the one hand, artificial intelligence applications put forward increasingly high demands on computing power, requiring high-performance hardware and software to meet the demand. On the other hand, the costs of AI applications also need to be controlled to ensure their economic benefits.

At the end of last year, the fifth-generation Xeon scalable processor was introduced, which significantly improved the number of cores and several performance indicators compared to the previous generation. The fifth-generation Xeon processors have a maximum of 64 cores and have further improved the frequency and overall performance by introducing new instruction sets optimized for AI, such as AMX and AVX, particularly in supporting generative AI applications.
At a media communication meeting, the fifth generation Xeon processor was further introduced, in terms of memory bandwidth, the fifth generation Xeon processor performed well in the industry data center level processor, reaching a high bandwidth of 5600MT/s. At the same time, the three-level cache capacity is increased by three times, so that data processing does not need to frequently access memory, improving processing efficiency.

In terms of software ecology, with the release of the fifth generation Xeon, more than 300 deep learning models were contributed to the community, and more than 50 machine learn-based models were optimized for developers to use. AI development software has also been updated to enable better application optimization on fifth-generation Xeon processors, and support for mainstream large model and generative AI frameworks such as PyTorch and TensorFlow has been enhanced.

In terms of performance, the fifth generation Xeon has achieved significant performance improvement in AI training, real-time reasoning, batch reasoning, etc., according to different algorithms, up to 40%. General-purpose servers based on the latest generative AI grand model fully meet the requirements in terms of performance, maintaining excellent performance even under high loads.

In terms of very important cost performance, the fifth-generation Xeon processor can maintain response times within 100ms while supporting concurrent access for multiple users. In addition, the practical application verification of partners, such as the tests of Alibaba Cloud and Baidu Cloud, also proved the excellent performance of the fifth generation Xeon in generative AI model reasoning.

"On the whole, for some general applications, such as meeting minutes extraction, outline summary, content analysis, and some content creation, especially the more recently discussed Vincenne chart, robot chat customer service, code writing, this productivity improvement application, the use of general computing power, especially based on the results of the fifth generation Xeon server is still more advantageous. So we are confident that Xeon 5 can meet the workload demands of these generative AI models."

The fifth generation Xeon - in-depth analysis of architecture

The fifth generation Xeon can play such an effective role in the field of AI, and the architecture behind it may be contributory. The improvement of key performance indicators of the fifth generation Xeon is mainly reflected in the following aspects:

1. Upgrade to the Raptor Cove core.
2. The number of cores increased from 60 to 64 cores at most.
3. The size of the LLC increased from 1.875MB to 5MB. This is a major step forward in Intel's history, which used to be that Intel's LLCS were basically in the 1M-2M range.
4.DDR speed increased from 4800MT/s to 5600MT/s.
5.UPI speed increased from 16GT/s to 20GT/s.
6. The SoC chip topology is changed, and the 4-chip package is changed to the 2-chip package (as described above).
7. The standby power consumption is reduced. The optimization of fully integrated voltage regulator (FIVR) and the enhancement of active idle mode have improved energy efficiency, especially during non-full load operation.

From the aspects of process technology, chip layout, performance and energy efficiency, final level cache, memory IO, etc., senior technical experts elaborated on the technical innovations made by the fifth-generation Xeon processor in these invisible places, as well as the actual performance improvements and energy efficiency optimization brought by these innovations.