AMD's Strix Point APUs showcase a strong performance advantage in AI LLM workloads against Intel's Lunar Lake offerings.
AMD Strix Point APUs show dominance in AI LLMs while reducing overall latency against competing Intel Lunar Lake SoCs
The demand for higher performance in AI workloads has not only forced many companies to bring their own specialized hardware to the market but also made the competition more fierce. Since LLMs(large language models) have evolved significantly, the need for faster hardware is also increasing.
To tackle this, AMD introduced its own AI-oriented processors for mobile platforms, known as Strix Point, a while back. In the latest blog post, the company claims that its Strix Point APUs can have a big lead over its rivals while decreasing the latency for quicker output. According to AMD, the Ryzen AI 300 processors can deliver higher Tokens per second than Intel's Lunar Lake chips, which are Intel's special mobile chips for AI workloads.
As per the comparison, the Ryzen AI 9 HX 375 offers up to 27% higher performance in consumer LLM applications in LM Studio than the Intel Core Ultra 7 258V. The latter isn't the fastest in the Lunar Lake lineup, but it's surely close to the higher-end Lunar Lake CPUs since the core/thread count remains the same except for the core clocks.
The LM Studio is AMD's consumer-friendly tool built on the llama.cpp that doesn't require its users to learn the technical side of the LLMs. Llama.cpp is a framework that is optimized for x86 CPUs and uses AVX2 instructions. While the framework doesn't need a GPU to run LLMs, it can surely be accelerated using a GPU.
In the latency department, the Ryzen AI 9 HX 375 can deliver up to 3.5x lower latency than its rival and can achieve up to 50.7 tk/s vs 39.9 tk/s by Core Ultra 7 258V in Meta Llama 3.2 1b Instruct.
As both Intel Lunar Lake and Strix Point APUs come with powerful integrated graphics, the LM Studio can offload the tasks to the iGPU to boost LLM performance using Vulkan API. Strix Point APUs bring powerful Radeon graphics based on the RDNA 3.5 architecture and can offer up to a 31% boost in performance for Llama 3.2.
Furthermore, using the VGM(Variable Graphics Memory) Ryzen AI 300 processors can allow memory reallocation for iGPU-oriented tasks, enhancing power efficiency, and resulting in a solid 60% higher performance combined with the GPU acceleration.
AMD said that to make the comparison fair, it also tested both CPUs in Intel AI Playground with the same settings and found that the Ryzen AI 9 HX 375 was up to 8.7% faster than Core Ultra 7 258V on Microsoft Phi 3.1 and up to 13% faster on Mistral 7b Instruct 0.3 model. Nonetheless, it would have been interesting to see the Ryzen AI 9 HX 375 go against the flagship Core Ultra 9 288V processor as the HX 375 is itself the fastest Strix Point CPU.
Currently, AMD is focusing on making LLMs accessible to most users who don't possess technical skills and this can only be achieved using the LM Studio, based on the llama.cpp framework.
News Source: AMD
About the author: Sarfraz Khan is a hardware reporter with a focus on PC components and the builder community. With years of experience writing about PC hardware and laptops, his work has been featured on several reputable technology publications. Sarfraz's hands-on experience is demonstrated through his first-person accounts of using and comparing different hardware configurations, providing practical and relatable insights for everyday users. His technical analysis is respected by peers in the enthusiast community and has been cited by specialized hardware sites such as Germany's Igor's Lab.
Follow Wccftech on Google to get more of our news coverage in your feeds.
