MemVerge and Micron Boost NVIDIA GPU Utilization with CXL® Memory

The joint solution increases GPU utilization by 77% and more than doubles the speed of OPT-66B batch inference.

SAN JOSE, Calif., March 18, 2024 /PRNewswire/ -- MemVerge®, a leader in AI-first Big Memory Software, has joined forces with Micron to unveil a groundbreaking solution that leverages intelligent tiering of CXL memory, boosting the performance of large language models (LLMs) by offloading from GPU HBM to CXL memory. This innovative collaboration is being showcased in Micron booth #1030 at GTC, where attendees can witness firsthand the transformative impact of tiered memory on AI workloads.

Charles Fan, CEO and Co-founder of MemVerge, emphasized the critical importance of overcoming the bottleneck of HBM capacity. "Scaling LLM performance cost-effectively means keeping the GPUs fed with data," stated Fan. "Our demo at GTC demonstrates that pools of tiered memory not only drive performance higher but also maximize the utilization of precious GPU resources."

The demonstration, conducted by engineers from MemVerge and Micron featured a FlexGen high-throughput generation engine and OPT-66B large language model running on a Supermicro Petascale Server equipped with an AMD Genoa CPU, Nvidia A10 GPU, Micron DDR5-4800 DIMMs, CZ120 CXL memory modules, and MemVerge Memory Machinetm X intelligent tiering software.

The results of the demonstration were impressive. The FlexGen benchmark, utilizing tiered memory, completed tasks in less than half the time compared to conventional NVMe storage methods. Simultaneously, GPU utilization soared from 51.8% to 91.8%, thanks to the transparent management of data tiering across DIMMs and CXL modules facilitated by MemVerge Memory Machine X software.

This collaboration between MemVerge, Micron, and Supermicro marks a significant milestone in advancing the capabilities of AI workloads, enabling organizations to achieve unprecedented levels of performance, efficiency, and time-to-insight. By harnessing the power of CXL memory and intelligent tiering, businesses can unlock new opportunities for innovation and accelerate their journey towards AI-driven success.

"Through our collaboration with MemVerge, Micron is able to demonstrate the substantial benefits of CXL memory modules to improve effective GPU throughput for AI applications resulting in faster time to insights for customers. Micron's innovations across the memory portfolio provide compute with the necessary memory capacity and bandwidth to scale AI use cases from cloud to the edge," said Raj Narasimhan, senior vice president and general manager of Micron's Compute and Networking Business Unit.

For more information about MemVerge, Micron, and Supermicro's collaborative efforts and the transformative potential of CXL memory for AI workloads, please visit www.memverge.com/gtc2024 

About MemVerge

MemVerge is a pioneering developer of Big Memory Computing software. In the cloud or on-premises, data-intensive workloads run faster, cost less, and recover automatically with the company's award-winning Memory Machinetm products. Memory Machine X is poised to revolutionize how CXL® memory will be used in the future, while Memory Machine Cloud stands out with its ability to continuously rightsize cloud cost and performance. Learn more about MemVerge and its Memory Machine software at www.memverge.com.

Media Contacts:
Frank Berry
MemVerge
[email protected]
949.300.8917

SOURCE MemVerge