During the latest CXL Consortium webinar, Steve Scargall (MemVerge) presented how CXL technology is revolutionizing data center memory performance and efficiency, highlighting the latest advancements in hardware and software. The webinar also explored how CXL is enhancing AI/ML workloads, including Retrieval-Augmented Generation (RAG), vector databases, stable diffusion, Large Language Models (LLMs) and more.
Watch On-Demand or Download the Slides
If you were not able to attend the live webinar, the recording is available via YouTube and BrightTALK and the webinar presentation slides are available for download on the CXL Consortium website.
| Webinar on YouTube | Webinar on BrightTALK | Download Slides |
 |
 |
 |
Webinar Q&A
We received great questions from the audience but were not able to address them all during the webinar. Below, we’ve included answers to the questions we didn’t get to during the live webinar.
Q: Where can I try CXL servers and devices?
Reach out to your preferred Server OEM. They can provide you with their current server portfolio and roadmaps along with a list of validated CXL devices.
Q: Are platform vendors committed to enabling CXL in the update cycle and when do you anticipate CXL support to be common? Are integration challenges delaying or preventing acceptance?
Integration isn’t the problem. It takes time (years) to bring new protocol standards, such as CXL, to market. CPU and Server platform vendors are committed to delivering servers with CXL support. AMD and Intel have CPUs on the market with CXL 1.1 support today, and ARM’s Neoverse V2 introduced CXL support. A few server vendors have x86 servers available with CXL 1.1 support. The server and device market are ramping up for CXL 2.0 adoption, as is evident from CXL device and server announcements. This will continue as future generations support CXL 3.X and beyond. The Linux Kernel continues to add new CXL versions and feature support as they are announced.
Q: Could you share the memory foot prints and bandwidth requirements for the workloads in the use cases you discussed like ray and other AI inference?
The Stable Diffusion Demo from MemVerge and Micron idea came from Anyscale’s “We Pre-Trained Stable Diffusion Models on 2 billion Images and Didn’t Break the Bank – Definitive Guides with Ray Series” blog.
We used the following configuration:
- 2 x Servers, each with Intel 5th Gen Xeon CPUs (Emerald Rapids), 1TiB DDR, 1 x NVidia A100, Ubuntu 22.04.04, 10Gbps Ethernet, 1 x 3.84TiB U.3 Micron 7450 Max NVMe
- 1 x XConn CXL 2.0 Switch
- 2 x 256GiB Micron CZ-122 CXL Memory Devices (interleaved)
- MemVerge GISMO, which natively supports Ray.io
- The number of training images was capped to 10,000 for no other reason than to reduce the training time.
CXL devices allow you to design the memory configuration, in conjunction with DRAM, to meet the [growing] needs of the applications. Adding more CXL devices, either locally and/or externally, adds capacity and bandwidth. You can choose between E3.s and CEM (PCIe® Add-in-card) form factors depending on your server or memory appliance chassis design. CXL device characteristics offer a choice between cost, performance, capacity, and features such as volatile/non-volatile, compression, security, etc.
Q: Is there cost-effective access to hardware for device development in CXL?
Hardware device development commonly starts on FPGAs. FPGA vendors can provide the necessary CXL IP Blocks and Bitstreams to get started with designing CXL devices. Xilinx, Altera, and AMD Versal FPGA products offer CXL IP Blocks. You should choose the FPGA board and IP Blocks that provide the features you need for your CXL device design. Once you have thoroughly tested and validated your design, then you can take it to a fab and produce ASICs.
Q: What additional work is needed to enable CXL NV (Persistent Memory) and when can we expect it in future devices?
Samsung announced their Persistent Memory CXL devices in March 2024. Their CXL Memory Module-Hybrid (CMM-H) device offers several modes. Visit https://semiconductor.samsung.com/news-events/tech-blog/samsung-cxl-solutions-cmm-h/ for more information.
Q: Are there any open-source management frameworks to handle memory allocation and coherence across different compute nodes while porting applications to use CXL?
There are two leading projects. FAMFS is an open-source shared CXL memory-aware file system from Micron that is currently under development. MemVerge’s GISMO is a proprietary object store. Both allow applications to Create, Read, Update, and Delete (CRUD) files/objects from multiple servers using CXL Shared Memory connected via a fabric. CXL 3.X introduces memory-centric fabric architectures, which will lead to the development of new distributed shared memory management frameworks and libraries.
Related Links on the CXL Consortium Website