Last month, CXL® Consortium Marketing Work Group Co-Chairs Kurt Lender and Kurtis Bowman presented a webinar covering the CXL device ecosystem and its use cases. The webinar also featured a Q&A panel of member representatives from AMD, Astera Labs, Elastics.cloud, Intel, IntelliProp, MemVerge, Samsung, Synopsys, Teledyne LeCroy, and UnifabriX. The panelists highlighted the CXL technology demonstrations they showcased at Supercomputing 2022 (SC’22). Learn more about each of these CXL demos here.
If you were not able to attend the live webinar, the recording is available via BrightTALK and YouTube. The presentation is also available on the CXL Consortium website for download here. We received great questions from the audience during the live Q&A but were not able to address them all during the webinar. Below, we answered the questions we didn’t get to during the live webinar.
Q: In CXL 3.0 there is a device type GFAM which can be shared across thousands of hosts in a cache-coherent manner. Is there any challenge in terms of scalability to thousands of hosts? Also, when can we expect a GFAM type of device?
[CXL Consortium]: GFAM (Global Fabric Attached Memory) is a new feature defined in the CXL 3.0 specification that enables a CXL device to support large number of hosts (1 to 4095) to access the device. Part (including all or none) of the GFAM memory can be simultaneously and coherently shared across multiple hosts. When memory is coherently shared by multiple hosts, it is wise to limit the number of hosts writes are expected in into the same memory space. This will keep the coherency management traffic (e.g., BISnps) to a reasonable level. Alternately, if there are only a few hosts that update the data less frequently and the others are consumers then the sharing domain can be made quite large.
CXL is an industry Consortium responsible for advancing the CXL specification and interoperability but does not make specific products, we can’t comment on product timelines. Our members implement the specification and release many outstanding products and we suggest you reach out to our members for specific product information.
Q: When will official CXL Compliance Testing start?
[CXL Consortium]: The Compliance Work Group expects to start compliance testing in Q2 – for CXL 1.1 and 2H’23 for CXL 2.0. Please monitor future blogs and announcements for upcoming Compliance events and what they will cover.
Q: Your demo showcased memory security, was there anything special required on the CXL memory device?
[AMD]: AMD showcased enabling SEV technology (Secure Encrypted Virtualization) as part of CXL demonstration by running confidential containers seamlessly spawning off DDR memory and CXL memory. In this case, the host handled the encryption of the data to make sure data leaving the SOC is always encrypted, either on DDR or on CXL. This did not require any device side IP changes and can be enabled on any CXL memory expander. This solution enables end-to-end data security, protecting data in-flight and at rest.
[Astera Labs]: The use cases Leo supports include memory expansion, memory pooling and memory sharing.
-
With memory expansion Leo solves performance bottlenecks in AI and ML type of workloads for cloud applications.
-
Memory pooling provides overall memory utilization and avoids memory stranding that is prevalent in today’s data centers, thereby reducing TCO.
-
Memory sharing will improve data flow efficiency in addition to all the benefits that memory sharing provides.
Q: How is Astera Labs addressing the reliability and security needs for these use-cases?
[Astera Labs]:
-
On the RAS front, we provide support for all of the CXL spec defined RAS features as well as advanced features that cloud service providers and enterprise servers would significantly benefit from such as telemetry and fleet management. To make the integration of these RAS features easier, we provide software solutions and APIs that can be easily integrated in the BMC or host. We also have launched our Cloud-Scale Interop Lab where we test with leading CXL CPUs from Intel and AMD and all the leading memory vendors. This will make a real difference for our customers because it will ensure seamless interoperability and reliability.
-
On the security side, we provide a complete set of end-to-end security features through CXL IDE, as well as other industry leading specifications such as OCP and TCG. These security measures apply to a wide variety of use models, offer broad interoperability, and align to industry best practices.
Q: How does Xeon Max handle the HBM Cache Only mode? Will it only support CXL Type 3 devices in the future as a Memory instead of CPU attached Memory?
[Intel]: This is a product specific question, please address with vendor directly (Intel in this case).
Q: Does Xeon 4th Gen not support CXL Type 3 devices now? Will this come before 5th Gen?
[Intel]: This is a product specific question, please address with vendor directly (Intel in this case).
Q: Are the demo systems available to test software applications with the IntelliProp memory fabric hardware and software?
[IntelliProp]: IntelliProp has engaged with various companies in the industry to test software applications utilizing the IntelliProp test servers, switches and fabric attached memory. Additionally, companies developing composability managers have used the IntelliProp hardware and software stack to dynamically allocate fabric attached memory to test servers.
IntelliProp supports two models of engagement. One method is via a dedicated virtual private network connection to the IntelliProp test lab in Longmont, CO. Once connected to the VPN, companies are able to run their software applications with the servers, switches and fabric attached memory in the IntelliProp lab. IntelliProp also works with companies to acquire the hardware and software stack needed for testing in their own labs.
Q: Is Samsung using a 3rd party CXL controller? Does Samsung plan to work on an in-house controller?
[Samsung]: Samsung is collaborating with multiple CXL controller suppliers for a multi-vendor strategy. We are also considering an in-house CXL controller.
Q: What media is used in Samsung’s CXL based memory expander?
[Samsung]: DDR5 is used in Samsung’s first generation of CXL based memory expander product. We continue to evaluate alternative media options to meet demand for cost-effective solution.
Q: What was the key message you were trying to convey with your demo at SC22?
[UnifabriX]: First and foremost, we were very excited to demo at SC’22, and we could not have wished for a better opportunity to exit stealth mode and show the rest of the world (other than our customers) what we have been working on during the past two years. At SC’22, we decided to focus on the market readiness of CXL technology. We wanted to convey that CXL products bring real value, working with real workloads, today and are ready for primetime.
That is why we insisted on showcasing a live demo of a working, fabric-connected system, so visitors can experience first-hand how such systems look, connect, and behave. Our demo showcased a demanding workload such as HPCG (used to rank the world’s fastest computers in the TOP500 list) and shared how we can accelerate performance by dynamically solving the fundamental memory bandwidth bottleneck, shattering the perception that memory exposed by CXL is slower and therefore can only degrade performance.
It was important for us to let visitors play with the memory pooling configuration during the demo and change the amount of memory exposed to a running system on the fly using a CXL 1.1 Intel platform, as we heard from customers telling us that it would not be usable. And finally, we showcased a live benchmark for a standard block device exposed over CXL, running at amazing speeds, to show that CXL can deliver multiple functionalities at the same time with the same connection.
The CXL Consortium is an industry standards body dedicated to advancing Compute Express Link® (CXL®) technology. For more information or to join, visit www.computeexpresslink.org.