Compute Express Link® (CXL®) is a high-speed interconnect technology that enables efficient communication between processors, memory devices, and accelerators. CXL allows for both shared and pooled memory to serve different purposes. In this post, I will explain the importance of memory pooling and sharing and how CXL provides the much-needed solution.
Shared memory is a type of memory that is accessible by all processors in a system. Each processor can read from and write to the shared memory, allowing for easy data sharing and communication between processors. Pooled memory, on the other hand, is a type of memory that is shared between processors but managed by a specific memory controller. The memory controller can allocate and de-allocate memory dynamically, allowing for more efficient use of memory resources.
There are several reasons why someone might want to use shared memory on CXL, including:
1. Efficient communication between processors. Shared memory enables efficient communication between processors in a system or data center by allowing multiple processors to access the same block of memory. As a result, data can be easily passed between them without the need for complex data transfer mechanisms.
2. Simplified programming model. Shared memory simplifies the programming model for servers with multiple processors (including CPUs, GPUs, DPUs, etc.). Rather than managing complex data transfer mechanisms between multiple processing units, developers can simply access the shared memory as if it were local to their processor.
3. Low latency. Shared memory has very low latency compared to other communication mechanisms, such as inter-processor communication over a network. This makes shared memory ideal for high-performance computing applications that require rapid data access and processing.
4. Reduced power consumption. Shared memory also helps reduce power consumption in the data center. Instead of constantly transferring data between processors, the data can be stored in shared memory and accessed by each processor as needed. This reduces the amount of energy consumed by the systems.
Alternatively, there are many reasons why someone might want to use pooled memory on CXL, including:
1. Efficient memory allocation. Pooled memory allows for efficient allocation of memory resources among multiple processors. The memory controller can dynamically allocate and de-allocate memory as needed, ensuring that memory resources are used efficiently. In the case of an individual server with CXL memory cards added, one processor can use CXL memory and release it to the OS when finished, allowing it to be reallocated to another process on either CPU. In the future, CXL memory could exist in a dedicated memory box on the CXL fabric with systems requesting and releasing memory from this pool as needed.
2. Guaranteed memory access. Pooled memory provides guaranteed memory access to each processor by delegating a specific pool of memory that it can access, ensuring that the necessary memory resources to execute its tasks are readily available. In this case, a processor will always have some amount of locally attached memory to ensure it has the required minimum amount of necessary memory to boot the OS and run applications. CXL memory would then be allocated to a server where it would benefit most from either more memory capacity or bandwidth. CXL memory would be added to a system in large 64GB or more chunks when the benefits come more from memory capacity. For more memory bandwidth, the admin would use more CXL links and less memory per link to maximize the total added memory bandwidth.
3. Memory isolation. Pooled memory provides memory isolation between processors. Each processor has its own pool of memory separate from the memory pools of other processors. This prevents memory access conflicts while ensuring that each processor can execute its tasks without interference from other processors.
4. Security. Pooled memory also helps improve system security. By isolating the memory pools of each processor, it is more difficult for malicious code to access and manipulate data in other memory pools.
Overall, pooled memory on CXL provides significant benefits in terms of memory allocation efficiency, guaranteed access, isolation, and security for multi-processor systems. It is particularly useful for applications that require a lot of memory resources or need to ensure that access is effectively managed.
Summary
Whether it is better to use pooled or shared memory on CXL depends upon the specific use case and requirements of the individual system.
Shared memory is ideal if an application requires frequent data sharing and communication between processors, such as in data management systems that store frequently accessed data in a memory cache to speed up database operations. Shared memory provides a simple programming model with low latency, making it suitable for high-performance computing applications needing fast data access and processing while avoiding the overhead of interprocess communication, leading to improved performance. Shared memory is also more power-efficient than other communication mechanisms, such as inter-processor communication over a network.
Pooled memory is better suited for applications that require efficient memory allocation and utilization among multiple processors, for example within in-memory databases which would benefit most from increased memory capacity. Pooled memory provides guaranteed memory access and isolation between processors, ensuring that each processor has the resources necessary to execute tasks efficiently. Isolated memory pools also mean a more secure system.
However, there may be cases where a combination of both shared and pooled memory is needed to achieve the best performance for a particular application or workload. In a future post, I will explain how both options working in tandem on CXL can meet unique system demands. Stay tuned!
