By: Yamini Shastry (VIAVI Solutions)
The growing demand for modern workloads, particularly in Artificial Intelligence (AI), Machine Learning (ML) and High-performance computing (HPC), has pushed the boundaries of traditional interconnect technologies. Compute Express Link® (CXL®) has emerged to meet these demands, offering high-speed, low-latency communication between processors, accelerators and memory devices. However, for CXL to reach its full potential, proper testing, analysis, and optimization are critical. This is where protocol analysis tools come into play. In this blog, we will discuss the CXL link protocol and the role of protocol analysis tools in troubleshooting CXL links.
CXL Link Protocol and Stages
In the figure below, we broadly categorize the CXL link into four stages. This provides a simplified view of a point-to-point CXL link as seen from an inline protocol analyzer.
Link Training Status State Machine (LTSSM): Upon power-on, the CXL link enters the LTSSM stage. During this stage, Training Sequences (TS) and Modified Training Sequences(mTS) are used to negotiate key link features, such as link width, speed, and CXL protocol capabilities. After a successful LTSSM, the link transitions to the L0 state, where the link partners establish CXL protocol connection and begin exchanging CXL FLITs.
ALMP (Arbitrator and Multiplexer): Next, the ALMP handshake begins. CXL leverages the existing PCIe® Physical Layer to facilitate the ALMP handshake. The Arb-Mux dynamically multiplexes data from multiple protocols (CXL.IO and CXL.Cache-Mem), routing it through the Physical Layer. This enables CXL to fully utilize the PCIe physical layer while maintaining synchronization of virtual Link State Machines (vLSMs) across all layers.
Enumeration (CXL.io): Once the link layer state machines are active, the enumeration and discovery process begin. The host reads the device’s Configuration Space Registers, to identify, and configure CXL functionality. Configuration and memory-mapped registers are accessed to configure various CXL subsystem components, such as RAS capabilities, security, IDE, and link features.
.Mem/.Cache Transactions: After the host gathers the necessary information and activates the correct register settings, actual CXL transactions commence. For CXL-attached memory, the memory is mapped into the host’s coherent space, enabling coherent read and write operations. These read and write operations are seen as CXL.mem transactions when captured on the wire using protocol analyzers.
CXL Transactions
Typically, a transaction consists of multiple FLITs, which are fixed-size units of transfer. The latest version of CXL, CXL 3.0, operates over PCIe 6.0, offering 64 GT/s bandwidth per lane and scaling up to 16 lanes. This allows for higher bandwidth and, in turn, more transactions with lower latencies. Additionally, CXL 3.0 supports 256-byte flits, providing more slots for data transfer and further optimizing data transfer efficiency. The Memory Read (MemRd) transaction shown below contains four FLITs: a Memory read command (MemRd), a completion (Cmp-E), and two data FLITs (MemData).
A FLIT contains multiple slots, and each slot can contain multiple headers. These headers indicate the data contained in the slot, such as commands, completions, or payload data for a transaction.
As you can see, multiple simultaneous transactions can occur within one FLIT and multiple FLITs are needed to complete a single transaction. This complexity makes CXL protocol analysis quite challenging. Increased visibility of traffic flow from transactions to FLITs to byte/bit level, along with insights into protocol issues are crucial for effective CXL testing and root cause analysis. This level of detail can be captured using inline protocol analyzers which are critical tools in debugging and verifying the operation of high-speed data protocols.
How protocol analysis tools can help
CXL protocol analyzers provide insights into the CXL traffic flowing across the interconnect, enabling developers to track, capture, and analyze protocol communication. These tools help identify issues such as latency, bandwidth, bottlenecks and misconfigurations that can hinder performance. As the adoption of CXL systems increases, debugging and optimizing the CXL protocol becomes critical.
A CXL analyzer equipped with analytical software can become a time-saving tool for troubleshooting CXL links. Identifying implementation bugs early in the product life cycle is crucial for ensuring stable and high-quality products. Analyzers offer an easy view of traffic flow and provide trace analytics necessary to pinpoint problems at every layer of the protocol stack. They also feature user-friendly interfaces that aid faster debugging by flagging protocol violations and anomalous/hidden behaviors. Key features of CXL protocol analyzers include deep trace buffers, stateful triggering/filtering capabilities, detailed decodes down to the bit level, error detection capabilities, protocol compliance and comprehensive post-capture analysis. The figure below provides an example of software tool analysis, specifically from the VIAVI Xgig® software suite:
Conclusion
As the adoption of CXL protocol grows, so does the need for tools that can verify its performance, reliability, and compliance. CXL protocol analyzers play an essential role in ensuring that the interconnect meets the high demands of modern computing environments. By leveraging these tools, organizations can ensure that their CXL devices adhere to the specifications, maintain high quality, achieve a short time to market, and are prepared for the future of data center architecture. Visit VIAVI’s exhibit at CXL DevCon 2025 to discuss how protocol analyzers can optimize your CXL implementations.
About the author
Yamini Shastry is the Head of Customer Success at VIAVI Solutions, ensuring customers receive comprehensive support in operating and applying VIAVI Xgig protocol analysis systems. With years of software engineering experience, Yamini specializes in developing and delivering test and analysis solutions for high-speed protocols.