By: Anil Godbole, Senior Product Planning & Marketing Manager, Intel and Daniel Byrne, Senior MTS, DCG, Intel
AI agent sandboxes are commonly deployed as microVMs with several gigabytes of memory allocated to each, yet only a small fraction of that memory is touched between tool calls. This access pattern makes sandboxes a natural fit for the added capacity of CXL memory.
At the SC26 CXL Pavilion (Booth no. 1218), Intel will showcase a demo featuring the pre-production Intel® Xeon® CPU running sandboxes, with 512 GB of DRAM and 256 GB of CXL memory. In the demo, we compare software-managed memory tiering against Intel Flat Memory Mode built in the CPU hardware. In the former case, CXL memory is treated as a separate NUMA node. With Intel Flat Memory Mode, the processor places data, at cache-line granularity, across DRAM and CXL regions as one combined NUMA node. We show that Intel Flat memory mode runs each tool call at DRAM-like latency (+0.3% at p50, at most +2% at p99), while software tiering ends up with up to 22% slower latency. By transparently leveraging CXL memory capacity, Flat Memory Mode enables 50% more sandboxes per server while maintaining DRAM class performance.
Introduction
LLM agents execute their tool calls inside sandboxes, which are commonly deployed as microVMs [1, 9]. Each microVM holds a guest OS, a filesystem, a shell, and the agent’s working state for the duration of a session. In a deployment of sandboxes, each sandbox is allocated 2 to 8 GB of memory that stays resident. This makes the memory capacity the bound for how many sandboxes a server holds [11].
CXL memory devices add capacity, but CXL memory latency is two to three times the latency of local DRAM [7]. Software tiering hides this by tracking which pages are hot/cold and migrating them between DRAM and CXL [6, 8, 10], which on a virtualized host is coarse and costly [11]. Intel Flat Memory Mode [4, 5] is a hardware approach. The memory controller pairs CXL memory with an equal amount of DRAM. It uses the local DRAM as a direct-mapped cache, keeping each 64-byte line in either local DRAM memory or CXL memory. Recently used lines are swapped in from CXL memory by their corresponding tag. As a result, the OS sees the combined capacity as a single NUMA node.
Experiment Setup
Platform. We use a pre-production Intel Xeon CPU with 512 GB of DDR5 DRAM, and two 128 GB CXL Type-3 memory expanders interleaved as one 256 GB region, running Linux 7.2 (includes upstream s/w tiering TPP* patches). Under software tiering the CXL region is a CPU-less NUMA node with the operating system deciding which pages are promoted or demoted between local DRAM and CXL based on ‘hot/cold’ page metrics.
Under flat memory mode the platform pairs the 256 GB of CXL with 256 GB of DRAM: node 0 keeps the remaining 256 GB of DRAM and node 1 is a 512 GB flat memory node with the most recently used cache-lines in the DRAM half, which provide DRAM-class latency for those lines.
Workload. Each guest is an 8 GB Cloud Hypervisor microVM with one vCPU on 2 MB pages. It loops over two agent tool steps, a python pandas analysis and a grep search, each followed by an idle period which represents the model’s think time (Figure 1).
Figure 1: The per-guest loop: two tool steps, each followed by an idle period that stands in for the model’s think time.
The baseline is 55 guests running with DRAM only. This provides us with the latencies of DRAM only tool performance. With the addition of CXL memory, we can run 31 additional guests for a total of 86.
Table 1: Initial guest placement per memory layout (8 GB guests).
Results
Figure 2: 86-guest vs. the 55-guest DRAM-only reference. Left: slowdown of the guests on CXL memory. Right: guests at DRAM-class latency.
Figure 2 shows the 86-guest configuration. Under software tiering, the guests with pages that reside on CXL run 17% slower at p50 and 19% at p99 for pandas and 21%/22% for ripgrep. Therefore, we are constrained to 55 guests at DRAM-class latency in software tiering. On flat memory mode the 63 guests on node 1 provide similar performance compared to DRAM, +0.3%/0% for pandas and +1%/+2% for ripgrep, and fleet throughput is the same in both layouts (443 steps/min).
Therefore, the same 256 GB of CXL adds 31 sandboxes (+56% increase) at DRAM latency in flat memory mode. The hot data for a sandbox is a few hundred MB, the rest of the memory is resident but untouched between steps. Flat memory mode swaps in the hot cache-lines but leaves the cold data in CXL which is how we achieve DRAM class latency in flat memory mode.
Related Work
Zhong et al. [11] examined flat memory mode for Azure VMs: 82% of 115 workloads lose at most 5%, outliers lose up to 34%, and Memstrata uses page coloring and an online slowdown estimator to hold the worst case under 6%. A sandbox fleet is a new case as our guests are mostly uniform, with hot sets that fit the cache.
Ahn et al. [2, 3] evaluate SAP HANA on flat memory mode, show that it beats directly attached CXL by exploiting locality; our two-node layout is their MIXED mode. TPP and MEMTIS [6, 8] place pages by hotness (TPP the method used in our software tiering since it is the upstream Linux tiering method).
Summary
Visit the CXL Pavilion (Booth no. 1218) to learn how Intel’s Flat Memory Mode enables more AI agents per server while maintaining DRAM-class performance.
References
[1] Alexandru Agache, Marc Brooker, Andreea Florescu, Alexandra Iordache, Anthony Liguori, Rolf Neugebauer, Phil Piwonka, and Diana-Maria Popa. 2020. Firecracker: Lightweight virtualization for serverless applications. In NSDI ’20, 2020.
[2] Minseon Ahn, Thomas Willhalm, Donghun Lee, Norman May, Jungmin Kim, Daniel Ritter, and Oliver Rebholz. 2025. Exploiting locality in flat memory with CXL for in-memory database management systems. In DaMoN ’25, 2025.
[3] Minseon Ahn, Thomas Willhalm, Donghun Lee, Norman May, Jungmin Kim, Daniel Ritter, Oliver Rebholz, Yash Gupta, Tarik Yuksek, Byeonghun Hwang, Kyumin Park, and Seungpyo Cho. 2026. Locality-aware memory tiering with CXL for in-memory database management systems. Datenbank-Spektrum (2026).
[4] Intel Corporation. 2026. The advantages of managing the CXL memory tier in hardware.
[5] Intel Corporation. 2026. What is Intel flat memory mode on Intel Xeon 6 and Xeon 6+ processors?
[6] Taehyung Lee, Sumit Kumar Monga, Changwoo Min, and Young Ik Eom. 2023. MEMTIS: Efficient memory tiering with dynamic page classification and page size determination. In SOSP ’23, 2023.
[7] Nanqinqin Li, Yuhong Zhong, Asaf Cidon, and Michael J. Freedman. 2026. Harvesting sub-microsecond CXL memory stalls with LiteSwitch. In OSDI ’26, 2026.
[8] Hasan Al Maruf, Hao Wang, Abhishek Dhanotia, Johannes Weiner, Niket Agarwal, Pallab Bhattacharya, Chris Petersen, Mosharaf Chowdhury, Shobhit Kanaujia, and Prakash Chauhan. 2023. TPP: Transparent page placement for CXL-enabled tiered-memory. In ASPLOS ’23, 2023.
[9] Mike A. Merrill and others. 2026. Terminal-bench: Benchmarking agents on hard, realistic tasks in command line interfaces.
[10] Midhul Vuppalapati and Rachit Agarwal. 2024. Tiered memory management: Access latency is the key! In SOSP ’24, 2024.
[11] Yuhong Zhong, Daniel S. Berger, Carl Waldspurger, Ryan Wee, Ishwar Agarwal, Rajat Agarwal, Frank Hady, Karthik Kumar, Mark D. Hill, Mosharaf Chowdhury, and Asaf Cidon. 2024. Managing memory tiers with CXL in virtualized environments. In OSDI ’24, 2024. 37–56.