Paper summary

Capacity-Latency Tradeoffs in CXL Memory Expander at Hyperscale

Haneul Park, Grant Ayers, Nam Sung Kim, Philip Levis, Brian Morris · University of Illinois Urbana-Champaign · Google · Stanford University

IEEE CAL, IEEE Computer Architecture Letters · Vol. 25, No. 1 · published March 2026

This page was written by an AI model to give a high-level overview of the paper. Please check the paper for detail: IEEE Xplore.

The big idea

DRAM now dominates the total cost of ownership (TCO) of hyperscale datacenter servers, and its price is rising further as manufacturing capacity shifts toward high-bandwidth memory (HBM) for AI datacenters. One way to cut that cost is to reuse older, slower DDR modules from retired servers by attaching them through a Compute Express Link (CXL) memory expander, such as the Open Compute Project (OCP) device proposed by Google and Meta. The OCP expander compresses data in hardware to raise its effective capacity. Because the CXL interface and decompression add latency, servers use it to hold cold pages, which are accessed infrequently.

When the host moves a page to the expander, the device splits it into blocks, compresses each block, and stores it in its DRAM. When the host later reads one cache line, the expander fetches the whole compressed block, decompresses it into an SRAM block cache, and returns the line. Two design choices trade capacity against latency: the compression algorithm (more complex algorithms compress better but decompress more slowly) and the block size (larger blocks compress better but take longer to decompress). Both also change read amplification, the amount read from DRAM to serve one cache line, which can saturate the expander's memory bandwidth. Commercial expanders use LZ4, a very simple algorithm, with 4 KiB blocks, choices inherited from software memory compression such as zswap. Expanders differ from software systems in three ways: no software page faults, compression in hardware, and much lower latencies. Whether they should make the same choices is the question this letter studies.

The key finding is counter-intuitive: a slower, more complex algorithm can lower access latency. Zstd takes longer per block than LZ4, but it compresses better, so each block cache miss reads fewer bytes from DRAM and the memory controller queues fewer requests. In an expander, memory bandwidth and queueing can dominate end-to-end latency, so the reduction in DRAM traffic more than offsets the extra decompression time. The paper captures this with bandwidth inflation: bytes the expander reads from DRAM to load a block, divided by the bytes of that block the host requests.

The study has two parts. First, it measures compressed size and decompression latency for four algorithms (LZ4, Snappy, Zstd, Deflate) and seven block sizes (64 B to 4 KiB) on HyperCompressBench, a benchmark built to reflect the compressibility seen in hyperscaler fleets, with latencies from FPGA designs scaled to ASIC speed. Second, it drives a gem5-based simulation of a commercial expander (four DDR4 channels, a 16 MB block cache) with ten cold-memory access traces from servers in Google's fleet, each server running hundreds of concurrent workloads.

Averaged over the ten traces and compared with LZ4 at 4 KiB, Zstd with 4 KiB blocks gives 35.8% more effective capacity and 11.5% lower average access latency, and Zstd with 2 KiB blocks gives 30.0% more capacity and 12.1% lower latency. LZ4 with 512 B blocks has the lowest latency (33.8% lower) but 24.4% less capacity. The paper also finds that for small Zstd blocks decompression becomes compute-bound, so the OCP guideline of matching decompression throughput to memory bandwidth is not enough. Its conclusion is that future expander specifications should be designed differently.

One cache-line read through a compressing CXL memory expander
where the two design knobs act on the read path
Hit path and miss path Host CPU 64 B read CXL CXL memory expander SRAM block cache 16 MB, 8-way, 30 ns hit Line served from SRAM no DRAM read, no decompression miss Read the block from DRAM: bytes = compressed size Decompress time grows with block size and algorithm Fill the cache and return the line to the host queues when DRAM bandwidth saturates queues when all engines are busy Knob 1 · compression algorithm LZ4, Snappy: fast to decompress, compress less Zstd, Deflate: 13 to 37% smaller, but slower Better compression means fewer DRAM bytes per miss, so less queueing at the DRAM Knob 2 · block size (64 B to 4 KiB) Larger blocks compress better and raise the block cache hit rate, but each miss reads and decompresses more; smaller blocks are more often incompressible and waste chunk space Commercial expanders use LZ4 with 4 KiB blocks. On ten fleet traces, Zstd with 2 or 4 KiB blocks gave both more effective capacity and lower average access latency.
Simplified from the paper's Fig. 1 (worst-case read path on a block cache miss) and Section I; block cache parameters from Section III-A, the 13 to 37% from Section II-B. The paper's Fig. 1 also shows a translation table, omitted here. Layout is this summary's.

Key takeaways

  • CXL memory expanders reuse legacy DDR DRAM and compress cold data in hardware; each cache-line miss reads and decompresses a whole block, which an SRAM block cache partly amortizes.
  • Commercial expanders use LZ4 with 4 KiB blocks, defaults inherited from software memory compression, even though hardware expanders have no page faults and much lower latencies.
  • Better compression reduces DRAM traffic per miss. Because DRAM bandwidth and queueing dominate expander latency, that saving can outweigh slower decompression.
  • Averaged over ten fleet traces, Zstd with 2 KiB blocks gives 30.0% more capacity and 12.1% lower latency than LZ4 at 4 KiB (35.8% and 11.5% at 4 KiB blocks); LZ4 at 512 B is fastest but loses 24.4% capacity.
  • For small Zstd blocks decompression turns compute-bound, so matching decompression throughput to memory bandwidth, as the OCP guideline does, is not sufficient.
Mechanism, step by step

How algorithm and block size set an expander's capacity and latency

The expander's read path, the two knobs in isolation, the fleet-trace simulation, and why DRAM traffic decides the outcome.

1 · How a compressing expander stores and serves data

The OCP expander sits behind a CXL link and holds cold pages in legacy DDR DRAM. Hardware compression is what distinguishes it from CXL memory pooling systems.

  1. Write a page. When the host moves a page to the expander, the device breaks it into blocks and compresses each block before storing it in DRAM.
  2. Skip incompressible blocks. If a block's compressed size is more than 80% of its original size, the expander stores it uncompressed. Such a block takes longer to read from DRAM but bypasses decompression.
  3. Quantize into chunks. The expander manages its DRAM in fixed-size chunks, so each compressed block occupies a whole number of chunks. The rounding lowers effective capacity.
  4. Read a cache line. On a request, the expander checks its SRAM block cache. On a miss it fetches the whole compressed block from DRAM, decompresses it into the block cache, and supplies the requested line. Later requests to the same block hit in SRAM.
Read amplification. Serving one 64 B line on a miss means reading a whole compressed block. Larger blocks and weaker compression read more bytes per miss, which can saturate DRAM bandwidth and raise latency. If many lines of a block are used before it is evicted from the block cache, the read is amortized over those accesses.

2 · The two knobs in isolation

Section II of the paper measures compressed size and decompression latency on HyperCompressBench for LZ4, Snappy, Zstd and Deflate at seven block sizes (64 B to 4 KiB in powers of 2). LZ4 and Snappy are designed for low latency; Deflate and Zstd for higher compression ratios. Latency comes from FPGA-based decompressor designs, scaled by a factor derived from an ASIC Deflate decompressor.

ObservationValue in the paper
Compressed size, Zstd and Deflate vs LZ4 and Snappy13 to 37% smaller; algorithms with the same computational foundations give similar sizes
Effect of larger blocks on sizeAll algorithms compress larger blocks more, with diminishing returns after 1 KiB; tiny blocks save little space
Zstd minimum decompression latency370 ns, 21× higher than the others
Deflate latencyCompetitive with LZ4 and Snappy for tiny blocks; for larger blocks Deflate and Zstd are more than 2.5× slower than LZ4 and Snappy
Incompressible LZ4 blocks18.5% at 512 B, 4.8% at 4 KiB
Decompression latency saved by incompressible blocks18% at 512 B (the paper's Fig. 3(c))

Chunk quantization. Smaller blocks waste more capacity in chunk rounding, and the waste depends on the distribution of compressed sizes. In the paper's example for LZ4 at 512 B, 27% of blocks compress to just over 128 bytes (25% of original size). With 64 B chunks these blocks are stored as 192 B; with 128 B chunks, as 256 B.

3 · The fleet-trace simulation

Raw decompression latency does not translate directly into access latency, for three reasons the paper gives: the CPU can use a line as soon as it is decompressed (so access position within a block matters), the block cache serves later accesses without DRAM traffic, and memory fetches themselves cost substantial latency that better compression reduces. Section III therefore simulates the whole expander with production access patterns.

ComponentSetup
TracesTen (Trace-0 to Trace-9) from Intel Optane-based far memory in a hyperscaler fleet; each from a different server running a different mix of hundreds of concurrent workloads
Sampling1% of memory accesses with Linux perf, limited by production overhead. Each record is a physical address and timestamp of a cold-page access; cache and host DRAM accesses are excluded
Data compressibilityEach access gets a compressed size and decompression latency pair sampled from the Section II results
SimulatorBased on gem5; idle latency matched to a commercial device: 0.00% difference on a cache hit, +2.61% on an incompressible block, −3.59% on a compressed block (Table I)
Expander DRAMFour 8 GB DDR4-2600 DIMMs, one per channel
Block cache16 MB, 8-way set-associative, LRU, 30 ns access
Decompression enginesEnough to match full DRAM bandwidth, per the OCP specification
ConfigurationsLZ4 and Zstd at each block size. The paper treats Snappy as interchangeable with LZ4 and notes Zstd is faster than Deflate at block sizes with sufficient compression

4 · Results across the ten traces

Capacity and latency of each configuration, relative to LZ4 with 4 KiB blocks
average over the ten fleet traces · redrawn from the paper's Table II
better on both metrics Pareto-optimal (bold in Table II) LZ4 Zstd label = block size −60−40−200+20+40 +40+200−20−40−60−80 effective capacity change vs. LZ4/4 KiB (%) access latency reduction (%) 4K (baseline) 2K 1K 512 256 128 64 4K 2K 1K 512 256 128 64
Values from Table II of the paper (averages across the ten fleet traces, simulated). Rings mark the configurations the table marks as Pareto-optimal. Block sizes are in bytes (K = KiB).
  1. Zstd at large blocks wins on both metrics. At 4 KiB it gives 35.8% more capacity and 11.5% lower latency than LZ4 at 4 KiB, despite Zstd's much higher single-block decompression latency. At 2 KiB it gives 30.0% more capacity and 12.1% lower latency.
  2. LZ4 at small blocks wins on latency only. LZ4 at 512 B has the lowest average latency (33.8% lower) and loses 24.4% capacity. Below 512 B, LZ4 latency stops improving while capacity keeps falling.
  3. Zstd at small blocks loses on both. From 512 B down, Zstd is slower than the baseline (−31.0% at 512 B, −82.3% at 128 B) and holds less data.

5 · Why a slower algorithm can be faster

To explain the result, the paper defines bandwidth inflation: bytes the expander reads from DRAM to load a block, divided by the bytes of that block the host requests. Read amplification counts the DRAM data read to serve a single cache line; bandwidth inflation also credits the block cache hits that one decompressed block serves. If every line of a compressed block were accessed, bandwidth inflation could fall below 1. The paper's worked example for LZ4 with 4 KiB blocks, on average:

  1. A 4 KiB block compresses to 44.5% of its size, so a miss reads 44.5% × 64 lines of data: read amplification of 28×.
  2. The average block cache hit rate is 71%, so 3.4 cache lines are served per block fetched.
  3. Bandwidth inflation is 64 × 44.5% × 29% = 8.2. Zstd at the same block size, compressing better, has a bandwidth inflation of 6.0.

On the representative trace the paper breaks down (Trace-3, its Fig. 4), larger blocks raise both the block cache hit rate and bandwidth inflation for both algorithms, and Zstd's inflation stays well below LZ4's.

The paper's explanation: because LZ4 compresses less, it must read more from DRAM to serve the same cache line, and that extra traffic causes significant queueing delay as DRAM bandwidth saturates. Better compression therefore buys more than capacity: it uses DRAM bandwidth more efficiently, which in this setting matters more than computational efficiency.

6 · Delayed hits and decompression queueing

Fig. 4 of the paper splits average latency into hit and miss latency for each block size. Two further effects shape the curves:

  1. Delayed hits. Average hit latency rises with block size for both algorithms, even though an SRAM read does not depend on block size. A request for a block that is still being decompressed counts as a hit but waits for the rest of the decompression. Larger blocks make it more likely that a later access lands in a block in flight.
  2. LZ4 misses. Miss latency rises with block size because higher bandwidth inflation adds DRAM queueing. Average latency grows more slowly than either, since larger blocks also raise the hit rate.
  3. Zstd misses at large blocks. Above 512 B, miss latency grows mainly from memory queueing, as with LZ4.
  4. Zstd misses at small blocks. Below 1 KiB, miss latency rises as blocks shrink. Halving the block from 1 KiB increases the number of blocks to decompress by 16% (a higher miss rate) but cuts per-block decompression time by only 4%. Decompression becomes compute-bound and requests queue at the engines. At 64 B, where all blocks are incompressible, engine load drops again.

7 · More decompression engines

The simulated expander follows the OCP guideline of enough decompression engines to match DRAM bandwidth. Because small Zstd blocks are compute-bound, the paper argues this guideline is insufficient, and its Fig. 5 repeats the Zstd runs with twice as many engines. Engine queueing falls, smaller blocks see much smaller increases in miss latency, and average hit latency also falls because delayed hits finish sooner. Reading the figure, the miss-latency change is visible from 64 B to 512 B, while at 1 KiB and above it is close to zero (this summary's reading of the chart).

Implication for the specification. The OCP specification requires LZ4 for its low latency. The paper concludes that under real access workloads, with queueing and bandwidth limits, DRAM bandwidth efficiency matters more than computational efficiency, and that future expander specifications should be designed differently.

8 · Scope of the evaluation

  1. Sampled traces. Production overhead limited sampling to 1% of memory accesses.
  2. Assigned compressibility. Performance and privacy concerns prevented measuring compressibility on production servers, so each access gets a size and latency from HyperCompressBench. Biased assignments that favored higher or lower compressibility for high-miss pages changed average latency by only 3 to 5%, leaving the conclusion unchanged.
  3. Simulated, two algorithms. Results come from a gem5-based model calibrated to a commercial device's idle latency, and only LZ4 and Zstd are simulated.
  4. One trace in detail. The latency breakdown and bandwidth inflation curves are shown for one representative trace (Trace-3); Table II averages all ten.
Common questions

FAQ

Q·How can a slower decompressor lower latency?▾

Decompression is only part of a miss. The expander must also read the compressed block from DRAM, and when DRAM bandwidth saturates, requests queue. Zstd compresses better than LZ4, so each miss reads fewer bytes and the queues are shorter. On the fleet traces, that saving outweighs Zstd's longer decompression: Zstd at 4 KiB has 11.5% lower average latency than LZ4 at 4 KiB, even though its single-block decompression latency is much higher.

Q·What is the difference between read amplification and bandwidth inflation?▾

Read amplification is the data read from DRAM to serve one cache line on a miss: for LZ4 at 4 KiB, 44.5% of 64 lines, or 28×. Bandwidth inflation divides the bytes read to load a block by the bytes of that block the host actually requests, so it credits the block cache hits a decompressed block serves. With a 71% hit rate, LZ4 at 4 KiB has a bandwidth inflation of 8.2, and Zstd at 4 KiB has 6.0.

Q·Isn't this the same tradeoff as in zswap?▾

The defaults (LZ4, 4 KiB) come from software memory compression such as zswap, but the paper lists three differences: expanders do not need software-handled page faults, they implement compression in hardware, and they have much lower latencies. When decompression runs in software behind a page fault, the fastest algorithm is the natural choice (inference). In a hardware expander, the paper finds that memory bandwidth and queueing can dominate end-to-end latency, which changes the answer.

Q·Why not use the smallest blocks for the lowest latency?▾

For LZ4, latency stops improving below 512 B (33.8% better at 512 B, 30.5% at 64 B) while capacity keeps falling (−24.4% at 512 B, −55.5% at 64 B). Small blocks compress less, are more often incompressible (18.5% of LZ4 blocks at 512 B vs 4.8% at 4 KiB) and waste more space to chunk rounding. For Zstd, small blocks are worse on both metrics because decompression becomes compute-bound and requests queue at the engines.

Q·What is a delayed hit, and why does it matter here?▾

A delayed hit is a request for a block that is already being fetched and decompressed after an earlier miss. It counts as a block cache hit, but it waits for the remaining decompression time. Larger blocks make delayed hits more likely, which is why average hit latency rises with block size even though an SRAM read takes the same time at any block size.

Q·Where do the traces come from, and what do they contain?▾

From Intel Optane-based far memory deployed in a hyperscaler fleet. There are ten traces, each from a different server running a different mix of hundreds of concurrent workloads. Each records the physical addresses and timestamps of accesses to cold pages, sampled at 1% with Linux perf; cache and host DRAM accesses are excluded. The paper notes that expander access patterns differ from those of individual programs because the OS selects cold pages globally across many workloads.

Q·How was compressibility assigned if it was not measured in production?▾

Each access in a trace is paired with a compressed size and decompression latency sampled from the HyperCompressBench results in Section II, because performance and privacy concerns ruled out measuring compressibility on production servers. As a check, the paper tried biased assignments that gave high-miss pages higher or lower compressibility; average latency changed by only 3 to 5%, and the qualitative conclusion held.

Q·Why are only LZ4 and Zstd simulated?▾

For brevity, based on Section II: Snappy and LZ4 behave interchangeably (algorithms with the same computational foundations give similar compressed sizes), and Zstd is faster than Deflate at the block sizes that offer sufficient compression.

Q·Would more decompression hardware help Zstd?▾

For small blocks, yes. With twice the OCP-recommended number of engines, engine queueing falls, small blocks see much smaller increases in miss latency, and hit latency drops as delayed hits complete sooner (the paper's Fig. 5). At 1 KiB and above, where latency is dominated by memory queueing, the figure shows little change. The paper's point is that matching decompression throughput to memory bandwidth, as the OCP guideline does, is not sufficient.

Q·Are these results from real hardware?▾

No. Compression sizes come from a benchmark corpus, decompression latencies from FPGA designs scaled to ASIC speed, and end-to-end latency from a gem5-based simulator. The simulator's idle latency was matched to a commercial expander: identical on a cache hit, within +2.61% for an incompressible block and −3.59% for a compressed block. The access traces are from production servers.

Background terms
CXL (Compute Express Link)A cache-coherent interconnect standard that runs over PCIe and lets a server attach memory devices outside its own memory channels.
CXL memory expanderA CXL device that holds DRAM and presents it to the host as additional memory. The OCP design studied here reuses legacy DDR modules and compresses data in hardware.
OCP (Open Compute Project)An industry foundation that publishes open hardware designs for datacenters, including the Hyperscale CXL Tiered Memory Expander specification proposed by Google and Meta.
Cold pageA memory page that is accessed infrequently, which the OS can place in slower, cheaper memory with little performance cost.
Far memoryA memory tier slower than local DRAM, such as compressed memory, an expander or Optane, used to hold cold pages.
Compression blockThe unit of data the expander compresses and decompresses as a whole, 64 B to 4 KiB in this paper.
LZ4 and SnappyLossless compression algorithms based on simple dictionary matching, designed for very fast decompression at modest compression ratios (general knowledge for the algorithm internals).
Zstd and DeflateLossless compression algorithms that add entropy coding to dictionary matching, giving higher compression ratios at the cost of slower decompression (general knowledge for the algorithm internals).
Block cacheAn SRAM cache inside the expander that holds recently decompressed blocks, so later reads to the same block avoid a DRAM fetch and decompression. 16 MB, 8-way, LRU in the simulated device.
Read amplificationThe ratio of data read from DRAM to the data requested when serving a single cache line, which is large because a miss reads a whole compressed block.
Bandwidth inflationThe paper's metric: bytes read from DRAM to load a block divided by bytes of that block the host requests, so it accounts for block cache hits. Values below 1 are possible in principle.
Incompressible blockA block whose compressed size exceeds 80% of its original size; the expander stores it uncompressed and skips decompression when reading it.
Chunk quantizationThe rounding of each compressed block up to a whole number of the fixed-size chunks the expander allocates, which wastes some capacity.
Delayed hitA cache request for data that is already being fetched after an earlier miss; it counts as a hit but waits for the fetch (here, decompression) to finish.
Decompression engineA hardware unit in the expander that decompresses blocks; the number of engines limits how many blocks can be decompressed at once.
HyperCompressBenchA compression benchmark corpus designed to reproduce the data compressibility observed in hyperscaler fleets.
gem5An open-source, cycle-level computer system simulator widely used in architecture research.
Pareto-optimal configurationA configuration for which no other configuration is better on one metric without being worse on another; here the metrics are capacity and latency.
Summary of H. Park, G. Ayers, N. S. Kim, P. Levis and B. Morris, "Capacity-Latency Tradeoffs in CXL Memory Expander at Hyperscale," IEEE Computer Architecture Letters, vol. 25, no. 1, 2026. Numbers are from the paper unless marked as this summary's arithmetic.