Paper summary

Towards Memory Specialization: A Case for Long-Term and Short-Term RAM

Peijing Li, Muhammad Shahir Abdurrahman, Rachel Cleaveland, Sergey Legtchenko, Philip Levis, Ioan Stefanovici, Thierry Tambe, David Tennenhouse, Caroline Trippel, H.-S. Philip Wong · Stanford University · Microsoft Research · Independent Researcher

DIMES '25, 3rd Workshop on Disruptive Memory Systems · Seoul, Republic of Korea · October 13, 2025, co-located with SOSP '25

This page was written by an AI model to give a high-level overview of the paper. Please check the paper for detail: ACM Digital Library.

The big idea

SRAM and DRAM have stopped getting cheaper per byte. DRAM cost trends have stagnated over the past 15 years, SRAM cells have barely shrunk since the 7 nm process node, and DRAM now accounts for over 50% of server hardware cost. Packaging advances such as HBM (High Bandwidth Memory) and LPDDR buy bandwidth or lower power, but they do not change the cost of a memory cell, so memory has become the main cost, power and performance bottleneck of modern systems.

Emerging memory technologies scale better. Gain cell embedded DRAM reaches 2 to 3 times SRAM's density at similar process nodes, and 3D vertical RRAM (V-RRAM) can reach up to 10 times the density of HBM4 at the same node. None of them is a drop-in replacement, though: they read and write asymmetrically and trade retention time against write endurance. RRAM's limited endurance and high write energy rule it out for frequently overwritten data; gain cells need periodic refresh. Each device is good for some data and bad for other data, and exposing every device's quirks directly to software would be unmanageable.

The paper's proposal is to organize emerging devices into just two new memory classes, defined by the data they serve rather than by device physics. Long-term RAM (LtRAM) is for read-heavy data with long lifetimes (minutes or longer): it accepts higher write latency and energy and lower endurance in exchange for dense, low-energy reads and long retention. Short-term RAM (StRAM) is for transient data with sub-second lifetimes: it needs symmetric reads and writes and very high endurance, but not long retention. Together with SRAM, DRAM and NAND flash they form a five-class memory system, and the operating system manages classes, not individual technologies.

The paper then maps workloads onto the classes. Model weights during inference, code pages, search indices and read-mostly caches such as Redis and Memcached fit LtRAM; neural network activations, gradients, message queues and short-lived cache data fit StRAM. Using them well breaks the assumption that memory forms a hierarchy ordered by distance from the processor, so the paper lays out the systems problems that follow: new OS abstractions, placement policies that reason about lifetime and write cost instead of hotness, memory consistency and coherence when StRAM can lose data or LtRAM writes lag, and power and thermal limits in dense, tightly integrated systems.

This is a position paper. It has no prototype or measurements of its own; its contributions are the two class definitions, a survey of device technologies that could implement them, and a list of research questions. The numbers it cites (scaling trends, densities, rack power) come from prior work.

From a three-level hierarchy to five memory classes
two new classes, each defined by the data it serves
Today: a hierarchy ordered by distance from compute SRAM on-chip caches DRAM off-chip main memory NAND flash storage Farther from compute means slower and cheaper; none of the three is getting cheaper per byte. Proposed: add two classes, chosen by data lifetime and access pattern StRAM: short-term RAM transient data, sub-second lifetimes Gives up: long retention (may need refresh) Keeps: symmetric reads and writes, very high or unlimited endurance Devices: gain cell eDRAM, 1T1C FeRAM Data: activations, gradients, queues, short-lived cache data LtRAM: long-term RAM read-heavy data, minutes or longer Gives up: write latency, write energy, endurance Keeps: dense, low-energy reads; retention Devices: RRAM, MRAM, FeRAM, managed-retention DRAM variants Data: model weights, code, indices, caches SRAM, DRAM and NAND flash stay. All five together are the paper's Table 1.
Summarized from Sections 2, 4.1 and 4.2 and Table 1 of the paper. The example data are drawn from Sections 4 and 5; the layout is this summary's.

Key takeaways

  • SRAM and DRAM have stopped scaling in cost per byte; DRAM is over 50% of server hardware cost, and packaging (HBM, LPDDR) does not fix cell cost.
  • Emerging memories (gain cell eDRAM, RRAM, MRAM, FeRAM) are denser but not drop-in replacements: reads and writes are asymmetric, and retention trades against endurance.
  • The proposal: two new classes defined by data, not device. LtRAM for read-heavy, long-lived data; StRAM for transient, frequently accessed data. With SRAM, DRAM and NAND flash, a five-class memory system.
  • Workloads split naturally: inference weights, code and indices toward LtRAM; activations, gradients, queues and short-lived cache data toward StRAM.
  • A position paper with no evaluation. Its research agenda: OS abstractions, lifetime-aware placement and profiling, consistency and coherence for StRAM and LtRAM, and power-aware integration.
Mechanism, step by step

From a memory hierarchy to specialized memory classes

Why SRAM and DRAM stalled, what emerging devices offer, the two proposed classes, and the systems questions they raise.

1 · Why SRAM and DRAM stopped getting cheaper

Today's systems use three kinds of memory: SRAM caches on the compute chip, off-chip DRAM as main memory, and NAND flash (or other non-volatile storage) for permanent data. The two volatile ones are hitting physical limits:

  1. SRAM uses six transistors per bit, is fast, holds its state while powered and integrates easily with logic. Its cell size has stopped shrinking significantly since the 7 nm node (the paper's Figure 1b), because balancing transistor sizing and threshold voltages gets harder at small nodes and low voltages.
  2. DRAM uses one transistor and one capacitor per bit. The capacitors are narrow and tall, which makes integration with high-performance logic impractical. Charge leaks and reads are destructive, so DRAM must be refreshed (every 32 ms for DDR5), which draws power even when idle. Smaller cells are still feasible, but capacitor manufacturing prevents lower per-cell cost: DRAM cost trends have stagnated over the past 15 years (Figure 1a).
  3. Packaging helps capacity and bandwidth, not cost per byte. LPDDR reaches 10 Gbps per pin versus 5.6 for conventional DDR but must be soldered to the board. HBM stacks dies on a silicon interposer for much higher bandwidth, but it is substantially more expensive than DDR DRAM, terabyte on-package configurations are not practical, and density gains may stop beyond 20 dies.

2 · What emerging memories offer, and why they are not drop-in

Gain cell embedded DRAM, ferroelectric RAM (FeRAM), magnetoresistive RAM (MRAM) and resistive RAM (RRAM) have different scaling roadmaps. The paper's density examples:

TechnologyDensity claim in the paperConstraint that blocks drop-in use
Gain cell eDRAM2 to 3× SRAM's density at similar process nodes; all-transistor cell allows on-chip integrationNeeds periodic refresh: it stores data on transistor capacitance instead of SRAM's feedback loop
Planar RRAMUp to 2× DRAM's densityLimited endurance and high write energy rule out frequently overwritten data
3D V-RRAMUp to 10× HBM4's density at the same process node; stacks up to 64 layers versus HBM's 16 dies; may keep scaling where DRAM struggles below 11 nmSame as RRAM

Two properties decide which data a device can hold. First, for a device with limited endurance, the data's write rate over the device's expected service life must fit within the cell endurance. Second, matching the retention of the cells to the lifetime of the data helps reduce energy consumption.

The paper's argument: because each device fits only some write patterns and data lifetimes, memory should be built from a heterogeneous set of device arrays, each matched to part of a workload's accesses. This is what the paper calls memory specialization.

3 · Two new classes instead of many devices

Specialized hardware cannot stay invisible to software, but making the OS reason about every device is unmanageable. The paper groups emerging devices into two classes that capture most device and workload trade-offs without excessive segmentation:

StRAM (short-term RAM)LtRAM (long-term RAM)
Optimized forTransient data that is frequently accessed, or read once and discardedPersistent, read-heavy data
Data lifetimeShort, sub-secondMinutes and longer
Reads and writesSymmetric performance; low write energyRead performance and read energy first; accepts higher write latency and energy
EnduranceVery high or unlimitedLower, accepted as a trade-off
RetentionNot necessarily highLong
Candidate devicesGain cell eDRAM; 1T1C FeRAM if its endurance can be brought high enoughRRAM, MRAM, FeRAM, managed-retention DRAM variants
Where it sitsOn-die to near-compute, depending on device and thermal constraintsOn-die, 3D-stacked packages or off-chip modules

LtRAM builds on managed-retention memory (MRM), a prior proposal to repurpose storage-class memories for workloads such as LLM inference by trading retention time and write performance for improved endurance and lower I/O energy. The key insight the paper gives for LtRAM is that long data lifetimes and read-heavy access allow optimizations unsuitable for general-purpose memory, and that today read-intensive data competes for the same DRAM as frequently modified data.

4 · The five-class memory system

The new classes are added to, not substituted for, the existing three. The paper's Table 1:

ClassStrengthsWeaknessesUses
SRAMLow read/write latency, long retention, low static powerLow densityFast read/write caches
DRAMVery denseOff-chip only, read/write energy, high static power, refresh overhead, destructive readsLarge, random-access read/write data
NAND flashExtremely dense, low static powerOff-chip only, low bandwidth, limited endurance, expensive erases, very high latencyStorage, rarely accessed data
StRAMDense, low write energy, low static powerShort retention, refresh overheadFast read/write caches, write-and-read scratchpads
LtRAMVery dense, low read energy, low static powerHigh write energy, high write latency, limited enduranceRead-mostly data, read-mostly caches

In the table, "power" is continual leakage and "energy" is the cost of one active read or write.

5 · Which data goes where

The paper walks through three application domains and assigns their data by lifetime and write pattern.

Matching data to StRAM and LtRAM
examples from the paper's Section 5, by domain
StRAM: short-lived, often written LtRAM: long-lived, read-mostly Server workloads ML inference ML training Processor cores Section 5.1 Section 5.2 Section 5.2 Section 5.3 logging and telemetry buffers, query-processing temporaries, serverless ephemeral data, hot DB pages Redis, Memcached, Ray Plasma, CDN and DNS data; search inverted indices; cold database pages; code pages activations and intermediate results, discarded right after computation model weights: immutable, read in large blocks at high bandwidth activations and gradients, used briefly, then discarded not assigned: weights see both read-heavy and write-heavy access short-lived L1/L2/L3 data: call stacks, temporaries, thread communication buffers hot instruction paths (listed among LtRAM uses in Section 4.2)
Assignments as stated in Sections 4.2 and 5.1 to 5.3 of the paper. The paper does not assign training weights to either class; the grid layout is this summary's.
  1. Server workloads. In-memory stores, CDNs and DNS servers are read-mostly with variable lifetimes and could adapt their caching to a mix of LtRAM and StRAM. Logging and telemetry produce write-once data with short-to-medium lifetimes, which StRAM can buffer before archival to flash. Code pages are read-intensive with very long lifetimes, yet today share DRAM with frequently modified data.
  2. Machine learning. Inference weights are typically immutable and read in large blocks; the paper notes that HBM and DRAM are over-provisioned in write performance for them. Activations are discarded right after use. Training mixes read-heavy and write-heavy access to weights with write-heavy access to gradients and optimizer states.
  3. Processor cores. Most L1/L2/L3 cache data lives briefly, so SRAM's indefinite retention is over-provisioned for it; StRAM could give comparable performance at higher density and lower static power.

6 · OS abstractions and data placement

Traditional memory hides device differences behind a flat address space, and placement policies assume a hierarchy where closer means faster. Specialized classes break that assumption: SRAM and StRAM could both be on-chip scratchpads chosen by application need, and LtRAM could be off-chip yet accessed directly for read-heavy, long-lived data despite lower bandwidth. The paper's questions:

  1. What to expose. How should the OS expose retention, endurance limits and read/write asymmetry while keeping memory programmable? The paper points to NAND flash exposing internal organization and GPUs exposing explicit SRAM instructions as evidence that low-level control pays off, at the cost of code complexity.
  2. Placement without a hierarchy. Linux NUMA nodes and tiering policies rank memory by access frequency, latency or memory-level parallelism. LtRAM's large read/write gap does not fit that ranking, and it interacts with the fact that reads load into the cache synchronously while writes reach memory asynchronously, on eviction.
  3. Who decides. Databases and ML frameworks know their buffer pools and dataflow and can place data themselves. Other applications can rely on the OS (hot-cold policies, placing pages mapped read-only in LtRAM, or inference from program behavior). The OS must support transparent management, hints and explicit control.
  4. Profiling. Page-level hot-cold and bandwidth tracking cannot express lifetime, endurance, caching or read/write trade-offs. Lightweight fine-grained profiling, and a way to pass its results down (compiler annotations or special instructions, as in the Expressive Memory interface), are open.

7 · Memory consistency and coherence

Memory consistency models (MCMs) define which values a load may return, and coherence protocols make all threads agree on the order of writes to one location. Both assume volatile memories with high retention times, low write latency and energy, and high endurance for all data. StRAM and LtRAM each break part of that. The paper's litmus tests (all values start at 0):

Paper figureThread 1Thread 2What goes wrong
Figure 3 (StRAM)data = 1data is corrupted to 42; read data returns 42An "out-of-thin-air" value that was never written, which most modern MCMs forbid
Figure 4 (StRAM)data = 1read data returns 1; data is corrupted to 0; read data returns 0Threads disagree on the order of writes to one location: a coherence violation
Figure 5 (LtRAM)data = 1; flag = 1read flag returns 1; read data returns 0Stale data despite an updated flag: forbidden by sequential consistency and total store order, allowed by ARMv8 and RISC-V's weak order (which provide fences)
  1. StRAM fixes. Cache controllers can refresh lines before retention runs out or evict them early. Software-initiated invalidations, with or without writebacks, could lower hardware cost. Coherence metadata stored in StRAM faces the same corruption risk.
  2. LtRAM fixes. Invalidation-based coherence writes cache-line metadata often, which is costly on LtRAM. Update-based protocols could suit read-mostly, few-producer, many-consumer data; alternatively, hardware could keep metadata in write-optimized StRAM and data in read-optimized LtRAM.
  3. Mixed caches. A single cache level built from SRAM, StRAM and LtRAM parts behaves like a multibanked memory that one thread can access concurrently, which is how the Figure 5 outcome can arise.

8 · Power and thermal limits

The paper argues that power is becoming a first-order design goal and that memory is a large part of it.

Rack power is rising, and memory is a large share
rack power from Section 6.4 · component shares from the paper's Figure 6
Power per rack (kW) typical CPU-based servers current AI racks (up to) next-gen AI racks (late 2027) power delivery being developed 20 120 600 over 1,000 Peak power by component, warehouse-scale computer 33% 10% 30% 5% 22% CPUs disks DRAM network other Bars are to scale within each panel. The component shares come from a survey the paper cites.
Rack power values from Section 6.4 of the paper (each from a cited industry source); the last bar marks power delivery being developed "for over 1 MW per rack". Component shares are the paper's Figure 6, from a data center energy survey.

Memory draws power through static leakage, refresh and data movement, and data movement energy, which grows superlinearly with interconnect distance, typically dominates. Matching cell retention to data lifetime cuts static power by avoiding unneeded SRAM leakage and DRAM refresh. Specialization therefore has to be co-optimized with interconnect, packaging and data assignment, and dense, tightly integrated arrays may need new cooling, such as microfluidics.

9 · What the paper leaves open

  1. No evaluation. The paper defines classes and maps workloads to them; it does not build or measure an LtRAM or StRAM system.
  2. Future devices. It focuses on technologies available today. Monolithic 3D DRAM (multiple cell layers on one wafer) is one emerging option; it needs at least 64 layers to match 2D density. Commercial viability, particularly cost, is uncertain.
  3. Integration. Unlike DRAM and NAND, which must be off-chip because of process incompatibilities, StRAM and LtRAM can be placed anywhere from on-chip to 2.5D chiplets to 3D stacks, which the paper sees as a chance to co-design cells, interconnect and placement.
  4. The research agenda in the conclusion: fine-grained access-pattern profiling, lifetime-aware placement policies, memory system abstractions for heterogeneous devices, and coherence protocols for specialized memories, with collaboration from materials science and device physics through architecture and systems.
Common questions

FAQ

Q·What do "long-term" and "short-term" refer to?▾

Data lifetime, the time data must stay in memory. LtRAM holds data that lives for minutes or longer and is mostly read; StRAM holds data that lives for under a second, such as activations or queue entries. "Memory specialization" in the title means building memory from several device types, each matched to the write patterns and lifetimes of part of a workload.

Q·Why two new classes and not one per technology?▾

The paper's concern is software complexity. Specialized hardware may differ from machine to machine, and operating systems, middleware and applications must keep working as devices change. Two classes give the OS a small set of abstractions while letting each class be built from different devices. The paper acknowledges that a finer continuum of retention and access characteristics is conceivable, but argues that two classes capture most device and workload trade-offs without excessive segmentation.

Q·Does LtRAM have to be non-volatile?▾

The paper does not require it. It describes LtRAM data as "persistent" in the sense of long-lived, and its candidate devices include non-volatile memories (RRAM, MRAM, FeRAM) and managed-retention DRAM variants. What defines the class is long retention, cheap reads and tolerance for costly writes.

Q·How is this different from memory tiering?▾

Tiering policies, including Linux NUMA placement and CXL memory tiering, assume memories can be ranked: a page moves to a slower tier based on access frequency, latency or memory-level parallelism. The paper argues that LtRAM and StRAM break this ranking. LtRAM is fast to read and slow to write, so whether a page fits depends on its write pattern and lifetime, not on how hot it is. Placement must reason about those properties directly, which today's page-level hot-cold tracking cannot express.

Q·How does LtRAM relate to managed-retention memory?▾

LtRAM builds on it. Managed-retention memory (MRM), from co-authors at Microsoft Research, proposes repurposing storage-class memories for workloads such as LLM inference by trading retention time and write performance for improved endurance and lower I/O energy. It found DRAM too slow for that workload and HBM over-provisioned in write performance and endurance. LtRAM generalizes the idea into a memory class that several technologies could implement.

Q·Who decides what goes into LtRAM or StRAM?▾

The paper wants a spectrum. Databases and ML frameworks know their buffer pools and dataflow and could place data explicitly. Other applications would rely on the OS, for example hot-cold policies, placing pages mapped read-only in LtRAM, or inferring behavior from the program. Application hints, compiler annotations or special instructions sit in between. Building the profiling and hint interfaces is an open problem the paper names.

Q·Can StRAM's short retention break program correctness?▾

Yes, if nothing manages it. If a value decays before it is used, a read can return a value that was never written (an "out-of-thin-air" result, forbidden by most consistency models), or two reads of the same location can disagree on the order of writes (a coherence violation). The paper's remedies: cache controllers refresh lines before retention expires or evict them early, or software issues invalidations. Coherence metadata kept in StRAM needs the same protection.

Q·Why would LtRAM affect cache coherence?▾

Invalidation-based coherence protocols write cache-line metadata frequently, and writes are what LtRAM is bad at. The paper suggests update-based protocols for read-mostly data with few producers and many consumers, or keeping metadata in write-optimized StRAM while data stays in read-optimized LtRAM. It also notes that a cache level mixing SRAM, StRAM and LtRAM parts can let a reader see an updated flag but stale data, an outcome some consistency models (sequential consistency, total store order) forbid and others (ARMv8, RISC-V's weak order) allow.

Q·Where would these memories physically sit?▾

Anywhere from on-die to off-chip. StRAM ranges from on-die to near-compute placement depending on device and thermal constraints. LtRAM can be on-die, in 3D-stacked packages or in off-chip modules. The paper sees this flexibility as an advantage over DRAM and NAND, which must be off-chip because their processes are incompatible with logic, and gives the example of LtRAM placed off-chip yet accessed directly for read-heavy data.

Q·Why does power come into it?▾

Rack power is climbing: the paper cites about 20 kW for typical CPU-based servers, up to 120 kW for current AI racks and 600 kW expected for next-generation AI racks in late 2027. In the warehouse-scale breakdown it cites, DRAM is 30% of peak power. Matching retention to data lifetime reduces SRAM leakage and DRAM refresh, but data movement energy typically dominates memory power, so specialization must be co-designed with interconnect and packaging.

Q·Does the paper evaluate LtRAM or StRAM?▾

No. It is a position paper. It defines the two classes, surveys candidate devices with numbers from prior work (scaling trends, densities, power), maps workloads to classes, and lists research questions. There is no prototype, simulation or application measurement.

Q·Where did the LtRAM and StRAM split come from?▾

The acknowledgments credit Shridhar Mukund of EMD with first suggesting LtRAM and StRAM as a useful decomposition, in the context of ML inference.

Background terms
LtRAM (Long-term RAM)A proposed memory class for read-heavy data with long lifetimes: dense memory with low read energy that accepts high write latency and energy and limited endurance.
StRAM (Short-term RAM)A proposed memory class for transient, sub-second data: dense memory with symmetric reads and writes, low write energy and very high endurance, but short retention.
SRAM (Static RAM)Memory that stores each bit in a six-transistor latch, holding its value while powered without refresh. Fast and logic-compatible, used for on-chip caches.
DRAM (Dynamic RAM) and refreshMemory that stores each bit as charge on a capacitor gated by one transistor. Charge leaks and reads drain it, so the contents must be rewritten periodically (refreshed), every 32 ms for DDR5.
RetentionThe time a memory cell reliably holds data without being refreshed.
EnduranceThe number of write cycles a memory cell supports before it permanently degrades.
Data lifetimeHow long data must stay persistent and accessible in memory hardware, from sub-second activations to long-lived code pages.
HBM (High Bandwidth Memory)DRAM dies stacked vertically and connected to the processor through a silicon interposer in the same package, giving very high bandwidth at higher cost and limited capacity.
LPDDRLow-power DRAM that runs at higher per-pin rates than conventional DDR but is soldered to the board instead of sitting in a removable module.
Gain cell eDRAMEmbedded DRAM whose cell stores charge on a transistor's own capacitance instead of a separate capacitor. It is all-transistor, so it can be built on a logic chip, but it needs refresh.
RRAM and 3D V-RRAMResistive RAM stores a bit as the resistance of a material switched by voltage; a cell is one transistor and one resistor, or one resistor. V-RRAM stacks such cells in vertical layers (8 or 64 in the paper's examples).
FeRAM and MRAMFerroelectric RAM stores a bit in the polarization of a ferroelectric material; magnetoresistive RAM stores it in the magnetic orientation of a layer, read as a resistance. Both are non-volatile.
Managed-retention memory (MRM)A prior proposal for a memory that trades retention time and write performance for improved endurance and lower I/O energy, aimed at workloads such as LLM inference. LtRAM builds on it.
Process nodeA semiconductor manufacturing generation, named by a nominal feature size such as 7 nm; smaller nodes have historically meant smaller, cheaper cells.
NUMA nodeIn Linux, a group of memory with similar performance associated with a group of cores, used to keep allocations local. Tiering systems extend it to CXL-attached memory.
Memory consistency model (MCM)The rules of an instruction set architecture that define which values a load may legally return when threads share memory, for example sequential consistency, total store order or ARMv8's weaker model.
Cache coherenceThe hardware protocol that makes all cores agree on a single order of writes to each memory location, by invalidating or updating other cores' cached copies.
Static power and active energyStatic power is the continual leakage a memory draws just by being powered; active energy is the cost of each read or write operation.
Summary of P. Li, M. S. Abdurrahman, R. Cleaveland, S. Legtchenko, P. Levis, I. Stefanovici, T. Tambe, D. Tennenhouse, C. Trippel and H.-S. P. Wong, "Towards Memory Specialization: A Case for Long-Term and Short-Term RAM," DIMES '25. Numbers are from the paper unless marked as this summary's arithmetic.