Paper summary

The Kernel's Write: Application Read-Only Memory

Hui Sub Shim, Katherine Mohr, Philip Levis · Stanford University

DIMES '26, 4th Workshop on Disruptive Memory Systems · Prague, Czech Republic · September 29 to October 2, 2026

This page was written by an AI model to give a high-level overview of the paper. Please check the paper for detail: ACM Digital Library.

The big idea

DRAM has become the most expensive part of a server. Microsoft Azure reports that it is over half of total server cost, and DRAM's cost per bit has stopped falling the way it used to. Long-term RAM (LtRAM) is a class of emerging memories (RRAM, PCM, FeFET, MRAM) that read at close to DRAM speed and cost less per bit. The catch is that every candidate has the same shape: writes are much slower than reads, writes happen in chunks larger than a 64-byte cache line, and each cell survives only a limited number of writes. That makes LtRAM a good fit for long-lived, read-mostly data such as code, machine learning inference weights, and in-memory caches.

Intel Optane, the only LtRAM-class memory ever sold as a DIMM, hid that shape from software. Its on-DIMM controller made the media look like DRAM using a write-combining buffer, a read buffer, an address remapping table (the AIT) and its own wear leveling. The paper's analysis of prior measurements shows what that cost: a read-modify-write on every small store, an extra media access on most reads, 60 µs latency spikes when the controller moved data for wear leveling, and a bandwidth collapse when reads and writes mix. The paper's conclusion is that the interface was the problem, not the memory technology.

The proposal is to stop pretending LtRAM is DRAM and let the operating system manage it. The design rests on one invariant, Application Read-Only Memory (AROM): applications can read LtRAM pages but never write them, and only the kernel writes LtRAM, a whole 4 KB page at a time, when it migrates a page there. If an application stores to an LtRAM page, the store faults, the kernel copies the page back to DRAM (copy-on-write), and the store runs on the DRAM copy. Because every write is now a page-sized kernel write, the controller can shrink to three jobs (cache-line reads, 4 KB page writes, and reporting each block's health), and the OS takes over placement, migration and wear leveling with the page tables and free lists it already has.

The authors prototype the interface on the Enzian research platform, with a 256 MB NOR flash chip behind an FPGA controller standing in for LtRAM. Projecting the measured controller cost onto a DDR-class bus, three candidate technologies (3D FeFET, 3D V-RRAM, STT-MRAM) would read 26 to 79% faster than Optane per access, ranging from 3.2 times DRAM's latency to 9% faster than DRAM. Application-level benchmarks are left for future work.

Two ways to put LtRAM on the memory bus
Optane's controller hides the media · AROM hands its management to the OS
Optane: hide the media behind a DRAM interface CPU: 64 B loads and stores, as to DRAM on-DIMM controller (XPController) Write-combining buffer read-modify-write for random 64 B stores (4×) Read buffer a random 64 B read fetches 256 B (4×) Indirection table (AIT) consulted on every read; +76 ns on a cache miss Wear-leveling engine moves data on its own; reads stall up to ~60 µs 3D XPoint media: 256 B lines (XPLines) Each controller box exists to make the media look like DRAM, and each one adds latency or cost (Section 3.2 of the paper). AROM: the OS manages the media Application Kernel DRAM all app writes thin controller no AIT, no wear leveling LtRAM media 1 2 3 4 1 loads: cache-line reads, as with DRAM 2 stores: a copy-on-write fault; the kernel moves the page to DRAM and the store runs there 3 the kernel is the only writer, 4 KB pages at a time 4 the controller reports block health; the OS levels wear
Left panel from the paper's Section 3.2 and Figure 1 (the 4× amplification, the 76 ns AIT miss penalty and the ~60 µs tail are the paper's numbers, taken from prior measurements). Right panel from Sections 4.1 and 4.2.

Key takeaways

  • LtRAM candidates share one shape: near-DRAM reads, slower and coarser writes, and limited endurance. They suit read-mostly data.
  • Optane's DRAM-compatible controller (buffers, AIT, hardware wear leveling) is what made it slow and costly; the paper locates the problem in the interface.
  • AROM: applications only read LtRAM; the kernel is the only writer, in 4 KB pages, during migration. Application stores trigger a copy-on-write migration back to DRAM.
  • The controller shrinks to cache-line reads, 4 KB page writes and block-health reporting. The OS handles placement (a new memory zone, dirty-bit scans) and wear (a token allocator that paces migrations to the endurance budget).
  • Prototype: Enzian with NOR flash. For the three projected technologies, per-access reads are 26 to 79% faster than Optane and 0.91× to 3.2× DRAM's latency; application-level results are future work.
Mechanism, step by step

From a DRAM-compatible DIMM to a kernel-managed LtRAM zone

What Optane's controller cost, the AROM invariant, what the controller and the OS each do, and the prototype.

1 · The shape every LtRAM candidate shares

The paper surveys four candidate technologies against DRAM (its Table 1). The numbers vary by orders of magnitude with materials and circuit choices, so the paper asks that they be read for shape, not precision. The shape: reads close to DRAM, writes one to two orders of magnitude slower, write units larger than a cache line, and bounded endurance.

DeviceReadWriteEndurance (writes)Write granularityCost per bit
DRAM80 ns80 nsunlimitedcache linehigh
3D V-RRAM100 ns1 µs106pagelow
3D FeFET200 ns10 µs105pagelow
PCM300 ns1 µs108256 Bmid
STT-MRAM20 ns20 ns1015wordhigh

Because writes are the weak side, the paper targets long-lived, read-mostly data: code pages, machine learning inference weights, and in-memory stores such as Redis and Memcached.

2 · What Optane's controller cost

Optane DC Persistent Memory Modules sat on the memory bus next to DRAM and accepted ordinary 64-byte loads and stores. Its media, 3D XPoint (a phase-change memory), works in 256-byte units Intel calls XPLines. The on-DIMM controller bridged the two with four mechanisms, and the paper (drawing on published measurements) assigns each a cost:

MechanismWhat it doesMeasured costReplaced in AROM by
Granularity bridging (write-combining and read buffers)Merges 64 B writes into 256 B XPLines; fetches whole XPLines on readsRandom 64 B writes need a read-modify-write: 4× write amplification, ~2.3 to ~0.56 GB/s (~75% drop). Random reads also 4× amplified: ~305 to 374 ns vs ~169 ns sequentialCache-line reads and 4 KB page writes only
Hardware wear levelingMoves data off cells that receive concentrated writesA migration about every 14,000 XPLine overwrites (~3.4 MB); reads stall ~60 µs, over 160× a typical random readWear leveling in the OS; no migration path in hardware
Address Indirection Table (AIT)Maps CPU physical addresses to media locations, for wear leveling and bad-block remapping. Full table in the media; a 16 MB on-DIMM DRAM cache holds recent entriesReads take 351 ns on an AIT-cache hit and 427 ns on a miss. The cache covers ~8 GB of a 128 GB DIMM, so ~94% of random reads missNo on-module map
Workload-agnostic designAccepts any read/write mix, as DRAM doesSequential bandwidth falls from ~30 GB/s (pure reads) to ~10 GB/s at a 50/50 mixRead-mostly placement; OS-paced writes
The paper's diagnosis: the media was not the main source of these costs; the DRAM-compatible contract was. Any LtRAM placed behind a controller that hides granularity, wear, indirection and mixed traffic inherits the same overheads.

3 · The AROM invariant

Under Application Read-Only Memory, LtRAM is read-only to applications, and the kernel writes LtRAM only when it migrates a page there from DRAM. That one rule changes what every LtRAM write looks like:

  1. Every write is page-sized. The kernel writes whole 4 KB pages, so no write is ever smaller than the media's write unit and the read-modify-write path disappears.
  2. Every write comes from one place. The kernel's migration path is the only writer, so it can decide when writes happen and how many happen.
  3. Every write is to a page believed to be read-mostly. Pages are moved only after they stop being written, which keeps LtRAM traffic mostly reads and avoids the mixed-workload collapse.
  4. Application stores still work. They are enforced with copy-on-write: a store to an LtRAM page faults, and the kernel migrates the page back to DRAM before the store is applied. The application never sees the difference except as latency on that store.

4 · What the controller still does

With AROM in place, the device-side controller has three responsibilities:

  1. Reads at cache-line granularity, where LtRAM is competitive with DRAM.
  2. Writes at 4 KB page granularity. 4 KB is a multiple of every candidate's write unit in Table 1, so writes always align to the media's boundary.
  3. Report each block's health to the OS, which uses it to seed its own wear-leveling state.

The controller does not wear-level. That removes the AIT and its DRAM cache (and with them the extra ~75 ns media access on the 94% of random reads that missed), and the ~60 µs stalls from hardware-driven migrations. What remains is the media's own access latency plus a thin controller.

5 · What the OS guarantees

  1. The only writes to LtRAM are 4 KB page writes from the kernel's migration path.
  2. An application store to an LtRAM page triggers a copy-on-write fault that migrates the page back to DRAM before the store applies.
  3. Pages migrated into LtRAM are read-mostly, avoiding the mixed-workload bandwidth collapse.
  4. Write traffic to LtRAM is balanced across cells, replacing the controller's wear leveling.

Guarantees 1 and 2 enforce AROM. Guarantees 3 and 4 move migration policy and wear leveling from the controller into the kernel, which already tracks pages, page tables and free lists.

6 · The life of a page

Where a page lives, and what moves it
allocation · migration to LtRAM · copy-on-write migration back to DRAM
page allocation everything else read-only by construction (code, shared libraries, read-only mapped files) DRAM page writable; dirty bit checked every scan interval T LtRAM page write-protected: applications can only read it migrate: the kernel writes the 4 KB page dirty bit clear at a scan, and a token available application store: copy-on-write fault allocate DRAM page, copy, update page table, run store Dirty at a scan: the bit is cleared and the page is checked again after T. No token: the migration waits for the next interval. After a copy-on-write migration, the old LtRAM page is freed and its block is scheduled for erase when the device is idle. If a phase turns write-heavy, back-pressure (admission control on migration, DRAM-occupancy watermarks) limits copy-on-write storms.
From Section 5.2.1 and Figure 2 of the paper. The scan interval T and the token budget are parameters; the paper does not fix their values.
  1. Allocation. LtRAM is a new Linux memory zone with its own free-page list. A page goes straight to LtRAM only when the kernel or application can establish that the data is read-only by construction: executable code, shared libraries, read-only memory-mapped files. Everything else starts in DRAM.
  2. Finding read-mostly pages. Every scan interval T, the kernel walks the target process's page table entries. Pages whose dirty bit is still clear become migration candidates; pages that were written have their dirty bit cleared for the next scan.
  3. Migration to LtRAM. Candidates move in scan order, one token per migration (Section 7 below). If tokens run out, the rest wait for the next interval. The paper notes this delay is harmless and even helps accuracy: a page that stays read-mostly for longer is a safer bet.
  4. Migration back to DRAM. All LtRAM pages are write-protected. On a write fault the kernel allocates a DRAM page, copies the LtRAM page, updates the page table and executes the store. The old LtRAM page is freed and its block scheduled for erase when the device is idle.
  5. When a phase changes. If a large LtRAM region turns write-heavy at once, every store triggers a copy-on-write migration and DRAM's free list can run out. The paper proposes standard back-pressure: admission control on migration and DRAM-occupancy watermarks. Under write-heavy workloads fewer pages qualify for LtRAM, and the system behaves like a DRAM-only machine that swaps colder pages to storage.
Misplacement costs more than in CXL tiering. In CXL tiering a misplaced page only makes reads slower. In LtRAM a misplaced page costs a page copy on the store's critical path, which is why the classifier matters. The paper calls its dirty-bit classifier "cheap but imprecise": a page that looks read-mostly in one window may be written in the next. A small online predictor (time since last write, recent write frequency) is one option it leaves open.

7 · Wear leveling by token allocation

Instead of tracking every block's erase count, the kernel paces writes. For an LtRAM with N pages, each rated for E erases, over a deployment of T seconds, tokens are released at

r = N · E / T  tokens per second
  1. Every migration into LtRAM consumes one token; if none is available the migration is deferred.
  2. Tokens can be banked during idle periods, so a quiet workload can spend them later in a burst while the long-run average stays within budget.
  3. Because the kernel is the only writer and the allocator paces it, the device meets its lifetime budget by construction. Only the running token balance is stored, not per-block erase counts.
  4. This bounds total wear. It does not make wear uniform across the chip; the paper leaves uniform wear to future work (see Section 10).

8 · The prototype: Enzian with NOR flash

Enzian is a research platform that pairs a server-class ARM CPU (48 ARMv8 cores) with a cache-coherent FPGA. The FPGA acts as the LtRAM memory controller, and the CPU runs Linux 6.8 with the authors' page-migration extensions. The LtRAM is a 256 MB Micron serial NOR flash chip on the FPGA board.

NOR is not a canonical LtRAM technology, but it has the same shape and needs no custom chip: byte-addressable reads, 256 B writes, 4 KB erases, and 105 erase cycles per block. Measured chip-level latencies are 500 ns for a 128 B read (Enzian's cache line), 80 µs for a 256 B write and 18 ms for a 4 KB erase.

  1. One size for everything. The 4 KB write unit equals both the OS base page and the NOR erase block, so the controller never merges writes.
  2. Writes do not block reads. Incoming writes land in an SRAM buffer and are acknowledged before they reach the NOR chip, so reads to other regions are not stalled by the slow program operation.
  3. Erase is the OS's call. The erase command is exposed to the OS instead of being triggered internally, so the OS can schedule erases when read pressure is low.
  4. What the controller keeps. Only a small bad-block remap table. Its coarse entries (Optane used 64 KB per entry) let it live entirely in SRAM, unlike the AIT.
  5. What the kernel changes. A new zone at the top of the zone fallback order keeps ordinary allocations off LtRAM unless explicitly requested. The existing copy-on-write fault path is extended so LtRAM pages are copy-on-write by default. The largest change is migration to LtRAM: choosing pages and the token allocator. Applications are unchanged and never learn whether a page is in DRAM or LtRAM.

9 · Projected read latency

The paper projects the latency of one 128 B read through the new interface on a DDR-class bus, split into interconnect, controller and media. The interconnect is 40 ns for every technology: DRAM's 80 ns minus a 30 ns cell access and a 10 ns on-DIMM controller. The prototype's controller costs 180 ns (500 ns measured minus 320 ns of media access), which is 36 cycles at NOR's 200 MHz clock; at a DDR5-5600 clock (2.8 GHz) the same 36 cycles take 13 ns, the value used for the projected technologies.

Projected read latency per 128 B access
interconnect + controller + media · redrawn from the paper's Figure 4
media controller interconnect 0200400600 read latency (ns) DRAM 80 to 120 ns Optane 305 to 374 ns 40180320 540 40200 13 253 40100 13 153 40 13 + 20 73 NOR3D FeFET3D V-RRAMSTT-MRAM (prototype)(projected)(projected)(projected) 6.8× DRAM3.2× DRAM1.9× DRAM0.91× DRAM
Values from the paper's Figure 4 and Section 5.3. Ratios are against DRAM's 80 ns. The paper describes the 13 ns controller as conservative: 16 of the 36 cycles are wait states of off-the-shelf NOR that a purpose-built controller would not need.

The three projected technologies improve on Optane by 26 to 79%. The paper is careful about what this shows: these are single-access latencies. In real workloads, caches, prefetching and memory-level parallelism overlap per-access costs, so application-level performance on read-mostly data may come closer to DRAM than these numbers suggest. Application benchmarks are future work.

10 · Open questions the paper raises

  1. Wear on pages that never move. Read-only pages such as code and model weights can sit on the same LtRAM blocks for a long time, and unmovable pages (kernel code) never rotate at all, so the remaining blocks absorb more wear. Free-list rotation alone may not even out wear. One option: when the wear gap exceeds a threshold, evict pages from the least-worn blocks so those blocks rejoin the rotation. When to trigger it, the threshold, and how to pick pages cheaply are open.
  2. Huge pages. Code, libraries and databases are often mapped as 2 MB huge pages for TLB efficiency. One write-heavy object makes the whole 2 MB page ineligible; once in LtRAM, one store triggers a 2 MB copy-on-write. Splitting into 4 KB pages restores fine-grained copy-on-write but costs page-table entries and TLB reach, and transparent huge pages (khugepaged) can merge read-mostly pages with a write-heavy one again.
  3. Application hints. AROM needs no application changes, but an application could hint that a region is read-only or read-mostly (for example with madvise) so the kernel places it immediately. A richer interface could let systems such as database engines pass their own knowledge of access patterns to placement.
Common questions

FAQ

Q·What does the title mean?▾

It names the invariant: in an AROM system, a write to LtRAM is always the kernel's write. Applications only read LtRAM; the kernel writes it, in whole pages, when it migrates data there.

Q·Isn't this just memory tiering, like TPP or Memtis on CXL memory?▾

The mechanism looks similar (the OS moves pages between kinds of memory), but the target is different. Tiering systems demote cold pages to a slower tier that sits between DRAM and storage, and both reads and writes to that tier are slower. None of them reason about endurance. AROM places read-mostly pages, which may be hot, in LtRAM as main memory alongside DRAM, and the only extra cost is a copy-on-write migration when a page in LtRAM is written. The paper also warns that tiering's page-selection policies may not carry over, because they select for coldness, not for an absence of writes.

Q·Which prior system is closest?▾

Google's software-defined far memory built on zswap: the OS manages a cheaper tier (compressed RAM) and defers the cost to access time (decompression on a fault). AROM extends that idea into a hardware/software co-design, and its deferred cost is paid only on writes to LtRAM pages, not on both reads and writes.

Q·Do applications have to change?▾

No. A process sees only its virtual address space and never learns whether a page is in DRAM or LtRAM. The kernel decides placement. Hints (for example madvise marking a region read-only or read-mostly) would let the kernel place data sooner, and richer hints from systems such as databases could reduce migrations back to DRAM, but both are optional and left as open design questions.

Q·What happens when a read-mostly page suddenly starts being written?▾

The first store write-faults. The kernel allocates a DRAM page, copies the LtRAM page into it, updates the page table and runs the store; later stores go to DRAM at full speed. The danger is a phase change that turns a large LtRAM region write-heavy at once: every page faults and DRAM's free list can run out. The paper's answer is back-pressure (admission control on migration and DRAM-occupancy watermarks). In a write-heavy period fewer pages qualify for LtRAM, and the system behaves like an ordinary DRAM-only machine.

Q·How does the kernel know a page is read-mostly?▾

In the current design, with the dirty bit in each page table entry. Every scan interval the kernel checks the bits: a clear bit makes the page a candidate, and a set bit is cleared so the next scan can see whether the page is written again. The paper calls this cheap but imprecise and leaves better classifiers, such as a small online predictor, as an open question.

Q·Why fix the write size at 4 KB?▾

Three reasons from the paper. It is a multiple of every candidate technology's write unit, so writes always align to the media. It is the OS's base page size, so the kernel can manage LtRAM with its existing page machinery. And on the NOR prototype it is also the erase block size, so the controller never has to merge writes.

Q·Does the token allocator spread wear evenly?▾

No, and the paper says so. Tokens bound the total number of writes to what the device's endurance allows over its lifetime. They do not control which blocks absorb them. Long-lived read-only pages and unmovable kernel pages stay on the same blocks, so the rest of the device takes more wear. How to even it out (for example by evicting pages from the least-worn blocks past a wear gap) is open.

Q·Why prototype with NOR flash if it is not an LtRAM technology?▾

Availability and cost: a 256 MB Micron serial NOR chip drops onto the Enzian FPGA board, while MRAM or RRAM would need a research tape-out. NOR has the same shape the interface targets: byte-addressable reads, 256 B writes, 4 KB erases and 105 erase cycles per block. It is slower than the projected technologies (500 ns reads), which is why the paper projects latency for the others instead of reporting NOR's.

Q·What about 2 MB huge pages?▾

They are an open problem. A single write-heavy object makes a whole 2 MB page ineligible for LtRAM, and once a huge page is in LtRAM one store triggers a 2 MB copy-on-write. Splitting it into 4 KB pages keeps copy-on-write fine-grained but costs page-table entries and TLB reach, and transparent huge pages can merge pages back together. Prior tiering work suggests splitting pages with skewed access or co-locating hot objects, but when to split or merge is left for future work.

Q·Does the paper show application speedups?▾

No. It reports measured chip-level latencies on the prototype and projects per-access read latency for other technologies. It notes that caches, prefetching and memory-level parallelism hide part of per-access latency, so applications on read-mostly data may come closer to DRAM than the per-access numbers suggest. Application-level evaluation is future work.

Q·Who handles bad blocks?▾

The controller keeps a small bad-block remap table. Its entries are coarse (Optane used 64 KB per entry), so the whole table fits in on-controller SRAM, unlike Optane's AIT, which needed a DRAM cache and media storage.

Background terms
LtRAM (Long-term RAM)A class of memory technologies meant as a denser, cheaper complement to DRAM: reads close to DRAM latency, but slower writes, writes coarser than a cache line, and limited endurance.
AROM (Application Read-Only Memory)The paper's invariant: LtRAM pages are read-only to applications and written only by the kernel, during page migration.
Cache lineThe unit in which a CPU reads and writes memory, 64 bytes on most x86 servers and 128 bytes on Enzian's ARM CPU.
Write granularityThe smallest unit a memory can write in one operation. Writing less than that unit requires reading the rest, merging, and writing it all back.
EnduranceThe number of write (or erase) cycles a memory cell survives before it wears out.
Wear levelingSpreading writes across a device's cells so that no region wears out early.
Intel Optane DCPMMA discontinued non-volatile DIMM built on 3D XPoint, a phase-change memory, that sat on the memory bus next to DRAM and accepted ordinary loads and stores.
XPLineOptane's 256-byte media access unit, four times a 64-byte cache line.
AIT (Address Indirection Table)Optane's on-DIMM table mapping CPU physical addresses to media locations, used for wear leveling and bad-block remapping. Stored in the media, with recent entries cached in 16 MB of on-DIMM DRAM.
Page table entry and dirty bitA page table entry is the record that maps one virtual page to a physical page. Its dirty bit is set by the hardware when the page is written, and the OS can clear it to watch for new writes.
Copy-on-write (CoW)A technique where a page is mapped read-only and a write to it faults; the kernel then copies the page and lets the write proceed on the copy. AROM uses it to move written pages from LtRAM to DRAM.
Memory zoneA region of physical memory that Linux manages with its own free-page lists. The design adds one for LtRAM so that ordinary allocations do not land there.
Memory tieringOS techniques that move cold pages from DRAM to a slower, cheaper memory such as CXL-attached memory, and back when they become hot.
NOR flashA flash memory that is byte-addressable for reads, programmed in small pages (256 B here) and erased in larger blocks (4 KB here).
EnzianA research computer from ETH Zurich that pairs a server-class ARM CPU with an FPGA on a cache-coherent interconnect, so the FPGA can act as a memory controller.
Huge pageA 2 MB page (instead of 4 KB) that lets one TLB entry cover more memory. Transparent huge pages are Linux's automatic merging of small pages into huge ones.
Summary of H. S. Shim, K. Mohr and P. Levis, "The Kernel's Write: Application Read-Only Memory," DIMES '26. Numbers are from the paper unless marked as this summary's arithmetic.