Memory Fit: Architectural Support for Page Placement by Learning Models of Good Behavior
This page was written by an AI model to give a high-level overview of the paper. Please check the paper for detail: PDF.
Server memory is no longer just DRAM. CXL memory expanders, phase-change memory (PCM), and software-compressed memory like zswap all let an operating system add cheaper capacity below fast DRAM — as long as it places pages well. A page placed on the wrong memory wastes the benefit of having a second tier at all.
To decide where a page belongs, an OS needs to know how that page's recent accesses would perform on each candidate memory. Today it has three unsatisfying options. Page-table access and dirty bits are cheap but coarse: they say a page was touched, not how it was touched. Out-of-band profiling with Intel PEBS or AMD IBS gives richer per-access events, but at a cost — HeMem's page-table scanning alone costs 18% of GUPS throughput, and aggressive PEBS sampling drops up to 30% of samples once the core falls behind. Runtime instrumentation such as NEMO is more precise still — it finds twice as many skewed "huge" pages and speeds migration decisions up to 10.4× — but sampling just 10% of cache misses in software can slow an application by more than 50%. Hardware counter tables sidestep the software cost but need real SRAM: Memtis's 1.4 MiB table takes about 37 seconds to sweep 16 GiB at 4 KiB granularity, and a full per-page counter table for 1 TiB of memory would need roughly 90 MiB of SRAM — more than the per-die caches on a modern multi-chiplet processor.
This paper's answer is memory fit: a hardware-computed, per-page, per-memory score that says how well a page's observed access pattern suits each memory in the system, independent of how busy that memory is. The mechanism that computes it, soft trace matching (STM), borrows decades of work on fast hardware data compressors. Instead of compressing the access stream, STM scores it: it compares a page's recent sequence of reads, writes, and reuse distances against a small model, learned offline, of what a "good" (fast) access sequence looks like on each memory. A lower score means the page's behavior fits that memory well. Because the OS never sees raw traces or counters — only an already-digested, comparable score per page per memory — the paper can keep the hardware's job (measuring fit) strictly separate from the OS's job (deciding what to do about it).
| System | Placement signal | Expressive | Simple | Efficient |
|---|---|---|---|---|
| TPP | Recency (hint faults) | ✓ | ||
| HeMem | Frequency (PEBS) | |||
| Memtis | Frequency (PEBS) | ✓ | ||
| M5 | Frequency (counter) | ✓ | ✓ | |
| NEMO | Frequency (counter) | ✓ | ✓ | |
| Memory Fit | Soft trace matching | ✓ | ✓ | ✓ |
Reproduced from the paper's Table 1. Expressive: distinguishes different memories' performance tradeoffs, not just "hot or cold." Simple: exposes a per-page value the OS policy can use without knowing the underlying memory details. Efficient: bounded on-chip work and state, no large SRAM tables, no added request latency.
Key takeaways
- Memory fit is a hardware-computed, per-page score for how well a page's access pattern suits each memory. Scores are comparable across memories (which wins for a page) and across pages (which page benefits most from moving), so the OS can prioritize scarce fast-memory capacity for the pages that gain the most.
- Soft trace matching (STM) computes the score from reads, writes, and reuse distance alone, using small tables learned offline — no per-access neural network, no online training, no PEBS or IBS sampling.
- On an extended GUPS microbenchmark, memory fit identifies the right pages to move with 100% accuracy (TPP and random placement both manage only 50%) and recovers 72.4% and 71.9% of the optimal-to-worst placement gap in static and dynamic phases, versus 19.2% and 33.3% for TPP.
- On full application runs, memory fit recovers 37.1–90.4% of Redis's placement gap and 29.9–77.7% of llama.cpp's across DRAM capacities, translating into measured throughput gains of up to 7.6% (Redis) and 13.3% (llama.cpp) over TPP.
- The whole mechanism — STM hardware plus its request monitor — costs 10,414 LUTs and 7,572 flip-flops, about 2.1–3.0% of one modern server core's logic budget, and adds only 1.2% execution overhead when active.
Measuring fit in hardware, acting on it in software
Soft trace matching, the hardware pipeline that computes it, the record format that carries it to Linux, the placement policy that consumes it, and the full set of evaluation results.
1 · What a memory fit score means
For one page, memory fit gives a score per candidate memory. A lower score is better: it means the page's recent access sequence resembles the kind of sequence that memory serves quickly. Two properties make the scores useful to an OS policy. First, for a single page, scores are comparable across memories: the difference between a page's DRAM score and its PCM score indicates how much faster (or slower) that page would run in each. Second, scores are comparable across pages: a bigger score reduction from moving page A to fast memory than from moving page B means A benefits more, so a capacity-constrained policy should move A first. The paper's implementation trains scores to track relative completion time, but the authors note the same mechanism could target other objectives, such as energy or a weighted mix of metrics (Appendix A.1, Section 7).
2 · Soft trace matching
STM turns an access stream into a score using small tables learned offline, once, from synthetic training traces — not from the workload being run and not updated online. Each observation is reduced to two things: whether it was a read or a write, and its normalized reuse distance — the number of other page accesses since this page was last touched, divided by an estimate of the current working-set size (so the same learned anchors apply whether the working set is kilobytes or gigabytes). The root table scores an observation using only this pair; a second, "conditional" table refines the score using the preceding observation's coarse class (read/write, short/long reuse) — useful because, for instance, a PCM write's cost depends on whether the cell was read first. The paper found two table levels (root plus one level of history) sufficient in practice, though the technique generalizes to more. STM's tabulation scheme is adapted from Context Tree Weighting, a technique from lossless data compression, repurposed here as a placement-fitness estimator rather than a compressor.
Each memory has its own set of learned tables. Hardware evaluates the same observation against every memory's tables in parallel and interpolates between the nearest "reuse anchors" the tables were trained on, producing one updated eight-bit score per memory. A page whose accesses look like the memory's learned "good" pattern gets a small update (its running score barely moves); a page that looks unlike that pattern gets pushed further. Because the same training procedure makes the faster memory, for a given access pattern, converge to a lower average score, the gap between a page's two scores doubles as a confidence-weighted estimate of how much placement actually matters for that page.
3 · The hardware pipeline
Three design choices keep this off the critical path. Non-blocking: the monitor copies request headers into a small ingress queue and drops copies when it's full, applying no back-pressure to the memory access itself — under heavy load, STM simply observes less, rather than slowing anything down. Bounded metadata traffic: the on-chip delta cache has a fixed number of entries and merges repeated updates to the same page into one read-modify-write, so STM's traffic to coherent memory doesn't scale with every individual access. Eventual consistency: a page's PFN record is written completely before its frame number is published to the notification ring, and an "outstanding notification" bit prevents a page from occupying more than one ring entry; the OS always re-reads and revalidates the current record before acting, so a stale or coalesced notification can only delay a useful migration, never cause an incorrect one.
4 · The per-page record Linux reads
| Field (64-bit record) | Purpose |
|---|---|
| Fit scores (two 8-bit fields used; room reserved for two more) | One decaying score per tracked memory, maintained by hardware on every observed access |
| Timestamp | Lets hardware resume the decaying-average computation correctly after a delta-cache entry is written back and later reinstalled |
| Count / context | Tracks enough history to resume STM's conditional (history-aware) scoring across delta-cache evictions |
| Notification + stability state | Marks an outstanding, unread notification and filters out low-confidence score reversals before they reach software (see §7) |
| Optional MLP class | A coarse, four-level memory-level-parallelism tag Linux can use to break ties between pages with similar fit scores; does not affect the fit scores themselves |
At eight bytes per tracked 4 KiB page, the table occupies 0.195% of the memory it covers — no change to a page table entry or the kernel's struct page is required.
5 · Turning scores into migrations
Linux's side of the interface is intentionally simple. A monitoring kernel thread drains the notification ring and, for each page, computes the benefit of moving it to another memory. It keeps one bounded max-heap per memory, keyed by that benefit; a later notification updates a page's existing heap entry instead of duplicating it, and once a heap is full a new page is admitted only if it beats the weakest current candidate. A separate migration thread wakes when a heap changes: if the destination memory has spare capacity it migrates the best candidate outright; if the destination is full, it pairs the best incoming page with a resident page from the opposite heap and accepts the swap only if it reduces their combined score. Selected pages are revalidated against the live PFN table, grouped into batches, and handed to Linux's existing page-migration code path.
This design gives the software path three properties: it is scanless (only pages Linux was notified about are ever examined, never a sweep of memory), incremental (each notification costs O(log k) work in a bounded k-entry heap, and each migration cycle processes a bounded batch), and convergent (for a fixed set of scores, every accepted move or swap strictly improves the placement, so the process reaches a local optimum where no candidate in the heaps can be improved further). The paper formalizes this as a steady-state bipartite-matching problem and shows the greedy strategy achieves at least half of the optimal matching's total benefit — the classical 1/2-approximation guarantee for greedy weighted matching.
6 · Prototype and evaluation methodology
| Part | Setup |
|---|---|
| Prototype | FireSim cycle-accurate FPGA simulation of a Small BOOMv3 (single-issue, out-of-order RISC-V) core running Linux 6.6 on an AWS F2 instance; 16 KiB 4-way L1s, 4 MiB 8-way LLC |
| Memories | Two 8 GiB NUMA-visible memories (16 GiB total); FireSim's memory-timing model gives each memory distinct bandwidth/latency so the single BOOM core can exercise tradeoffs it couldn't by itself saturate real HBM (PCM write bandwidth modeled at 10% of DRAM's, per prior measurements) |
| Baseline compared | Linux Transparent Page Placement (TPP). Memtis and HeMem were not run — both depend on Intel PEBS, unsupported on this RISC-V target |
| Metric | Cycle-gap recovery: the percentage of the gap between worst-case and optimal (best-case) placement that a policy recovers for a fixed workload, capacity, and migration budget. 100% = optimal; 0% = worst placement; a naive random placement is also reported as a sanity check |
| Overhead accounting | All reported end-to-end numbers include the full cost of online monitoring, scoring, and migration, run on the same single core as the workload (the paper expects this cost to largely disappear on a multicore chip with a dedicated notification-processing core, as in prior systems) |
7 · Results walkthrough
Placement accuracy and reaction to phase changes (§5.2–5.3). An extended GUPS microbenchmark splits memory into a read-only, a write-only, and an untouched region; the optimal placement keeps the write-only pages in DRAM. Memory fit identifies the write-heavy pages with 100% accuracy; TPP and random placement both land at only 50%. When which region is write-heavy flips halfway through the run (the "dynamic" case), memory fit still recovers 71.9% of the cycle gap, essentially unchanged from its 72.4% in the unchanging ("static") case — it reacts to the phase change almost immediately, where TPP takes much longer to catch up. Enabling STM's observation and scoring costs 1.2% in execution cycles when placement is held fixed.
End-to-end applications (§5.4). Redis (a mix of batched, memory-level-parallel key lookups and pointer-chasing traversal with per-node updates) and llama.cpp (an 8-context TinyStories-15M inference workload, with its compute disabled so the single-core prototype doesn't become compute-bound) were each run across a sweep of DRAM capacities:
| Workload | DRAM capacity | TPP recovery | Fit recovery | Fit throughput gain |
|---|---|---|---|---|
| Redis | 25% | 22.4% | 37.1% | 1.8% |
| 50% | 7.9% | 56.0% | 6.0% | |
| 75% | 31.8% | 90.4% | 7.6% | |
| llama.cpp | 15% | −35.8% | 29.9% | 9.0% |
| 25% | −25.1% | 67.3% | 13.3% | |
| 50% | 26.4% | 77.7% | 7.5% |
TPP's negative recovery on llama.cpp at low DRAM capacities means it leaves the workload worse off than doing nothing: its migration and PEBS-like access-sampling overhead, measured on the same single core as the workload, outweighs any benefit from the pages it does move correctly. Replaying Redis's 50%-capacity placement decisions statically, with no online controller or migration cost, recovers 80.0% of the cycle gap for memory fit versus 18.7% for TPP — evidence that memory fit's advantage is mostly in which pages it picks, not just in reacting faster.
Retargeting to a different memory (§5.5). Re-training STM's tables (not redesigning any hardware) for a higher-latency, CXL-like memory recovers 86.5% (static) and 80.2% (dynamic) of the cycle gap, against 11.2% and 22.1% for TPP — evidence the same mechanism generalizes across memory technologies rather than being tuned specifically to a DRAM/PCM pair.
Composing with other placement techniques (§5.7). Combined with Colloid's bandwidth-aware placement and ALTO's per-page memory-level-parallelism ranking, on a three-region extended-GUPS variant, memory fit recovers 68.7% (static) and 69.3% (dynamic) of the cycle gap versus 14.8% and 27.1% for TPP, improving measured workload throughput over TPP by 13.2% and 10.3% respectively — memory fit's score is additional ranking information, not a replacement for bandwidth- or parallelism-aware policy.
Hardware cost (§5.6). STM and its request monitor together use 10,414 LUTs, 7,572 flip-flops, and 13 DSP blocks (no block RAM), adding 15.1% LUTs and 15.9% flip-flops to the Small BOOM tile. Scaled against a reference budget of 350,000–500,000 LUT-equivalents for a large server core, that's 2.1–3.0% of one core. Because STM is shared, uncore logic that serves every core's observed memory traffic, one instance amortizes across a whole chip: shared across 96 cores (an EPYC-9004-class reference), it is 0.022–0.031% of the combined budget.
Overhead-reduction filters (§5.8). Two hardware filters cut the OS-visible traffic without changing placement quality: coalescing repeated updates to a resident page cuts ring publications and interrupts by 46.0%, and a randomized low-pass filter that suppresses low-confidence score reversals (see the stability mechanism in Appendix B.2) cuts decision notifications by 83.9%. Together, on llama.cpp, they cut the mechanism's late-run cycle overhead from 3.7% down to 0.26%, without a measurable loss in placement quality.
8 · Scope and limits the paper states
- Single-core prototype. All reported overheads run monitoring and migration on the same core as the workload. The paper expects this cost to largely vanish on a multicore system by dedicating a separate core to interrupt handling and notification processing, as prior systems running on commodity hardware already do, but does not measure that configuration directly.
- Training objective is fixed to completion time. STM's tables in this paper are trained so a lower score means faster completion time. The authors are explicit that the underlying method is more general — it could target energy, a weighted composite of several metrics, or other objectives — but only the completion-time objective is evaluated.
- Offline, fixed tables. STM's learned tables are trained once on synthetic traces and compiled into the hardware image; unlike a dynamic branch predictor, they are not adapted online while the system runs. Whether other, possibly adaptive, scoring algorithms could outperform STM is left as future work.
- Scaling beyond this prototype. The paper notes that much of memory fit's implementation complexity comes from guaranteeing coherent, precise estimates to the OS, and that relaxing some of those guarantees might reduce overhead further; scaling to multi-chiplet, high-core-count, or multi-socket systems may require exactly such relaxations, which the paper does not evaluate.
- Evaluated on two physical memories at a time. Each experiment uses two memories (DRAM/PCM, or DRAM/CXL-like); the PFN record format reserves room for up to four memory-fit scores, but only two are exercised in this paper's evaluation.
FAQ
Q·How is this different from what Linux's Transparent Page Placement (TPP) already does?▾
TPP infers how "hot" a page is mainly from recency, using access-bit hint faults, and treats placement largely as hot-versus-cold. Memory fit instead scores a page's actual read/write and reuse-distance pattern against a learned model of what performs well on each specific memory, so it can tell apart two equally "hot" pages that would behave very differently on, say, DRAM versus PCM. In the paper's extended-GUPS test, this is the difference between 100% and 50% placement accuracy.
Q·Does memory fit replace access/dirty bits or hardware counters like Intel PEBS?▾
It's a different kind of signal, not an extension of either. Access/dirty bits and PEBS-style counters report what happened (how often, whether dirty); memory fit directly reports a comparison — how well the access pattern matches each memory's profile — computed continuously in dedicated hardware rather than sampled. The paper didn't run Memtis or HeMem in its own evaluation specifically because they depend on Intel PEBS, which the RISC-V prototype doesn't support.
Q·What new hardware does a system need to support this?▾
A monitor at the last-level-cache output that copies (not intercepts) post-cache request headers, the STM scoring pipeline with its small learned tables, a delta cache, and a coherent PFN-indexed state table plus notification ring that the OS reads over ordinary memory accesses. No page-table entry or kernel struct page changes are needed. The prototype's full implementation uses 10,414 LUTs and 7,572 flip-flops — about 2–3% of one modern server core's logic budget, shared across all cores on a chip.
Q·How much does it slow the system down?▾
With placement held fixed, enabling STM's observation and scoring adds 1.2% to execution cycles in the paper's measurement. Two hardware filters — coalescing repeated updates and a low-pass filter on low-confidence score changes — cut OS-visible interrupts by 46.0% and notifications by 83.9%, bringing llama.cpp's late-run overhead down to 0.26% without hurting placement quality.
Q·Why does TPP sometimes make performance worse than doing nothing?▾
On llama.cpp at 15% and 25% DRAM capacity, TPP's cycle-gap recovery is negative (−35.8% and −25.1%). The paper attributes this to TPP's migration and access-sampling overhead, measured on the same single core as the workload in this prototype: it moves pages more often and less accurately than memory fit, and that overhead outweighs whatever placement benefit it does achieve.
Q·Why does TPP's recovery go up (33.3% vs 19.2%) in the dynamic GUPS case, when memory fit's stays flat?▾
This is a quirk of TPP's low accuracy, not an improvement in its judgment: TPP identifies the correct write-heavy region only about half the time regardless of phase. When the workload's write-heavy region flips midway, TPP's existing (largely arbitrary) placement happens to benefit from the change about as often as it's hurt by it, which raises its average recovery number. Memory fit's accuracy is 100% in both phases, so its recovery is essentially unaffected by the flip.
Q·Is this evaluated on real silicon?▾
It's implemented as a complete FireSim target — a cycle-accurate FPGA simulation of a Small BOOMv3 RISC-V core booting Linux 6.6 on an AWS F2 instance, not a software model. The target runs at 18 MHz on the FPGA host, but FireSim's timing model reports performance as if the design ran at its intended 1 GHz core / 500 MHz STM clocks, which is how the paper's cycle counts and overhead percentages should be read.
Q·Does it only work for DRAM and PCM?▾
No — the paper re-trains STM's tables (without changing any hardware) for a higher-latency, CXL-like memory and finds memory fit recovers 86.5% and 80.2% of the cycle gap in static and dynamic settings, versus 11.2% and 22.1% for TPP. The authors present this as evidence the interface generalizes across memory technologies rather than being specific to a DRAM/PCM pair.
Q·Can memory fit be combined with bandwidth-aware or parallelism-aware placement policies?▾
Yes. Combined with Colloid (bandwidth-aware placement) and ALTO's per-page memory-level-parallelism ranking, memory fit recovers 68.7% and 69.3% of the cycle gap (static/dynamic) versus 14.8% and 27.1% for TPP, with measured throughput gains over TPP of 13.2% and 10.3%. Memory fit supplies an additional ranking signal; it doesn't replace bandwidth- or parallelism-aware decisions.
Q·Does using memory fit require changes to page tables or kernel data structures?▾
No. Memory fit state lives in a separate, coherent, PFN-indexed table (eight bytes per tracked 4 KiB page, 0.195% overhead on the memory it covers) and a notification ring that Linux polls through its existing page-management and migration code paths. No page-table entry or struct page field is added or changed.
Q·Where can I read the full paper, and has it appeared at a conference yet?▾
The paper is posted as an arXiv preprint (arXiv:2611.07506, submitted November 5, 2026) and is not yet tied to a specific conference or journal venue. See the PDF for the full text, including the formal model and approximation-bound proofs in Appendix A.