Paper summary

GainSight: A Unified Framework for Data Lifetime Profiling and Heterogeneous Memory Composition

Peijing Li, Matthew Hung, Yiming Tan, Konstantin Hoßfeld, Jake Cheng Jiajun, Shuhan Liu, Lixian Yan, Xinxin Wang, Philip Levis, H.-S. Philip Wong, Thierry Tambe · Stanford University

arXiv preprint, arXiv:2504.14866 · version 5, August 5, 2025

This page was written by an AI model to give a high-level overview of the paper. Please check the paper for detail: arXiv.

The big idea

AI accelerators want more on-chip memory, and on-chip memory today means SRAM. SRAM has stopped shrinking: the paper cites a cell size that has plateaued at 0.021 µm² across the 5 nm to 3 nm process nodes, while language model parameter counts grow almost 400× every two years and accelerator memory capacity only doubles. At the same time, much of the data an accelerator keeps on chip is short-lived. In LLM inference, attention score matrices exist only until the softmax consumes them, decoding generates candidate tokens and keeps one, and KV-cache data is staged into SRAM just before the tensor cores use it; the paper estimates that many such structures persist for mere microseconds. An SRAM cell holds its value for as long as it is powered, so for this data it pays, in area and leakage, for retention nobody uses.

Short-term RAM (StRAM) trades retention for density. Gain-cell RAM (GCRAM) and embedded DRAM (eDRAM) store each bit as charge on a small node that slowly leaks away, so a cell holds its value only for a retention time, from about a microsecond to milliseconds depending on the technology. They are 2 to 4× denser than SRAM, with lower leakage. Normally a leaky memory needs refresh (reading and rewriting each cell before it decays), which costs energy. The key observation is that if data dies before the charge does, no refresh is needed. Exploiting this requires knowing, access by access, how long each piece of data actually lives. Existing profilers report kernel-level or aggregated cache statistics, and earlier accelerators with eDRAM or gain cells estimated lifetimes from static dataflow heuristics.

GainSight is a profiling and design-exploration framework that makes data lifetime (the time from a value's first write to its last read before it is overwritten or invalidated) a first-class design metric. Retargetable backends run workloads on modified cycle-accurate simulators (Accel-Sim for GPU L1 and L2 caches, SCALE-Sim for systolic-array scratchpads) and emit a common trace of every on-chip memory access. An architecture-agnostic frontend turns the trace into lifetimes, refresh counts, live capacity and active energy for each cell type in a library of 5 nm models (SRAM, silicon GCRAM, hybrid oxide-silicon GCRAM, oxide eDRAM). It then builds a heterogeneous composition by assigning each access to the cell type with the shortest retention that still exceeds the data's lifetime, and reports the capacity, area and energy of the resulting mix.

On MLPerf Inference and PolyBench workloads, 64.3% of GPU L1 accesses and 18.4% of L2 accesses have lifetimes within Si-GCRAM's 1 µs retention, and 97.9% and 52.0% fit Hybrid-GCRAM's 10 µs. On a 256×256 systolic array running ResNet-50, at least 79.01% of scratchpad accesses live under 1 µs. The generated compositions reduce active energy by an average of 1.89× for GPU L1 caches and 1.3× for L2 caches, with about 3× smaller memory arrays (up to 3.1× energy and 4.3× area across workloads), and by about 3× in energy and 2.4× in area for the systolic-array scratchpads. All results are simulated, against uniform SRAM, from device models rather than fabricated arrays. GainSight is open source.

How long the data lives versus how long each cell remembers
cell retention on a log axis · share of accesses that fit each retention
Retention time of each cell type (5 nm models used in the case studies) Si-GCRAM 1 µs Hybrid-GCRAM 10 µs eDRAM ms-scale SRAM while powered 0.1 µs1 µs10 µs100 µs1 ms10 ms indefinite Si-GCRAM, then Hybrid-GCRAM, then eDRAM: longer retention, higher active energy per access Share of on-chip accesses whose data lifetime fits within that retention lifetime ≤ 1 µs (fits Si-GCRAM) lifetime ≤ 10 µs (fits Hybrid-GCRAM) GPU L1 cache all GPU workloads 64.3% 97.9% GPU L2 cache all GPU workloads 18.4% 52.0% Systolic scratchpads ResNet-50, 256×256 array ≥ 79.01% 0%50%100%
Retention values from Sections 2.1 and 6.1.2 of the paper (eDRAM is given only as ms-scale; the axis is schematic). GPU shares are over all L1 and L2 accesses of the workloads in Table 5 (Section 6.1.2, Figure 5); the systolic-array share covers the input, weight and output buffers across all three dataflows (Section 6.2.2, Figure 8). Hybrid-GCRAM retention also falls at high write frequency (Figure 3).

Key takeaways

  • SRAM cell area has stopped scaling, and much accelerator data lives only microseconds, so SRAM's indefinite retention is overprovisioned for it.
  • Gain-cell RAM and eDRAM are 2 to 4× denser than SRAM but lose data after a retention time. Data that dies sooner never needs a refresh.
  • GainSight measures per-access data lifetimes with retargetable simulator backends (GPU caches, systolic-array scratchpads) and correlates them with 5 nm cell models in an architecture-agnostic frontend.
  • Composition rule: each access goes to the cell type with the shortest retention that still exceeds its lifetime; the frontend sizes each type and projects area and active energy.
  • Simulated against uniform SRAM: GPU L1 caches average 1.89× lower active energy and L2 caches 1.3×, with about 3× smaller arrays (up to 3.1× and 4.3×); systolic scratchpads for ResNet-50 save about 3× energy and 2.4× area.
Mechanism, step by step

From memory access traces to a heterogeneous on-chip memory

The cells, the lifetime metric, GainSight's backends and frontend, and what the two case studies found.

1 · Why uniform SRAM is a poor fit

Accelerators keep weights, activations and intermediate results in on-chip memory to avoid slow, energy-hungry off-chip accesses. Two trends squeeze that memory. Capacity demand grows fast: the paper cites language model parameter counts growing almost 400× every two years while accelerator memory capacity only doubles. Supply stalls: SRAM cell area has plateaued at 0.021 µm² from 5 nm to 3 nm, and even the 2 nm gate-all-around process struggles to restore the earlier trend. On a reticle-limited die, every extra square millimeter of SRAM is taken from compute.

The second half of the argument is about what the data needs. Data lifetime is highly uneven within and across workloads, and a large share of AI data is transient. A memory whose cells hold data forever treats every byte as if it might be needed indefinitely. The paper's phrase for this is that SRAM is "overprovisioned in retention capability."

2 · The memory cells: SRAM, gain cells and eDRAM

For a systems reader, the difference is how each cell stores a bit (general knowledge, consistent with the paper's Section 2.1). An SRAM cell is a pair of cross-coupled inverters that actively hold each other's value, so it never decays while powered but needs six transistors. A gain cell stores the bit as charge on a tiny storage node (the gate of a read transistor); a separate write transistor puts the charge there, and the read transistor amplifies it into a bitline current, so a read does not destroy the value. An eDRAM cell stores the charge on a capacitor behind one access transistor, like commodity DRAM but on the logic die. In both, the stored charge leaks, and the cell's retention time is how long it stays readable. Data kept longer than that must be refreshed (read and rewritten), which costs energy and access slots.

CellStructureRetentionWhat the paper says about it
6T SRAM6 transistors, cross-coupled invertersindefinite while poweredFast reads and writes; area scaling has stalled; high static power
Si-GCRAM (silicon gain cell)2 to 3 silicon transistors (the library uses a 3T cell)µs-scale; 1 µs in the 5 nm case studies2 to 4× denser and lower leakage than SRAM, SRAM-comparable latency, standard FinFET process
Hybrid-GCRAM2T: an oxide (indium tin oxide, ITO) write transistor stacked above a silicon read transistorlonger than Si-GCRAM; 10 µs in the case studiesDenser through 3D stacking in the back-end-of-line layers; retention falls as write frequency rises
Oxide eDRAM1 transistor and 1 capacitor (1T1C), built in the back-end-of-linems-scaleHigh density; higher active energy than GCRAM because reads share charge with the bitline

Moving down the StRAM rows, retention gets longer and active energy per access gets higher. That ordering is what the composition step later exploits.

Short-term versus long-term RAM. The paper's Table 1 also lists long-term RAM (LtRAM: MRAM, RRAM, FeRAM), which is dense and keeps data for a long time but has expensive writes and limited endurance. It suits rarely written data. GainSight's case studies leave LtRAM out because the target workloads are write-intensive and transient (see Section 10).

3 · Data lifetime, and when refresh disappears

The paper's Definition 2.1: the lifetime of a data structure at a given memory address is the time between its first write and last read before being overwritten or invalidated. In a scratchpad, that runs from a store to the final read before the next overwrite. In a cache, it runs from a store or a miss fill to the last hit before the line is replaced or written again.

The rule that follows is simple. If a value's lifetime is shorter than the cell's retention, the cell never needs a refresh for it. If the lifetime is longer, the value needs either refresh or a cell with longer retention. The paper's Figure 3 shows this for Llama-3.2-11B operations on a GPU backend: GEMM data lives under Si-GCRAM's retention, tensor-transpose data fits Hybrid-GCRAM, and normalization data outlives both and needs eDRAM or SRAM. The same figure shows Hybrid-GCRAM's retention dropping at high write frequencies, which is why the device models are frequency-dependent.

Why profiling matters: earlier accelerators with eDRAM or gain cells (DaDianNao, RANA, CAMEL) estimated lifetimes from static heuristics about predictable dataflows. Those estimates must be redone for each new model family. Existing profilers (perf mem, Cachegrind, Nsight Compute, rocProfiler, GMProf and others in the paper's Table 2) report counters or cache statistics but do not correlate accesses with memory-cell properties or generate a memory composition.

4 · The framework at a glance

GainSight's pipeline
retargetable backend · common trace · device library · architecture-agnostic frontend
Retargetable backend 1 · Workload MLPerf Inference, PolyBench 2 · Binary and sampling NVBit SASS trace; PKA (GPU) 3 · Cycle-accurate simulator GPU: Accel-Sim / GPGPU-Sim systolic array: SCALE-Sim Logged memories GPU: L1 and L2 caches array: 3 scratchpad buffers 5 · Memory access trace time · read/write · address one stream per subpartition same format from every backend 6 · Device library (N5) SRAM · Si-GCRAM Hybrid-GCRAM · eDRAM retention vs write frequency, bit-cell area, energy per bit Analytical frontend 7 to 9 · Per-device analysis lifetimes and refresh counts live capacity active energy (R, W, refresh) 10 · Composition each access goes to the cell with the shortest retention above its lifetime; then size each type's capacity and area 11 to 13 · Outputs lifetime histograms, JSON report, composition summary Step numbers follow the paper's Figure 4. Any backend that emits this trace format can feed the frontend.
Redrawn and simplified from the paper's Figure 4 (Sections 3 to 5). Box contents are condensed from the text; step 4 in the paper is the simulator itself. The device library's models are used by both the per-device analysis and the composition step.
  1. Capture fine-grained on-chip memory access traces for arbitrary workloads on a retargetable hardware backend.
  2. Extract lifetime statistics, read and write frequencies, and capacity use from the trace.
  3. Correlate lifetimes with each cell type's retention and other properties to project array area and active energy.
  4. Compose a heterogeneous StRAM mix that minimizes energy through refresh-free operation.
  5. Visualize the distributions so designers can explore them across workloads and kernels.

5 · The backends: GPU and systolic array

Because lifetimes depend on dataflow and memory organization, the same workload can look very different on two architectures, so GainSight uses one backend per architecture and simulation instead of hardware counters.

  1. GPU backend. CUDA workloads are traced with an NVBit-based tool to capture each kernel's SASS assembly. Representative kernels are then chosen offline with Principal Kernel Analysis (PKA), and simulated in GPGPU-Sim 4.2.1 through a modified Accel-Sim 1.3.0. The modification inserts logging at the interfaces between the streaming multiprocessors (SMs) and the L1 and L2 caches, recording address, access type, time and hit or miss for every cache operation.
  2. Why sampling. Accel-Sim runs 6 to 7 orders of magnitude slower than native GPU hardware. Deep learning repeats similar layers and identical operations across batches, so a few representative kernels stand in for many.
  3. Systolic-array backend. An extended SCALE-Sim models the three scratchpad buffers of a systolic array (input feature map or ifmap, weights or filter, output feature map or ofmap), with configurable buffer sizes, array dimensions and dataflow (input, weight or output stationary).
  4. Bring your own backend. Any simulator or physical accelerator that emits timestamps, address ranges and read/write types in the trace format can plug in. The frontend treats each distinct memory (a cache level, a scratchpad buffer) as an independent subpartition.

The paper's Table 4 measures what PKA sampling costs in accuracy on three workloads:

WorkloadKernels simulatedSpeedupLifetime error (MAE)Write-frequency error (MAE)
bert-base-uncased8.37%10.89×0.36 µs11.59 MHz
llama-3-8b0.15%412.76×3.76 µs21.39 MHz
resnet-5010.31%60.97×3.73 µs10.09 MHz

MAE is mean absolute error against the unsampled run. The paper describes the errors in these memory metrics as bounded and sufficient for memory-system analysis.

6 · The frontend: device library, analysis, composition

Device library. All four cells are characterized for TSMC's N5 (5 nm) node. SRAM numbers come from TSMC publications; Si-GCRAM from fabricated arrays (Giterman et al., Pentecost et al.) scaled to 5 nm; Hybrid-GCRAM from post-layout simulations of 256×256 arrays (Liu et al.); oxide eDRAM from Chiang et al., with DRAM operations such as precharge folded into per-bit energy. Each model gives retention time (write-frequency dependent for GCRAM), bit-cell area, and read and write energy per bit. Adding a technology means adding a model.

  1. Statistics. From the trace: read and write frequencies, unique addresses and operation counts.
  2. Lifetimes and refreshes. For each write, the time until the data is overwritten or invalidated. The lifetime distribution is compared with each cell's write-frequency-dependent retention; any data object that outlives retention contributes bit-level refresh operations.
  3. Capacity. The number of unique addresses live at the same time, scaled by block size. This can be well below the physical capacity.
  4. Active energy. Reads, writes and refreshes weighted by per-bit device costs, with each refresh modeled as an extra read followed by a write.
  5. Composition. Each access goes to the cell type with the shortest retention that still exceeds its lifetime, preferring Si-GCRAM, then Hybrid-GCRAM, then eDRAM. The resulting share of accesses per type, combined with the capacity analysis, gives each type's capacity and area and the mix's total area and energy.
  6. Outputs. Interactive lifetime histograms (built with Tableau), a JSON report of refresh counts, energy and area for each single-device design and for the composition, and a composition summary.
Scope of the projections: the paper presents these as first-order design guidance for memories that do not yet exist as fabricated prototypes. Circuit-level effects that would need SPICE simulation are abstracted away.

7 · Evaluation setup

GPU case studySystolic-array case study
BackendAccel-Sim / GPGPU-Sim, L1 and L2 cachesSCALE-Sim, input, weight and output buffers
WorkloadsPolyBench 2D and 3D convolution; MLPerf Inference v5.0 models: Llama-3.2-1B, Llama-3-8B, Llama-3.2-11B-Vision, ResNet-18, ResNet-50, BERT-base-uncased, GPT-J-6B, Stable DiffusionResNet-50, BERT and a 1B-parameter Llama (GEMM operations only); results shown for ResNet-50
InputsText models: 7-word prompts, 20 generated tokens. Vision: 224×224 images, batch 1256-token sequences; 224×224 images, batch 1
Hardware parameterssimulated GPU cache hierarchyPE arrays from 32×32 to 256×256; 4 kB input and weight buffers, 8 kB output buffer; three dataflows
BaselineThe same memories built entirely from SRAM

The inputs were chosen to match common inference scenarios while keeping simulation tractable.

8 · Results on the GPU

  1. Lifetimes. Across all workloads, 64.3% of L1 and 18.4% of L2 accesses fall within Si-GCRAM's 1 µs retention, and 97.9% of L1 and 52.0% of L2 accesses within Hybrid-GCRAM's 10 µs. Some data lives much longer (up to 70 µs) and needs eDRAM or SRAM.
  2. Which kernels. Short-lived data comes mostly from linear algebra such as GEMM; longer lifetimes appear in memory transfers and nonlinear computations.
  3. Compositions. Most workloads get Si-GCRAM-dominated L1 caches (over 90% of capacity). Transformer models such as the Llama variants need substantial longer-retention capacity for KV-cache storage. L2 caches need a more balanced mix of technologies.
  4. Savings over SRAM. Si-GCRAM-dominant L1 caches average 1.89× lower active energy and Hybrid-GCRAM-integrated L2 caches 1.3×. Memory array area improves by 3× across workloads. The best workloads reach up to 3.1× energy and 4.3× area reductions.

9 · Results on the systolic array

  1. Lifetimes. For ResNet-50 on a 256×256 array, at least 79.01% of accesses across the three buffers live under 1 µs, so they fit 5 nm Si-GCRAM without refresh.
  2. Dataflow matters for stationary data. Output feature map data is short-lived under every dataflow (nearly all of it fits Hybrid-GCRAM), because its generation is tightly coupled to the PE array's timing. The dataflow mainly changes the maximum lifetime of whichever operand is held stationary, which can sit idle in its buffer once the array is saturated.
  3. Compositions and savings. The vast majority of scratchpad capacity goes to Si-GCRAM, with little Hybrid-GCRAM or eDRAM. Savings are about 3× in active energy and 2.4× in area under all three dataflows. The paper attributes the consistency to the deterministic access patterns of systolic arrays, in contrast with the more variable GPU results.
  4. Array size. Larger PE arrays slightly reduce average lifetimes (data is consumed faster) and shrink the maximum lifetime of stationary data, narrowing the distribution. Fewer long-lived outliers means more room for cells with even shorter retention.
Reductions against uniform SRAM
active energy and memory array area · simulated · redrawn from Sections 6.1.3 and 6.2.3
active energy array area GPU, L1 and L2 caches L1 energy, average L2 energy, average L1/L2 area, across workloads energy, best workload area, best workload 1.89× 1.3× 3× up to 3.1× up to 4.3× Systolic array, ResNet-50 scratchpads energy, input stationary energy, output stationary energy, weight stationary area, input stationary area, output stationary area, weight stationary 2.97× 2.98× 2.99× 2.39× 2.38× 2.37× 01×2×3×4×5× dashed line: 1×, same as SRAM
GPU values from Section 6.1.3 and Takeaway 6.2 (averages across the workloads in Table 5; the dashed bars are the best single-workload results). Systolic-array values from the paper's Figure 10 (ResNet-50, 256×256 array). Higher is better.

10 · Limitations and directions the paper states

  1. Gain-cell scaling. Smaller process nodes raise leakage and cut retention in both Si-GCRAM and Hybrid-GCRAM, even though gate-all-around transistors reduce off-current. The paper expects GCRAM to stay viable: higher clock rates at smaller nodes shorten real data lifetimes, and circuit techniques can extend retention.
  2. Long-term RAM is not yet modeled. RRAM and MRAM are excluded because the studied workloads are transient and write-intensive. Evaluating them properly needs endurance modeling (write frequency, spatial patterns, temporal clustering over long periods), which the paper names as a future extension.
  3. Systolic-array coverage. The SCALE-Sim study covers GEMM operations; nonlinear operations are future extensions.
  4. Model fidelity. Results are first-order projections from cycle-accurate simulation with kernel sampling and device models, not measurements of fabricated heterogeneous arrays.
  5. Beyond AI. The paper argues that the same lifetime-aware approach applies to servers (HTTP buffers and session state with µs-scale lifetimes, cached content suited to LtRAM), databases (transaction logs, query results), scientific computing (intermediate arrays) and media pipelines (frame buffers). These are not evaluated.
Common questions

FAQ

Q·Is GainSight a chip design or a tool?▾

A tool. It profiles workloads on simulated accelerators and, from a library of cell models, recommends how much of each memory technology an on-chip memory should contain and what that would save in area and active energy. No heterogeneous memory is fabricated in the paper. The code is open source (code.stanford.edu/tambe-lab/gainsight), with documentation and interactive visualizations at gainsight.stanford.edu.

Q·How is short-term RAM different from the long-term RAM (LtRAM) work on emerging memories?▾

They sit at opposite ends of the same trade. LtRAM technologies (MRAM, RRAM, FeRAM) are dense and keep data for a long time, but writes are expensive and endurance is limited, so they suit data that is rarely written. StRAM technologies (gain cells, eDRAM) are dense, cheap to write and low in leakage, but forget their contents after microseconds to milliseconds, so they suit data that is written, read and discarded quickly. GainSight focuses on StRAM; the paper names LtRAM evaluation, with endurance modeling, as a future extension.

Q·What happens to data that lives longer than the cell's retention?▾

On a single-technology design, it must be refreshed, and the frontend counts those bit-level refreshes and charges each as an extra read and write. The composition step avoids refresh altogether by sending such data to a cell with longer retention: Hybrid-GCRAM if it lives under 10 µs, otherwise eDRAM (ms-scale) or SRAM. In the GPU study some data lives up to 70 µs.

Q·Why not use eDRAM for everything, since it retains data for milliseconds?▾

Because longer retention costs active energy. The paper notes that Si-GCRAM, Hybrid-GCRAM and eDRAM offer increasing retention at the cost of higher active energy, and eDRAM's charge-sharing reads make it the most expensive of the three per access. Assigning each access to the shortest-retention cell that is still long enough maximizes use of the cheapest cells while keeping operation refresh-free.

Q·Why does write frequency appear in the device models?▾

For gain cells, retention is not a fixed number. The paper cites an inverse trade-off for Hybrid-GCRAM between retention and write frequency, and its Figure 3 shows Hybrid-GCRAM retention dropping at high write frequencies. The frontend therefore compares each workload's lifetimes with a write-frequency-dependent retention, using the write frequencies it measures from the trace.

Q·Why are the GPU results more variable than the systolic-array results?▾

The paper attributes it to determinism. A systolic array moves data through its buffers according to a fixed dataflow, so lifetimes are predictable and the savings are nearly identical across dataflows (about 3× energy and 2.4× area for ResNet-50). GPU kernels have dynamic control flow and cache behavior, so lifetimes, compositions and savings vary more from workload to workload.

Q·Does the right composition depend on the workload?▾

Yes. Most workloads in the GPU study get L1 caches that are over 90% Si-GCRAM, but the Llama variants need substantial longer-retention capacity for KV-cache storage, and L2 caches need a more balanced mix. That is why the paper argues for profiling each target workload set instead of relying on fixed heuristics.

Q·Cycle-accurate GPU simulation is very slow. How is this tractable?▾

Two ways. The GPU backend uses Principal Kernel Analysis to cluster kernels and simulate only representatives, which cut the simulated kernels to as little as 0.15% of the original (Llama-3-8B) with speedups of 10.89× to 412.76× on the three workloads checked; the mean absolute error in lifetime was 0.36 to 3.76 µs. The experiments also use small inputs: 7-word prompts with 20 generated tokens for text models, and single 224×224 images for vision models.

Q·How does this differ from Nsight Compute, perf mem or Cachegrind?▾

Those tools are retrospective: they report counters, sampled misses or cache statistics, often at kernel granularity, for an existing CPU or GPU. The paper's Table 2 shows none of them profiles non-GPU accelerators, correlates accesses with memory-cell properties, or generates a memory composition. GainSight records every cache or scratchpad access with timing on a simulated architecture and turns that into a design recommendation.

Q·Can I add my own accelerator or memory technology?▾

Yes, by design. Any simulator or physical accelerator that can emit a trace with timestamps, address ranges and read/write types can serve as a backend, with each of its memories mapped to a subpartition. A new cell type needs a model of its retention, bit-cell area and per-bit read and write energy.

Q·Why do larger systolic arrays help?▾

A larger PE array consumes data faster, which slightly lowers average lifetimes, and it sharply reduces the maximum lifetime of the stationary operand. With fewer long-lived outliers, a designer can consider cells that trade even more retention for energy efficiency, since outliers would otherwise dominate refresh overhead.

Q·What does the evaluation not cover?▾

All numbers are first-order projections from simulation and device models, without circuit-level effects or fabricated arrays. The systolic-array study covers GEMM operations only and reports detailed results for ResNet-50. Long-term RAM devices are out of scope. The applicability to server, database, scientific and media workloads is argued in the discussion but not measured.

Background terms
SRAM (static random-access memory)A memory whose cell is a pair of cross-coupled inverters plus access transistors (six transistors in a 6T cell). It holds its value as long as power is applied and is the standard on-chip memory.
Retention timeHow long a charge-based memory cell stays readable after being written before leakage corrupts it.
RefreshReading a cell and writing the value back before its retention time runs out, so that data can be kept longer than the cell's retention. It costs energy and memory bandwidth.
StRAM (short-term RAM)The paper's term for memory technologies that trade retention time for density and energy, such as gain-cell RAM and eDRAM.
LtRAM (long-term RAM)Memory technologies such as MRAM, RRAM and FeRAM that are dense and retain data for long periods, but have expensive writes and limited endurance.
Gain cell (GCRAM)A memory cell of 2 to 3 transistors that stores a bit as charge on the gate of a read transistor. The read transistor converts the stored voltage into an amplified bitline current, so reading does not drain the stored charge (general knowledge).
Si-GCRAMA gain cell built entirely from standard silicon transistors in a FinFET logic process, with µs-scale retention (1 µs in the paper's 5 nm model).
Hybrid-GCRAMA gain cell whose write transistor is an oxide-semiconductor transistor (indium tin oxide) stacked above the silicon read transistor. It retains data longer than Si-GCRAM (10 µs in the paper's model) and is denser through 3D stacking.
eDRAM (embedded DRAM)DRAM built on the logic die: each cell is one access transistor and one capacitor (1T1C). The paper's oxide eDRAM builds both in the back-end-of-line and retains data for milliseconds.
Back-end-of-line (BEOL)The upper layers of a chip, the metal wiring built above the silicon transistors. Devices fabricated there can be stacked on top of logic instead of taking silicon area.
Data lifetimeThe time between a value's first write to a memory address and its last read before it is overwritten or invalidated (the paper's Definition 2.1).
Heterogeneous memory compositionAn on-chip memory built from several cell technologies in chosen proportions, with each technology holding the data whose lifetime matches its retention.
Systolic arrayAn accelerator made of a grid of processing elements (PEs) that pass operands to their neighbors in lockstep, typically used for matrix multiplication, and fed from on-chip scratchpad buffers.
Dataflow (input, weight, output stationary)The rule for which operand stays fixed inside each processing element while the others stream through: the inputs, the weights or the partial outputs.
Scratchpad (ifmap, filter, ofmap)A software- or compiler-managed on-chip memory without cache tags. A systolic array has one for the input feature map (ifmap), one for the weights (filter) and one for the output feature map (ofmap).
Accel-Sim and GPGPU-SimCycle-accurate simulators of NVIDIA-style GPUs; GPGPU-Sim models the GPU microarchitecture and Accel-Sim is a framework around it that runs traces of real GPU binaries.
SCALE-SimAn open-source, Python-based cycle-accurate simulator of systolic-array accelerators for convolution and GEMM layers, with configurable array size, buffer sizes and dataflow.
Principal Kernel Analysis (PKA)A sampling method for GPU simulation that clusters a program's kernels by their characteristics and simulates only representatives of each cluster.
Summary of P. Li, M. Hung, Y. Tan, K. Hoßfeld, J. Cheng Jiajun, S. Liu, L. Yan, X. Wang, P. Levis, H.-S. P. Wong and T. Tambe, "GainSight: A Unified Framework for Data Lifetime Profiling and Heterogeneous Memory Composition," arXiv:2504.14866, 2025. Numbers are from the paper unless marked as this summary's arithmetic.