Paper summary

Federation of Experts: Communication Efficient Distributed Inference for Large Language Models

Muhammad Shahir Abdurrahman, Chun Deng, Azalia Mirhoseini, Philip Levis · Stanford University

arXiv preprint, arXiv:2605.06206 [cs.LG] · May 2026

This page was written by an AI model to give a high-level overview of the paper. Please check the paper for detail: arXiv.

The big idea

Most recent large language models (LLMs) are mixture-of-experts (MoE) models: each transformer layer holds many small feed-forward networks ("experts"), and a router sends each token to only its top-k of them. That keeps compute per token low while the parameter count grows. When the model is spread over many GPUs with expert parallelism, though, the experts a token picks usually live on other GPUs, so every layer needs an all-to-all exchange to send tokens out (dispatch) and bring results back (combine). Prior work the paper cites puts this communication at 47% to 68% of inference time, and across servers it gets worse: the paper compares up to 900 GB/s of NVLink inside a node with 400 Gb/s per InfiniBand link between nodes, a gap of roughly 18×.

Existing fixes work around the all-to-all rather than removing it. Schedulers such as Sem-MoE, ExFlow and MoETuner place experts near the tokens that tend to pick them; COMET, FlashMoE and DeepEP overlap the communication with compute; BigMac and LatentMoE shrink what is sent. The paper's point is that all of them still run one global expert-parallel group, so a large share of each token's expert selections still crosses GPUs and, in multi-node deployments, the slow inter-node network.

The proposal, Federation of Experts (FoE), changes the model so the traffic pattern is local by construction. Each MoE layer is split into H independent expert groups, with H set to the model's number of key/value (KV) attention heads. Each group owns one KV head (and its share of the KV cache), 1/H of the experts and 1/H of the GPUs, and keeps its own copy of every token's hidden state. Instead of choosing k experts from the whole pool, each token chooses k/H experts inside every group. The only exchange between groups is a sum (an all-reduce) of the groups' post-attention residuals once per layer, which gives every group the same routing input.

The paper's analysis is in terms of the local activation rate (the fraction of expert selections served on the token's own GPU). With expert parallelism over G GPUs it is 1/G; with FoE it is min(H/G, 1). On one node with no more GPUs than KV heads (the usual case: 8 GPUs, 8 or more KV heads), every selection is local, the all-to-all disappears, and per-layer communication volume drops by exactly k×. Across nodes, each group is kept inside one node, so dispatch and combine stay on NVLink and only the all-reduce crosses nodes; the paper's bandwidth-weighted model predicts about 6.3× less per-layer communication time for k = 8, H = 8 and a 20× intra/inter-node bandwidth ratio.

Because FoE changes the architecture, existing checkpoints cannot be used, so the authors pretrain 1B- and 7B-parameter OLMoE-like models from scratch together with size-matched MoE baselines, and serve both in their own engine, FlexServe, on one 8×H100 node and on two such nodes over InfiniBand. On LongBench requests with Poisson arrivals, FoE lowers mean time to first token (TTFT) by 2.66× to 3.11× on one node and by up to 3.62× on two nodes, and mean time between tokens (TBT) by up to 1.95×. On synthetic batches the forward pass is up to 5.2× faster at 24,576 tokens, with little difference below 2,048. Training loss and zero-shot accuracy on six benchmarks are comparable to the MoE baselines.

Where the expert traffic goes
one global all-to-all · or H groups that only share an all-reduce
MoE with expert parallelism each token's top-k experts can sit on any GPU GPU 0GPU 1GPU 2GPU 3 attentionattentionattentionattention E/4 expertsE/4 expertsE/4 expertsE/4 experts global all-to-all over all G GPUs dispatch and combine at every MoE layer; only 1/G of expert selections are local per-layer volume 2k(G−1)/G (units of S·d) Federation of Experts H groups, one KV head each (here G = H = 4) group 0group 1group 2group 3 KV head 0KV head 1KV head 2KV head 3 E/4 expertsE/4 expertsE/4 expertsE/4 experts all-reduce: sum of post-attention residuals no expert traffic between groups each token picks k/H experts in every group; with G ≤ H, every selection is local per-layer volume 2(G−1)/G, k× less
Redrawn after the paper's Figure 2 and Section 3.2, for a single node. Four GPUs and four groups are illustrative; the paper's models use H = 8 on 8 GPUs. Volumes are the paper's single-node formulas (its Table 3), in units of tokens per batch × hidden dimension.

Key takeaways

  • In expert-parallel MoE inference, the all-to-all that sends tokens to experts on other GPUs is a major cost, and it grows with the number of GPUs and nodes.
  • FoE splits each MoE layer into H expert groups (one per KV head); each token picks k/H experts in every group, and groups exchange only an all-reduce of residuals.
  • On one node with G ≤ H, all expert selections are local and per-layer communication volume falls by exactly k×; across nodes, the all-to-all stays on the intra-node fabric.
  • Measured on 8×H100 and 2×8×H100 with LongBench: mean TTFT up to 3.62× lower and mean TBT up to 1.95× lower; synthetic forward passes up to 5.2× faster at large batches.
  • FoE is a model change: the 1B and 7B FoE models were pretrained from scratch and match size-matched MoE baselines on loss and zero-shot accuracy; larger models and clusters are not yet tested.
Mechanism, step by step

From one global expert-parallel group to a federation of local expert groups

MoE and its parallelism, why the all-to-all dominates, the FoE layer, its communication model, and the measurements.

1 · Background: mixture-of-experts layers

A transformer layer has two parts: attention, which mixes information across the tokens of a sequence, and a feed-forward network (FFN), which transforms each token's hidden state (a vector of dimension d) on its own. The FFN holds most of the parameters. An MoE layer replaces the single FFN with E smaller expert FFNs and a router (gate network) that scores all experts for each token. In the common token-choice design the paper builds on, each token runs through its k highest-scoring experts, and their outputs are added with the router's weights. Total parameters grow with E while compute per token grows only with k.

Attention in these models uses grouped key/value (KV) heads: many query heads share a smaller number of KV heads, and the keys and values computed for past tokens are kept in a KV cache. The paper's models have 64 experts, top-8 routing and 8 KV heads (its Table 8).

2 · Background: splitting a model across GPUs

When a model and its batch do not fit one GPU, serving engines combine several kinds of parallelism (general knowledge, except where the paper is cited):

SchemeWhat is splitCommunication it needs
Data parallelismThe batch: each GPU holds a full copy of the weights and serves different requestsNone during inference
Tensor parallelismEach weight matrix, for example attention by heads; every GPU sees every tokenAn all-reduce of partial results after each split layer
Expert parallelism (EP)The experts of each MoE layer: each GPU holds E/G of them (the paper's main baseline, as in vLLM and SGLang)An all-to-all to dispatch tokens to their experts' GPUs, and another to combine the results

In an all-to-all, every GPU sends a different piece of data to every other GPU. In an all-reduce, every GPU contributes a tensor of the same shape and every GPU receives the sum. The paper's hardware argument is that GPUs inside a server talk over NVLink at up to 900 GB/s, while servers talk over InfiniBand at 400 Gb/s per link, roughly 18× slower; its analysis uses an intra/inter-node bandwidth ratio r of 15 to 20 for H100-class machines.

3 · Why the all-to-all dominates

The paper's central quantity is the local activation rate ρ: the fraction of a token's expert selections served by experts on the GPU that already holds the token's input and KV cache. Only non-local selections go through the all-to-all, so under balanced routing the per-layer all-to-all volume is 2k(1 − ρ) · S · d for a batch of S tokens.

  1. Standard EP has ρ = 1/G. A token's experts are spread uniformly over G GPUs, so at G = 8 only 1 in 8 selections is local (this summary's arithmetic), and the share shrinks as the deployment grows.
  2. Full locality needs k ≤ E/G. Only then could one GPU hold all of a token's experts. Fine-grained MoE models with large k on many GPUs violate this for most tokens.
  3. Scale makes it worse. The paper cites DeepSeek's V3/R1 deployment, which spreads 256 experts over 18 nodes of 8 GPUs for decode, and prior measurements that put all-to-all at 47% to 68% of inference time.
  4. Imbalance adds stragglers. Tokens do not pick experts evenly, so some GPUs receive more tokens than others, and every GPU waits for the slowest at the all-to-all. In the paper's traces, the busiest GPU averages 1.64× the median GPU's token count.
The paper's diagnosis: placement schedulers, overlap techniques and low-dimensional projections all still operate inside one global EP group, so they can raise ρ statistically or hide the cost, but not bound it. FoE instead changes the model so that ρ is high by design.

4 · The FoE layer

FoE replaces each MoE layer with H expert groups and sets H equal to the number of KV heads. Group h owns KV head h (and the matching slice of the query projection), the contiguous block of experts Eh = {h·E/H, ..., (h+1)·E/H − 1}, and 1/H of the GPUs. Between layers the model carries H hidden states per token, one per group, instead of one.

One FoE layer, for one token i
per-group attention · one sync · shared routing · per-group experts
per-group stateper-group attentionsyncshared routingper-group expertsnext layer x[0, i]x[1, i]x[H−1, i] attentionKV head 0 attentionKV head 1 attentionKV head H−1 all-reduce r_full = average of all r[h, i] router scores g[i] over all E experts top-k/H of group 0weighted expert sum top-k/H of group 1weighted expert sum top-k/H of group H−1weighted expert sum x′[0, i]r_full + sum x′[1, i]r_full + sum x′[H−1, i]r_full + sum Only the all-reduce crosses groups; each group's expert dispatch and combine stay on that group's own GPUs. First layer: one shared attention feeds every group. Last layer: a final cross-group average returns a single state.
Drawn from the paper's Section 3.1 and Table 2 (intermediate layer, 1 < l < L). Three rows stand for the H groups.
  1. Per-group attention. Group h runs attention on its own state with its own KV head and adds the result to form a post-attention residual r[h, i].
  2. One synchronization. An all-reduce across groups averages the H residuals into rfull[i], which every group now holds. This is the only cross-group step in the layer.
  3. Shared scores, per-group choice. One router computes scores over all E experts from rfull. Each group then takes the top k/H experts within its own block, so every group contributes to every token at every layer (k must be a multiple of H).
  4. Local expert update. Each group applies its chosen experts to rfull and adds the weighted sum to rfull, producing its own next-layer state. Dispatch and combine never leave the group.
  5. Layer boundaries. The first MoE layer starts from the single embedding, runs one shared attention and fans out into H states; the last layer averages the H states back into one before the output head.

Because every token visits every group, each group receives the same number of tokens. On a single node, where each group maps to its own GPU, this makes per-GPU load equal by construction.

5 · The communication model

FoE raises the local activation rate to ρ = min(H/G, 1), relaxing the locality condition from k ≤ E/G to k/H ≤ E/G, and adds an all-reduce of volume 2(min(G, H) − 1)/min(G, H). Volumes are aggregate egress across GPUs, in units of S · d.

Single node (G ≤ H). Production models typically have 8 or more KV heads and single nodes rarely have more than 8 GPUs, so FoE's all-to-all term vanishes and only the all-reduce remains. The ratio is exactly k, whether G = H or the node has fewer GPUs than groups:

ArchitecturePer-layer volume (paper's Table 3)At k = 8, G = 8 (this summary's arithmetic)
MoE with EP2k · (G − 1)/G14
FoE2 · (G − 1)/G1.75 (8× less)

Multiple nodes (1 < N ≤ H). Each expert group is placed within one node, so its all-to-all stays on the intra-node fabric however many nodes there are. MoE's single EP group sends a fraction (N − 1)/N of its all-to-all across nodes, so its inter-node volume grows linearly in k; FoE's inter-node traffic is only the all-reduce, bounded by about 2 regardless of k. Weighting inter-node volume by r gives a time model whose MoE/FoE ratio tends to kHr / (kH + r(H − 1)) as N grows; for k = 8, H = 8, r = 20 that is about 6.3×, which the paper says is within about 5% of the exact formula for N ≥ 2. Plugging the paper's two-node test configuration into its Table 4:

N = 2, G = 16, H = 8, k = 8, r = 20Intra-node volumeInter-node volumeWeighted time
MoE with EP7.57.5157.5
FoE8.8750.87526.4
Ratio8.6×about 6.0×

All values in this table are this summary's arithmetic from the paper's formulas; the bandwidth ratio r = 20 is the top of the paper's stated 15 to 20 range. FoE moves slightly more data inside the node but far less across it.

6 · Models, engine and setup

  1. Models trained from scratch. FoE changes routing and partitions the query projection, so open MoE checkpoints cannot be reused. The authors pretrain OLMoE-style models of 1.02B and 6.85B parameters on FineWeb-Edu (4.7B and 4.9B tokens), each with a size-matched MoE baseline sharing all other settings: 16 layers, 64 experts, top-8, 8 KV heads, and H = 8 for FoE. The training-loss curves of FoE and MoE follow the same trajectory; at 1B they are indistinguishable at the end of training.
  2. Serving engine. Both expert parallelism and FoE run in FlexServe, the authors' MoE inference engine built on FlexAttention, which includes PagedAttention, chunked prefill, data and pipeline parallelism. Running the EP baseline in the same engine is meant to isolate the architectural difference from framework differences.
  3. Hardware. One server with 8 NVIDIA H100 SXM GPUs (80 GB each), and two such servers connected by InfiniBand.
  4. Workload and metrics. LongBench requests arriving as a Poisson process at three arrival scales; TTFT (arrival to first token, the paper's prefill metric), TBT (average time per generated token in decode) and end-to-end request latency, each as mean, p50 and p99.
  5. Baseline choice. Predictive schedulers (MoETuner, Sem-MoE) are left out of the latency runs, which the paper treats as orthogonal, and are compared on local activation rate instead.

7 · Latency results

SetupArrival scaleMean TTFT (s), MoE → FoEMean TBT (s), MoE → FoEMean end-to-end (s), MoE → FoE
1 node0.53.710 → 1.394 (2.66×)0.780 → 0.566 (1.38×)202.7 → 145.7 (1.39×)
1 node0.753.233 → 1.041 (3.11×)0.841 → 0.458 (1.83×)217.7 → 117.9 (1.85×)
1 node1.52.264 → 0.790 (2.87×)0.371 → 0.334 (1.11×)96.9 → 86.0 (1.13×)
2 nodes0.53.883 → 1.128 (3.44×)0.736 → 0.543 (1.36×)191.5 → 139.5 (1.37×)
2 nodes0.753.403 → 0.939 (3.62×)0.876 → 0.449 (1.95×)226.7 → 115.4 (1.96×)

Means from the paper's Tables 5 and 6. Tails improve too: on two nodes at scale 0.5, p99 TTFT is 2.575 s for FoE against 7.066 s for MoE. The TTFT gain at scale 0.5 grows from 2.66× on one node to 3.44× on two, which the paper attributes to MoE's dispatch crossing InfiniBand while FoE's stays inside each node. The paper links the tail improvement to FoE not waiting on stragglers at a global all-to-all.

Forward-pass latency vs. batch size
synthetic batches · redrawn from the paper's Table 9 (Appendix A.3)
MoE with expert parallelism Federation of Experts 0123 forward-pass latency (s) 3.49 s 0.67 s 1281285121,0242,0484,0968,19212,28816,38420,48024,576 1.04×1.00×0.96×1.01×0.97×0.92×1.30×2.19×3.59×4.48×4.88×5.14×5.21× batch speedup Batch sizes are evenly spaced categories, not a linear axis. Speedup is MoE latency ÷ FoE latency.
All values from the paper's Table 9. The x-axis spacing is this summary's choice for readability.

Up to a batch of 1,024 the speedup ranges from 0.92× to 1.04×, so FoE is sometimes slightly slower. From 2,048 on, MoE latency climbs with the all-to-all while FoE's stays near-constant before scaling smoothly, reaching the 5.2× headline at 24,576. A profile of one layer with an 8,192-token batch on the 8-GPU node (Appendix A.2) shows 12.3× less time on the communication stream.

8 · Locality and load balance

At an expert-parallel size of 8 (the paper's Figure 4), FoE's local activation rate is 1.0, since each group is self-contained on one GPU. The predictive placement methods it compares against reach 0.39 (Sem-MoE) and 0.21 (ExFlow), with MoETuner also well below Sem-MoE. For load, the paper measures max/median tokens per GPU: standard EP averages 1.64×, and FoE is 1.0 because every group receives every token. Values for MoETuner and Sem-MoE in that comparison come from their own papers, and neither reaches 1.0. The paper attributes its latency gains to both effects together: less off-GPU and off-node traffic, and no straggler GPU holding up a global synchronization.

9 · Model quality

ModelARC-EasyBoolQCOPAHellaSwagPIQASciQ
MoE (7B)59.6%60.4%65.0%33.7%66.9%80.0%
FoE (7B)58.2%60.9%66.0%33.1%66.6%80.0%
MoE (1B)53.5%57.7%62.0%30.9%63.9%77.7%
FoE (1B)52.7%61.4%61.0%30.1%63.8%74.6%

Zero-shot accuracy from the paper's Table 7. FoE is ahead on some tasks and behind on others, and the paper states that the differences are within the range of typical training-seed variance. Both models are small by production standards and trained on about 5B tokens.

10 · Limitations the paper states

  1. Training complexity. FoE trained to similar quality with MoE's hyperparameters, but partitioning experts and KV heads into groups may need its own hyperparameter tuning for the best results.
  2. Distributed deployments only. On a single GPU there is no all-to-all to remove, and the cross-group all-reduce is pure overhead.
  3. Topology. The evaluation assumes homogeneous nodes; with highly asymmetric bandwidths, how groups are placed across nodes decides how much traffic stays local, and skewed setups may need placement-aware tuning.
  4. Scale of evidence. Two model sizes (1B and 7B) and two hardware configurations (one 8×H100 node and two nodes over InfiniBand). Larger models and longer datasets are left for future work.
Common questions

FAQ

Q·Is FoE a serving-system change or a model change?▾

A model change. Each layer carries H per-group hidden states, routing picks experts per group, and the query projection is partitioned by group. A model has to be trained as an FoE model; existing MoE checkpoints cannot be run this way off the shelf. That is why the paper pretrains its own 1B and 7B models with size-matched MoE baselines.

Q·How is this different from expert-placement schedulers such as Sem-MoE, ExFlow or MoETuner?▾

Those keep the model unchanged and try to put experts on the GPUs where the tokens that select them already live, using observed or predicted routing patterns. They raise the local activation rate statistically; at an expert-parallel size of 8, the best of them in the paper's comparison reaches 0.39. FoE restricts which experts a token can pick so that locality holds by construction (1.0 on one node). The paper treats the two as orthogonal: placement and scheduling can still be applied on top of FoE.

Q·Isn't this just tensor parallelism of attention by heads?▾

The paper does not make this comparison, so this answer is inference. The attention side looks similar: each group handles one KV head, and an all-reduce follows attention, as in head-wise tensor parallelism. The difference is on the expert side. Each group keeps its own hidden state and only uses its own block of experts, so no token has to be sent to another group's GPUs for its experts. The all-reduce produces a shared routing input, and the all-to-all disappears because routing itself is confined to the group.

Q·Why set the number of groups equal to the number of KV heads?▾

The paper defines H = nkv, so each group owns one KV head and its slice of the KV cache, and the attention for that head runs where the group's experts are. The paper's single-node argument also rests on it: modern production models typically have 8 or more KV heads and single nodes rarely exceed 8 GPUs, so G ≤ H and every expert selection is local.

Q·Does restricting each token to k/H experts per group hurt quality?▾

Not measurably in the paper's tests. Training loss follows the MoE baseline's trajectory, and zero-shot accuracy on ARC-Easy, BoolQ, COPA, HellaSwag, PIQA and SciQ is within what the paper describes as typical seed variance, with FoE ahead on some tasks and behind on others. The tests cover only 1B and 7B models trained on about 5B tokens, and the paper notes FoE may need its own hyperparameter tuning to reach its best quality.

Q·What happens with more GPUs than KV heads?▾

Then each group spans several GPUs (G/H of them), and FoE's local activation rate is H/G, so some all-to-all returns, but only among the GPUs of one group. The paper places each group within a single node, so that all-to-all stays on NVLink and only the all-reduce crosses nodes. In the paper's two-node test (16 GPUs, 8 groups), each group has 2 GPUs.

Q·Does FoE help on a single GPU?▾

No. With one GPU there is no all-to-all to remove, and the cross-group all-reduce becomes extra work with nothing to offset it. The paper lists this as a limitation: FoE's benefits apply only to distributed deployments.

Q·Why does FoE barely change latency at small batch sizes?▾

In the paper's synthetic forward-pass test, both architectures take about 0.06 to 0.08 s up to a batch of 1,024, and FoE is between 0.92× and 1.04× of MoE's speed there. The paper's account is that MoE's latency only escalates once batches grow beyond 2,048, when the all-to-all becomes the bottleneck; FoE's latency stays near-constant to moderately large batches. The 5.2× headline is at the largest batch tested, 24,576.

Q·Where do the headline numbers come from?▾

The 5.2× forward-pass figure is from synthetic batches (Appendix A.3, batch 24,576). The 3.62× TTFT and 1.95× TBT figures are mean values on the two-node InfiniBand setup at arrival scale 0.75 on LongBench (Table 6). On one node, mean TTFT improves 2.66× to 3.11× and mean TBT 1.11× to 1.83× across the three arrival scales tested.

Q·Why does the tail latency improve so much?▾

The paper attributes it to load balance. With standard expert parallelism, tokens pile up unevenly (the busiest GPU averages 1.64× the median), and every GPU waits for the slowest one at the global all-to-all. In FoE every group receives every token, so on a single node every GPU gets the same number of tokens and there is no straggler to wait for.

Q·Does FoE use more GPU memory?▾

The paper reports a negligible increase at inference, from the extra activation memory of the H independent per-group hidden states.

Q·What does the evaluation not cover?▾

By the paper's own account: models larger than 7B, more than two nodes, heterogeneous or skewed network topologies, and longer datasets. The latency baseline is expert parallelism implemented inside the authors' FlexServe engine (to remove framework differences), not vLLM or SGLang themselves, and predictive schedulers are compared only on local activation rate, not on latency.

Background terms
Large language model (LLM)A neural network, usually a stack of transformer layers, trained to predict the next token of text and used to generate text one token at a time.
Token and hidden stateA token is a unit of text (a word piece) that the model processes; its hidden state is the vector of dimension d that represents it between layers.
Feed-forward network (FFN)The per-token part of a transformer layer: a small multilayer perceptron applied to each token's hidden state independently. It holds most of a layer's parameters.
Mixture of experts (MoE)A layer that contains many expert FFNs and a router; each token is processed by only a few of them, so total parameters grow faster than compute per token.
Router and top-kThe router (gate network) is a small learned layer that scores every expert for a token; top-k routing sends the token to the k highest-scoring experts and mixes their outputs by those scores.
Attention and KV headsAttention is the part of a layer that lets each token read from earlier tokens. It is computed in several heads; with grouped attention, several query heads share one key/value (KV) head.
KV cacheThe stored keys and values of all previous tokens in a sequence, kept on the GPU so that each new token's attention does not recompute them.
Residual streamThe running hidden state that each sublayer adds its output to; a post-attention residual is the state after attention's output has been added.
Expert parallelism (EP)A way to distribute an MoE model in which each GPU holds a different subset of the experts, so tokens must be sent to the GPUs holding the experts they selected.
Tensor parallelismA way to distribute a model in which individual weight matrices (for example, attention heads) are split across GPUs, each computing a slice for every token, followed by an all-reduce (general knowledge).
All-to-allA collective communication operation in which every participating GPU sends a distinct block of data to every other GPU. In EP it is used twice per MoE layer: dispatch (tokens to experts) and combine (results back).
All-reduceA collective operation in which every participant contributes a tensor of the same shape and all receive the elementwise sum (or average).
Local activation rate (LAR, ρ)The fraction of a token's expert selections that are served on the GPU already holding the token, and so need no all-to-all. The paper's central metric.
Expert groupFoE's unit of partitioning: one KV head, a contiguous block of E/H experts, a per-group hidden state and the GPUs that run them.
NVLink and InfiniBandNVLink is NVIDIA's high-bandwidth GPU-to-GPU interconnect inside a server (up to 900 GB/s on H100, per the paper); InfiniBand is a datacenter network between servers (400 Gb/s per link, per the paper).
Prefill and decodeThe two phases of generating a response: prefill processes the whole prompt at once to produce the first token; decode then produces one token per step.
TTFT and TBTTime to first token is the time from a request's arrival to its first generated token; time between tokens is the average time per generated token during decode.
LongBenchA public benchmark of long-context tasks, bilingual and multitask, that the paper uses as its request workload with Poisson arrivals.
Summary of M. S. Abdurrahman, C. Deng, A. Mirhoseini and P. Levis, "Federation of Experts: Communication Efficient Distributed Inference for Large Language Models," arXiv preprint arXiv:2605.06206, May 2026. Numbers are from the paper unless marked as this summary's arithmetic.