We Need to Talk About the Actual Guarantees of Rust-based Systems
This page was written by an AI model to give a high-level overview of the paper. Please check the paper for detail: ACM Digital Library.
Over the past decade, systems researchers have used Rust to enforce security and isolation guarantees over user-provided code, which the paper calls extensions: kernel subsystems, network and serverless functions, eBPF alternatives, web application endpoints. The appeal is that Rust code performs comparably to C/C++ while the compiler checks properties such as memory safety, so a system can build a lighter enforcement mechanism on top of what the compiler already guarantees. RedLeaf, for example, isolates kernel subsystems so the kernel can recover from a crashed one, and relies on the rule that a subsystem holding an immutable reference cannot modify the data behind it. Sesame enforces privacy policies in web applications and relies on the rule that application code cannot read the private fields of its policy containers (PCons).
The paper's point is that these rules are not what Rust promises. Rust's only promise is soundness: a valid program never exhibits undefined behavior. Code inside unsafe blocks may do things the compiler would otherwise reject, as long as it stays sound, and it can be wrapped behind a safe interface that callers cannot tell apart from ordinary safe code. The properties systems depend on, which the paper calls incidental properties (encapsulation, lifetimes as a bound on aliases, immutability through references), hold in entirely safe code, but sound unsafe code is allowed to break them; a Mutex or RefCell mutates data behind shared references by design. Systems are left with two poor options: require globally safe extensions with no unsafe anywhere in their dependency tree, which almost no real code satisfies, or require only locally safe extensions, with unsafe banned in the top-level code but allowed in libraries, which the paper shows is too brittle.
The case studies make this concrete, including in the authors' own work. A locally safe Sesame endpoint that uses a logging helper built on serde's remote-type feature and mem::transmute leaks PCon-protected data, and Miri confirms the code is sound. A locally safe RedLeaf scheduler that keeps thread metadata in a RefCell on the shared heap can mutate that metadata through an immutable reference; if it then crashes, the kernel cannot roll the change back and a dequeued thread is lost forever. The authors report confirmed violations of the same kind in five more systems: Flowistry, NetBricks, Cocoon, Boucher et al.'s serverless isolation, and Rex.
The proposed path forward is tooling that reasons about incidental properties directly. Relative to one property, code is either unsound, sound but not property-preserving, or property-preserving, and only some unsafe patterns can break a given property (UnsafeCell::get can break immutability but not encapsulation; transmute and pointer arithmetic the reverse). A system's invariant is usually a property scoped to its own types, such as "PCons stay encapsulated". The authors' prototype, Sniffer, is a Rust compiler plugin that builds a monomorphized call graph of an extension and all its dependencies and flags every reachable path from a developer-annotated entity to an unsafe operation that may violate the specified property, for a developer to audit.
The preliminary evaluation covers the two case studies. Sniffer finds both guarantee-breaking operations. On Sesame's WebSubmit endpoint it flags 12 calls, one of them the real leak; flagging every unsafe block yields 33,735 regions, and every reachable one 9,982. On small RedLeaf scheduler excerpts (the full RedLeaf code base does not build with the compiler versions Sniffer supports) it reports no false positives. The paper presents these as early evidence that invariant-targeted analysis keeps audit burden low, not as a full evaluation.
Key takeaways
- Rust promises only soundness (no undefined behavior). Rust-based systems also rely on incidental properties (encapsulation, lifetimes as alias bounds, immutability of references) that safe code keeps but sound
unsafecode may break. - Requiring globally safe extensions is impractical, and requiring locally safe ones is brittle:
unsafein a trusted library can still break the system's invariant. - Case studies: a sound, locally safe Sesame endpoint leaks policy-protected data, and a locally safe RedLeaf scheduler can leave the kernel unable to recover after a crash. Violations were also confirmed in Flowistry, NetBricks, Cocoon, Boucher et al. and Rex.
- Proposal: name the incidental properties a system depends on, scope them to the system's types, and map each to the
unsafepatterns that can violate it. - Prototype Sniffer (a MIR-level compiler plugin) found both case-study violations; on Sesame it flagged 12 calls (one real) against 9,982 reachable
unsaferegions. The evaluation is preliminary.
From incidental properties to invariant-targeted auditing
What Rust promises, where Rust-based systems assume more, two case studies, the proposed vocabulary, and the Sniffer prototype.
1 · What Rust actually promises
Rust's central, and only, promise is that valid Rust programs never exhibit undefined behavior. That implies memory safety and the absence of data races. The paper describes the enforcement as two tiers:
- Safe code. For the bulk of a program, the compiler statically rules out undefined behavior.
unsafeblocks. The compiler suspends some of its checks, and the developer becomes responsible for avoiding undefined behavior, possibly with tools such as Miri or Kani.- Safe interfaces. A developer can wrap sound
unsafecode behind a safe interface. To a caller, such an interface is indistinguishable from a function with nounsafecode unless the caller reads its source. The paper notes this is critical for modularity.
The paper uses standard Rust terminology: unsafe means the unsafe keyword is present, unsound means the code may exhibit undefined behavior, and safe and sound are their counterparts.
2 · Incidental properties
The paper defines a property as incidental if it holds in entirely safe code but violating it does not cause undefined behavior. The term is the paper's own, not standard Rust terminology. It gives three examples:
| Incidental property | What it says | Why safe code keeps it | Sound unsafe that can break it |
|---|---|---|---|
| Encapsulation | Private fields of a struct are accessible only inside the defining module | The compiler enforces it directly | mem::transmute, pointer arithmetic |
| Lifetimes as aliases | The aliases of a reference of type &'a T can be soundly approximated from the lifetime 'a | Follows from the borrow checker's rules (aliasing XOR mutability) | Code that extends lifetimes, e.g. with mem::transmute |
| Immutability of references | Data reachable through an immutable reference &'a T never changes during 'a | Follows from the borrow checker's rules | Interior mutability via UnsafeCell::get (used by RefCell, Mutex) |
Breaking these properties is often the point. A Rust Mutex uses unsafe behind a safe interface precisely to let several threads mutate data they each reach through a shared reference. It is sound, because it uses atomic operations and is designed to avoid undefined behavior and data races, yet it violates immutability of references. The paper's key sentence: while Rust only mandates that unsafe code maintains soundness, Rust-based systems often depend on incidental properties that sound Rust code is under no obligation to uphold.
3 · Two unsatisfactory options
| Option | Rule for extensions | Who uses it | Problem |
|---|---|---|---|
| Globally safe | No unsafe anywhere, including the whole dependency tree | The system can always meet its goals, since the compiler enforces every incidental property | Such code is extremely rare. A cited study of the 500 most-used Rust crates found more than half call into unsafe code even after excluding the standard library |
| Locally safe | No unsafe in the top-level extension; libraries may use it | Systems that assume unsafe "is confined to trusted libraries" or lint only the top-level code | Too brittle: sound, idiomatic extensions that use only widely trusted libraries such as std can still break incidental properties |
unsafe code is sound, and soundness decides whether a function may be a safe interface. There is no equivalent "incidental property" interface, and no conventions or tools for reasoning about these properties.4 · Case study: Sesame leaks private data
Sesame, built by some of the authors, enforces end-to-end privacy policies (for example access control or k-anonymity) in Rust web applications. Its central abstraction is the policy container PCon<T,P>, which holds two private members: the sensitive data of type T and a policy of type P. If encapsulation holds, the application can reach the data only through Sesame's API, which checks the policy before data leaves.
The leak starts from a common pain point: if a third-party type does not implement Debug or Serialize, an application cannot log or serialize it directly. serde's remote feature lets the developer write a surrogate "remote type" that tells serde how to reach the third-party type's fields, normally through public getters. With unsafe code it can also reach fields that have no getters. The authors built such a library with serde-remote and std::mem::transmute, able to serialize any third-party type regardless of field visibility. The paper's Figure 2 uses it like this:
| Line | Code (simplified WebSubmit endpoint) | What it does |
|---|---|---|
| 1 to 6 | #[derive(Serialize)] struct SubmitForm { username: String, #[serialize_any(String, NoLoggingPolicy)] answer: PCon<String, NoLoggingPolicy> } | The library macro on line 4 serializes the PCon's contents through transmute, breaking its encapsulation |
| 10 | println!("{}", serde_json::serialize(&request)); | Logs the sensitive answer, which its policy forbids; it looks like a standard serialization call |
| 11 | SesameDB.insert(request.username, request.answer); | The intended, policy-checked use |
- The authors confirmed that
serialize_anylets Sesame's WebSubmit example log sensitive data, and confirmed with Miri that the code is sound. - They replicated the leak with another serialization library (Abomonation), and believe other libraries may implement similar patterns, for example for stack inspection or rewinding after panics.
- The paper frames this as a mistake made "out of confusion or oversight", exactly the kind Sesame aims to protect against.
5 · Case study: RedLeaf loses kernel state
RedLeaf is an operating system that isolates kernel subsystems so the kernel can recover when one crashes. Subsystems exchange data only through a shared heap, either by transferring ownership or by passing immutable references. On a crash the kernel cleans up only data the crashed subsystem owns, since that is the only data it could have modified. That reasoning depends on immutability of references. RedLeaf's partial protection is an auto trait, RRefable, that restricts which types can go on the shared heap: it rules out raw pointers but not interior mutable types such as RefCell.
- The authors wrote a locally safe, pluggable scheduler as a RedLeaf kernel subsystem. Its interface takes the metadata of the queued threads as input.
- Thread metadata must be modifiable by both the scheduler and the base kernel (which creates and destroys threads), so it is allocated on the shared heap inside a
RefCell. RedLeaf's original scheduler usesRefCellthe same way, but it is part of the trusted core kernel rather than an isolated subsystem. - The scheduler meets RedLeaf's shared-heap restrictions, yet through interior mutation it can modify data it reaches only by an immutable reference, for example by removing a thread from the queue.
- If the scheduler crashes after such a change, the kernel does not roll it back. The kernel is left inconsistent: it permanently loses the dequeued thread's metadata, and that thread never runs again.
The root is UnsafeCell, Rust's core interior-mutability primitive, whose get() and raw_get() are the only sound ways to interior-mutate a type. Banning UnsafeCell from the shared heap would also ban RefCell and common structures that contain interior mutable types, such as doubly-linked lists and ring buffers, even when a subsystem never mutates them. The less restrictive rule (allow UnsafeCell on the shared heap but forbid calling get()/raw_get() on it inside a subsystem) needs tooling beyond Rust's type system and auto traits.
6 · The pattern across systems
The authors confirmed that sound, locally safe extensions break the guarantees of each system in the paper's Figure 3. For the five beyond the case studies, the paper only outlines the violations.
| System | Goal | Stated scope | Incidental property | Violation |
|---|---|---|---|---|
| Sesame | Privacy policy enforcement | no "malicious" unsafe code | encapsulation | data leaks, policy violations |
| RedLeaf | Kernel subsystem isolation | "restricted to safe Rust" | immutability of references | no isolation between subsystems; kernel cannot recover after crashes |
| Flowistry | Analyzing information flow | "safe subset of Rust programs" | lifetimes + immutability | misses existing flows (unsound) |
| NetBricks | Packet isolation between network functions | no "unsafe pointer arithmetic" | immutability of references | packet modified after a network function sent it; memory bugs |
| Cocoon | Information flow control | "absence of unsafe Rust" | encapsulation + immutability | data leakage |
| Boucher et al. | Serverless function isolation | "safe Rust" | "language isolation" | data leaks between serverless functions |
| Rex | Safe kernel extensions (an eBPF alternative) | "safe Rust with selected features" | lifetimes + "resource safety" | kernel corruptions and crashes |
Rex shows the locally safe trap clearly: it uses a lint to forbid unsafe code and leak-prone functions like mem::forget, but the lint checks only the top-level extension, so a dependency can still call mem::forget and silently leak resources.
7 · A vocabulary for property preservation
The paper's framework for reasoning about unsafe code relative to one incidental property:
- Three classes of code. Unsound: has undefined behavior and no guarantees. Sound but not property-preserving: no undefined behavior, but violates the property in question. Property-preserving: no undefined behavior, and upholds the property.
- Only some patterns matter. Each property can be violated only by a subset of sound
unsafepatterns, so an extension can callunsafecode and still meet a system's requirements. A sound subsystem that respects immutability of references meets RedLeaf's requirements whatever otherunsafecode it invokes. - Invariants refine properties. A system usually needs a property only over its own entities. Sesame needs encapsulation of PCons and is unaffected if an application breaks encapsulation of unrelated types; RedLeaf needs immutability for shared-heap data and is fine with a subsystem that interior-mutates its own local data, such as a doubly-linked list.
- A shared library of properties. Many systems reuse the same incidental properties, so tools can offer a library of properties, each mapped to the
unsafepatterns that can violate it, for systems to cite when describing their invariants.
| Threat model | Who checks the extension | What tooling must provide | Examples in the paper |
|---|---|---|---|
| Cooperative extensions | Trusted extension developers run the system's tools and audit the output | Exhaustive flagging with a manageable audit burden | Sesame, RedLeaf, Rex, Cocoon |
| Adversarial extensions | The system itself, before loading or running untrusted code | Near-perfect precision (no human resolves ambiguity), possibly plus separate undefined-behavior detection | NetBricks, Boucher et al. (code from untrusted tenants) |
The paper surveys what exists. Lints and runtime mechanisms such as Sesame's obfuscated pointers have the weaknesses shown above. The unstable Freeze marker trait, implemented only for types that contain no UnsafeCell, lets a system forbid interior mutability through trait bounds, but it restricts types rather than behavior: it rejects a type containing an UnsafeCell even when the code never mutates through it. Static analysis and auditing tools for soundness, formal methods (Verus, for example, reasons about UnsafeCell via ghost state but not about transmute) and runtime isolation could, the paper suggests, be extended toward incidental properties.
8 · Sniffer: invariant-targeted static analysis
Sniffer is the authors' prototype auditing tool for the cooperative model. Its aim is to be exhaustive (surface every potential violation) while separating benign unsafe code from code that may violate the system's invariant, so a human reviews little code.
- Specify once per system. The system developer states which incidental properties the system cares about and, where applicable, the system types over which they must hold: all PCons stay encapsulated, or shared-heap data in RedLeaf is never interior-mutated. The prototype encodes checks for encapsulation and immutability of references as all the
unsafepatterns that may violate them. The system can reuse the specification for every extension, for example through a build script, and can tell extension developers which top-level functions (such as RedLeaf's subsystem interface) to annotate as entry points. - Build the monomorphized call graph (MCG). Sniffer is a compiler plugin that works on the mid-level intermediate representation (MIR). It extends the compiler's built-in reachability analysis to record, at each call site, what is needed to resolve dynamic dispatch soundly: trait and method signatures of
dyncalls and signatures of function pointers. The MCG instantiates generic functions and soundly over-approximates dynamic dispatch. - Search for violating patterns. A custom MIR visitor walks each reachable function body and matches the property-violating
unsafepatterns. Results can be narrowed by the types of the arguments passed to a flagged operation and limited to call graphs rooted at annotated entry points. No path means the extension upholds the invariant. - Audit. The extension developer reviews each flagged location and call chain, revises the code if the violation is real, and otherwise exempts it (a spurious flow, or code that is benign in context).
mem::transmute on a PCon that does not leak).9 · Preliminary evaluation
Setup: Sniffer runs on Sesame's full WebSubmit homework-submission endpoint, modified with the logging-library call from Section 4. The RedLeaf code base does not build with the compiler versions Sniffer supports, so the authors extract small compatible excerpts of the scheduler case study. These place a RefCell on the shared heap directly and nested in other structs, and mutate it directly, in transitive helpers, and after storing it in and retrieving it from collections.
- Q1, detection. Sniffer detects both guarantee-breaking operations: the
mem::transmuteinside the logging library that leaks PCon data, and theUnsafeCell::getthat allows interior mutation of shared-heap data in RedLeaf. - Q2, developer burden (Sesame). Sniffer flags 12 calls. One is the real encapsulation-breaking
transmute, which sits inside a macro expansion and which the paper says a manual audit would likely miss unless all macros were expanded. Five false positives involve pointer arithmetic that is harmless in context, such as iterators advancing through aVec<PCon>. Six come from over-approximated dynamic dispatch and can be dismissed by mechanically checking that no PCon reaches the flagged function. On the RedLeaf excerpts there are no false positives, though the paper notes they do not capture a practical subsystem's complexity. - Q3, design effectiveness. Compared with simpler alternatives on WebSubmit, below.
The paper reads this as validation that targeting the invariant, not just reachability, cuts false positives: from 9,982 reachable regions to 12 (roughly an 830-fold reduction, this summary's arithmetic). It does not report Sniffer's running time.
10 · Limitations and directions the paper names
- Scope of the evaluation. Two case studies; RedLeaf only as small excerpts, because of compiler-version incompatibility. Violations in the other five systems were confirmed but only outlined, and Sniffer was not run on them.
- Richer annotations. The prototype annotates types and top-level functions. Future work would support specific variables or all implementations of a trait, and integrate variable-level information flow analysis (as in Flowistry) to filter more false positives, for example to tell a
RefCellinternal to a RedLeaf subsystem from one passed in through the shared heap. - Adversarial extensions. When no trusted auditor exists, the paper envisions combining Sniffer with other mechanisms. Sniffer narrows the regions at risk (12 instead of 9,982 in the Sesame example) and could emit local proof obligations for formal tools, or add runtime checks only where it cannot dismiss a flag: a check before a flagged
transmutethat rejects PCon arguments, or memory protection keys (MPK) that make RedLeaf's shared-heap data read-only to non-owners so flaggedUnsafeCell::getcalls trap. - Beyond incidental properties. Which invariants code may reasonably rely on is an active Rust community discussion (for example "language" versus "library" invariants). Sniffer-like analysis could target safety invariants of custom types, or application invariants such as panic freedom, as a middle ground between syntactic checks and full formal verification.
FAQ
Q·Isn't this just the familiar problem of buggy unsafe code?▾
No. The paper separates its problem from the widely studied one of finding unsafe code that violates memory safety or causes data races, which tools such as Miri, Kani, Rudra and RustBelt address. Its focus is code that is memory-safe and meets all of Rust's requirements, yet still breaks a property a system depends on. The Sesame leak passes Miri.
Q·Is this a bug in Rust?▾
The paper does not call it one. Rust promises soundness and nothing more, and it explicitly lets idiomatic, widely accepted unsafe code violate incidental properties; Mutex and RefCell exist to do so. The mismatch is between that promise and what research systems assume.
Q·Why not simply forbid unsafe code in extensions?▾
Forbidding it everywhere, including dependencies (globally safe code), does make the guarantees hold, but such code is extremely rare. A study the paper cites found that more than half of the 500 most-used Rust crates call into unsafe code even excluding the standard library. Forbidding it only in the top-level extension (locally safe code), as lints do, leaves the dependencies free to break the invariant, which is what both case studies show.
Q·Were the case-study extensions malicious?▾
No. The Sesame endpoint is the kind of mistake a developer could make "out of confusion or oversight" while working around a type that lacks a Serialize implementation, and Sesame's stated scope excludes only "malicious" unsafe code. The RedLeaf scheduler uses RefCell the same way RedLeaf's own built-in scheduler does; the difference is that it runs as an isolated subsystem.
Q·Couldn't RedLeaf just ban RefCell and UnsafeCell from the shared heap?▾
It could, but every interior mutable type would go with them, so subsystems could no longer share common structures such as doubly-linked lists or ring buffers, even ones they never mutate through a shared reference. The more precise rule, allowing UnsafeCell on the shared heap but forbidding calls to its get() and raw_get() inside a subsystem, is about behavior rather than types, and needs tooling beyond Rust's type system and auto traits.
Q·What about the Freeze marker trait?▾
The paper calls the unstable Freeze trait a leading contender in the Rust community for dealing with interior mutability: it is implemented only for types that contain no UnsafeCell, so a trait bound can rule interior mutability out. Its limit is the same as above: it restricts types, not behavior, and rejects a type containing an UnsafeCell even if the code never mutates through it.
Q·Does Sesame's pointer obfuscation not already stop the leak?▾
It stops this particular leak: PCons store the data pointer XORed with a private constant, so serialization shows only an obfuscated address. But it costs 2× in Sesame's microbenchmarks and does not prevent other guarantee-breaking patterns, such as swapping a PCon's policy for a more permissive one. The paper argues that tooling which rules out encapsulation-breaking operations on PCons would make the overhead unnecessary and cover those other patterns too.
Q·If Sniffer finds nothing, is the extension proven correct?▾
Relative to what it checks, yes in the sense the paper states: Sniffer is sound, so any reachable code that actually violates the specified property is flagged, and no path from an annotated entity to a violating operation means the extension upholds the invariant. The guarantee is bounded by the specification (which properties, which types, which entry points) and by the encoded set of violating patterns. When Sniffer does flag something, a human still decides whether it is real.
Q·Where do Sniffer's false positives come from?▾
From over-approximating dynamic dispatch in the call graph, and from the per-property patterns being heuristic. In the Sesame run, 11 of 12 flags were false positives: 5 harmless uses of pointer arithmetic (iterators over a Vec<PCon>) and 6 due to dynamic dispatch, which a mechanical check that no PCon reaches the call site can dismiss.
Q·How does Sniffer differ from cargo-geiger or cargo-scan?▾
Syntactic tools such as cargo-geiger and count-unsafe flag every unsafe block: 33,735 regions for WebSubmit. cargo-scan tracks potentially dangerous effects across crate boundaries and reported 43,761 items when applied to each dependency separately; its cross-package mode uses an AST-level call graph that does not resolve trait implementations and dynamic dispatch into other crates, so the paper calls it unsound and not meaningfully comparable. Sniffer combines a sound call graph with the specific patterns that can violate the system's invariant, which brought the count to 12.
Q·Can Sniffer protect against an untrusted extension?▾
Not on its own. It is designed for cooperative extensions, whose developers run it and audit the flags. Untrusted extensions, as in NetBricks or Boucher et al., need near-perfect precision because no human resolves ambiguity. The paper proposes combining Sniffer with formal methods (local proof obligations for the few flagged regions) and runtime mechanisms applied only to flags it cannot dismiss, such as a check that rejects PCon arguments to a flagged transmute, or MPK protection for RedLeaf's shared heap.
Q·What does the evaluation not show?▾
It is preliminary by the paper's own description. Sniffer was run on one Sesame endpoint and on small RedLeaf excerpts (the full RedLeaf code base is incompatible with the compiler versions Sniffer supports), not on the other five systems in the table. The paper does not report Sniffer's analysis time or a user study of audit effort; audit burden is measured as the number of flagged locations.
unsafe code is sound if its author made it so.unsafe blockA region of Rust code, marked with the unsafe keyword, where the compiler permits operations it cannot check (such as dereferencing raw pointers) and the programmer takes responsibility for soundness.unsafe code. Callers cannot tell it from fully safe code without reading the source.unsafe anywhere, including all its dependencies. Locally safe code has none in its own top-level code but may call libraries that do.UnsafeCell, the primitive whose get() returns a mutable pointer to its contents; RefCell and Mutex wrap it with run-time checks or locking.mem::transmuteAn unsafe standard-library function that reinterprets the bits of a value as another type. It can expose private fields or extend a reference's lifetime.RRefable or the unstable Freeze.unsafe in a crate. Lints apply to the code they are enabled on, not to dependencies.Serialize itself.PCon<T,P>, holding sensitive data of type T and its privacy policy of type P as private fields, released only through Sesame's policy-checked API.dyn trait objects or function pointers are resolved conservatively to every possible target.