Catalog
subetha-cxc master catalog
Every MMF-backed primitive in subetha-cxc, grouped by category,
with a one-line description and a “use when…” hint per type. The
Type column links to the per-category page where the primitive’s
prose doc lives; the Source column links to its canonical
in-source-tree .md (the per-type design doc).
For the alphabetised lookup (every name A-Z), see index-all . For the role-pair-driven selection guide, see Pick the right primitive .
Rings, stacks, and queues
Bounded, lock-free FIFO / LIFO / pub-sub structures.
| Type | What it is | Use when | Source |
|---|---|---|---|
SharedRing<P> | Cross-thread / cross-process lock-free MPMC ring | Multiple producers AND multiple consumers compete on one bounded queue | SHARED_RING.md |
SharedBroadcastRing | Single-producer, multi-consumer pub/sub ring | One process broadcasts events; many subscribers each consume the full stream independently | SHARED_BROADCAST_RING.md |
SharedTreiberStack<T> | Cross-process lock-free LIFO stack | LIFO ordering matters and contention is moderate; one CAS per push/pop | SHARED_TREIBER_STACK.md |
BlockingSpscRing | SPSC ring + 2 CrossProcessWaker for cross-process blocking send / recv | Single producer + single consumer want to park kernel-side instead of spinning when the ring is empty / full; cross-process safe on Linux via SHARED futex | blocking_spsc_ring.rs |
BlockingMpscRing | Composed-SPSC MPSC fan-in + per-ring producer wakers + shared consumer waker | N producers + 1 consumer want cross-process blocking semantics; per-producer FIFO; consumer parks on a shared waker any producer can fire | blocking_mpsc_ring.rs |
BlockingMpmcRing | Composed-SPSC MPMC grid + per-ring producer wakers + per-subset consumer wakers | N producers + M consumers want cross-process blocking semantics; each consumer owns a subset of rings and parks on its own waker | blocking_mpmc_ring.rs |
Maps, lists, and sequences
Keyed lookup and ordered storage.
| Type | What it is | Use when | Source |
|---|---|---|---|
SharedHashMap<K, V> | Cross-process open-addressed hash map | Key-value with O(1) lookup; FNV-1a hashing for cross-process determinism | SHARED_HASH_MAP.md |
SharedBTreeMap<K, V> | Cross-process ordered map via B-tree | Key-value with ordered iteration; range queries needed | SHARED_BTREE_MAP.md |
SharedLinkedList<T> | Cross-process doubly-linked list | Need stable iterator positions across mutations; not random access | SHARED_LINKED_LIST.md |
SharedVec<T> | Cross-process bounded indexable sequence | Push/index/pop with a known capacity ceiling | SHARED_VEC.md |
Atomics and cells
Scalar shared state.
| Type | What it is | Use when | Source |
|---|---|---|---|
SharedAtomicU32 / SharedAtomicU64 / SharedAtomicBool | Cross-process atomic counter / flag | Single integer or bool flag shared across processes; cheaper than any map | SHARED_ATOMIC.md |
SharedCell<T> | Cross-process single-value cell | One typed value updated atomically; reads and writes from any process | SHARED_CELL.md |
SharedOnceCell<T> | Cross-process init-once cell | Initialise a value exactly once; subsequent processes read the cached result | SHARED_ONCE_CELL.md |
SharedAsyncPointer<T> | Cross-process lazy / speculative pointer | Speculative reads; the first process to materialise wins, others race-free observe | SHARED_ASYNC_POINTER.md |
Caches
| Type | What it is | Use when | Source |
|---|---|---|---|
SharedLRUCache<K, V> | Cross-process LRU cache | Bounded keyed cache with eviction; shared by many processes | SHARED_LRU_CACHE.md |
Locks and synchronisation
Mutual-exclusion and rate-limiting primitives.
| Type | What it is | Use when | Source |
|---|---|---|---|
SharedRWLock | Cross-process reader-writer lock with writer preference | Many readers, occasional writer; readers must not block each other | SHARED_RW_LOCK.md |
SharedSemaphore | Cross-process counting semaphore | Bounded resource pool (N concurrent users); acquire / release pattern | SHARED_SEMAPHORE.md |
SharedRateLimiter | Cross-process token-bucket rate limiter | Throttle requests across many processes against one shared budget | SHARED_RATE_LIMITER.md |
SharedFenceClock | Hybrid Logical Clock (HLC) lifted to a process boundary | Need a monotonic timestamp that orders events across processes | SHARED_FENCE_CLOCK.md |
Probabilistic sketches
Approximate aggregations - sub-linear memory for the cardinality of values they see.
| Type | What it is | Use when | Source |
|---|---|---|---|
SharedBitVec | Cross-process bit-packed boolean array | Dense set membership over a known small key space | SHARED_BIT_VEC.md |
SharedBloomFilter | Cross-process probabilistic set membership | Approximate “has key X been seen?” with controlled false-positive rate | SHARED_BLOOM_FILTER.md |
SharedBlockedBloomFilter | Cache-blocked probabilistic set membership | Large-scale membership where one cache line per query matters (past L3) | SHARED_BLOCKED_BLOOM_FILTER.md |
SharedCountMinSketch | Cross-process probabilistic frequency counter | Approximate counts per key without keeping the keys themselves | SHARED_COUNT_MIN_SKETCH.md |
SharedHyperLogLog | Cross-process probabilistic distinct-count | Count unique elements with very low memory; merges across processes | SHARED_HYPER_LOG_LOG.md |
SharedHistogram | Cross-process bucketed counter | Latency / value distributions binned at fixed buckets | SHARED_HISTOGRAM.md |
SharedReservoirSampler<T> | Cross-process uniform random sample | Sample N items from an unknown-size stream | SHARED_RESERVOIR_SAMPLER.md |
Arenas and region storage
Pool allocators backed by an MMF.
| Type | What it is | Use when | Source |
|---|---|---|---|
SharedStringArena | Append-only position-independent string arena | Many small strings pooled in one MMF; refer to them by offset | SHARED_STRING_ARENA.md |
SharedHandleTable<T> | Cross-process ECS-style slotmap | Generational handles to slot-allocated entities; like an ECS world shared across processes | SHARED_HANDLE_TABLE.md |
SharedRegion<T> | Cross-process typed arena with position-independent pointers | Bulk allocation of T inside an MMF; offset pointers between regions | SHARED_REGION.md |
Ownership and election
Who-holds-the-token primitives.
| Type | What it is | Use when | Source |
|---|---|---|---|
OwnerLease<T> | Cross-process Mutex with auto-failover | Exclusive resource access where the holder might die; lease auto-reassigns | OWNER_LEASE.md |
SharedLeaderElection | Cross-process leader election | Exactly one process plays the leader role; auto-elect a replacement on death | SHARED_LEADER_ELECTION.md |
LazyConfig<T> | Thundering-herd-proof distributed config fetch | Many processes need the same config; only ONE actually fetches it; rest read | LAZY_CONFIG.md |
Liveness, failover, and barriers
Coordination across process boundaries.
| Type | What it is | Use when | Source |
|---|---|---|---|
HeartbeatTable | Per-process heartbeat slots in an MMF | Discover which peer processes are alive; the table backs failover | HEARTBEAT.md |
FailoverWatchdog | Scans the heartbeat table and reclaims work from dead peers | Reassign owner-leases / leader-roles when a process dies | FAILOVER.md |
EpochBarrier | Multi-process phase synchronisation | All N processes must finish phase K before any starts phase K+1 | EPOCH_BARRIER.md |
Work distribution
Higher-level coordination layered on the substrate.
| Type | What it is | Use when | Source |
|---|---|---|---|
EventStateLog<E, S> | Event-sourced state with cross-process replay | Append-only event log + materialised state; readers reconstruct from log | EVENT_STATE_LOG.md |
PriorityFanout | Tiered work queue with O(1) priority selection | N priority classes; consumers grab work from the highest non-empty class | PRIORITY_FANOUT.md |
ProgressTask<R> | Distributed work with live cross-process progress reporting | Long-running task split across processes; UI watches aggregated progress | PROGRESS_TASK.md |
BackgroundScheduler | Autonomous Pass executor backed by the MMF | Schedule periodic / triggered work; survives process restart | SCHEDULER.md |
pass_registry | Closure registry for cross-process Pass dispatch | Register handlers in process A; process B fires them via execute | PASS_REGISTRY.md |
CrossProcessWaker | Userspace-futex slot list in MMF. Every wait runs the hardware monitor tier first (MONITORX/UMONITOR on x86-64, WFE on aarch64); kernel parks are SHARED futex (Linux), non-PRIVATE _umtx_op (FreeBSD), os_sync_wait_on_address SHARED (macOS 14.4+), WaitOnAddress (Windows anon backings; cross-process Windows wakes ride the monitor tier) | Backs the Blocking{Spsc,Mpsc,Mpmc}Ring wrappers; usable directly by callers who need cross-process park / wake with a per-slot target sequence | cross_process_waker.rs |
SharedCondvar | Cross-process Mesa-style condition variable; one generation counter + CrossProcessWaker | Callers want condvar semantics across processes; predicate atom is caller-owned (any MMF-resident bool / counter); cross-process wake on Linux/WSL via SHARED futex | shared_condvar.rs |
BlockingSemaphore | Cross-process counting semaphore with kernel-park slow path | Callers want SharedSemaphore semantics but with zero CPU at idle and microsecond wake latency on release | blocking_semaphore.rs |
BlockingRWLock | Cross-process reader-writer lock with kernel-park slow path | Callers want SharedRWLock semantics with zero CPU at idle; readers and writers both park on the same waker | blocking_rw_lock.rs |
AsyncSpscRing | Future-shaped adapter on BlockingSpscRing | Callers want .recv().await / .send().await semantics with any async executor (tokio, smol, async-std, custom); short-lived std::thread per in-flight future bridges kernel-park to Rust Waker | async_ring.rs |
BlockingTcpBridge | TCP bridge whose forwarder uses recv_blocking / send_blocking via spawn_blocking | Callers want the existing TcpBridge’s wire format but with zero CPU at idle on both sides; replaces tokio::task::yield_now polling with cross-process futex park | blocking_tcp_bridge.rs |
Specialised data structures
Less common shapes for specific workloads.
| Type | What it is | Use when | Source |
|---|---|---|---|
SharedVersionedChain<T> | Cross-process MVCC linked list | Time-travel reads at a versioned snapshot; writers append new versions | SHARED_VERSIONED_CHAIN.md |
SharedTimePointTile<T> | BSPA + Versioned tile (16-slot snapshot-isolation scan) | Time-point queries over a small set of slots; SIMD lane mask scan | SHARED_TIME_POINT.md |
SharedNaNValue | 64-bit NaN-boxed heterogeneous value cell | Pack a small typed value (int / float / short string) into one f64 slot | SHARED_NAN_VALUE.md |
SharedNaNTaggedValue | NaN-boxed value where the pointer bits identify the payload type | Polymorphic value cell with no out-of-line type tag | SHARED_NAN_TAGGED_VALUE.md |
SharedGraph<N, E> | Cross-process directed graph | Cross-process graph adjacency; nodes and edges in one MMF | SHARED_GRAPH.md |
SharedUniversal<T> | Layer-2 cross-process container that adapts strategy | Single container that auto-picks among the IPC families based on observed load | SHARED_UNIVERSAL.md |
SharedTopologyMap | K_process axis observer + recommendation surface | Watch peer-process distribution; surface placement hints for cross-process work | SHARED_TOPOLOGY_MAP.md |
KTowerCascade<T, DEPTH> | Recursive pow2-of-pow2 cascading container | Multi-resolution indexed access; each tower level halves resolution | K_TOWER_CASCADE.md |
SharedUmbraPointer<T> | Cross-process content-prefixed pointer | Pointer comparisons that short-circuit on content prefix before deref | SHARED_UMBRA_POINTER.md |
IPC pointers (addressing primitives)
Low-level pointer types that other primitives compose into. Use these directly only when building a new MMF-backed type.
| Type | What it is | Use when | Source |
|---|---|---|---|
OffsetPtr<T> | File-relative offset pointer (no tag bits) | Pointing into the same MMF from another process; offset from base | OFFSET_PTR.md |
TaggedOffsetPtr<T, TAG_BITS> | High-bit-stealing tagged offset pointer | Same as OffsetPtr but you need to pack a small tag (state, type, generation) alongside the offset | TAGGED_OFFSET_PTR.md |
Polymorphic substrate (Locale x Protocol x Shape x Capacity x Ordering)
Cross-axis primitives that compose under one pin-protocol contract. Each entry’s “Use when” is the situation that the substrate’s default-composed stack does NOT cover automatically.
| Type | What it is | Use when | Source |
|---|---|---|---|
AdaptiveRing | Shape-morphing ring with all 4 shapes pre-allocated; peers register / unregister at runtime and the per-producer backings grow on demand (shared peer directory) | Default ring type; shape auto-morphs SPSC -> MPSC -> MPMC to the live peer counts, Vyukov on declaration; registration errors only under an explicit with_contract ceiling | adaptive_ring.rs |
| Adaptive ordering | Ordering axis on stamped AdaptiveRings: push stamps (TSC / counter / monotonic), cross-producer inversion metric, MMF-resident merge flag, strict watermark gate, single-drainer lease | Global FIFO as a runtime decision on the composed rings: flip the flag, the backlog orders retroactively | ordering.rs |
| Reorder consumer | Consumer-side EXACT delivery for the best-effort by-stamp merge: ReorderBuffer (bounded min-by-stamp, adaptive window that also widens with producer growth), ReorderingReceiver, AdaptiveOrderedReceiver (auto reorder-vs-MergeStrict) | You need exact global FIFO on a SharedCounter stamped ring without the strict merge’s slowest-producer tax | reorder.rs |
PeerDirectory | The AdaptiveRing’s shared topology substrate: producer / consumer slot bitmaps (claim / release / recycle), published backing count, MPMC ring-ownership table (claim / handoff / crash takeover via pid liveness), topology epoch | Consumed by AdaptiveRing automatically; reach for it directly when composing a new multi-peer primitive that needs cross-process peer accounting | peer_directory.rs |
LocaleAdaptiveRing | Three-locale wrapper (Anon / File / ShmFs) around AdaptiveRing; ships with LocaleAdaptiveRingSidecar + DefaultLocalePolicy for hysteresis-gated migrations | You want runtime morphability across storage locales | locale_adaptive_ring.rs |
CapacityAdaptiveRing | Runtime-resizable AdaptiveRing wrapper; ArcSwap state-swap + stale-list; ships with CapacityAdaptiveRingSidecar + DefaultCapacityPolicy (fill-ratio thresholds + hysteresis) | Workload’s queueing depth has wide dynamic range; sidecar-driven elastic capacity | capacity_adaptive_ring.rs |
CapacityBroadcastRing | Capacity-morph wrapper around SharedBroadcastRing; same ArcSwap state-swap with lag(idx) == 0 spin discipline; subscribers stay in lockstep across morphs | Elastic-capacity 1P/NC fan-out broadcast | capacity_broadcast_ring.rs |
CapacityPubSubRing + CapacityPubSubSubscriber | Capacity-morph wrapper around PubSubRing; chain-of-backings; subscribers carry (backing_idx, position) and advance through the chain | Elastic-capacity 1P/NC pub/sub with per-subscriber absolute positions | capacity_pubsub_ring.rs |
PubSubRing + PubSubSubscriber | One-publisher many-subscriber broadcast with per-subscriber positions | Independent subscribers walking the same producer stream at independent rates | protocol_pubsub.rs |
VirtualEndpoint + VirtualEndpointRegistry | Substrate-level identity that resolves to local or remote at runtime | Application code wants one addressing surface covering both same-host and cross-host peers | virtual_endpoint.rs |
QosPolicy + QosSnapshot | DDS-inspired runtime-mutable QoS knobs | Sidecar-driven morphs that depend on durability / reliability / history / latency wishes | qos_policy.rs |
RingContract | Declared ring contract: producer/consumer count ceilings, an ordering contract, and a capacity ceiling as one validated artifact; UNBOUNDED unless declared - the declared contract is the only source of registration errors on an AdaptiveRing | Pin a peer ceiling (the user override on the otherwise grow-on-demand ring), or pin an ordering contract the auto-morph cannot violate (a Fifo contract forbids the partitioned per-producer-lane shapes) | ring_contract.rs |
SubscriberPosition | MMF-resident position counter for resumable subscribers | Subscriber must survive a process restart + resume from its last position | replay_positions.rs |
ShmFile | Cross-platform named shared-memory backing | Building a custom cross-process primitive on top of named shm | shm_file.rs |
Cross-host bridges (Cargo features)
Substrate primitives that ferry bytes between two AdaptiveRing instances on different hosts. Gated behind Cargo features.
| Type | Cargo feature | Transport | Source |
|---|---|---|---|
QuicBridgeClient / QuicBridgeServer | quic-bridge | QUIC over UDP (TLS via rustls) | quic_bridge.rs |
TcpBridgeClient / TcpBridgeServer | tcp-bridge | Plain TCP | tcp_bridge.rs |
OS-specific substrate primitives
Primitives whose implementation is platform-gated but whose surface is shared across the targets each supports. Compiled away where unsupported; the workspace stays buildable everywhere.
| Type | Cargo gate | What it is | Source |
|---|---|---|---|
DirectFileRing | cfg(any(unix, windows)) | Non-mmap pread/pwrite ring with page-cache bypass: O_DIRECT (Linux/FreeBSD), F_NOCACHE (macOS), FILE_FLAG_NO_BUFFERING (Windows) | protocol_direct_file.rs |
fd_handoff::send_fd / recv_fd | cfg(any(unix, windows)) | SCM_RIGHTS fd passing over a Unix socket (unix, incl. macOS); DuplicateHandle (Windows) | fd_handoff.rs |
HugepageRegion | cfg(target_os = "linux") | MAP_HUGETLB anon mmap (2 MB or 1 GB pages) | hugepages.rs |
VsockSocket | cfg(any(target_os = "linux", windows)) | AF_VSOCK SOCK_STREAM for host-VM byte streaming | locale_vsock.rs |
WireSocket | wire-locale feature (Linux / Windows / FreeBSD / macOS) | Raw-L2 NIC-bypass socket: AF_XDP (Linux), XDP (Windows), netmap (FreeBSD), BPF (macOS) | locale_wire.rs |
Two further OS-specific primitives, referenced here by source: SuperPageRegion
(super_pages.rs
,
cfg(any(target_os = "freebsd", target_os = "macos"))) - the superpage anon
mmap (FreeBSD MAP_ALIGNED_SUPER, macOS x86_64 VM_FLAGS_SUPERPAGE_SIZE_2MB)
that backs AdaptiveRing::create_hugepage on those OSes; and KernelAsyncRing
(kernel_async_ring.rs
,
cfg(any(target_os = "linux", windows, target_os = "freebsd", target_os = "macos")))
- the kernel async-I/O ring (io_uring on Linux, IoRing on Windows, POSIX
aioon FreeBSD / macOS).
Windows-only substrate primitives
OS-specific primitives gated on cfg(windows).
| Type | Cargo gate | What it is | Source |
|---|---|---|---|
LargePageRegion | cfg(windows) | VirtualAlloc(MEM_LARGE_PAGES) private memory (2 MB pages); Windows parity for HugepageRegion | large_pages.rs |
LargePageSection | cfg(windows) | SEC_LARGE_PAGES named pagefile-backed section: cross-process large-page sharing by section name (huge memory tables shared between processes) | large_pages.rs |
See also
- Alphabetical index - every name A-Z without category grouping.
- Pick the right primitive - role-pair-driven selection.
subetha-pointersreference - the exotic-pointer sibling family.- Architecture - where this family sits in the four-crate stack.
- The MMF substrate - why one byte layout serves three deployment modes.