Skip to content

Async Engine

block_on, reactor, executors, WakerRing

Rust Runtime

The async conventions on Channel / AdaptiveIpc are plain std::future::Futures. They run on tokio, smol, async-std, or the crate’s own runtime-free driver, and a suspended consumer never costs a thread per future. This page documents the engine the futures ride on.

reactor::block_on - the runtime-free driver

pub fn block_on<F: Future>(future: F) -> F::Output;

Drives one future to completion on the calling thread, parking the thread (not spinning) while the future is Pending. No runtime, no dependency. It is what lets the substrate’s async surface run with no tokio in the build:

use subetha_cxc::reactor::block_on;

let sum = block_on(async {
    let mut s = 0u64;
    for _ in 0..n {
        s += chan.recv_async().await?;
    }
    Ok::<_, subetha_cxc::ApiError>(s)
})?;

The same recv_async() future also drives from inside a #[tokio::main] runtime, so the substrate sits under an existing async app when you want it to (tokio_interop ).

The reactor - bridging a shared-memory wake to a Waker

A future parked on a ring needs something to fire its Waker when an item arrives. In-process, the producer’s push fires it directly. Across a process boundary, a per-receiver reactor thread blocks on the shared CrossProcessWaker and fires the local Waker when the producing process publishes.

    flowchart LR
  subgraph PA["Process A - producer"]
    TS["ReactiveSender::try_send"]
  end
  subgraph SH["Shared memory-mapped files"]
    RING[("SPSC ring")]
    XW[("CrossProcessWaker")]
  end
  subgraph PB["Process B - consumer"]
    RT["reactor thread<br/>blocked on the waker"]
    WK["local Waker"]
    RX["recv_async&lpar;&rpar;.await<br/>suspended task"]
    RT --> WK --> RX
  end
  TS -- push bytes --> RING
  TS -- wake_up_to --> XW
  XW -- unblocks --> RT
  RING -. bytes read on resume .-> RX
  classDef shared fill:#1e3a8a,color:#fff
  classDef thread fill:#0f766e,color:#fff
  class RING,XW shared
  class RT,WK,RX thread
  
ItemSignatureRole
anon_pair(capacity)-> Result<(ReactiveSender, ReactiveReceiver), RingError>In-process pair over an anonymous ring; the sender’s push wakes the receiver’s local Waker directly.
sender_cross(ring, xwaker)-> ReactiveSenderCross-process producer half over a shared MMF ring + named waker.
receiver_cross(ring, xwaker)-> ReactiveReceiverCross-process consumer half; spawns the reactor thread that turns a shared-memory wake into a local Waker fire.

ReactiveSender::try_send(payload: &[u8]) -> Result<(), RingError> pushes and signals (the signal is sent only on a successful push); published() -> u64 is the producer’s published count. ReactiveReceiver::recv() -> ReactiveRecv is the awaitable future, resolving to [u8; SPSC_PAYLOAD_BYTES]; try_recv(out: &mut [u8]) -> Result<usize, RingError> drains without awaiting. The ReactiveRecv future owns Arc clones, so it is Send + 'static and can be spawned onto any executor rather than awaited in place.

This bridge is what Channel / AdaptiveIpc use internally: the first recv_async() / send_async() call spins up the reactor so a wake crosses the process boundary; an in-process wake takes the direct path.

WakerRing - the minimal awaitable ring

The smallest async ring: one SPSC lane with a Waker cell. It is the primitive the fixed-pool scaling story is built on, with no marshalling or shape dispatch in the way.

pub fn create_anon_pair(capacity) -> Result<(WakerProducer, WakerConsumer), RingError>;

capacity is a precondition, not an argument to be validated: it must be a power of two and at least 2, and the underlying SpscRingCore asserts rather than returning Err. The Result covers the mapping, not the capacity. Round before you call if the number comes from configuration - AutoIpc::capacity does that for you on the high-level path.

  • WakerProducer::try_push(payload: &[u8]) -> Result<(), RingError> - push and fire the consumer’s registered Waker.
  • WakerConsumer::recv() -> WakerRecv - a Future resolving to [u8; SPSC_PAYLOAD_BYTES].
  • WakerConsumer::try_recv(out: &mut [u8]) -> Result<usize, RingError> - drain without awaiting.

Executors - driving many futures on few threads

Two bounded executors drive awaiting tasks without a thread per task.

TaskPool

A fixed pool of worker threads with a shared ready queue.

MethodPurpose
TaskPool::new(n_workers)Spawn n_workers worker threads.
spawn(future)Enqueue a Future<Output = ()> + Send + 'static.
worker_count()Worker thread count.
shutdown(self)Stop the workers and join.

RingExecutor

A work-stealing executor whose per-worker ready queues are themselves Vyukov rings, with workers pinned to cores.

MethodPurpose
RingExecutor::new(n_workers, max_tasks)Construct with an explicit worker count and task ceiling.
RingExecutor::with_available_parallelism(max_tasks)Construct sized to available_parallelism.
spawn(future)Enqueue a Future<Output = ()> + Send + 'static.
pending()Tasks not yet complete.
worker_count() / shard_count() / pinned_workers() / shard_capacity()Topology inspection.
wait_idle()Block until the executor drains.
shutdown(self)Stop the workers and join.

Cross-host: net_bridge

The async wake crosses a machine boundary over blocking std::net sockets, with no async runtime in the build.

FunctionSignatureRole
shipship(addr: SocketAddr, producer_ring: Arc<SpscRingCore>, n_items: u64) -> io::Result<()>Drain a producer ring to a remote addr.
serve_oneserve_one(listener: &TcpListener, consumer_ring: &Arc<SpscRingCore>, xwaker: &Arc<CrossProcessWaker>) -> io::Result<u64>Accept one connection and feed arriving bytes into a consumer ring, firing the waker so a parked recv_async() resolves.

E2E proofs

  • channel_async - block_on driving sync + blocking + async, 300k items.
  • xproc_async_ring - a recv_async().await parked in one process, woken by a push from a separate process through the reactor bridge.
  • net_async_ring - the same await woken by a TCP packet through net_bridge, on blocking std::net sockets.
  • async_subscriber_scale - 10,000 WakerRing consumers on a TaskPool, no thread per subscriber.
  • ring_async_executor - the RingExecutor driving awaiting tasks.

See also