Skip to content
Choose a structure

Choose a structure

Choose a structure

The module exports 135 cmdlets, and most scripts need two or three of them. This page goes the other way round from the reference : name the shape you have, and read off what fits it.

Start at the front door

Four cmdlets pick the shape underneath from what you describe. If one of them matches, take it and stop reading; the families below are for when you want to choose yourself.

You wantTake
a queue between processes that a reader can wait onNew-SubEthaChannel
work one process owns, that idle processes help withNew-SubEthaWorkQueue
a lookup from one number to anotherNew-SubEthaKvMap
a queue, and you cannot predict the trafficNew-SubEthaAdaptiveQueue

New-SubEthaAdaptiveQueue counts the sizes it is sent and moves between a ring and a work-stealing deque while it runs. Reach for it when the shape is genuinely unknown, not as a hedge: a shape you do know is cheaper named outright.

Moving items between processes

Answer two questions: how many processes write, and does every reader need every item.

WritersReadersEvery reader sees everythingTake
oneone-New-SubEthaSpscRing
onemanyyesNew-SubEthaBroadcastRing
manyone-New-SubEthaMpscPool
manymanyno, items are shared outNew-SubEthaMpmcGrid
onemanyyes, and a slow reader must be told what it missedNew-SubEthaPubSub
anyanyyou would rather not decideNew-SubEthaRing

New-SubEthaRing is the adaptive ring and the reasonable default: it registers producers and consumers by id, carries payloads bigger than a slot through SendFrame and RecvFrame, and changes shape under the traffic it sees.

The constructors that serve several ends hand out every end at once, because the shape is what makes them correct. Say how many of each you want when you build it:

$pool = New-SubEthaMpscPool -Path C:\ipc\pool -Producers 4 -Capacity 4096
$pool.Producers[0].Send($bytes)
$pool.Consumer.Recv()

$grid = New-SubEthaMpmcGrid -Path C:\ipc\grid -Producers 4 -Consumers 2 -Capacity 4096

The Lamport pair is the one that writes out as two objects rather than one holding both, producer first:

$p, $c = New-SubEthaLamportPair -Path C:\ipc\lamport -Capacity 4096

PubSub is the one to take when falling behind must be visible. A subscriber that drops too far back gets an error with id SubEthaLagged naming what it lost, rather than silently resuming at whatever survived.

When order matters

A ring does not promise that items from different senders arrive in the order their senders made them. If that matters, say so when you build it and read through an ordered receiver:

$ring = New-SubEthaRing -Path C:\ipc\events -Capacity 4096 -Stamps Counter
$ordered = $ring.OrderedReceiver($consumer)
$ordered.Drain()

OrderedReceiver refuses a ring built without stamps rather than waiting forever for an order it can never establish. Use Drain rather than Recv: Recv answers $null both when the window is still filling and when the ring is empty, and Drain takes the ring and the held-back tail together so the two cases never need telling apart.

For items that come from somewhere other than a ring, New-SubEthaReorderWindow is the same window standing alone.

Holding state rather than moving items

You wantTake
one number several processes updateNew-SubEthaAtomic
one value replaced wholeNew-SubEthaCell
an indexable array of fixed-size elementsNew-SubEthaVec
numbered slots that come and goNew-SubEthaSlab
a keyed lookup where entries are removedNew-SubEthaHashMap
the same, in key orderNew-SubEthaBTreeMap
a block of memory you address yourselfNew-SubEthaRegion

KvMap from the front door has no removal at all, which is why HashMap exists: reach for KvMap when keys only ever accumulate, and HashMap the moment anything has to go away.

Vec and Slab are seqlocked, so a reader retries rather than blocks. Reading a run of them goes through ReadRange, which honors the retry on every element and moves them packed end to end in one call.

When readers must not see a half-finished write

The versioned family keeps history so a reader gets one unchanging view while writers carry on:

You wantTake
one value’s past versionsNew-SubEthaVersionChain
numbered slots, each with recent historyNew-SubEthaVersionedSlab
a keyed index scanned while it is writtenNew-SubEthaVersionedMap
the same, with several writers at onceNew-SubEthaLanedMap

Reading takes a pin, and the pin is what holds the view still. The reading methods are on the pin rather than on the structure, which is what makes it impossible to read without one:

$pin = $map.Pin()
try {
    $pin.Scan(0, 1000, 100)      # low, high, limit; a SubEtha.Entry each
} finally {
    $pin.Release()
}

LanedMap splits the index across several trees so writers do not queue behind one another. A key belongs to one lane for its whole life; writing it through another is an error named SubEthaWrongLane that names the right lane. Claim with ClaimLaneFor($key) when the key already exists and ClaimLane() when it does not. Reading claims nothing.

Two names in this family mean what the Rust says rather than what they sound like. VoidEpoch does not reclaim: it undoes the writes stamped at exactly one epoch, for a writer that died partway through. Sweep and SweepSlot reclaim.

Making processes wait for each other

You wantTake
exclusive or shared access to somethingNew-SubEthaRWLock
at most N processes doing something at onceNew-SubEthaSemaphore
to wait until a condition holdsWait-SubEthaCondition
one process to compute a value the rest useNew-SubEthaLazyValue
one process to own something, another to take over when it diesNew-SubEthaOwnerLease
everyone to reach the same point before continuingNew-SubEthaEpochBarrier
to know which processes are aliveNew-SubEthaHeartbeat
to agree which process leadsNew-SubEthaLeaderElection

Every waiting form that takes a timeout sleeps rather than spins, so a long wait costs no processor, and none of them waits past its deadline even if whoever holds the lock never gives it back.

The condition variable is a cmdlet rather than a method because its condition is a script block, and a script block runs only on the pipeline thread:

Wait-SubEthaCondition -Path C:\ipc\cv -Until { $ready.Load() -eq 1 } -Timeout 5

Counting and sampling without keeping everything

BloomFilter and BlockedBloomFilter for set membership, CountMinSketch for frequencies, HyperLogLog for distinct counts, Histogram for distributions, Reservoir for a bounded unbiased sample, RateLimiter for a budget spent across processes.

Size the two filters and the sketch from what you can tolerate rather than guessing:

Measure-SubEthaBloomSize -Items 10000 -FalsePositiveRate 0.01
Measure-SubEthaSketchSize -Epsilon 0.01 -Delta 0.01

Still not sure

Take New-SubEthaChannel if items move and New-SubEthaHashMap if state sits. Both are ordinary, both are easy to replace once the shape is clearer, and neither commits you to anything the others do not.

Where to go next

  • Make it fast once the structure is right and the per-item cost is not.
  • The reference for every method on whichever one you picked.
  • Get-Help New-SubEthaRing -Full for the parameters and an example, without leaving the shell.