synvael/docs/chunk_streaming.md

Chunk streaming

How the server keeps the set of resident chunks in sync with where players are, and how chunk generation is kept off the simulation tick. The generating side lives in crates/shared/src/generator.rs; the streaming and storage side in crates/server/src/world_server.rs, driven from crates/server/src/main.rs.

Overview

Chunk residency is reconciled every tick against a desired set: the union of a cylinder of chunks around every player anchor. Chunks inside the desired set are made resident; chunks outside it are evicted. Generation of a missing chunk is expensive, so it is performed on a dedicated worker-thread pool rather than inline on the tick. Reconciliation is therefore non-blocking: each pass drains whatever chunks the pool has finished, evicts what is no longer wanted, and dispatches what is still missing, without ever waiting for a chunk to be generated.

Desired set

cylinder_chunks(center, radius, out) inserts every chunk position within the streaming cylinder around center into out. The shape is a disc in XZ (dx² + dz² ≤ radius²) extruded vertically to ±radius/2, reflecting the fact that horizontal view distance exceeds vertical.

Two producers build desired sets. During the startup loading gate, the ECS streaming system stream_chunks (in main.rs) queries every entity carrying Player, Position, and ViewDistance and unions each anchor's cylinder, pre-warming the origin region before the network is up. During steady-state play, the desired set is instead the union of every connected client's subscription (each a cylinder_chunks(center, radius) around its camera), assembled from the ClientStream map in the main loop — see Network delivery. In both cases the sets are unioned, so overlapping cylinders deduplicate automatically and a chunk is evicted only when no subscriber wants it. See Multiplayer below.

The worker pool

ServerWorld::new builds the pool once. Its structure:

• Generator sharing: the VoxelGenerator is wrapped in an Arc and a handle is cloned into every worker. generate_chunk(&self) is read-only, so no Mutex is required; the workers share one immutable generator.
• Job channel (main → workers): a crossbeam-channel carrying ChunkPos. crossbeam-channel is used rather than std::sync::mpsc because the pool needs multiple consumers: every worker clones the Receiver and pulls from the one shared queue, and each job is delivered to exactly one worker. std::sync::mpsc permits only a single consumer.
• Result channel (workers → main): a crossbeam-channel carrying (ChunkPos, Chunk). Each worker clones the Sender; the main thread holds the single Receiver.
• Worker loop: each worker blocks on job_rx.recv(), generates the chunk, and sends (pos, chunk) back. A blocking recv on a worker thread is acceptable because it is not the simulation thread.

Channel disconnection and shutdown

A crossbeam-channel reports disconnection (its recv returns Err) only once all senders — or, symmetrically, all receivers — have been dropped. After the spawn loop, the template job_rx and result_tx that were cloned from are dropped immediately. Retaining either would keep its channel open forever: workers would never observe job-channel shutdown, and the main thread would never observe the result channel closing. Worker JoinHandles are retained on ServerWorld for a future graceful-stop path that drops job_tx and joins the threads; the process currently relies on OS teardown at exit.

Reconcile: drain → evict → dispatch

ServerWorld::reconcile(&mut self, desired) runs three non-blocking phases per pass:

1. Drain. result_rx.try_recv() is pulled in a loop until empty (try_recv never blocks). Each returned position is removed from in_flight. A returned chunk is inserted into the resident map only if it is still in desired — see the eviction race.
2. Evict. Resident chunks absent from desired are removed. In-flight chunks that are no longer wanted need no handling here; the drain guard discards them when they arrive.
3. Dispatch. For every position in desired that is neither resident nor already in in_flight, the position is inserted into in_flight and sent on job_tx. The in_flight set is what prevents the same position being re-queued on every pass while a worker is still generating it.

in_flight therefore tracks positions dispatched but not yet returned, and is the single source of truth for "work outstanding."

The eviction race

Between a chunk being dispatched and the worker returning it, the anchor may move so that the chunk is no longer wanted. Without a guard, the drain phase would insert the now-unwanted chunk, resurrecting a chunk that the evict phase had already discarded (or would never be asked to discard, since it was never resident). The guard in phase 1 — insert only if desired.contains(&pos) — makes a late arrival harmless: an unwanted chunk is dropped on arrival rather than made resident.

Startup loading gate

Startup reuses the same worker pool and schedule; there is no separate synchronous loading path. Before granting player control, main runs the streaming schedule in a loop and polls ServerWorld::streaming_idle() (true when in_flight is empty). Once the initial region has at least one resident chunk and no work in flight, the region is ready. Waiting here is acceptable because no gameplay is running yet. During play the same reconcile runs every tick but is never waited on. A loading progress fraction is available as resident / (resident + in_flight).

Network delivery (client ↔ server)

Residency (above) keeps chunks in the server's memory; delivery streams them to each client. The two are decoupled: the reconcile pool does not know about clients, and delivery does not generate. Delivery is implemented in crates/net/src/chunk.rs (transport) and crates/server/src/client_stream.rs (per-client bookkeeping), driven from main.rs; the client side lives in crates/client/src/chunks.rs.

The chunk stream

After the handshake, the client opens one bidirectional QUIC stream (the canonical StreamLayout::chunk_lod0, stream 3) and the server accepts it, mirroring the control-stream convention. Both directions ride this one stream: client → server carries ChunkSubscribe { center, radius }, server → client carries ChunkMessage::{Chunk { pos, data }, Drop { pos }}. Frames use the existing length-prefixed postcard codec with a dedicated MAX_CHUNK_FRAME_LEN (1 MiB) cap, larger than the 64 KiB control cap.

The async/sync bridge

The QUIC pump is async on the network thread; the simulation loop (server) and winit loop (client) are synchronous. Two channels cross the boundary per connection, in opposite directions, and use different primitives for that reason:

• Inbound (ChunkSubscribe arriving async, consumed by the sync loop) reuses the crossbeam ServerEvent channel, surfaced as ServerEvent::ChunkSubscribe { id, request }. The async send is non-blocking; the sync loop drains with try_iter.
• Outbound (a ChunkMessage produced by the sync loop, consumed async) uses a tokio unbounded MPSC. Its send is synchronous, so the non-async loop pushes without a runtime, while the pump's recv().await composes into its tokio::select!. A blocking crossbeam receiver would freeze the current-thread runtime and cannot appear in a select! arm. The tokio sender is wrapped so neither server nor client names a tokio type: ChunkSink (server → client deliveries) and ChunkSubscriber (client → server subscriptions). See ADR-0010.

The server-side pump is chunk_stream_task; its client mirror is client_chunk_task. Each is one select! loop over "a frame arrived to read" and "a message is queued to write." The client's ClientLink bundles the handshake outcome, the ChunkSubscriber, and a crossbeam ChunkStream receiver of deliveries.

Per-client state and the diff

Each connected client is tracked by a ClientStream holding its ChunkSink, its current desired set (radius-clamped to SERVER_MAX_RADIUS), and its sent set. On each subscription, desired_diff(previous, new) yields the load list (new − previous) and drop list (previous − new); a ChunkMessage::Drop is emitted for every already-sent chunk that left the set. Newly-desired chunks are not sent immediately — chunk loads are async, so ClientStream::flush runs each tick and delivers every desired-but-unsent chunk that has since become resident, retrying on later ticks until the pool returns it.

Delivery is bounded by MAX_DELIVERIES_PER_TICK (32 chunks per client per tick). Encoding a chunk is the expensive part of flush, and a client whose subscription has just jumped can have hundreds of chunks pending at once; without a cap that backlog is encoded in a single tick and shows up directly as a tick overrun. The budget counts chunks actually encoded, so a tick where most of the desired set is still in flight is not charged for work it did not do. The fixed count is a placeholder for a time budget, which becomes necessary once per-chunk cost varies with LOD.

Self-contained payloads (all-air diff)

ChunkMessage::Chunk carries a ChunkData (the sparse, baseline-relative form; see ADR-0009). Because the client runs no worldgen (the server owns world content; worldgen never runs client-side), it cannot reconstruct a worldgen baseline to diff against. So delivered chunks are diffed against an all-air baseline (Chunk::default()): the edits become the chunk's full non-air content, and the client materializes each payload against its own all-air Chunk::default(). This makes every delivery self-contained, at the cost of not exploiting the deterministic baseline for compression — a compression concern deferred to the LOD/compression pass.

Client application

The client subscribes with its own LOAD_RADIUS (so the server's per-client resident set matches what the client keeps) whenever its center chunk changes. Deliveries are drained under per-frame budgets: a ChunkMessage::Chunk is materialized and the position (plus its six neighbours) is queued for meshing, while a ChunkMessage::Drop removes the mesh. Meshing itself runs on a worker pool rather than inline, so the client retains chunk voxels after upload; that pipeline is described in meshing.md. The client also evicts chunks outside LOAD_RADIUS locally, independent of the server Drop, so memory stays bounded even if the server is slow.

Multiplayer

Residency is a single shared pipeline: every client's subscription cylinder is unioned into one desired set, reconciled against one chunk store served by one worker pool, so a chunk is generated once no matter how many clients want it. Delivery, by contrast, is per-client: each ClientStream independently tracks what that client has been sent and diffs its own subscription (see Network delivery). A client joining or leaving is a ClientStream entering or leaving the map on the connect/disconnect events. Backpressure and fairness across clients (a bounded job channel, nearest-first priority, per-chunk ack/flow-control) remain deferred.

Level of detail

Each job is currently a full-detail (LOD0) chunk. When LOD is introduced, the job payload grows from ChunkPos to (ChunkPos, Lod); the worker-pool plumbing is LOD-agnostic and does not change.

Testing

The pure cylinder math and the async reconcile behaviour are unit-tested in world_server.rs:

• cylinder_chunks symmetry, boundary inclusion, and translation invariance.
• reconcile_converges_over_multiple_passes: an initial pass dispatches work and leaves nothing resident; repeated passes drain the pool until every desired position is resident.
• evicted_chunk_is_not_repopulated_on_arrival: a dispatched chunk that stops being wanted is discarded on arrival and never becomes resident. This test is timing-independent because every pass after dispatch reconciles against an empty desired set.