Synvael/docs/chunk_streaming.md

Chunk streaming

This covers how the server keeps resident chunks in sync with player positions and how we keep chunk generation completely off the main simulation tick. The generation code lives in crates/shared/src/generator.rs, while the streaming and storage logic is in crates/server/src/world_server.rs, driven from crates/server/src/main.rs.

Overview

Every tick, the server reconciles chunk residency against a desired set, which is the union of cylinders around every player's anchor. If a chunk is in the desired set, we make it resident; if it falls outside, we evict it. Because generating a missing chunk is expensive, we hand that work off to a dedicated thread pool rather than blocking the tick. This makes the reconciliation process entirely non-blocking: each pass drains whatever the pool has finished, evicts chunks that are no longer needed, and dispatches requests for anything still missing, all without ever waiting for a chunk to finish generating.

Desired set

The cylinder_chunks(center, radius, out) function takes every chunk position within the streaming cylinder around the center and inserts it into out. This shape is a horizontal disc (dx² + dz² ≤ radius²) extruded vertically to ±radius/2. We use this shape because horizontal view distance matters much more than vertical.

We build these desired sets in two places. During the startup loading gate, the stream_chunks ECS system queries every entity that has Player, Position, and ViewDistance components, unioning each anchor's cylinder to pre-warm the origin region before the network is even up. During steady-state play, the desired set becomes the union of every connected client's active subscription (calculated as cylinder_chunks(center, radius) around its camera), which we assemble from the ClientStream map in the main loop. In both cases, overlapping cylinders deduplicate automatically because we union the sets, meaning a chunk only gets evicted when absolutely no subscriber wants it anymore. See the Multiplayer section for more on this.

The worker pool

The ServerWorld::new function builds the worker pool once. Here is how it is structured:

• Generator sharing: We wrap the VoxelGenerator in an Arc and clone a handle into every worker. Because generate_chunk(&self) is read-only, we don't need a Mutex; all workers share a single immutable generator safely.
• Job channel (main to workers): This is a crossbeam-channel that carries ChunkPos. We specifically use crossbeam instead of std::sync::mpsc because the pool requires multiple consumers. Every worker clones the Receiver to pull from a single shared queue, and each job goes to exactly one worker. The standard library's mpsc only allows a single consumer.
• Result channel (workers to main): Another crossbeam-channel carrying (ChunkPos, Chunk). Every worker clones the Sender, and the main thread holds the single Receiver.
• Worker loop: Each worker blocks on job_rx.recv(), generates the requested chunk, and sends the result back. Blocking on recv here is perfectly fine since this runs on a worker thread, not the main simulation thread.

Channel disconnection and shutdown

A crossbeam-channel will only report disconnection (returning an Err on recv) when all senders or receivers have been completely dropped. Right after spawning the pool, we immediately drop the original job_rx and result_tx templates. If we held onto them, the channels would stay open forever, meaning the workers would never realize the job channel shut down and the main thread would never see the result channel close. We do keep the worker JoinHandles on ServerWorld for a future graceful shutdown path, but right now the process just relies on the OS tearing down threads on exit.

Reconcile: drain, evict, dispatch

The ServerWorld::reconcile(&mut self, desired) function runs three non-blocking phases during each pass:

1. Drain: We pull from result_rx.try_recv() in a loop until it is empty (which never blocks). We remove each returned position from the in_flight set. A returned chunk only gets inserted into the resident map if it is still present in the desired set (see the eviction race).
2. Evict: Any resident chunks that are no longer in the desired set are removed. If an in-flight chunk is no longer wanted, we don't need to handle it here; the drain phase will naturally discard it when it finally arrives.
3. Dispatch: For every position in the desired set that isn't resident and isn't already in_flight, we insert it into in_flight and send it down job_tx. The in_flight set ensures we don't spam the same position into the queue on every single pass while a worker is busy generating it.

This makes in_flight our single source of truth for tracking work that has been dispatched but hasn't returned yet.

The eviction race

While a chunk is off being generated by a worker, the player might move away, meaning the chunk is no longer wanted. If we weren't careful, the drain phase would insert this unwanted chunk into the map, effectively resurrecting a chunk that the evict phase had already tossed out. By guarding the drain phase (only inserting if desired.contains(&pos)), a late arrival is totally harmless. The unwanted chunk just gets dropped on arrival instead of becoming resident.

Startup loading gate

Startup uses the exact same worker pool and schedule; we intentionally avoid building a separate synchronous loading path. Before handing control over to the player, main runs the streaming schedule in a tight loop and polls ServerWorld::streaming_idle() (which returns true when in_flight is empty). Once the starting region has at least one resident chunk and zero work in flight, it is ready to go. It is perfectly fine to wait here because actual gameplay hasn't started yet. Once the game is running, this exact same reconcile logic runs every tick but is never waited on. We calculate loading progress as a simple fraction: resident / (resident + in_flight).

Network delivery (client and server)

While residency keeps chunks loaded in the server's memory, delivery is responsible for streaming those chunks to clients. These concepts are strictly decoupled. The reconcile pool knows nothing about clients, and the delivery system never generates chunks. Delivery logic lives in crates/net/src/chunk.rs for transport and crates/server/src/client_stream.rs for server-side bookkeeping. The client counterpart is in crates/client/src/chunks.rs.

The chunk stream

Once the handshake finishes, the client opens a single bidirectional QUIC stream (specifically stream 3, StreamLayout::chunk_lod0) and the server accepts it. Both directions ride on this one stream. The client sends ChunkSubscribe { center, radius } to the server, and the server replies with ChunkMessage::{Chunk { pos, data }, Drop { pos }}. These frames use our length-prefixed postcard codec, capped at a dedicated 1 MiB limit (MAX_CHUNK_FRAME_LEN), which is much larger than the 64 KiB control limit.

The async/sync bridge

The QUIC network pump is completely async, but the server simulation and client winit loops are totally synchronous. To bridge this gap, we use two channels per connection (one for each direction) and pick different primitives based on the direction:

• Inbound: A ChunkSubscribe arrives via async and is consumed by the sync loop. We route this through the crossbeam ServerEvent channel, surfacing it as ServerEvent::ChunkSubscribe { id, request }. The async send is non-blocking, and the sync loop simply drains it with try_iter.
• Outbound: A ChunkMessage is produced by the sync loop and consumed via async. We use an unbounded MPSC from tokio for this. The synchronous send works without needing an async runtime, while the pump can await the recv() cleanly inside a tokio::select!. If we used a blocking crossbeam receiver here, it would freeze the current-thread runtime and wouldn't work inside a select!. We wrap the tokio sender in ChunkSink (server to client) and ChunkSubscriber (client to server) so neither the client nor server code ever explicitly names a tokio type (refer to ADR-0010).

The server runs this in chunk_stream_task while the client runs it in client_chunk_task. Each one is just a select! loop evaluating whether a frame arrived to read or a message is queued to write. The client bundles this into a ClientLink containing the handshake outcome, the ChunkSubscriber, and the crossbeam receiver for incoming deliveries.

Per-client state and the diff

Every connected client gets a ClientStream tracker holding its ChunkSink, its current desired set (clamped to SERVER_MAX_RADIUS), and a record of what it has already been sent. When a subscription updates, we calculate the diff (new - previous for loads, previous - new for drops). For every chunk in the drop list that was previously sent, we emit a ChunkMessage::Drop.

However, we do not send newly desired chunks immediately. Chunk generation is async, so ClientStream::flush checks each tick and delivers any newly resident chunks that the client wants but hasn't received yet. If the chunk isn't ready, it simply waits and tries again on a future tick.

We bound this delivery to MAX_DELIVERIES_PER_TICK (currently 32 chunks per client per tick) to prevent lag spikes. Encoding chunks is expensive. If a client teleports and suddenly needs hundreds of chunks, trying to encode them all at once would immediately blow out the tick budget. Importantly, this limit only applies to chunks we actually encode. If most of the desired set is still generating in the worker pool, we don't penalize the tick budget for work that hasn't happened yet. This fixed chunk count is a stopgap until we implement a proper time-based budget, which will be necessary once chunk costs start varying by LOD.

Self-contained payloads (all-air diff)

The ChunkMessage::Chunk payload carries a ChunkData in a sparse, baseline-relative format (detailed in ADR-0009). Because the client does absolutely zero worldgen (the server is strictly authoritative), it has no way to reconstruct the worldgen baseline to diff against. Instead, we diff all delivered chunks against a completely empty all-air baseline (Chunk::default()). The edits effectively become the chunk's entire non-air content, and the client reconstructs the chunk by applying those edits to its own empty baseline. This keeps every delivery completely self-contained. While we sacrifice the compression benefits of diffing against the true worldgen baseline, we're deferring heavy compression work to the future LOD pass.

Client application

Whenever the client's center chunk changes, it subscribes using its own LOAD_RADIUS. This ensures the server's per-client resident set perfectly matches what the client intends to keep. The client drains incoming deliveries under its own per-frame budgets. When a ChunkMessage::Chunk arrives, it materializes the data and queues the position (along with its six neighbors) for meshing. When a ChunkMessage::Drop arrives, it discards the mesh. Since meshing runs on a separate worker pool, the client safely holds onto the chunk voxels even after the mesh is uploaded (this pipeline is covered in meshing.md). Crucially, the client proactively evicts chunks outside its LOAD_RADIUS on its own. It doesn't strictly wait for the server's Drop message, ensuring memory usage stays strictly bounded even if the server lags behind.

Multiplayer

Residency operates as a single, shared pipeline. We union every client's subscription cylinder into one massive desired set, which is then reconciled against a single chunk store and a single worker pool. This guarantees a chunk is only generated once, regardless of how many clients requested it.

Delivery, however, is strictly per-client. Each ClientStream independently tracks what that specific client has received and diffs against its personal subscription. When a client connects or disconnects, its ClientStream is simply added to or removed from the map. Features like backpressure, fairness across clients, and per-chunk flow control are deferred for now.

Level of detail

Currently, every job processes a full-detail LOD0 chunk. When we eventually introduce LODs, the job payload will just grow from ChunkPos to (ChunkPos, Lod). The entire worker-pool plumbing is already LOD-agnostic and won't need to change.

Testing

We unit-test the pure cylinder math and async reconcile behavior inside world_server.rs:

• cylinder_chunks is tested for symmetry, boundary inclusion, and translation invariance.
• reconcile_converges_over_multiple_passes verifies that an initial pass dispatches work but leaves nothing resident, while subsequent passes drain the pool until everything is resident.
• evicted_chunk_is_not_repopulated_on_arrival confirms our eviction race guard works. If a chunk stops being wanted while it's in flight, it gets discarded upon arrival and never enters the resident set. This test works without any timing hacks because subsequent passes simply reconcile against an empty desired set.