# Chunk streaming How the server keeps the set of resident chunks in sync with where players are, and how chunk generation is kept off the simulation tick. The generating side lives in [`crates/shared/src/generator.rs`](../crates/shared/src/generator.rs); the streaming and storage side in [`crates/server/src/world_server.rs`](../crates/server/src/world_server.rs), driven from [`crates/server/src/main.rs`](../crates/server/src/main.rs). ## Overview Chunk residency is *reconciled* every tick against a **desired set**: the union of a cylinder of chunks around every player anchor. Chunks inside the desired set are made resident; chunks outside it are evicted. Generation of a missing chunk is expensive, so it is performed on a dedicated worker-thread pool rather than inline on the tick. Reconciliation is therefore non-blocking: each pass *drains* whatever chunks the pool has finished, *evicts* what is no longer wanted, and *dispatches* what is still missing, without ever waiting for a chunk to be generated. ## Desired set `cylinder_chunks(center, radius, out)` inserts every chunk position within the streaming cylinder around `center` into `out`. The shape is a disc in XZ (`dx² + dz² ≤ radius²`) extruded vertically to `±radius/2`, reflecting the fact that horizontal view distance exceeds vertical. Two producers build desired sets. During the **startup loading gate**, the ECS streaming system `stream_chunks` (in `main.rs`) queries every entity carrying `Player`, `Position`, and `ViewDistance` and unions each anchor's cylinder, pre-warming the origin region before the network is up. During **steady-state play**, the desired set is instead the union of every connected client's subscription (each a `cylinder_chunks(center, radius)` around its camera), assembled from the `ClientStream` map in the main loop — see [Network delivery](#network-delivery-client--server). In both cases the sets are unioned, so overlapping cylinders deduplicate automatically and a chunk is evicted only when *no* subscriber wants it. See [Multiplayer](#multiplayer) below. ## The worker pool `ServerWorld::new` builds the pool once. Its structure: - **Generator sharing:** the `VoxelGenerator` is wrapped in an `Arc` and a handle is cloned into every worker. `generate_chunk(&self)` is read-only, so no `Mutex` is required; the workers share one immutable generator. - **Job channel (main → workers):** a `crossbeam-channel` carrying `ChunkPos`. `crossbeam-channel` is used rather than `std::sync::mpsc` because the pool needs **multiple consumers**: every worker clones the `Receiver` and pulls from the one shared queue, and each job is delivered to exactly one worker. `std::sync::mpsc` permits only a single consumer. - **Result channel (workers → main):** a `crossbeam-channel` carrying `(ChunkPos, Chunk)`. Each worker clones the `Sender`; the main thread holds the single `Receiver`. - **Worker loop:** each worker blocks on `job_rx.recv()`, generates the chunk, and sends `(pos, chunk)` back. A blocking `recv` on a worker thread is acceptable because it is not the simulation thread. ### Channel disconnection and shutdown A `crossbeam-channel` reports disconnection (its `recv` returns `Err`) only once *all* senders — or, symmetrically, all receivers — have been dropped. After the spawn loop, the template `job_rx` and `result_tx` that were cloned from are dropped immediately. Retaining either would keep its channel open forever: workers would never observe job-channel shutdown, and the main thread would never observe the result channel closing. Worker `JoinHandle`s are retained on `ServerWorld` for a future graceful-stop path that drops `job_tx` and joins the threads; the process currently relies on OS teardown at exit. ## Reconcile: drain → evict → dispatch `ServerWorld::reconcile(&mut self, desired)` runs three non-blocking phases per pass: 1. **Drain.** `result_rx.try_recv()` is pulled in a loop until empty (`try_recv` never blocks). Each returned position is removed from `in_flight`. A returned chunk is inserted into the resident map **only if it is still in `desired`** — see [the eviction race](#the-eviction-race). 2. **Evict.** Resident chunks absent from `desired` are removed. In-flight chunks that are no longer wanted need no handling here; the drain guard discards them when they arrive. 3. **Dispatch.** For every position in `desired` that is neither resident nor already in `in_flight`, the position is inserted into `in_flight` and sent on `job_tx`. The `in_flight` set is what prevents the same position being re-queued on every pass while a worker is still generating it. `in_flight` therefore tracks positions dispatched but not yet returned, and is the single source of truth for "work outstanding." ### The eviction race Between a chunk being dispatched and the worker returning it, the anchor may move so that the chunk is no longer wanted. Without a guard, the drain phase would insert the now-unwanted chunk, resurrecting a chunk that the evict phase had already discarded (or would never be asked to discard, since it was never resident). The guard in phase 1 — insert only if `desired.contains(&pos)` — makes a late arrival harmless: an unwanted chunk is dropped on arrival rather than made resident. ## Startup loading gate Startup reuses the *same* worker pool and schedule; there is no separate synchronous loading path. Before granting player control, `main` runs the streaming schedule in a loop and polls `ServerWorld::streaming_idle()` (true when `in_flight` is empty). Once the initial region has at least one resident chunk and no work in flight, the region is ready. Waiting here is acceptable because no gameplay is running yet. During play the same reconcile runs every tick but is **never** waited on. A loading progress fraction is available as `resident / (resident + in_flight)`. ## Network delivery (client ↔ server) Residency (above) keeps chunks in the server's memory; **delivery** streams them to each client. The two are decoupled: the reconcile pool does not know about clients, and delivery does not generate. Delivery is implemented in [`crates/net/src/chunk.rs`](../crates/net/src/chunk.rs) (transport) and [`crates/server/src/client_stream.rs`](../crates/server/src/client_stream.rs) (per-client bookkeeping), driven from `main.rs`; the client side lives in [`crates/client/src/chunks.rs`](../crates/client/src/chunks.rs). ### The chunk stream After the handshake, the client opens one **bidirectional** QUIC stream (the canonical `StreamLayout::chunk_lod0`, stream 3) and the server accepts it, mirroring the control-stream convention. Both directions ride this one stream: client → server carries `ChunkSubscribe { center, radius }`, server → client carries `ChunkMessage::{Chunk { pos, data }, Drop { pos }}`. Frames use the existing length-prefixed `postcard` codec with a dedicated `MAX_CHUNK_FRAME_LEN` (1 MiB) cap, larger than the 64 KiB control cap. ### The async/sync bridge The QUIC pump is async on the network thread; the simulation loop (server) and winit loop (client) are synchronous. Two channels cross the boundary per connection, in opposite directions, and use different primitives for that reason: - **Inbound** (`ChunkSubscribe` arriving async, consumed by the sync loop) reuses the `crossbeam` `ServerEvent` channel, surfaced as `ServerEvent::ChunkSubscribe { id, request }`. The async `send` is non-blocking; the sync loop drains with `try_iter`. - **Outbound** (a `ChunkMessage` produced by the sync loop, consumed async) uses a **`tokio` unbounded MPSC**. Its `send` is synchronous, so the non-async loop pushes without a runtime, while the pump's `recv().await` composes into its `tokio::select!`. A blocking `crossbeam` receiver would freeze the current-thread runtime and cannot appear in a `select!` arm. The tokio sender is wrapped so neither `server` nor `client` names a tokio type: `ChunkSink` (server → client deliveries) and `ChunkSubscriber` (client → server subscriptions). See [ADR-0010](adr/0010-net-crate-async-runtime.md). The server-side pump is `chunk_stream_task`; its client mirror is `client_chunk_task`. Each is one `select!` loop over "a frame arrived to read" and "a message is queued to write." The client's `ClientLink` bundles the handshake outcome, the `ChunkSubscriber`, and a `crossbeam` `ChunkStream` receiver of deliveries. ### Per-client state and the diff Each connected client is tracked by a `ClientStream` holding its `ChunkSink`, its current desired set (radius-clamped to `SERVER_MAX_RADIUS`), and its `sent` set. On each subscription, `desired_diff(previous, new)` yields the load list (`new − previous`) and drop list (`previous − new`); a `ChunkMessage::Drop` is emitted for every already-**sent** chunk that left the set. Newly-desired chunks are **not** sent immediately — chunk loads are async, so `ClientStream::flush` runs each tick and delivers every desired-but-unsent chunk that has since become resident, retrying on later ticks until the pool returns it. Delivery is bounded by `MAX_DELIVERIES_PER_TICK` (32 chunks per client per tick). Encoding a chunk is the expensive part of `flush`, and a client whose subscription has just jumped can have hundreds of chunks pending at once; without a cap that backlog is encoded in a single tick and shows up directly as a tick overrun. The budget counts chunks **actually encoded**, so a tick where most of the desired set is still in flight is not charged for work it did not do. The fixed count is a placeholder for a time budget, which becomes necessary once per-chunk cost varies with LOD. ### Self-contained payloads (all-air diff) `ChunkMessage::Chunk` carries a `ChunkData` (the sparse, baseline-relative form; see [ADR-0009](adr/0009-baseline-relative-sparse-chunk-persistence.md)). Because the client runs **no** worldgen (the server owns world content; worldgen never runs client-side), it cannot reconstruct a worldgen baseline to diff against. So delivered chunks are diffed against an **all-air baseline** (`Chunk::default()`): the edits become the chunk's full non-air content, and the client materializes each payload against its own all-air `Chunk::default()`. This makes every delivery self-contained, at the cost of not exploiting the deterministic baseline for compression — a compression concern deferred to the LOD/compression pass. ### Client application The client subscribes with its own `LOAD_RADIUS` (so the server's per-client resident set matches what the client keeps) whenever its center chunk changes. Deliveries are drained under per-frame budgets: a `ChunkMessage::Chunk` is materialized and the position (plus its six neighbours) is queued for meshing, while a `ChunkMessage::Drop` removes the mesh. Meshing itself runs on a worker pool rather than inline, so the client retains chunk voxels after upload; that pipeline is described in [`meshing.md`](meshing.md). The client **also** evicts chunks outside `LOAD_RADIUS` locally, independent of the server `Drop`, so memory stays bounded even if the server is slow. ## Multiplayer Residency is a single shared pipeline: every client's subscription cylinder is unioned into one desired set, reconciled against one chunk store served by one worker pool, so a chunk is generated once no matter how many clients want it. **Delivery**, by contrast, is per-client: each `ClientStream` independently tracks what that client has been sent and diffs its own subscription (see [Network delivery](#network-delivery-client--server)). A client joining or leaving is a `ClientStream` entering or leaving the map on the connect/disconnect events. Backpressure and fairness across clients (a bounded job channel, nearest-first priority, per-chunk ack/flow-control) remain deferred. ## Level of detail Each job is currently a full-detail (LOD0) chunk. When LOD is introduced, the job payload grows from `ChunkPos` to `(ChunkPos, Lod)`; the worker-pool plumbing is LOD-agnostic and does not change. ## Testing The pure cylinder math and the async reconcile behaviour are unit-tested in `world_server.rs`: - `cylinder_chunks` symmetry, boundary inclusion, and translation invariance. - `reconcile_converges_over_multiple_passes`: an initial pass dispatches work and leaves nothing resident; repeated passes drain the pool until every desired position is resident. - `evicted_chunk_is_not_repopulated_on_arrival`: a dispatched chunk that stops being wanted is discarded on arrival and never becomes resident. This test is timing-independent because every pass after dispatch reconciles against an empty desired set.