docs(server): document chunk streaming and async worker pipeline

docs/chunk_streaming.md

@@ -0,0 +1,60 @@
+# Chunk streaming
+
+How the server keeps the set of resident chunks in sync with where players are, and how chunk generation is kept off the simulation tick. The generating side lives in [`crates/shared/src/generator.rs`](../crates/shared/src/generator.rs); the streaming and storage side in [`crates/server/src/world_server.rs`](../crates/server/src/world_server.rs), driven from [`crates/server/src/main.rs`](../crates/server/src/main.rs).
+
+## Overview
+
+Chunk residency is *reconciled* every tick against a **desired set**: the union of a cylinder of chunks around every player anchor. Chunks inside the desired set are made resident; chunks outside it are evicted. Generation of a missing chunk is expensive, so it is performed on a dedicated worker-thread pool rather than inline on the tick. Reconciliation is therefore non-blocking: each pass *drains* whatever chunks the pool has finished, *evicts* what is no longer wanted, and *dispatches* what is still missing, without ever waiting for a chunk to be generated.
+
+## Desired set
+
+`cylinder_chunks(center, radius, out)` inserts every chunk position within the streaming cylinder around `center` into `out`. The shape is a disc in XZ (`dx² + dz² ≤ radius²`) extruded vertically to `±radius/2`, reflecting the fact that horizontal view distance exceeds vertical.
+
+The ECS streaming system `stream_chunks` (in `main.rs`) queries every entity carrying `Player`, `Position`, and `ViewDistance`, and unions each anchor's cylinder into one `HashSet<ChunkPos>`. Because the sets are unioned, overlapping cylinders deduplicate automatically and a chunk is evicted only when *no* player wants it. See [Multiplayer](#multiplayer) below.
+
+## The worker pool
+
+`ServerWorld::new` builds the pool once. Its structure:
+
+- **Generator sharing:** the `VoxelGenerator` is wrapped in an `Arc` and a handle is cloned into every worker. `generate_chunk(&self)` is read-only, so no `Mutex` is required; the workers share one immutable generator.
+- **Job channel (main → workers):** a `crossbeam-channel` carrying `ChunkPos`. `crossbeam-channel` is used rather than `std::sync::mpsc` because the pool needs **multiple consumers**: every worker clones the `Receiver` and pulls from the one shared queue, and each job is delivered to exactly one worker. `std::sync::mpsc` permits only a single consumer.
+- **Result channel (workers → main):** a `crossbeam-channel` carrying `(ChunkPos, Chunk)`. Each worker clones the `Sender`; the main thread holds the single `Receiver`.
+- **Worker loop:** each worker blocks on `job_rx.recv()`, generates the chunk, and sends `(pos, chunk)` back. A blocking `recv` on a worker thread is acceptable because it is not the simulation thread.
+
+### Channel disconnection and shutdown
+
+A `crossbeam-channel` reports disconnection (its `recv` returns `Err`) only once *all* senders — or, symmetrically, all receivers — have been dropped. After the spawn loop, the template `job_rx` and `result_tx` that were cloned from are dropped immediately. Retaining either would keep its channel open forever: workers would never observe job-channel shutdown, and the main thread would never observe the result channel closing. Worker `JoinHandle`s are retained on `ServerWorld` for a future graceful-stop path that drops `job_tx` and joins the threads; the process currently relies on OS teardown at exit.
+
+## Reconcile: drain → evict → dispatch
+
+`ServerWorld::reconcile(&mut self, desired)` runs three non-blocking phases per pass:
+
+1. **Drain.** `result_rx.try_recv()` is pulled in a loop until empty (`try_recv` never blocks). Each returned position is removed from `in_flight`. A returned chunk is inserted into the resident map **only if it is still in `desired`** — see [the eviction race](#the-eviction-race).
+2. **Evict.** Resident chunks absent from `desired` are removed. In-flight chunks that are no longer wanted need no handling here; the drain guard discards them when they arrive.
+3. **Dispatch.** For every position in `desired` that is neither resident nor already in `in_flight`, the position is inserted into `in_flight` and sent on `job_tx`. The `in_flight` set is what prevents the same position being re-queued on every pass while a worker is still generating it.
+
+`in_flight` therefore tracks positions dispatched but not yet returned, and is the single source of truth for "work outstanding."
+
+### The eviction race
+
+Between a chunk being dispatched and the worker returning it, the anchor may move so that the chunk is no longer wanted. Without a guard, the drain phase would insert the now-unwanted chunk, resurrecting a chunk that the evict phase had already discarded (or would never be asked to discard, since it was never resident). The guard in phase 1 — insert only if `desired.contains(&pos)` — makes a late arrival harmless: an unwanted chunk is dropped on arrival rather than made resident.
+
+## Startup loading gate
+
+Startup reuses the *same* worker pool and schedule; there is no separate synchronous loading path. Before granting player control, `main` runs the streaming schedule in a loop and polls `ServerWorld::streaming_idle()` (true when `in_flight` is empty). Once the initial region has at least one resident chunk and no work in flight, the region is ready. Waiting here is acceptable because no gameplay is running yet. During play the same reconcile runs every tick but is **never** waited on. A loading progress fraction is available as `resident / (resident + in_flight)`.
+
+## Multiplayer
+
+No per-player streaming pipeline exists. Every player anchor's cylinder is unioned into one desired set, reconciled against one chunk store served by one worker pool. A player joining or leaving is simply an entity entering or leaving the ECS query; it requires no streaming-specific code. The only per-player concern is the loading gate, which for a joining player checks that player's cylinder against the resident set rather than the global set. Backpressure and fairness across players (a bounded job channel, nearest-first priority) are shared-pipeline concerns deferred for later.
+
+## Level of detail
+
+Each job is currently a full-detail (LOD0) chunk. When LOD is introduced, the job payload grows from `ChunkPos` to `(ChunkPos, Lod)`; the worker-pool plumbing is LOD-agnostic and does not change.
+
+## Testing
+
+The pure cylinder math and the async reconcile behaviour are unit-tested in `world_server.rs`:
+
+- `cylinder_chunks` symmetry, boundary inclusion, and translation invariance.
+- `reconcile_converges_over_multiple_passes`: an initial pass dispatches work and leaves nothing resident; repeated passes drain the pool until every desired position is resident.
+- `evicted_chunk_is_not_repopulated_on_arrival`: a dispatched chunk that stops being wanted is discarded on arrival and never becomes resident. This test is timing-independent because every pass after dispatch reconciles against an empty desired set.