43 lines
5 KiB
Markdown
43 lines
5 KiB
Markdown
# Save format
|
||
|
||
How modified chunks are framed, stored, and read back from disk. This note describes the implementation.
|
||
|
||
The pure, in-memory framing (the `SYNR` region index and `SYNC` chunk records) lives in [`crates/shared/src/save/`](../crates/shared/src/save/). The filesystem side — reading a region file, mutating its chunks, and flushing it back crash-safely — lives in [`crates/server/src/save/`](../crates/server/src/save/). The runtime load path that turns a `ChunkPos` into a resident chunk lives in [`crates/server/src/world_server.rs`](../crates/server/src/world_server.rs) and is driven by the streaming reconcile loop documented in [`chunk_streaming.md`](chunk_streaming.md).
|
||
|
||
## What is persisted
|
||
|
||
Only chunks that diverge from their deterministic worldgen baseline are stored; the rationale is recorded in [ADR-0009](adr/0009-baseline-relative-sparse-chunk-persistence.md). A modified chunk is a [`ChunkData`](../crates/shared/src/world/chunk_data.rs): the chunk position, the `worldgen_version` its baseline is pinned to, and a sparse `local_index → BlockId` edit map. An unmodified chunk stores no voxel data and is absent from its region file.
|
||
|
||
## On-disk layout
|
||
|
||
Voxel storage is partitioned into **region files**, each covering a 32×32 grid of chunk columns in the XZ plane (the grid is 2D; Y is not partitioned). A chunk's region is `(cx.div_euclid(32), cz.div_euclid(32))` — `div_euclid`, not truncating division, so negative columns floor toward negative infinity rather than toward zero. All multi-byte integers are little-endian.
|
||
|
||
A region file is a `SYNR` index followed by the `SYNC` records the index points at.
|
||
|
||
- **`SYNR` region index** ([`region.rs`](../crates/shared/src/save/region.rs)): magic tag, framing version, and three side tables — a **header table** (`ChunkPos → offset+length+flags` for every resident record), a **free list** (reclaimable spans left by removed or shrunken records), and a **stamp table** (per-chunk `worldgen_version` exceptions; the region pins a `base_worldgen_version` and stores only chunks that differ from it). The header and stamp tables are `BTreeMap`s so their serialization order is deterministic.
|
||
- **`SYNC` chunk record** ([`record.rs`](../crates/shared/src/save/record.rs)): a fixed header (magic, chunk-format version, flags, `last_modified` timestamp in unix-ms, and the compressed and uncompressed payload lengths) followed by a zstd-compressed, postcard-serialized `ChunkData`. The header is never compressed, so a repair tool can read framing without decompressing. Compression is zstd level 3, favouring speed.
|
||
|
||
## Durability layer
|
||
|
||
[`RegionFile`](../crates/server/src/save/region_file.rs) reads a region file into memory, mutates its chunks (`write_chunk`, `remove_chunk`), and flushes it back. The flush strategy is a **whole-file atomic rewrite**: the complete file image is serialized, written to a `.tmp` sibling, fsynced, renamed over the target, and the containing directory is fsynced. This is a simpler alternative to an incremental append-plus-header-rewrite scheme; the deviation is noted at the `serialize` site and tracked for revision as follow-on work. The on-disk format is unchanged, so the switch requires no migration (the free list and absolute record offsets already support it).
|
||
|
||
## Concurrency: the save actor
|
||
|
||
Region files are owned by a single dedicated thread, the **save actor** ([`region_actor.rs`](../crates/server/src/save/region_actor.rs)). It holds the map of open `RegionFile`s and is their sole owner, so no region file needs a lock of its own. Worker threads never touch a region file directly; they hold cloned senders on the actor's request channel and communicate by message. This is the message-passing-over-shared-state concurrency stance from `AGENTS.md` applied to persistence: one queue thread serializes all region I/O, keeping it off both the simulation tick and the worker pool. A region file is opened on first access and its contents are served from memory thereafter.
|
||
|
||
## Load pipeline
|
||
|
||
A load of `ChunkPos` runs on the worker pool (off the tick thread), in [`world_server.rs`](../crates/server/src/world_server.rs) `load_chunk`:
|
||
|
||
1. The worker asks the save actor for the stored record at the position.
|
||
2. **Hit** (`Some(ChunkData)`): the baseline is regenerated (via the LRU baseline cache) and the stored diff is materialized over it.
|
||
3. **Miss** (`None`): the chunk was never modified, so the regenerated baseline is the chunk.
|
||
4. **Save-layer error**: streaming must not wedge, so the chunk falls back to a fresh baseline and the error is logged.
|
||
|
||
The regenerated baseline currently uses the current worldgen version rather than the record's stored `worldgen_version`; while a single version exists these coincide. Honouring the stored version on both load and write-back is follow-on work.
|
||
|
||
## Related decisions
|
||
|
||
- [ADR-0003](adr/0003-seed-deterministic-worldgen.md): seed-deterministic worldgen — the invariant that makes regen-on-load sound.
|
||
- [ADR-0009](adr/0009-baseline-relative-sparse-chunk-persistence.md): baseline-relative sparse persistence — why only diffs are stored.
|