docs(server): add save-format subsystem note
This commit is contained in:
parent
6310c384e5
commit
b89b73b277
|
|
@ -46,5 +46,6 @@ Subsystem notes:
|
|||
- [`packs.md`](packs.md): data packs & resource packs (load order, layout, resolution).
|
||||
- [`rendering.md`](rendering.md): rendering & coordinate gotchas (Vulkan clip space, Blender/glTF import).
|
||||
- [`chunk_streaming.md`](chunk_streaming.md): chunk streaming and async worker pipeline.
|
||||
- [`save_format.md`](save_format.md): chunk persistence, region-file layout, save actor, and load pipeline.
|
||||
|
||||
Further subsystem notes are added here as systems are implemented and locked.
|
||||
|
|
|
|||
42
docs/save_format.md
Normal file
42
docs/save_format.md
Normal file
|
|
@ -0,0 +1,42 @@
|
|||
# Save format
|
||||
|
||||
How modified chunks are framed, stored, and read back from disk. This note describes the implementation.
|
||||
|
||||
The pure, in-memory framing (the `SYNR` region index and `SYNC` chunk records) lives in [`crates/shared/src/save/`](../crates/shared/src/save/). The filesystem side — reading a region file, mutating its chunks, and flushing it back crash-safely — lives in [`crates/server/src/save/`](../crates/server/src/save/). The runtime load path that turns a `ChunkPos` into a resident chunk lives in [`crates/server/src/world_server.rs`](../crates/server/src/world_server.rs) and is driven by the streaming reconcile loop documented in [`chunk_streaming.md`](chunk_streaming.md).
|
||||
|
||||
## What is persisted
|
||||
|
||||
Only chunks that diverge from their deterministic worldgen baseline are stored; the rationale is recorded in [ADR-0009](adr/0009-baseline-relative-sparse-chunk-persistence.md). A modified chunk is a [`ChunkData`](../crates/shared/src/world/chunk_data.rs): the chunk position, the `worldgen_version` its baseline is pinned to, and a sparse `local_index → BlockId` edit map. An unmodified chunk stores no voxel data and is absent from its region file.
|
||||
|
||||
## On-disk layout
|
||||
|
||||
Voxel storage is partitioned into **region files**, each covering a 32×32 grid of chunk columns in the XZ plane (the grid is 2D; Y is not partitioned). A chunk's region is `(cx.div_euclid(32), cz.div_euclid(32))` — `div_euclid`, not truncating division, so negative columns floor toward negative infinity rather than toward zero. All multi-byte integers are little-endian.
|
||||
|
||||
A region file is a `SYNR` index followed by the `SYNC` records the index points at.
|
||||
|
||||
- **`SYNR` region index** ([`region.rs`](../crates/shared/src/save/region.rs)): magic tag, framing version, and three side tables — a **header table** (`ChunkPos → offset+length+flags` for every resident record), a **free list** (reclaimable spans left by removed or shrunken records), and a **stamp table** (per-chunk `worldgen_version` exceptions; the region pins a `base_worldgen_version` and stores only chunks that differ from it). The header and stamp tables are `BTreeMap`s so their serialization order is deterministic.
|
||||
- **`SYNC` chunk record** ([`record.rs`](../crates/shared/src/save/record.rs)): a fixed header (magic, chunk-format version, flags, `last_modified` timestamp in unix-ms, and the compressed and uncompressed payload lengths) followed by a zstd-compressed, postcard-serialized `ChunkData`. The header is never compressed, so a repair tool can read framing without decompressing. Compression is zstd level 3, favouring speed.
|
||||
|
||||
## Durability layer
|
||||
|
||||
[`RegionFile`](../crates/server/src/save/region_file.rs) reads a region file into memory, mutates its chunks (`write_chunk`, `remove_chunk`), and flushes it back. The flush strategy is a **whole-file atomic rewrite**: the complete file image is serialized, written to a `.tmp` sibling, fsynced, renamed over the target, and the containing directory is fsynced. This is a simpler alternative to an incremental append-plus-header-rewrite scheme; the deviation is noted at the `serialize` site and tracked for revision as follow-on work. The on-disk format is unchanged, so the switch requires no migration (the free list and absolute record offsets already support it).
|
||||
|
||||
## Concurrency: the save actor
|
||||
|
||||
Region files are owned by a single dedicated thread, the **save actor** ([`region_actor.rs`](../crates/server/src/save/region_actor.rs)). It holds the map of open `RegionFile`s and is their sole owner, so no region file needs a lock of its own. Worker threads never touch a region file directly; they hold cloned senders on the actor's request channel and communicate by message. This is the message-passing-over-shared-state concurrency stance from `AGENTS.md` applied to persistence: one queue thread serializes all region I/O, keeping it off both the simulation tick and the worker pool. A region file is opened on first access and its contents are served from memory thereafter.
|
||||
|
||||
## Load pipeline
|
||||
|
||||
A load of `ChunkPos` runs on the worker pool (off the tick thread), in [`world_server.rs`](../crates/server/src/world_server.rs) `load_chunk`:
|
||||
|
||||
1. The worker asks the save actor for the stored record at the position.
|
||||
2. **Hit** (`Some(ChunkData)`): the baseline is regenerated (via the LRU baseline cache) and the stored diff is materialized over it.
|
||||
3. **Miss** (`None`): the chunk was never modified, so the regenerated baseline is the chunk.
|
||||
4. **Save-layer error**: streaming must not wedge, so the chunk falls back to a fresh baseline and the error is logged.
|
||||
|
||||
The regenerated baseline currently uses the current worldgen version rather than the record's stored `worldgen_version`; while a single version exists these coincide. Honouring the stored version on both load and write-back is follow-on work.
|
||||
|
||||
## Related decisions
|
||||
|
||||
- [ADR-0003](adr/0003-seed-deterministic-worldgen.md): seed-deterministic worldgen — the invariant that makes regen-on-load sound.
|
||||
- [ADR-0009](adr/0009-baseline-relative-sparse-chunk-persistence.md): baseline-relative sparse persistence — why only diffs are stored.
|
||||
Loading…
Reference in a new issue