synvael/docs/save_format.md

# Save format

This document explains how modified chunks are framed, stored, and read back from disk.

The purely in-memory framing logic (handling the `SYNR` region index and `SYNC` chunk records) lives inside [`crates/shared/src/save/`](../crates/shared/src/save/). The filesystem operations (reading a region file, mutating its chunks, and flushing it back crash-safely) live in [`crates/server/src/save/`](../crates/server/src/save/). The actual runtime load path that turns a `ChunkPos` into a resident chunk is in [`crates/server/src/world_server.rs`](../crates/server/src/world_server.rs), driven by the streaming reconcile loop we documented in [`chunk_streaming.md`](chunk_streaming.md).

## What is persisted

We only store chunks that diverge from their deterministic worldgen baseline. (The full rationale for this is recorded in [ADR-0009](adr/0009-baseline-relative-sparse-chunk-persistence.md)). A modified chunk is saved as a [`ChunkData`](../crates/shared/src/world/chunk_data.rs), which contains the chunk position, the `worldgen_version` its baseline is pinned to, and a sparse map of edits (`local_index → BlockId`). If a chunk hasn't been modified, it stores absolutely no voxel data and won't even appear in its region file.

## On-disk layout

We partition voxel storage into **region files**. Each file covers a 32×32 grid of chunk columns in the XZ plane (this grid is 2D; we don't partition the Y axis). You can calculate a chunk's region using `(cx.div_euclid(32), cz.div_euclid(32))`. Notice we use `div_euclid` instead of standard truncating division; this ensures negative columns correctly floor toward negative infinity rather than snapping toward zero. We store all multi-byte integers as little-endian.

A region file consists of a single `SYNR` index, followed by all the `SYNC` records that the index points to.

- **`SYNR` region index** ([`region.rs`](../crates/shared/src/save/region.rs)): This contains a magic tag, the framing version, and three side tables. The **header table** maps `ChunkPos` to `offset + length + flags` for every resident record. The **free list** tracks reclaimable file spans left behind by removed or shrunken records. The **stamp table** tracks per-chunk `worldgen_version` exceptions (the region pins a `base_worldgen_version` globally and only stores exceptions for chunks that differ from it). Both the header and stamp tables are `BTreeMap`s, ensuring their serialization order is perfectly deterministic.
- **`SYNC` chunk record** ([`record.rs`](../crates/shared/src/save/record.rs)): This starts with a fixed header (magic, chunk-format version, flags, a `last_modified` Unix-ms timestamp, and the compressed/uncompressed payload lengths). Following the header is a zstd-compressed, postcard-serialized `ChunkData` payload. We intentionally never compress the header itself, so repair tools can parse the framing without having to decompress the entire file. We use zstd level 3 for compression to heavily favor speed.

## Durability layer

The `RegionFile` struct ([`region_file.rs`](../crates/server/src/save/region_file.rs)) reads a region file into memory, applies chunk mutations (`write_chunk`, `remove_chunk`), and flushes it back to disk.

Right now, our flush strategy is a **whole-file atomic rewrite**. We serialize the complete file image, write it to a `.tmp` sibling file, `fsync` it, rename it directly over the target file, and then `fsync` the containing directory. This is vastly simpler than trying to build an incremental append-plus-header-rewrite scheme. We've noted this deviation right at the `serialize` call site and logged it for future revision. Because the on-disk format itself remains unchanged (the free list and absolute record offsets already fully support incremental appends), switching to an incremental strategy later will require absolutely no save migrations.

## Concurrency and the save actor

Region files are owned by a single dedicated thread: the **save actor** ([`region_actor.rs`](../crates/server/src/save/region_actor.rs)). It holds the map of all open `RegionFile`s and acts as their sole owner, meaning individual region files don't need their own locks.

Worker threads never touch a region file directly. Instead, they hold cloned senders for the actor's request channel and communicate purely by message passing. This directly applies the "message-passing over shared-state" rule from `DEVELOPMENT.md` to our persistence layer. Having one queue thread serialize all region I/O keeps that heavy lifting entirely off both the simulation tick and the main worker pool. A region file is opened lazily upon first access, and its contents are served directly from memory after that.

## Load pipeline

When we need to load a `ChunkPos`, the work runs on the worker pool (safely off the tick thread) via `load_chunk` in [`world_server.rs`](../crates/server/src/world_server.rs):

1. The worker asks the save actor for the stored record at the requested position.
2. **Hit (`Some(ChunkData)`):** We regenerate the chunk's baseline (using the LRU baseline cache) and materialize the stored diff straight over it.
3. **Miss (`None`):** The chunk was never modified, so the freshly regenerated baseline *is* the final chunk.
4. **Save-layer error:** Streaming must never wedge the game, so if reading fails, the chunk safely falls back to a fresh baseline and we log the error.

Right now, the regenerated baseline uses the *current* worldgen version rather than the record's specific stored `worldgen_version`. This is fine while only a single version exists, but honoring the stored version on both load and write-back is tracked as follow-on work.

## Related decisions

- [ADR-0003](adr/0003-seed-deterministic-worldgen.md): Covers seed-deterministic worldgen, which is the foundational invariant that makes regeneration-on-load mathematically sound.
- [ADR-0009](adr/0009-baseline-relative-sparse-chunk-persistence.md): Covers baseline-relative sparse persistence and explains exactly why we only store diffs.