Worldshaper/docs/OPEN_WORLD_CHUNK_V1.md

# Open World Chunking V1

This document proposes a concrete v1 data model and editor/runtime architecture for replacing the current finite `map` model with a streamed open world.

The goal is not "fake endless maps glued together." The goal is:

- one continuous world coordinate space
- chunked storage and loading
- seamless editing across chunk boundaries
- sparse height overrides
- chunk-local save/write operations
- renderer and editor surfaces that only load and draw what is nearby

## Plain-English Model

Think of the world as graph paper that goes on forever.

- The player and editor work in one global coordinate space: `(worldX, worldY)`.
- That graph paper is cut into fixed-size square sheets called `chunks`.
- Each chunk stores only the tiles, height patches, and instances that belong to that square.
- The game and editor only load the nearby chunks.

The user should never feel like they are "switching maps." They should feel like they are scrolling around one world.

## V1 Recommendation

V1 should introduce a new top-level concept: `world`.

Current:

- `maps.json`
- one map entry owns rows, layers, height layers, instances

Proposed:

- `worlds.json`
- one world entry defines chunk rules and metadata
- chunk files hold actual terrain/layer/instance data

Keep support for multiple worlds. Even in an "open world" game, multiple worlds are still useful for:

- overworld
- interiors or special dimensions
- test worlds
- developer sandboxes

## Goals

- Seamless editing across arbitrary world coordinates
- Automatic chunk creation when painting or placing far away
- No giant monolithic world file for tile data
- Chunk-local save/load for performance
- Reuse current room-layer and height-patch ideas where possible
- Replace minimap with world navigation, coordinates, and bookmarks

## Non-Goals For V1

- Infinite procedural terrain generation
- Networked world streaming
- Region compression or advanced binary storage
- Fully general LOD terrain synthesis
- Rewriting the whole engine to 3D or voxel storage

## Chosen Chunk Size

The chosen v1 chunk size is `32x32`.

Reason:

- simpler mental math in tooling and code
- slightly better fit for common viewport batching
- cleaner chunk index math
- easier future optimization

`25x25` would also work, but v1 should standardize on `32x32` and avoid making chunk size configurable until the new architecture is stable.

## Coordinate Model

The world uses two coordinate spaces:

1. World coordinates
- Absolute tile coordinates in the world
- Example: `(73, -12)`

2. Chunk-local coordinates
- Tile coordinates within a chunk
- Example: chunk `(2, -1)` local tile `(9, 20)`

Conversion:

```txt
chunkX = floor(worldX / chunkWidth)
chunkY = floor(worldY / chunkHeight)

localX = worldX - (chunkX * chunkWidth)
localY = worldY - (chunkY * chunkHeight)
```

This is the core of the whole system.

## Proposed File Structure

```txt
content/
  worlds.json
  worlds/
    overworld/
      world.json
      bookmarks.json
      chunks/
        0_0.json
        1_0.json
        0_1.json
        -1_0.json
```

### `content/worlds.json`

Index of available worlds.

```json
{
  "schemaVersion": 1,
  "worlds": [
    {
      "id": "overworld",
      "name": "Overworld",
      "worldDir": "worlds/overworld"
    }
  ]
}
```

### `content/worlds/overworld/world.json`

World-level metadata only.

```json
{
  "schemaVersion": 1,
  "id": "overworld",
  "name": "Overworld",
  "chunkWidth": 32,
  "chunkHeight": 32,
  "tileSize": 32,
  "defaultBackgroundTileId": "tile_grass_01",
  "spawn": { "x": 0, "y": 0 },
  "editor": {
    "defaultZoom": 1,
    "gridVisible": true
  }
}
```

### `content/worlds/overworld/bookmarks.json`

Replaces the current minimap's "jump around the finite map" role with saved navigation targets.

```json
{
  "schemaVersion": 1,
  "worldId": "overworld",
  "bookmarks": [
    { "id": "town_center", "label": "Town Center", "x": 120, "y": 84 },
    { "id": "north_tower", "label": "North Tower", "x": 145, "y": 32 }
  ]
}
```

### Chunk file

```json
{
  "schemaVersion": 1,
  "worldId": "overworld",
  "chunkX": 2,
  "chunkY": -1,
  "width": 32,
  "height": 32,
  "backgroundTileId": "tile_grass_01",
  "roomLayers": [
    {
      "layer": 0,
      "name": "Ground",
      "rows": ["................................", "..."],
      "instanceIds": []
    },
    {
      "layer": 1,
      "name": "Walls",
      "rows": ["                                ", "..."],
      "instanceIds": ["inst_gatekeeper_001"]
    }
  ],
  "heightLayers": [
    {
      "id": "height_001",
      "name": "Tower Level 1",
      "z": 1,
      "x": 10,
      "y": 8,
      "rows": [
        " 4444 ",
        " 4  4 ",
        " 4444 "
      ]
    }
  ],
  "instances": [
    {
      "id": "inst_gatekeeper_001",
      "templateId": "npc_gatekeeper_bubbles",
      "layer": 1,
      "x": 12,
      "y": 9,
      "record": {
        "name": "Bubbles",
        "spriteId": "npc_human_style_13",
        "faction": "dangerous_gatekeeper"
      }
    }
  ]
}
```

## Why This Structure

This splits data by ownership:

- `world.json`: rules and metadata
- `bookmarks.json`: editor navigation helpers
- chunk files: actual editable world content

That keeps save operations local and avoids turning one file into a bottleneck.

## Editor Data Model

The editor runtime should stop treating one "map" as the active document.

Instead it should have:

```ts
type OpenWorldSession = {
  worldId: string;
  chunkWidth: number;
  chunkHeight: number;
  tileSize: number;
  cameraWorldX: number;
  cameraWorldY: number;
  loadedChunks: Record<string, WorldChunk>;
  dirtyChunks: Record<string, boolean>;
  activeLayer: number;
  activeHeightLayerId: string;
  editingTargetKind: "room" | "height";
  bookmarks: WorldBookmark[];
};
```

The main change is that the editor should think in `world coordinates` first and `chunk ownership` second.

## Editing Rules

### Painting tiles

When the user paints:

1. Convert pointer position to `(worldX, worldY)`
2. Resolve chunk and local coordinate
3. Create chunk if it does not exist
4. Edit the right chunk layer row
5. Mark that chunk dirty
6. Patch only the visible surfaces touched by the stroke

If a rectangle, circle, or fill crosses chunk boundaries, the brush simply touches multiple chunks in one action.

### Placing instances

When the user drops an instance:

1. Resolve world tile
2. Resolve chunk
3. Create chunk if missing
4. Add instance to that chunk
5. Mark chunk dirty

### Painting far away

Yes, the editor should create all needed empty chunks automatically if the user paints or places content in a region that has never been created before.

It should not create a giant range of intermediate chunks just because the camera moved there. Chunks should be created lazily:

- create on first edit
- optionally create on explicit "stamp empty region" tooling

## Runtime Load Model

The game should not load the whole world.

Instead:

1. Determine the player's current chunk
2. Load a chunk radius around the player
3. Keep nearby chunks hot
4. Unload distant chunks

Example:

- player is in chunk `(10, 4)`
- active chunk radius is `1`
- load chunks from `(9..11, 3..5)`

That gives a `3x3` neighborhood.

This is the same core idea the editor should use for panning and visible drawing.

## Renderer Model

The renderer should operate in visible chunk slices, not full-world arrays.

Suggested v1 pipeline:

1. Determine visible world tile bounds from camera
2. Determine which chunks intersect that area
3. Ensure those chunks are loaded
4. Draw visible base layers from those chunks
5. Apply active height patches for current Z
6. Draw instances
7. Draw overlays and cursors

## Surface Caching

The current tile-surface work already points in the right direction.

V1 should move to:

- one cached surface per visible chunk layer
- one cached patch surface per height patch
- redraw only invalidated chunks or patches

That is much better than treating the whole world as one giant canvas.

## Height Layer Model In Open World

The current direction of sparse patch-based height overrides is still the right one.

V1 should keep height layers separate from ordinary draw layers.

Normal chunk data:

- base room layers define the world at Z0

Height patch data:

- sparse overrides define what appears at higher Z values

When the player goes up a ladder:

- current Z changes
- renderer continues drawing visible Z0 world
- renderer applies any visible patch data matching that Z

This is exactly the kind of thing chunking helps with, because only nearby patches need to be considered.

## Instance Model

Instances should remain data-driven and chunk-owned.

Recommended v1 storage:

- store instances inside the chunk that owns their tile origin
- keep instance `x` and `y` local to the chunk file
- derive world position in memory

Alternative:

- store world `x` and `y` directly in the chunk file

Either can work. For consistency with chunk-local tile rows, chunk-local instance positions are cleaner on disk.

Example:

```json
{
  "id": "inst_blacksmith_001",
  "templateId": "npc_blacksmith",
  "layer": 1,
  "x": 18,
  "y": 7,
  "record": {
    "name": "Mera"
  }
}
```

## World Navigation UI

The current minimap should be replaced.

Recommended v1 navigation tools:

- current world coordinate display
- jump to X/Y
- bookmarks / waypoints
- "center on player"
- chunk grid overlay toggle
- optional chunk overview panel showing loaded and dirty chunks

This is much more useful for an open world than a single minimap squeezed into a corner.

## Save Model

The editor should save only dirty chunks and world metadata.

Recommended split:

- save `world.json` only when world-level settings change
- save `bookmarks.json` only when bookmarks change
- save only touched chunk files when content changes

This is the core performance and safety advantage.

## Undo / History Model

History can no longer assume one finite map snapshot as the main unit.

Recommended v1 history operation format:

```ts
type ChunkEditOperation = {
  type: "chunk_edit";
  chunkKey: string;
  before: Partial<WorldChunk>;
  after: Partial<WorldChunk>;
};
```

For multi-chunk brushes:

```ts
type MultiChunkEditOperation = {
  type: "multi_chunk_edit";
  chunks: ChunkEditOperation[];
};
```

This is important because world editing will regularly cross chunk boundaries.

## API Shape

Suggested API direction:

- `GET /api/worlds`
- `GET /api/world/:worldId`
- `GET /api/world/:worldId/bookmarks`
- `GET /api/world/:worldId/chunks?x=10&y=4&radius=1`
- `POST /api/world/:worldId/chunk/:chunkX/:chunkY`
- `POST /api/world/:worldId/chunks/batch-save`
- `POST /api/world/:worldId/bookmarks`

The important change is that APIs become chunk-aware instead of map-aware.

## Migration From Current Map Model

The easiest migration path is not "rewrite everything at once."

Recommended migration steps:

1. Introduce `world` and `chunk` data structures alongside current maps
2. Build an importer that converts one existing map into one or more chunks
3. Keep existing tile/layer/height patch formats as similar as possible
4. Build a separate open-world editor mode first
5. Retire the old map model only after the new one is stable

For migration:

- one `100x100` map can become `4x4` chunks at `25x25`
- or `4x4` chunks at `32x32` with padded edges

## What Must Be Rewritten

This is a major architecture change. The following systems would need meaningful refactors:

- popup bootstrap and document loading
- editor runtime state
- renderer surface caching
- minimap/navigation UI
- history model
- persistence API
- map switcher UX

The following ideas can be reused:

- tile symbol rendering
- tile/instance palettes
- sparse height patch logic
- layer editing concepts
- chunk/surface invalidation patterns

## V1 Risks

- Trying to preserve too much of the finite-map abstraction
- Storing too much redundant empty chunk data
- Keeping full-world snapshots in history
- Letting chunk creation happen implicitly on camera movement
- Rebuilding all visible chunk surfaces too often

## Recommended V1 Boundaries

To keep this realistic, v1 should do only this:

- one open world
- fixed chunk size
- chunk-local room layers
- sparse chunk-local height patches
- chunk-owned instances
- bookmarks instead of minimap
- batch save of dirty chunks
- visible-chunk-only rendering

That is already a large and meaningful overhaul.

## Implementation Order

The safest order is:

1. Add new world/chunk schemas and API routes without removing current map routes
2. Build a converter that turns one current map into one chunked world
3. Add a chunk-aware loader in parallel with the current popup bootstrap
4. Replace the popup minimap with coordinates, bookmarks, and jump controls
5. Convert rendering from one-map surfaces to per-chunk surfaces
6. Convert tile painting, height painting, and instance placement to world-coordinate edits
7. Convert save/history to batch dirty chunks
8. Only then remove the old finite-map assumptions

## Final Recommendation

This direction makes sense.

The strongest version of it is:

- stop thinking in "many maps stitched together"
- start thinking in "one world partitioned into chunks"

That keeps the user experience seamless while keeping storage and rendering manageable.

If we choose to proceed, the next best step is:

1. finalize the exact chunk JSON schema
2. build the map-to-world importer
3. add a parallel open-world editor path before deleting the current map editor