# Data Management

## The Rule

**Default to files. Only move to SQL when data is structured enough at scale that SQL earns its complexity.**

Files can always be migrated to SQL later. The reverse is harder. Don't prematurely structure data just because it feels tidy — freeform data in files is cheap, flexible, and the butler reads it naturally.

SQL is justified when:
- The schema is fixed and consistent across every record
- You need to query, filter, or join at scale
- A WHERE clause is the right way to find something

If you'd hand it to the butler to read whole, it's a file. If you'd write a WHERE clause against it, it might be SQL — but only when the scale makes it worth it.

## What's in SQL Now

Only identity and auth. These are SQL from day one because login and palace routing require structured lookup before anything else can happen.

- **masters** — who has an account
- **palaces** — what palace and butler belong to each master

Everything else is files.

## What's in Files

All palace data lives in the palace filesystem. The butler reads and writes these directly.

**Per palace:**
- Diary entries
- Health logs
- Family messages
- Butler memory (temporal layers: daily → weekly → monthly → yearly)
- App data of any kind

**Structure:** Each palace gets its own directory on the server. Same layout as Junwon's palace. The butler knows the layout from the app's BUTLER.md.

## Platform vs Palace Files

**Platform-level files** (shared infrastructure, not per-master):
- App code, BUTLER.md definitions, config
- These never contain master data

**Palace-level files** (per master, isolated):
- Everything the master generates or the butler writes on their behalf
- No palace reads another palace's files

## Migration Path

When a file-based data type grows large enough that querying it becomes slow or cumbersome, migrate it to SQL. The signal is: you find yourself scanning every file to answer a question that should be instant. Until that happens, files are correct.