Aurora-Core/docs/database.md

# Aurora Core — Database

PostgreSQL is authoritative (principle 4). Nothing important lives only in
Redis. See [erd.md](erd.md) for the full entity-relationship diagram.

## Aurora IDs vs. primary keys

Every table's primary key is a UUIDv7 (`app/models/base.py:UUIDPrimaryKeyMixin`),
generated in the application, not the database. UUIDv7 is time-ordered, so it
stays btree-friendly (unlike UUIDv4) and can double as a rough creation-order
cursor.

The public "Aurora ID" strings clients see (`aur_movie_01m1q7p1c...`) are a
lossless, sortable Crockford-base32 encoding of that same UUID, produced by
`app/core/ids.py:aurora_id()`. Encoding (not a second stored column) keeps
the public ID and the primary key from ever drifting apart. See section 3 of
`docs/aurora-core-spec.md` for why Aurora never uses a provider's ID (TMDb,
IMDb, ...) as a primary key.

## Reference data

- `languages` (ISO 639-1), `countries` (ISO 3166-1 alpha-2), `certifications`
  (country + rating system + rating, e.g. US/MPA/PG-13) — populated
  incrementally as ingestion encounters new codes (`get_or_create_*` in
  `app/repositories/reference_repository.py`), not preloaded. A curated
  seed list is a reasonable follow-up (see Known limitations).
- `genres`, `keywords` — deduplicated by a slugified name. Two providers'
  "Sci-Fi" and "Science Fiction" are **not** currently merged; that requires
  a curated synonym table, deliberately deferred (section 33 lists dedup
  *foundations*, not a finished thesaurus).

## Movies / TV

`movies` and `series` hold the record itself; per-locale and per-relationship
data lives in satellite tables (`movie_alternate_titles`,
`movie_spoken_languages`, `movie_production_countries`, `release_dates`,
`movie_recommendations`, and the `series_*` equivalents) rather than JSON
blobs, per section 26. `seasons` and `episodes` form a strict
`series -> season -> episode` hierarchy; `episodes.series_id` is denormalized
(in addition to `season_id`) purely so episode queries don't need to hop
through `seasons` to filter/join by series.

`movies.status` / `series.status` are native Postgres enums, values `.value`
of `app/models/enums.py:MovieStatus`/`SeriesStatus` — see "Enum storage"
below.

## Credits

`credits` is the flexible cast/crew system from section 7: one row per
(person, media entity, department/job/character), not per-movie text blobs.
`department == "Acting"` rows carry `character`/`billing_order`; everything
else carries `job`. Series-level "Creator" credits and episode
`is_guest` (guest stars) reuse the same table rather than adding new ones.

## Companies / Collections

A single `companies` table serves production companies, studios, networks,
and distributors, distinguished by `company_type` (section 8) — a TV network
and a movie studio have the same shape (name, logo, country, parent
company), so a separate `Network` entity would just duplicate columns.
`collections` (franchises) are first-class, with their own artwork and
`collection_relations` for shared-universe links between collections
(section 9).

## Artwork / Videos

Never a `poster_url` column (section 10/11). `artwork` and `videos` are
their own tables carrying provider, provider-native asset ID, dimensions,
and (for artwork) `attribution`/`license` fields — Aurora doesn't assume
externally-sourced images can be freely redistributed. Aurora never
downloads video media; `videos.url` is a reference to the provider's
hosted copy (YouTube, Vimeo, ...).

## Polymorphic associations

`credits`, `artwork`, `videos`, `external_ids`, and `release_dates` attach to
more than one entity type via an `(entity_type, entity_id)` pair instead of a
typed foreign key, because a single column can't reference "movies.id or
series.id or people.id depending on a sibling column" in Postgres. The
tradeoff: **the schema does not enforce that `entity_id` actually points at
a real row of `entity_type`.** That integrity is enforced by the
repository/service layer (`app/repositories/polymorphic.py`,
`app/services/ingestion/*`), which is the only code path allowed to write
these tables. If Core grows enough that this becomes a real risk, the
standard fix is per-type junction tables (`movie_credits`, `series_credits`,
...) at the cost of losing the single shared query path — deliberately not
done for 0.1.

## External IDs & provenance

`external_ids` is the generalized external-identifier system (section 12):
`(provider, entity_type, external_id)` is unique (one Aurora entity per
provider-native ID), and `(entity_type, entity_id, provider)` is unique (one
mapping per provider per entity).

`provider_records` is the full provenance ledger (section 14): one row per
`(provider, entity_type, external_id)` holding the raw payload
(`raw_data` JSONB), `match_confidence`, and which `ingestion_job` produced
it. Entity-level provenance is coarse — `movies.source_provider_id` /
`series.source_provider_id` point at "the provider that was authoritative at
creation time" for quick display; full field-level provenance (which
provider supplied *this specific* field) is not tracked in 0.1. See Known
limitations.

`ingestion_jobs` tracks async work with a unique `idempotency_key` (e.g.
`tmdb:movie:603`) so re-triggering the same sync doesn't double-enqueue —
see `app/services/ingestion/runner.py`.

## Enum storage

Python enums in `app/models/enums.py` are `enum.StrEnum` (member value ==
member name's meaning, e.g. `MovieStatus.RELEASED == "released"`).
SQLAlchemy's `Enum` type stores `.name` (`"RELEASED"`) by default, which
would silently diverge from the lowercase strings the API and ingestion code
use everywhere. `app/models/base.py:_enum_column()` overrides this via
`values_callable` so the database, the ORM, and the API all agree on the
same lowercase strings.

## Migrations

Alembic (section 27); the application **never** creates tables at startup —
see `app/core/database.py`. The initial migration
(`migrations/versions/0001_initial_schema.py`) is hand-written rather than
autogenerated (no throwaway Postgres was available in the environment this
was built in — see the root README's Known Limitations), but was written
directly from the same models this document describes and validated with
`alembic history`; treat a first `alembic upgrade head` against a scratch
database as the real acceptance test, and autogenerate future migrations
against it (`alembic revision --autogenerate`) rather than the empty
database this one was written from.

```bash
alembic upgrade head          # apply all migrations
alembic revision --autogenerate -m "add xyz"   # generate the next one
alembic downgrade -1          # roll back one revision
```

Integration/API tests build schema directly from `Base.metadata.create_all`
(see `tests/conftest.py`) for speed and to catch model drift quickly — this
is a testing shortcut, not how the running application ever gets its schema.

## Search indexes

`pg_trgm` (trigram similarity, for typo-tolerant matching) plus functional
GIN indexes over `to_tsvector('simple', title || ' ' || original_title)` on
`movies`/`series`, `name` on `people`/`collections`/`companies`, and trigram
indexes on the alternate-title tables. See
`app/services/search/postgresql_provider.py` and section 16 of the spec.

## Known limitations (tracked, not hidden)

- No curated language/country seed data — rows are created on first
  encounter during ingestion, so `english_name`/`name` are sometimes just
  the code uppercased until backfilled.
- No fuzzy/candidate entity matching — only exact external-ID matches are
  auto-linked (section 14: never silently merge ambiguous matches). A
  review queue for AMBIGUOUS matches is the natural next step once Core has
  an admin UI to act on it.
- Field-level provenance (which provider supplied *this* value) is not
  tracked, only entity-level (`source_provider_id`) and per-provider raw
  payloads (`provider_records.raw_data`).
- Satellite associations (credits, artwork, videos, genre/keyword/company
  links) are replaced wholesale on each re-sync rather than diffed, so their
  Aurora IDs are not stable across re-ingestion — see
  `app/services/ingestion/movie_ingestion.py`.