Aurora-Core

Every get_or_create_* helper (genre, keyword, language, country,
company, collection, person) followed check-then-act: look up by
natural key, insert if missing. That was safe as long as nothing
ingested concurrently for overlapping data -- which stopped being
true the moment ingest_series_task could cascade season ingestion
(include_seasons), since arq runs multiple queued jobs concurrently.
Two seasons of the same series sharing a guest star, crew member, or
company can now race to create it: both see "not found", both try to
INSERT, and the loser hits a unique-constraint violation instead of
finding the winner's row.

Reproduced live: cascading Better Call Saul (60059) with
include_seasons crashed a season job with
"duplicate key value violates unique constraint uq_external_id_source"
on a recurring guest actor shared across two seasons ingesting at the
same time.

_create_or_recover wraps each insert in a SAVEPOINT and, on that
specific failure, rolls back just the savepoint and re-queries for
what the concurrent job committed -- recovering instead of aborting
the whole ingestion job. Added unit tests for the retry mechanism
itself; genuinely reproducing the underlying race in an automated test
would need real Postgres MVCC timing, which isn't something a
deterministic test can force reliably.

Co-Authored-By: Claude Sonnet 5 <[email protected]>

2026-09-05 02:27:07 -05:00

__init__.py

Initial Aurora Core 0.1 implementation

2026-09-04 17:34:03 -05:00

test_ids.py

Initial Aurora Core 0.1 implementation

2026-09-04 17:34:03 -05:00

test_normalizer.py

Add series/season/episode ingestion pipeline

2026-09-04 19:43:56 -05:00

test_reference_repository.py

Fix race condition in get_or_create_* under concurrent ingestion

2026-09-05 02:27:07 -05:00