Commit Graph
33 Commits
Author SHA1 Message Date
NixReaperandClaude Sonnet 5 905cd202eb Replace Traefik with host-webserver reverse proxy in prod deployment
CI / lint (push) Canceled after 0s
CI / typecheck (push) Canceled after 0s
CI / unit-tests (push) Canceled after 0s
CI / integration-tests (push) Canceled after 0s
CI / container-build (push) Canceled after 0s
CI / dependency-audit (push) Canceled after 0s
The production host runs cPanel, which already owns ports 80/443 for
other sites on the box -- there's no Traefik there. docker-compose.prod.yml
now binds the api container to 127.0.0.1:8000 only; the host's own
Apache/LiteSpeed (via cPanel) reverse-proxies api.amdbs.net to it and
terminates TLS itself (AutoSSL). Dropped the external traefik-public
network requirement and Traefik labels entirely.

Also corrected a separate latent inaccuracy while touching this: a bare
`docker compose up` (not `docker stack deploy`) never actually honored
deploy.replicas -- docs now say so and point at `--scale` instead of
implying free multi-replica scaling that was never really happening.

README's Production deployment section rewritten with concrete steps:
Cloudflare DNS record, cPanel subdomain + reverse proxy, then compose up.

Co-Authored-By: Claude Sonnet 5 <[email protected]>
2026-09-05 20:49:55 -05:00
NixReaperandClaude Sonnet 5 ec9b1c28e3 Move CI workflow from .github/workflows to .gitea/workflows
CI / lint (push) Canceled after 0s
CI / typecheck (push) Canceled after 0s
CI / unit-tests (push) Canceled after 0s
CI / integration-tests (push) Canceled after 0s
CI / container-build (push) Canceled after 0s
CI / dependency-audit (push) Canceled after 0s
auroradragon.dev is self-hosted Gitea (v1.27.3) -- confirmed via its
public API (/api/v1/version, /api/v1/repos/.../actions/tasks showing
zero workflow runs ever recorded). Gitea only reads .gitea/workflows/
by default, not .github/workflows/, so this CI pipeline (lint,
typecheck, unit tests, integration tests against real Postgres/Redis,
container builds, dependency audit) has never actually executed.

Still needs confirming: whether an Actions runner is registered for
this repo -- Actions being enabled doesn't guarantee a runner exists
to pick up queued jobs.

Co-Authored-By: Claude Sonnet 5 <[email protected]>
2026-09-05 20:44:08 -05:00
NixReaperandClaude Sonnet 5 a8cb807d56 Document production domain split in README
CI / lint (push) Canceled after 0s
CI / typecheck (push) Canceled after 0s
CI / unit-tests (push) Canceled after 0s
CI / integration-tests (push) Canceled after 0s
CI / container-build (push) Canceled after 0s
CI / dependency-audit (push) Canceled after 0s
api.amdbs.net (backend) / www.amdbs.net (frontend) -- records the
required AURORA_API_HOST and CORS_ORIGINS values for the production
.env, and the external-Traefik prerequisite docker-compose.prod.yml
already assumed but never spelled out.

Co-Authored-By: Claude Sonnet 5 <[email protected]>
2026-09-05 20:33:25 -05:00
NixReaperandClaude Sonnet 5 0a50e0ffb6 Fix two more test bugs surfaced by the full integration run
CI / lint (push) Canceled after 0s
CI / typecheck (push) Canceled after 0s
CI / unit-tests (push) Canceled after 0s
CI / integration-tests (push) Canceled after 0s
CI / container-build (push) Canceled after 0s
CI / dependency-audit (push) Canceled after 0s
1. test_ingestion_cache_invalidation.py: the job was created through
   the test's db_session fixture, whose commit() only ends a savepoint
   that gets rolled back at teardown (see tests/conftest.py) -- it's
   never a real commit. _run_ingestion looks the job up through its
   own session_scope() (a genuinely separate connection), which could
   never see it, so the very first _require_job call raised "job
   vanished mid-run" every time. Worse, test_failed_ingestion_does_not_
   invalidate_the_cache still passed, coincidentally -- it expected a
   RuntimeError and got one, just the wrong one, masking the bug.
   Fixed by having _make_job create the job through session_scope()
   itself, matching test_ingestion_retry.py's existing pattern.

2. Same class of bug as the video-collision fix in the last commit:
   FAKE_MOVIE's fixed imdb_id ("tt0133093"), not just its video, also
   collides across two different fake movies sharing it unmodified --
   uq_external_id_source is a real unique constraint on
   (provider_id, entity_type, external_id). Fixed by also overriding
   imdb_id on the second movie in both affected tests.

All caught by running the actual integration suite against Postgres --
these paths were previously only collect-only'd locally (no local DB).

Co-Authored-By: Claude Sonnet 5 <[email protected]>
2026-09-05 20:17:02 -05:00
NixReaperandClaude Sonnet 5 29fffc8919 Fix test fixtures: two movies can't share FAKE_MOVIE's embedded video
CI / lint (push) Canceled after 0s
CI / typecheck (push) Canceled after 0s
CI / unit-tests (push) Canceled after 0s
CI / integration-tests (push) Canceled after 0s
CI / container-build (push) Canceled after 0s
CI / dependency-audit (push) Canceled after 0s
FAKE_MOVIE carries a fixed provider_video_id ("abc123"); two different
fake movies both spreading it unmodified and getting ingested in the
same test collide on the real uq_video_provider_source unique
constraint (a video id belongs to exactly one entity). Found by
actually running the tests against Postgres on the devbox -- these
paths were only ever collect-only'd locally before now (no local DB).

Affected: test_movie_recommendations_only_include_already_ingested_movies
and test_ingest_collection_only_links_already_ingested_movies, both of
which ingest two FAKE_MOVIE-derived movies in one test. Fixed by
overriding videos to empty on the second movie in each.

Co-Authored-By: Claude Sonnet 5 <[email protected]>
2026-09-05 20:12:03 -05:00
NixReaperandClaude Sonnet 5 f1ac072794 Add GET /collections browse endpoint; fix embedded collection poster_path
CI / lint (push) Canceled after 0s
CI / typecheck (push) Canceled after 0s
CI / unit-tests (push) Canceled after 0s
CI / integration-tests (push) Canceled after 0s
CI / container-build (push) Canceled after 0s
CI / dependency-audit (push) Canceled after 0s
Completes browse/discover across all five entity types: movies, series,
people, companies, and now collections. No filter (no categorical field
like company_type exists for a collection) -- just page/page_size,
always alphabetical by name.

While adding poster_path to CollectionBriefOut for the browse listing,
found the same class of bug fixed earlier for company logo_path:
movie_service's embedded `collection` field never looked up its poster
artwork either, so it stayed null even after ingest_collection populated
one. Fixed alongside since it's the same schema field this feature was
already touching.

Also removed PageMeta from schemas/common.py -- scaffolding from early
spec work that was never actually wired into any endpoint; all five
browse endpoints shipped this session use a flat page/page_size/
total_results/total_pages shape instead.

Co-Authored-By: Claude Sonnet 5 <[email protected]>
2026-09-05 20:08:26 -05:00
NixReaperandClaude Sonnet 5 4cf3556433 Fix test fixture: seed countries before companies referencing them
CI / lint (push) Canceled after 0s
CI / typecheck (push) Canceled after 0s
CI / unit-tests (push) Canceled after 0s
CI / integration-tests (push) Canceled after 0s
CI / container-build (push) Canceled after 0s
CI / dependency-audit (push) Canceled after 0s
companies.country_code has a real FK to countries -- real ingestion
always calls get_or_create_country first (see _replace_movie_companies),
but the company browse test fixture called get_or_create_company
directly with a raw country code, violating that FK. Found by actually
running the tests against Postgres on the devbox.

Co-Authored-By: Claude Sonnet 5 <[email protected]>
2026-09-05 19:15:09 -05:00
NixReaperandClaude Sonnet 5 92b0f4e077 Add GET /companies browse/discover endpoint
CI / lint (push) Canceled after 0s
CI / typecheck (push) Canceled after 0s
CI / unit-tests (push) Canceled after 0s
CI / integration-tests (push) Canceled after 0s
CI / container-build (push) Canceled after 0s
CI / dependency-audit (push) Canceled after 0s
Completes the browse/discover set across all four entity types.
Companies have no popularity or date field, so unlike movies/series/
people there's no sort_by choice -- always alphabetical by name.
Filters are company_type (validated against the enum) and country
(a plain ISO 3166-1 code comparison, nothing to validate).

build_company_summary refactored to share row-to-summary logic with
the new list path, same pattern used for movies/series/people.

Co-Authored-By: Claude Sonnet 5 <[email protected]>
2026-09-05 19:08:14 -05:00
NixReaperandClaude Sonnet 5 6aa9f0b840 Add GET /people browse/discover endpoint
CI / lint (push) Canceled after 0s
CI / typecheck (push) Canceled after 0s
CI / unit-tests (push) Canceled after 0s
CI / integration-tests (push) Canceled after 0s
CI / container-build (push) Canceled after 0s
CI / dependency-audit (push) Canceled after 0s
Extends the browse/discover pattern already built for movies/series
to the fourth major entity type. No genre/year/status equivalent for
people -- known_for_department is the one categorical filter that
actually means something for a cast/crew list, and sort_by is just
popularity (default) or name. No invalid-filter case either: unlike
genre/status, known_for_department is a plain string compared
directly, nothing to parse or validate.

build_person_summary refactored to share row-to-summary logic with
the new list path, same pattern used for movies/series/companies.

Co-Authored-By: Claude Sonnet 5 <[email protected]>
2026-09-05 10:32:16 -05:00
NixReaperandClaude Sonnet 5 74adcf77bc Add series endpoint parity: /credits, /images, /videos
CI / lint (push) Canceled after 0s
CI / typecheck (push) Canceled after 0s
CI / unit-tests (push) Canceled after 0s
CI / integration-tests (push) Canceled after 0s
CI / container-build (push) Canceled after 0s
CI / dependency-audit (push) Canceled after 0s
Movies have separate /credits, /images, /videos sub-endpoints; series
only had /seasons. Adds the matching three for series, reusing the
same cast/crew/creators three-way split series detail already does
(extracted into _split_series_credits so both share it).

Co-Authored-By: Claude Sonnet 5 <[email protected]>
2026-09-05 09:52:08 -05:00
NixReaperandClaude Sonnet 5 6956bf7406 Add movies/series filmography to company detail
CI / lint (push) Canceled after 0s
CI / typecheck (push) Canceled after 0s
CI / unit-tests (push) Canceled after 0s
CI / integration-tests (push) Canceled after 0s
CI / container-build (push) Canceled after 0s
CI / dependency-audit (push) Canceled after 0s
GET /companies/{id} showed only the company's own info. Adds movies
and series lists (any role: production, distribution, network),
ordered by release/first-air date, read straight off the existing
movie_companies/series_companies junction rows -- no separate sync
needed since those only ever exist as a side effect of ingesting the
movie or series itself.

Co-Authored-By: Claude Sonnet 5 <[email protected]>
2026-09-05 09:34:31 -05:00
NixReaperandClaude Sonnet 5 606d2532c4 Add per-episode breakdown to guest-star filmography credits
CI / lint (push) Canceled after 0s
CI / typecheck (push) Canceled after 0s
CI / unit-tests (push) Canceled after 0s
CI / integration-tests (push) Canceled after 0s
CI / container-build (push) Canceled after 0s
CI / dependency-audit (push) Canceled after 0s
FilmographyCreditOut already aggregated a guest star's episode
appearances into one row per series with an episode_count, but never
exposed which episodes those were even though the individual episode
Credit rows exist in the database. Adds an `episodes` list (season/
episode number, title, air date, sorted chronologically) alongside the
existing count -- closes a known limitation tracked in docs/api.md.

Co-Authored-By: Claude Sonnet 5 <[email protected]>
2026-09-05 09:19:52 -05:00
NixReaperandClaude Sonnet 5 20f5e82d1b Add GET /movies and GET /series browse/discover endpoints
CI / lint (push) Canceled after 0s
CI / typecheck (push) Canceled after 0s
CI / unit-tests (push) Canceled after 0s
CI / integration-tests (push) Canceled after 0s
CI / container-build (push) Canceled after 0s
CI / dependency-audit (push) Canceled after 0s
The complement to /search for browsing without a query: paginated,
filterable (genre, year, status) and sortable (popularity,
release/air date, rating, title) listings. Offset-based pagination
rather than the Aurora ID cursor scheme, since sorting by
popularity/rating/date doesn't line up with creation-order cursors.

Total count is computed via a window function in the same query as
the page of results, avoiding a second round trip per request. An
invalid filter (malformed genre id, unknown status) is a 422 via a
per-service InvalidMovieFilterError/InvalidSeriesFilterError, caught
before any DB query runs.

build_movie_summary/build_series_summary refactored to share their
row-to-summary logic with the new list path, avoiding a redundant
get_by_id lookup when the row is already in hand.

Co-Authored-By: Claude Sonnet 5 <[email protected]>
2026-09-05 09:01:11 -05:00
NixReaperandClaude Sonnet 5 892f78016c Add collection/franchise ingestion
CI / lint (push) Canceled after 0s
CI / typecheck (push) Canceled after 0s
CI / unit-tests (push) Canceled after 0s
CI / integration-tests (push) Canceled after 0s
CI / container-build (push) Canceled after 0s
CI / dependency-audit (push) Canceled after 0s
Collections were bare stubs (name only) lazily created the moment a
movie's belongs_to_collection reference was seen. Adds a real
ingest_collection pipeline: TMDb's /collection/{id} for overview and
poster/backdrop artwork, plus ordered CollectionMovie membership --
linking only movies already ingested into Aurora, the same
never-fabricate-a-stub rule movie recommendations already follow.

get_or_create_collection now refreshes name/overview on an existing
stub instead of only creating, guarded so a bare movie-stub call
(overview=None) never clobbers real data a full ingestion already
filled in.

New POST /api/v1/admin/ingest/collection endpoint and
ingest_collection_task; GET /api/v1/collections/{id} needed no changes
since its read path (collection_service.py) already assembled the full
detail shape.

Co-Authored-By: Claude Sonnet 5 <[email protected]>
2026-09-05 08:37:27 -05:00
NixReaperandClaude Sonnet 5 4af4ba40ef Add GET /movies/{id}/recommendations
CI / lint (push) Canceled after 0s
CI / typecheck (push) Canceled after 0s
CI / unit-tests (push) Canceled after 0s
CI / integration-tests (push) Canceled after 0s
CI / container-build (push) Canceled after 0s
CI / dependency-audit (push) Canceled after 0s
Ingestion already stored movie_recommendations edges from TMDb's
recommendations list (linking only to movies already ingested into
Aurora, per the never-fabricate-a-stub rule), but nothing exposed them.
Adds the repository lookup, service assembly, and route to read them
back in ranked order.

Co-Authored-By: Claude Sonnet 5 <[email protected]>
2026-09-05 08:16:18 -05:00
NixReaperandClaude Sonnet 5 73488a0d80 Fix movie/series API responses never surfacing company logo_path
CI / lint (push) Canceled after 0s
CI / typecheck (push) Canceled after 0s
CI / unit-tests (push) Canceled after 0s
CI / integration-tests (push) Canceled after 0s
CI / container-build (push) Canceled after 0s
CI / dependency-audit (push) Canceled after 0s
movie_service.get_movie_detail() hardcoded companies[].logo_path = None
instead of looking up the company's logo artwork, and tv_service's
networks[]/companies[] never set logo_path at all -- both just built
CompanyBriefOut without the artwork lookup that company_service.py (and
_season_brief for posters) already does correctly. Ingestion and storage
were working the whole time; the API layer just never read the result.

Co-Authored-By: Claude Sonnet 5 <[email protected]>
2026-09-05 08:01:56 -05:00
NixReaperandClaude Sonnet 5 2138d4afc3 Invalidate movie/series/person detail cache on re-ingestion
CI / lint (push) Canceled after 0s
CI / typecheck (push) Canceled after 0s
CI / unit-tests (push) Canceled after 0s
CI / integration-tests (push) Canceled after 0s
CI / container-build (push) Canceled after 0s
CI / dependency-audit (push) Canceled after 0s
cached_response() had no write-side invalidation, so re-ingesting an
entity kept serving its pre-ingestion cached response for up to
CACHE_DEFAULT_TTL. Found live: re-ingesting a movie to test the new
company-logo field still showed logo_path: None because the sliding
5-minute cache from earlier testing never expired.

Co-Authored-By: Claude Sonnet 5 <[email protected]>
2026-09-05 07:56:00 -05:00
NixReaperandClaude Sonnet 5 c03a75989d Ingest and self-host company/network logos
CI / lint (push) Canceled after 0s
CI / typecheck (push) Canceled after 0s
CI / unit-tests (push) Canceled after 0s
CI / integration-tests (push) Canceled after 0s
CI / container-build (push) Canceled after 0s
CI / dependency-audit (push) Canceled after 0s
CompanyBriefOut.logo_path and CompanyDetailOut.logo_path have existed
in the API schemas since Core 0.1's first pass, but nothing ever
actually populated them -- TMDb's production_companies/networks
entries carry a logo_path field that the normalizer simply never read,
so every response has silently returned null there.

- NormalizedCompanyRef now captures logo_path from both movie
  production_companies and series networks/production_companies.
- normalizer.company_logo_artwork() builds the single-image artwork
  entry for it (a company has exactly one logo ingested, so it's
  always is_primary) -- centralized there rather than duplicated in
  both ingestion modules, matching how movie/series artwork URLs are
  already built in one place.
- movie_ingestion.py and series_ingestion.py sync it via the same
  sync_artwork() the rest of section 10's self-hosting already uses,
  so it gets the same re-ingestion-safe upsert behavior for free.
- tasks.py cascades a download_artwork_task per referenced company
  after movie/series ingestion, same fan-out pattern as
  movie/series/season/person artwork.

Co-Authored-By: Claude Sonnet 5 <[email protected]>
2026-09-05 07:46:30 -05:00
NixReaperandClaude Sonnet 5 69ce41d54f Fix media volume permission denied for the non-root worker/api user
CI / lint (push) Canceled after 0s
CI / typecheck (push) Canceled after 0s
CI / unit-tests (push) Canceled after 0s
CI / integration-tests (push) Canceled after 0s
CI / container-build (push) Canceled after 0s
CI / dependency-audit (push) Canceled after 0s
Neither Dockerfile created /app/media before switching to the
non-root `aurora` user, so a fresh named volume mounted there
(docker-compose.yml's aurora_media_data) defaulted to root ownership
-- Docker only auto-populates a volume's initial content/ownership
from the image the first time it's mounted, and nothing in the image
owned that path yet. The worker's first artwork download then failed
with a permission error trying to mkdir a subdirectory under it.

Creating and chowning /app/media to aurora before USER aurora fixes
this for any volume created fresh after this build; an already-created
volume from a prior `up --build` needs a one-time manual fix
(`docker compose exec -u root worker chown -R aurora:aurora /app/media`)
since Docker won't retroactively re-populate an existing volume.

Co-Authored-By: Claude Sonnet 5 <[email protected]>
2026-09-05 07:35:50 -05:00
NixReaperandClaude Sonnet 5 7c4bfe6cf4 Fix sync_artwork clobbering is_primary via duplicate list entries
CI / lint (push) Canceled after 0s
CI / typecheck (push) Canceled after 0s
CI / unit-tests (push) Canceled after 0s
CI / integration-tests (push) Canceled after 0s
CI / container-build (push) Canceled after 0s
CI / dependency-audit (push) Canceled after 0s
TMDb's images.posters/backdrops alternates commonly include the *same*
image as the top-level poster_path/backdrop_path -- so normalize_movie
legitimately produces the same provider_asset_id twice in one
artwork_list: once from the top-level field (is_primary=True), once
from the general alternates loop (is_primary=False, the default).
sync_artwork matched both occurrences to the same existing DB row by
provider_asset_id and did a plain overwrite each time, so whichever
occurrence it processed last won -- the non-primary duplicate always
came second in list order, so it silently un-set is_primary right
after the first occurrence had just set it.

Caught by re-ingesting The Matrix on a devbox with the artwork
download feature live: the movie ingested successfully, but zero
download jobs were ever enqueued because get_primary_artwork_ids
found nothing marked is_primary -- the flag had been clobbered back to
false immediately after being set.

Fixed by resetting is_primary on all existing rows up front (so a
provider dropping its "primary" pick on an image actually clears, not
just adds) and accumulating with OR within the batch instead of a
plain overwrite, so any primary occurrence in one sync sticks
regardless of processing order. Also dedupes brand-new rows within the
same batch the same way, closing the equivalent bug for images that
don't have an existing row yet.

Co-Authored-By: Claude Sonnet 5 <[email protected]>
2026-09-05 07:30:40 -05:00
NixReaperandClaude Sonnet 5 fef914db36 Add artwork self-hosting: local-disk download worker + storage
CI / lint (push) Canceled after 0s
CI / typecheck (push) Canceled after 0s
CI / unit-tests (push) Canceled after 0s
CI / integration-tests (push) Canceled after 0s
CI / container-build (push) Canceled after 0s
CI / dependency-audit (push) Canceled after 0s
Section 10: every image API responses reference has pointed straight
at TMDb's CDN since day one -- Aurora never actually downloaded or
served artwork itself, despite storing dimensions/checksum/local_path
columns for exactly that. Closes the gap:

- is_primary was declared on the Artwork model but never actually set
  anywhere -- every row defaulted to False. Now the normalizer marks
  the one image each provider designates as *the* image for an entity
  (TMDb's poster_path/backdrop_path/profile_path) as primary,
  distinct from the dozens of alternates also ingested, including
  correctly marking an existing row primary (not inserting a
  duplicate) when that image already appears in the alternates list.
- app/services/artwork/: an ArtworkStorage interface with a
  LocalDiskStorage implementation (checksum-named files, single-host
  deployment via a Docker volume shared between api and worker -- see
  docs/architecture.md for the S3/R2 upgrade path), and a download
  pipeline with content-type validation, a size cap, and path-traversal
  protection.
- After movie/series/person ingestion succeeds, tasks.py cascades one
  download_artwork_task per primary artwork id via arq's ctx['redis']
  (same fan-out pattern as the season cascade) -- fire-and-forget, not
  tracked in ingestion_jobs, since a failed image cache is a
  known-safe fallback to the provider URL, not a user-facing failure.
- app/main.py serves MEDIA_ROOT at MEDIA_URL_PREFIX; local_path is
  preferred over original_url once set (already-existing fallback
  logic in app/services/common.py:artwork_url, now actually exercised).

Also fixes a real interaction bug this surfaced: every ingestion
pipeline deleted and recreated all of an entity's Artwork rows on each
re-sync, which would have silently wiped local_path/checksum on every
re-ingestion, forcing a full re-download every time and defeating the
point of caching locally at all. app/repositories/polymorphic.py's new
sync_artwork upserts by provider_asset_id (TMDb's stable per-image
path) instead, preserving downloaded copies across re-sync, and
replaces three near-identical delete+recreate helpers that used to
live separately in movie_ingestion.py, series_ingestion.py, and
person_ingestion.py.

Co-Authored-By: Claude Sonnet 5 <[email protected]>
2026-09-05 03:26:28 -05:00
NixReaperandClaude Sonnet 5 cd4dd98adb Make failed ingestion jobs actually retry, not just say "retrying"
CI / lint (push) Canceled after 0s
CI / typecheck (push) Canceled after 0s
CI / unit-tests (push) Canceled after 0s
CI / integration-tests (push) Canceled after 0s
CI / container-build (push) Canceled after 0s
CI / dependency-audit (push) Canceled after 0s
arq only auto-retries a job on its own Retry exception (or
CancelledError) -- never on an arbitrary exception a task raises.
_run_ingestion caught failures, recorded them via mark_job_failed
(which sets ingestion_jobs.status to "retrying" when attempts hasn't
hit max_attempts), and then just re-raised the original exception --
which arq treats as a terminal failure. The DB said "retrying"
forever; nothing ever put the job back on the queue.

Confirmed live: cascading Better Call Saul's 6 seasons hit the
get_or_create race fixed in the previous commit on 4 of them, and the
worker's own shutdown summary showed "4 failed, 0 retries" -- every
one of those was permanently lost despite max_attempts=5 suggesting
otherwise.

_run_ingestion now raises arq.Retry (capped exponential backoff) when
mark_job_failed left the job in RETRYING state, so arq actually
requeues it; only once attempts are exhausted does the original
exception propagate as a genuine terminal failure.

Co-Authored-By: Claude Sonnet 5 <[email protected]>
2026-09-05 02:31:43 -05:00
NixReaperandClaude Sonnet 5 dd3ff9764b Fix race condition in get_or_create_* under concurrent ingestion
CI / lint (push) Canceled after 0s
CI / typecheck (push) Canceled after 0s
CI / unit-tests (push) Canceled after 0s
CI / integration-tests (push) Canceled after 0s
CI / container-build (push) Canceled after 0s
CI / dependency-audit (push) Canceled after 0s
Every get_or_create_* helper (genre, keyword, language, country,
company, collection, person) followed check-then-act: look up by
natural key, insert if missing. That was safe as long as nothing
ingested concurrently for overlapping data -- which stopped being
true the moment ingest_series_task could cascade season ingestion
(include_seasons), since arq runs multiple queued jobs concurrently.
Two seasons of the same series sharing a guest star, crew member, or
company can now race to create it: both see "not found", both try to
INSERT, and the loser hits a unique-constraint violation instead of
finding the winner's row.

Reproduced live: cascading Better Call Saul (60059) with
include_seasons crashed a season job with
"duplicate key value violates unique constraint uq_external_id_source"
on a recurring guest actor shared across two seasons ingesting at the
same time.

_create_or_recover wraps each insert in a SAVEPOINT and, on that
specific failure, rolls back just the savepoint and re-queries for
what the concurrent job committed -- recovering instead of aborting
the whole ingestion job. Added unit tests for the retry mechanism
itself; genuinely reproducing the underlying race in an automated test
would need real Postgres MVCC timing, which isn't something a
deterministic test can force reliably.

Co-Authored-By: Claude Sonnet 5 <[email protected]>
2026-09-05 02:27:07 -05:00
NixReaperandClaude Sonnet 5 64169c4aee Fix test factory Provider collision from ingestion-cascade tests
CI / lint (push) Canceled after 0s
CI / typecheck (push) Canceled after 0s
CI / unit-tests (push) Canceled after 0s
CI / integration-tests (push) Canceled after 0s
CI / container-build (push) Canceled after 0s
CI / dependency-audit (push) Canceled after 0s
tests/integration/factories.py:make_provider did a raw INSERT
assuming no "tmdb" Provider row existed yet. That held as long as
every test used the db_session fixture's isolated, rolled-back
transaction -- but the new ingestion-cascade tests call
get_or_create_ingestion_job, which really commits via
session_scope()'s own connection, outside that per-test isolation.
Once that ran, every later test using the raw-insert factory hit
"duplicate key value violates unique constraint uq_providers_slug".

Fixed by using the app's own get_or_create_provider in the factory
instead of a bare insert, which is also just correct: provider slugs
are unique in production too, so the test helper should behave the
same way real ingestion code does rather than assume a pristine table.

Co-Authored-By: Claude Sonnet 5 <[email protected]>
2026-09-05 02:21:43 -05:00
NixReaperandClaude Sonnet 5 b4d1e8f755 Add opt-in season cascade to series ingestion
CI / lint (push) Canceled after 0s
CI / typecheck (push) Canceled after 0s
CI / unit-tests (push) Canceled after 0s
CI / integration-tests (push) Canceled after 0s
CI / container-build (push) Canceled after 0s
CI / dependency-audit (push) Canceled after 0s
Ingesting a series only ever produced season stubs -- getting actual
episodes meant calling POST /admin/ingest/season once per season by
hand, which is exactly what manual end-to-end testing kept requiring
for every multi-season show.

POST /api/v1/admin/ingest/series now takes include_seasons (default
false, since a show can have dozens of seasons and that's dozens of
TMDb calls fired from one request). When true, ingest_series_task
fans out one ingest_season job per season once the series succeeds,
using arq's own ctx['redis'] connection for the job (the same pattern
arq's docs recommend for job chaining) rather than a second unrelated
connection. Each fanned-out season job still goes through
get_or_create_ingestion_job, so re-running a cascaded series ingestion
reuses the same season job rows instead of creating duplicates.

Co-Authored-By: Claude Sonnet 5 <[email protected]>
2026-09-05 02:03:25 -05:00
NixReaperandClaude Sonnet 5 ec6b5bdd7d Surface series/episode credits in person filmography
CI / lint (push) Canceled after 0s
CI / typecheck (push) Canceled after 0s
CI / unit-tests (push) Canceled after 0s
CI / integration-tests (push) Canceled after 0s
CI / container-build (push) Canceled after 0s
CI / dependency-audit (push) Canceled after 0s
/people/{id}/credits only ever looked at entity_type == "movie",
silently dropping the series-level cast/crew and episode-level
guest-star Credit rows that series_ingestion.py has been writing all
along -- a person ingested via a TV show would show an empty or
movie-only filmography no matter how much TV work was actually in the
database.

Regular series credits map straight to a filmography row, same shape
as movies. Guest-star episode credits are aggregated to one row per
(series, department/job/character) with an episode_count, rather than
one row per episode -- a prolific guest actor would otherwise flood
the list with dozens of near-identical entries. Added
tv_repository.get_series_by_ids/get_episodes_by_ids (bulk, mirroring
the existing movie_repository pattern) to support it.

Co-Authored-By: Claude Sonnet 5 <[email protected]>
2026-09-04 20:00:15 -05:00
NixReaperandClaude Sonnet 5 e6841badf8 Fix worker never picking up code changes after restart
CI / lint (push) Canceled after 0s
CI / typecheck (push) Canceled after 0s
CI / unit-tests (push) Canceled after 0s
CI / integration-tests (push) Canceled after 0s
CI / container-build (push) Canceled after 0s
CI / dependency-audit (push) Canceled after 0s
docker compose restart worker looked like it should reflect new code
(docker-compose.yml bind-mounts ./app:/app/app for dev), but the
worker Dockerfile installed the app package non-editably (pip install
.), which copies a frozen snapshot into site-packages at image build
time. The bare `arq` console-script just imports whatever
site-packages says `app` is -- unlike uvicorn --reload, which
explicitly resolves its import string against the CWD -- so the
worker silently kept running whatever code existed at the last
`docker compose up --build`, no matter how many times it was
restarted or how recent the git pull was.

Confirmed live: after pulling the series/season ingestion pipeline
and restarting, the worker log still read "Starting worker for 2
functions: ingest_movie_task, ingest_person_task" -- proof it never
saw ingest_series_task/ingest_season_task at all.

Switching to `pip install -e .` makes site-packages point at /app by
absolute path instead of a frozen copy, so the same bind mount that
already worked for the API now actually reaches the worker process
too -- a plain restart after `git pull` is enough going forward.
Applied to both Dockerfiles for consistency, since the API's working
behavior was incidental to uvicorn, not something to rely on either.

Co-Authored-By: Claude Sonnet 5 <[email protected]>
2026-09-04 19:51:52 -05:00
NixReaperandClaude Sonnet 5 3acf00e9f0 Fix test bug: await inside a generator expression passed to all()
CI / lint (push) Canceled after 0s
CI / typecheck (push) Canceled after 0s
CI / unit-tests (push) Canceled after 0s
CI / integration-tests (push) Canceled after 0s
CI / container-build (push) Canceled after 0s
CI / dependency-audit (push) Canceled after 0s
Python turns a generator expression containing await into an async
generator (since it's evaluated inside an async def), which the
built-in all() can't consume -- it raised "TypeError: 'async_generator'
object is not iterable" rather than testing anything. Replaced with a
plain loop. Found by actually running the new integration tests.

Co-Authored-By: Claude Sonnet 5 <[email protected]>
2026-09-04 19:45:29 -05:00
NixReaperandClaude Sonnet 5 3f6332559a Add series/season/episode ingestion pipeline
CI / lint (push) Canceled after 0s
CI / typecheck (push) Canceled after 0s
CI / unit-tests (push) Canceled after 0s
CI / integration-tests (push) Canceled after 0s
CI / container-build (push) Canceled after 0s
CI / dependency-audit (push) Canceled after 0s
Movie and person ingestion existed, but the series/season/episode API
endpoints had no way to actually populate them from TMDb -- the
biggest remaining gap in Core 0.1's television scope.

- normalize_series/normalize_season in the metadata normalizer,
  including two TMDb quirks that would otherwise silently produce
  empty data: /tv nests keywords and alternative_titles under
  "results" (not "keywords"/"titles" like /movie), and series-level
  creators come from a separate created_by array, not the credits
  cast/crew lists.
- ingest_series upserts the Series row plus season *stubs* (number,
  title, air date, poster -- everything TMDb inlines on /tv/{id});
  ingest_season fetches and upserts one season's full episode list,
  and requires the series to already exist (TMDb has no single
  endpoint for a whole series' episodes, so this has to be a separate
  per-season call).
- Wired through the same job-lifecycle path as movie/person ingestion
  (runner.py, arq tasks, admin endpoints), with the shared boilerplate
  between them factored into runner._run_ingestion and
  admin.py:_enqueue rather than copy-pasted a third and fourth time.
- Unit tests for both normalizers (runnable without a DB) and
  integration tests for the full ingest_series -> ingest_season flow,
  including the series-must-exist-first error and idempotency on
  re-ingestion.

Co-Authored-By: Claude Sonnet 5 <[email protected]>
2026-09-04 19:43:56 -05:00
NixReaperandClaude Sonnet 5 669034896a Fix job_id mismatch between ingestion enqueue and status endpoints
CI / lint (push) Canceled after 0s
CI / typecheck (push) Canceled after 0s
CI / unit-tests (push) Canceled after 0s
CI / integration-tests (push) Canceled after 0s
CI / container-build (push) Canceled after 0s
CI / dependency-audit (push) Canceled after 0s
POST /api/v1/admin/ingest/movie returned arq's own internal queue
job id, but GET /api/v1/admin/ingest/jobs/{job_id} looked up our
IngestionJob DB row by its own UUID -- a completely different id that
didn't even exist yet at enqueue time (the row was only created once
the worker picked up the task). Every status poll 404'd.

Fix: create the IngestionJob row synchronously in the enqueue endpoint
(reusing the existing idempotency-key logic, so repeat enqueues for
the same tmdb_id reuse the same row rather than creating duplicates),
pass its real id into the arq task, and have the worker update that
row instead of creating a new one.

Found by actually running the milestone end-to-end: enqueue a real
TMDb ingestion, then poll its status.

Co-Authored-By: Claude Sonnet 5 <[email protected]>
2026-09-04 19:28:31 -05:00
NixReaperandClaude Sonnet 5 dd6ecce78d Fix asyncpg cross-event-loop errors in the integration test suite
CI / lint (push) Canceled after 0s
CI / typecheck (push) Canceled after 0s
CI / unit-tests (push) Canceled after 0s
CI / integration-tests (push) Canceled after 0s
CI / container-build (push) Canceled after 0s
CI / dependency-audit (push) Canceled after 0s
tests/conftest.py's `engine` fixture is session-scoped (one asyncpg
engine/schema for the whole run, for speed), but pytest-asyncio's
default is a fresh event loop per test function. asyncpg connections
are bound to the event loop that created them, so every test after the
first failed with "cannot perform operation: another operation is in
progress" or "attached to a different loop" the moment it touched the
database. Pinning both the fixture and test loop scope to "session"
in pyproject.toml keeps everything on one loop, matching the
session-scoped engine.

Found by actually running the integration suite for the first time,
against real PostgreSQL + Redis on a devbox.

Co-Authored-By: Claude Sonnet 5 <[email protected]>
2026-09-04 19:23:18 -05:00
NixReaperandClaude Sonnet 5 5b38226818 Fix duplicate enum type creation in initial migration
CI / lint (push) Canceled after 0s
CI / typecheck (push) Canceled after 0s
CI / unit-tests (push) Canceled after 0s
CI / integration-tests (push) Canceled after 0s
CI / container-build (push) Canceled after 0s
CI / dependency-audit (push) Canceled after 0s
Postgres ENUM columns auto-create their type as a side effect of the
first CREATE TABLE that uses them; since the migration also explicitly
pre-creates all 7 enum types up front (to group and control that DDL
alongside their explicit drop in downgrade()), the two collided with
"type already exists" on the first table past the first enum's owner.
create_type=False on each ENUM tells SQLAlchemy the migration manages
that DDL itself.

Found by running the migration against a real PostgreSQL for the first
time (docker compose up on a devbox) — this repo had no Docker/Postgres
available to test against when the migration was originally written.

Co-Authored-By: Claude Sonnet 5 <[email protected]>
2026-09-04 18:48:44 -05:00
NixReaperandClaude Sonnet 5 c20df1c503 Initial Aurora Core 0.1 implementation
CI / lint (push) Canceled after 0s
CI / typecheck (push) Canceled after 0s
CI / unit-tests (push) Canceled after 0s
CI / integration-tests (push) Canceled after 0s
CI / container-build (push) Canceled after 0s
CI / dependency-audit (push) Canceled after 0s
Production-grade media metadata backend per docs/aurora-core-spec.md:
Aurora-owned UUIDv7 identity system, normalized movie/TV/people/credits
schema, provider abstraction with a TMDb adapter, an idempotent async
ingestion pipeline (normalize -> match -> upsert -> provenance), a
Postgres FTS/trigram search layer, a stateless FastAPI v1 API with
Redis caching/rate-limiting, arq workers, dev/prod Docker Compose
topologies, unit + integration + load tests, CI, and architecture/
database/API/ERD documentation.

Co-Authored-By: Claude Sonnet 5 <[email protected]>
2026-09-04 17:34:03 -05:00