37 Commits

Include renames

Author	SHA1	Message	Date
NixReaperandClaude Sonnet 5	0ab31a314c	Add movie/series ratings and watchlist ... Builds on the accounts foundation: PUT/DELETE a rating (0.5-10.0) or a watchlist entry on any movie or series, all scoped to the bearer token's own user, plus GET /account/ratings|watchlist/{movies,series} to list them back. Both are polymorphic entity_type/entity_id tables, same shape as watch_provider_offerings and trending_entries, so movie and series share one repository/router each rather than duplicating per type. Co-Authored-By: Claude Sonnet 5 <[email protected]>	2026-09-06 10:47:41 -05:00
NixReaperandClaude Sonnet 5	1cdfbc59ac	Fix test isolation: trending tests were leaking data into other tests ... Not a bug in production code -- refresh_trending()'s tests use session_scope() for real, permanent commits (necessary, since refresh_trending manages its own sessions), unlike every other integration test's db_session fixture, which rolls back after each test. FakeTrendingProvider reused FAKE_MOVIE/FAKE_SERIES unmodified except for id/title, so every synthetic trending movie/series carried the same embedded video id and imdb_id -- values meant to be globally unique per real TMDb entity. Those commits were never rolled back, so they permanently occupied uq_video_provider_source/uq_external_id_source in the shared test database for the rest of the run, causing test_watch_providers.py (which runs later alphabetically and ingests the same fixtures unmodified via the normal rolled-back db_session) to collide against leaked data with 14 failures that had nothing to do with watch providers. Found by actually running the full suite against real Postgres -- confirms it was worth doing before this deploys. Co-Authored-By: Claude Sonnet 5 <[email protected]>	2026-09-06 07:45:39 -05:00
NixReaperandClaude Sonnet 5	7bee596a0e	Add user accounts: registration, login, and JWT auth ... The first authenticated-user concept in Aurora Core -- everything else in this codebase is public, read-only catalog data. Foundation for ratings and a watchlist (not yet built): POST /auth/register, POST /auth/login, GET /auth/me. Security specifics worth knowing: - Passwords hashed with argon2 (via argon2-cffi), never stored plain or logged. - Login returns an identical 401 for a nonexistent email and a wrong password on a real one, including matching response timing (burn_time_like_a_password_check) -- otherwise either the status code or the timing alone lets a client enumerate registered emails. - AURORA_SECRET_KEY now actually signs JWTs (previously defined but unused anywhere) -- .env.example's existing token_urlsafe(48) guidance already produces a long enough key, just flagged its new stakes. - Email matched case-insensitively via explicit normalization in the service layer, not relied on from Pydantic's EmailStr (which does not lowercase the local part). 12 new integration tests (register/login/me over real HTTP via the `client` fixture) plus 10 unit tests for the hashing/JWT logic itself, following the same-day lesson that only unit tests actually execute locally without Postgres -- the auth core is tested for real here rather than only through integration tests that might not run before a deploy. Co-Authored-By: Claude Sonnet 5 <[email protected]>	2026-09-06 06:15:39 -05:00
NixReaperandClaude Sonnet 5	4403305da6	Add trending movies/series, refreshed on a schedule ... Mirrors TMDb's own /trending/{movie|tv}/{day|week} ranking rather than Aurora computing trending itself from popularity deltas -- that would require re-syncing the entire catalog on a schedule, infeasible at any real scale, whereas TMDb already computes this well from signals (real view/search velocity) Aurora has no access to. New: TrendingEntry model + migration 0003, MetadataProvider.get_trending (concrete/optional, like aclose(), so existing test fakes don't all need updating), refresh_trending() which actually ingests each trending item (only ~20 per window, cheap enough to fully ingest rather than apply the never-fabricate-a-stub rule recommendations uses), a twice-daily arq cron job (WorkerSettings.cron_jobs, new to this codebase), a manual admin trigger for right after a deploy, and GET /movies/trending + GET /series/trending. Co-Authored-By: Claude Sonnet 5 <[email protected]>	2026-09-06 05:18:02 -05:00
NixReaperandClaude Sonnet 5	6c95e18aad	Add watch providers (where to stream/rent/buy) ... New TMDb integration end to end: append_to_response "watch/providers", normalizer parsing grouped by country/monetization type, a WatchProvider catalog entry (external-id keyed, same get_or_create pattern as Company/ Collection/Person) + polymorphic WatchProviderOffering table (new migration 0002), ingestion write (including provider logos via the existing Artwork table), and API exposure -- embedded on movie/series detail plus a dedicated /watch-providers sub-route, shaped to mirror TMDb's own response for easy porting. Note: this data comes from JustWatch via TMDb and carries its own attribution requirement separate from TMDb's general one -- flagged in docs/api.md, must be satisfied before showing this in the frontend. Co-Authored-By: Claude Sonnet 5 <[email protected]>	2026-09-06 04:59:02 -05:00
NixReaperandClaude Sonnet 5	74ebad379a	Add now-playing and upcoming movie endpoints ... Buildable now that release_dates exists: GET /movies/now-playing and /movies/upcoming filter on release_type="theatrical" per region, windowed around today. A movie can carry more than one theatrical-type row in the same region (limited release then wide release both map to "theatrical" per normalizer._RELEASE_TYPE_MAP) -- collapsed to one row per movie via a grouped subquery rather than joining ReleaseDate directly, which would duplicate the movie in the page and inflate the total count. Co-Authored-By: Claude Sonnet 5 <[email protected]>	2026-09-06 04:45:09 -05:00
NixReaperandClaude Sonnet 5	567b4ef068	Add keyword browsing/filtering and expose alternate titles ... Keywords were ingested and shown read-only on detail pages, but had no directory (GET /keywords) or filter param -- the same gap genres had before today. Mirrors the genre browse pattern exactly: repository, service, route, plus a `keyword` filter on GET /movies and GET /series. Alternate/translated titles ("Also Known As") were normalized down to bare title strings and never written to MovieAlternateTitle/ SeriesAlternateTitle at all. Normalizer now captures country_code + type per TMDb's actual shape; ingestion writes them; movie/series detail now expose an alternate_titles list. Co-Authored-By: Claude Sonnet 5 <[email protected]>	2026-09-06 04:40:32 -05:00
NixReaperandClaude Sonnet 5	e1a36ce227	Expose certifications and per-country release dates ... TMDb's release_dates/content_ratings blocks were already fetched via append_to_response but discarded at normalization -- every downstream layer (ingestion write, repository read, schema, service) was unbuilt. Adds the full pipeline: NormalizedReleaseDate + get_or_create_certification, a shared polymorphic sync/read (movie and series both use it), and MovieDetailOut/SeriesDetailOut certification + release_dates fields. Series content_ratings has no date of its own, so each rating is anchored to the show's first_air_date with release_type="tv". Co-Authored-By: Claude Sonnet 5 <[email protected]>	2026-09-06 04:07:18 -05:00
NixReaperandClaude Sonnet 5	340cadd99d	Add GET /genres directory endpoint ... GET /movies and GET /series accept a genre filter (an Aurora genre id), but nothing exposed what those ids actually are -- genres otherwise only ever appear embedded in a movie/series response's own genres list. This is the resolve-by-name-first-then-filter step the frontend needs to build real genre browsing. No filter, no sort_by -- always alphabetical by name, same pattern as collections/companies (genres have no categorical field or popularity/ date to sort by). page_size defaults to 50 (max 200) rather than the 20/50 used elsewhere, since genres are a small, fixed reference set (TMDb has ~19-20 total) a client typically wants in full for a filter dropdown, not paginated like a catalog of millions of movies. Also fixed two stale "behind Traefik" mentions in app/main.py's module docstring while touching this file, left over from the earlier cPanel/host-webserver deployment change. Co-Authored-By: Claude Sonnet 5 <[email protected]>	2026-09-06 00:05:10 -05:00
NixReaperandClaude Sonnet 5	71388e254e	Fix get_or_create_provider race crashing concurrent bulk import workers ... Found live: running bulk_import.py's fixed 50-worker pool against the real movie export, 49 of the first 50 concurrent workers crashed immediately with "duplicate key value violates unique constraint uq_providers_slug" -- all racing to create the one-and-only "tmdb" Provider row on first run. get_or_create_provider was a plain check-then-insert with no race recovery, unlike its sibling get_or_create_* helpers in reference_repository.py, which already handle exactly this class of race (arq's own concurrency and season- ingestion cascades already exercise it there). Extracted the SAVEPOINT-based create_or_recover helper out of reference_repository.py into app/repositories/concurrency.py so both modules share it, and applied it to get_or_create_provider and create_ingestion_job (the latter's idempotency-key race is less likely under bulk_import's own access pattern -- each of its workers processes a distinct external_id -- but a concurrent admin-triggered ingestion of the same id during the weeks this bulk import runs is a realistic way to hit it too). reference_repository.py re-exports the helper under its old private name so its 8 existing call sites and test_reference_repository.py need no changes. Co-Authored-By: Claude Sonnet 5 <[email protected]>	2026-09-05 23:21:47 -05:00
NixReaperandClaude Sonnet 5	b05d34db5f	Fix bulk_import.py stalling on real-scale exports (1M+ ids) ... asyncio.gather(*(_worker(id) for id in ids)) eagerly wraps every argument into a Task before awaiting any of them -- with a semaphore bounding concurrent *execution* but not concurrent Task *creation*, running this against the real movie export (1,240,787 ids) meant constructing 1.24 million live Task + coroutine objects up front, which looked indistinguishable from hanging for minutes before any work could start, confirmed live in production tonight. Replaced with a fixed pool of _CONCURRENCY (50) consumer tasks pulling from a plain asyncio.Queue pre-loaded with every id -- only 50 Task objects ever exist regardless of catalog size. Verified: 500K enqueues against no-op stubs now complete in 0.59s (previously: the setup loop alone dominated), and the real bottleneck is correctly back to the per-id DB round-trip + Redis push, gated by the pool of 50 -- not Python-level Task scheduling overhead. Co-Authored-By: Claude Sonnet 5 <[email protected]>	2026-09-05 23:08:37 -05:00
NixReaperandClaude Sonnet 5	a52ac676ff	Add full-catalog bulk import from TMDb's daily ID exports ... app/services/bulk_import.py fetches and parses TMDb's daily ID export files (published once a day specifically for full-catalog sync, unlike /discover which caps at 10,000 results per query) -- today's file with a fallback to yesterday's, since TMDb publishes at a fixed time and "today's" may not exist yet. app/scripts/bulk_import.py is a standalone ops script (not an arq task, since enqueueing millions of jobs would blow past arq's job_timeout) that fans every id from the export into the existing per-entity ingestion pipeline -- get_or_create_ingestion_job + the same arq task each admin endpoint already uses, so a single id's behavior is identical whether it came from the admin API or this script. Bounded concurrency (50) on the enqueue phase; idempotency-keyed jobs make interrupting and re-running the script safe. Real TMDb catalog size (checked live): ~1.24M movies, 231K series, 4.9M people, 9.8K collections -- ~6.4M entities. At default worker concurrency this is realistically weeks of continuous processing, not hours, documented as such in README.md. Also added 429 (rate-limited) handling to TMDbProvider before running this at volume: previously a 429 fell into the generic >=400 branch and failed the request outright with zero retry, since only httpx.TransportError triggered the existing retry decorator. Now a 429 raises ProviderRateLimitedError carrying the response's own Retry-After hint, and tenacity's wait strategy honors that hint directly instead of guessing via exponential backoff. Co-Authored-By: Claude Sonnet 5 <[email protected]>	2026-09-05 22:12:12 -05:00
NixReaperandClaude Sonnet 5	0a50e0ffb6	Fix two more test bugs surfaced by the full integration run ... 1. test_ingestion_cache_invalidation.py: the job was created through the test's db_session fixture, whose commit() only ends a savepoint that gets rolled back at teardown (see tests/conftest.py) -- it's never a real commit. _run_ingestion looks the job up through its own session_scope() (a genuinely separate connection), which could never see it, so the very first _require_job call raised "job vanished mid-run" every time. Worse, test_failed_ingestion_does_not_ invalidate_the_cache still passed, coincidentally -- it expected a RuntimeError and got one, just the wrong one, masking the bug. Fixed by having _make_job create the job through session_scope() itself, matching test_ingestion_retry.py's existing pattern. 2. Same class of bug as the video-collision fix in the last commit: FAKE_MOVIE's fixed imdb_id ("tt0133093"), not just its video, also collides across two different fake movies sharing it unmodified -- uq_external_id_source is a real unique constraint on (provider_id, entity_type, external_id). Fixed by also overriding imdb_id on the second movie in both affected tests. All caught by running the actual integration suite against Postgres -- these paths were previously only collect-only'd locally (no local DB). Co-Authored-By: Claude Sonnet 5 <[email protected]>	2026-09-05 20:17:02 -05:00
NixReaperandClaude Sonnet 5	29fffc8919	Fix test fixtures: two movies can't share FAKE_MOVIE's embedded video ... FAKE_MOVIE carries a fixed provider_video_id ("abc123"); two different fake movies both spreading it unmodified and getting ingested in the same test collide on the real uq_video_provider_source unique constraint (a video id belongs to exactly one entity). Found by actually running the tests against Postgres on the devbox -- these paths were only ever collect-only'd locally before now (no local DB). Affected: test_movie_recommendations_only_include_already_ingested_movies and test_ingest_collection_only_links_already_ingested_movies, both of which ingest two FAKE_MOVIE-derived movies in one test. Fixed by overriding videos to empty on the second movie in each. Co-Authored-By: Claude Sonnet 5 <[email protected]>	2026-09-05 20:12:03 -05:00
NixReaperandClaude Sonnet 5	f1ac072794	Add GET /collections browse endpoint; fix embedded collection poster_path ... Completes browse/discover across all five entity types: movies, series, people, companies, and now collections. No filter (no categorical field like company_type exists for a collection) -- just page/page_size, always alphabetical by name. While adding poster_path to CollectionBriefOut for the browse listing, found the same class of bug fixed earlier for company logo_path: movie_service's embedded `collection` field never looked up its poster artwork either, so it stayed null even after ingest_collection populated one. Fixed alongside since it's the same schema field this feature was already touching. Also removed PageMeta from schemas/common.py -- scaffolding from early spec work that was never actually wired into any endpoint; all five browse endpoints shipped this session use a flat page/page_size/ total_results/total_pages shape instead. Co-Authored-By: Claude Sonnet 5 <[email protected]>	2026-09-05 20:08:26 -05:00
NixReaperandClaude Sonnet 5	4cf3556433	Fix test fixture: seed countries before companies referencing them ... companies.country_code has a real FK to countries -- real ingestion always calls get_or_create_country first (see _replace_movie_companies), but the company browse test fixture called get_or_create_company directly with a raw country code, violating that FK. Found by actually running the tests against Postgres on the devbox. Co-Authored-By: Claude Sonnet 5 <[email protected]>	2026-09-05 19:15:09 -05:00
NixReaperandClaude Sonnet 5	92b0f4e077	Add GET /companies browse/discover endpoint ... Completes the browse/discover set across all four entity types. Companies have no popularity or date field, so unlike movies/series/ people there's no sort_by choice -- always alphabetical by name. Filters are company_type (validated against the enum) and country (a plain ISO 3166-1 code comparison, nothing to validate). build_company_summary refactored to share row-to-summary logic with the new list path, same pattern used for movies/series/people. Co-Authored-By: Claude Sonnet 5 <[email protected]>	2026-09-05 19:08:14 -05:00
NixReaperandClaude Sonnet 5	6aa9f0b840	Add GET /people browse/discover endpoint ... Extends the browse/discover pattern already built for movies/series to the fourth major entity type. No genre/year/status equivalent for people -- known_for_department is the one categorical filter that actually means something for a cast/crew list, and sort_by is just popularity (default) or name. No invalid-filter case either: unlike genre/status, known_for_department is a plain string compared directly, nothing to parse or validate. build_person_summary refactored to share row-to-summary logic with the new list path, same pattern used for movies/series/companies. Co-Authored-By: Claude Sonnet 5 <[email protected]>	2026-09-05 10:32:16 -05:00
NixReaperandClaude Sonnet 5	74adcf77bc	Add series endpoint parity: /credits, /images, /videos ... Movies have separate /credits, /images, /videos sub-endpoints; series only had /seasons. Adds the matching three for series, reusing the same cast/crew/creators three-way split series detail already does (extracted into _split_series_credits so both share it). Co-Authored-By: Claude Sonnet 5 <[email protected]>	2026-09-05 09:52:08 -05:00
NixReaperandClaude Sonnet 5	6956bf7406	Add movies/series filmography to company detail ... GET /companies/{id} showed only the company's own info. Adds movies and series lists (any role: production, distribution, network), ordered by release/first-air date, read straight off the existing movie_companies/series_companies junction rows -- no separate sync needed since those only ever exist as a side effect of ingesting the movie or series itself. Co-Authored-By: Claude Sonnet 5 <[email protected]>	2026-09-05 09:34:31 -05:00
NixReaperandClaude Sonnet 5	606d2532c4	Add per-episode breakdown to guest-star filmography credits ... FilmographyCreditOut already aggregated a guest star's episode appearances into one row per series with an episode_count, but never exposed which episodes those were even though the individual episode Credit rows exist in the database. Adds an `episodes` list (season/ episode number, title, air date, sorted chronologically) alongside the existing count -- closes a known limitation tracked in docs/api.md. Co-Authored-By: Claude Sonnet 5 <[email protected]>	2026-09-05 09:19:52 -05:00
NixReaperandClaude Sonnet 5	20f5e82d1b	Add GET /movies and GET /series browse/discover endpoints ... The complement to /search for browsing without a query: paginated, filterable (genre, year, status) and sortable (popularity, release/air date, rating, title) listings. Offset-based pagination rather than the Aurora ID cursor scheme, since sorting by popularity/rating/date doesn't line up with creation-order cursors. Total count is computed via a window function in the same query as the page of results, avoiding a second round trip per request. An invalid filter (malformed genre id, unknown status) is a 422 via a per-service InvalidMovieFilterError/InvalidSeriesFilterError, caught before any DB query runs. build_movie_summary/build_series_summary refactored to share their row-to-summary logic with the new list path, avoiding a redundant get_by_id lookup when the row is already in hand. Co-Authored-By: Claude Sonnet 5 <[email protected]>	2026-09-05 09:01:11 -05:00
NixReaperandClaude Sonnet 5	892f78016c	Add collection/franchise ingestion ... Collections were bare stubs (name only) lazily created the moment a movie's belongs_to_collection reference was seen. Adds a real ingest_collection pipeline: TMDb's /collection/{id} for overview and poster/backdrop artwork, plus ordered CollectionMovie membership -- linking only movies already ingested into Aurora, the same never-fabricate-a-stub rule movie recommendations already follow. get_or_create_collection now refreshes name/overview on an existing stub instead of only creating, guarded so a bare movie-stub call (overview=None) never clobbers real data a full ingestion already filled in. New POST /api/v1/admin/ingest/collection endpoint and ingest_collection_task; GET /api/v1/collections/{id} needed no changes since its read path (collection_service.py) already assembled the full detail shape. Co-Authored-By: Claude Sonnet 5 <[email protected]>	2026-09-05 08:37:27 -05:00
NixReaperandClaude Sonnet 5	4af4ba40ef	Add GET /movies/{id}/recommendations ... Ingestion already stored movie_recommendations edges from TMDb's recommendations list (linking only to movies already ingested into Aurora, per the never-fabricate-a-stub rule), but nothing exposed them. Adds the repository lookup, service assembly, and route to read them back in ranked order. Co-Authored-By: Claude Sonnet 5 <[email protected]>	2026-09-05 08:16:18 -05:00
NixReaperandClaude Sonnet 5	73488a0d80	Fix movie/series API responses never surfacing company logo_path ... movie_service.get_movie_detail() hardcoded companies[].logo_path = None instead of looking up the company's logo artwork, and tv_service's networks[]/companies[] never set logo_path at all -- both just built CompanyBriefOut without the artwork lookup that company_service.py (and _season_brief for posters) already does correctly. Ingestion and storage were working the whole time; the API layer just never read the result. Co-Authored-By: Claude Sonnet 5 <[email protected]>	2026-09-05 08:01:56 -05:00
NixReaperandClaude Sonnet 5	2138d4afc3	Invalidate movie/series/person detail cache on re-ingestion ... cached_response() had no write-side invalidation, so re-ingesting an entity kept serving its pre-ingestion cached response for up to CACHE_DEFAULT_TTL. Found live: re-ingesting a movie to test the new company-logo field still showed logo_path: None because the sliding 5-minute cache from earlier testing never expired. Co-Authored-By: Claude Sonnet 5 <[email protected]>	2026-09-05 07:56:00 -05:00
NixReaperandClaude Sonnet 5	c03a75989d	Ingest and self-host company/network logos ... CompanyBriefOut.logo_path and CompanyDetailOut.logo_path have existed in the API schemas since Core 0.1's first pass, but nothing ever actually populated them -- TMDb's production_companies/networks entries carry a logo_path field that the normalizer simply never read, so every response has silently returned null there. - NormalizedCompanyRef now captures logo_path from both movie production_companies and series networks/production_companies. - normalizer.company_logo_artwork() builds the single-image artwork entry for it (a company has exactly one logo ingested, so it's always is_primary) -- centralized there rather than duplicated in both ingestion modules, matching how movie/series artwork URLs are already built in one place. - movie_ingestion.py and series_ingestion.py sync it via the same sync_artwork() the rest of section 10's self-hosting already uses, so it gets the same re-ingestion-safe upsert behavior for free. - tasks.py cascades a download_artwork_task per referenced company after movie/series ingestion, same fan-out pattern as movie/series/season/person artwork. Co-Authored-By: Claude Sonnet 5 <[email protected]>	2026-09-05 07:46:30 -05:00
NixReaperandClaude Sonnet 5	7c4bfe6cf4	Fix sync_artwork clobbering is_primary via duplicate list entries ... TMDb's images.posters/backdrops alternates commonly include the *same* image as the top-level poster_path/backdrop_path -- so normalize_movie legitimately produces the same provider_asset_id twice in one artwork_list: once from the top-level field (is_primary=True), once from the general alternates loop (is_primary=False, the default). sync_artwork matched both occurrences to the same existing DB row by provider_asset_id and did a plain overwrite each time, so whichever occurrence it processed last won -- the non-primary duplicate always came second in list order, so it silently un-set is_primary right after the first occurrence had just set it. Caught by re-ingesting The Matrix on a devbox with the artwork download feature live: the movie ingested successfully, but zero download jobs were ever enqueued because get_primary_artwork_ids found nothing marked is_primary -- the flag had been clobbered back to false immediately after being set. Fixed by resetting is_primary on all existing rows up front (so a provider dropping its "primary" pick on an image actually clears, not just adds) and accumulating with OR within the batch instead of a plain overwrite, so any primary occurrence in one sync sticks regardless of processing order. Also dedupes brand-new rows within the same batch the same way, closing the equivalent bug for images that don't have an existing row yet. Co-Authored-By: Claude Sonnet 5 <[email protected]>	2026-09-05 07:30:40 -05:00
NixReaperandClaude Sonnet 5	fef914db36	Add artwork self-hosting: local-disk download worker + storage ... Section 10: every image API responses reference has pointed straight at TMDb's CDN since day one -- Aurora never actually downloaded or served artwork itself, despite storing dimensions/checksum/local_path columns for exactly that. Closes the gap: - is_primary was declared on the Artwork model but never actually set anywhere -- every row defaulted to False. Now the normalizer marks the one image each provider designates as *the* image for an entity (TMDb's poster_path/backdrop_path/profile_path) as primary, distinct from the dozens of alternates also ingested, including correctly marking an existing row primary (not inserting a duplicate) when that image already appears in the alternates list. - app/services/artwork/: an ArtworkStorage interface with a LocalDiskStorage implementation (checksum-named files, single-host deployment via a Docker volume shared between api and worker -- see docs/architecture.md for the S3/R2 upgrade path), and a download pipeline with content-type validation, a size cap, and path-traversal protection. - After movie/series/person ingestion succeeds, tasks.py cascades one download_artwork_task per primary artwork id via arq's ctx['redis'] (same fan-out pattern as the season cascade) -- fire-and-forget, not tracked in ingestion_jobs, since a failed image cache is a known-safe fallback to the provider URL, not a user-facing failure. - app/main.py serves MEDIA_ROOT at MEDIA_URL_PREFIX; local_path is preferred over original_url once set (already-existing fallback logic in app/services/common.py:artwork_url, now actually exercised). Also fixes a real interaction bug this surfaced: every ingestion pipeline deleted and recreated all of an entity's Artwork rows on each re-sync, which would have silently wiped local_path/checksum on every re-ingestion, forcing a full re-download every time and defeating the point of caching locally at all. app/repositories/polymorphic.py's new sync_artwork upserts by provider_asset_id (TMDb's stable per-image path) instead, preserving downloaded copies across re-sync, and replaces three near-identical delete+recreate helpers that used to live separately in movie_ingestion.py, series_ingestion.py, and person_ingestion.py. Co-Authored-By: Claude Sonnet 5 <[email protected]>	2026-09-05 03:26:28 -05:00
NixReaperandClaude Sonnet 5	cd4dd98adb	Make failed ingestion jobs actually retry, not just say "retrying" ... arq only auto-retries a job on its own Retry exception (or CancelledError) -- never on an arbitrary exception a task raises. _run_ingestion caught failures, recorded them via mark_job_failed (which sets ingestion_jobs.status to "retrying" when attempts hasn't hit max_attempts), and then just re-raised the original exception -- which arq treats as a terminal failure. The DB said "retrying" forever; nothing ever put the job back on the queue. Confirmed live: cascading Better Call Saul's 6 seasons hit the get_or_create race fixed in the previous commit on 4 of them, and the worker's own shutdown summary showed "4 failed, 0 retries" -- every one of those was permanently lost despite max_attempts=5 suggesting otherwise. _run_ingestion now raises arq.Retry (capped exponential backoff) when mark_job_failed left the job in RETRYING state, so arq actually requeues it; only once attempts are exhausted does the original exception propagate as a genuine terminal failure. Co-Authored-By: Claude Sonnet 5 <[email protected]>	2026-09-05 02:31:43 -05:00
NixReaperandClaude Sonnet 5	dd3ff9764b	Fix race condition in get_or_create_* under concurrent ingestion ... Every get_or_create_* helper (genre, keyword, language, country, company, collection, person) followed check-then-act: look up by natural key, insert if missing. That was safe as long as nothing ingested concurrently for overlapping data -- which stopped being true the moment ingest_series_task could cascade season ingestion (include_seasons), since arq runs multiple queued jobs concurrently. Two seasons of the same series sharing a guest star, crew member, or company can now race to create it: both see "not found", both try to INSERT, and the loser hits a unique-constraint violation instead of finding the winner's row. Reproduced live: cascading Better Call Saul (60059) with include_seasons crashed a season job with "duplicate key value violates unique constraint uq_external_id_source" on a recurring guest actor shared across two seasons ingesting at the same time. _create_or_recover wraps each insert in a SAVEPOINT and, on that specific failure, rolls back just the savepoint and re-queries for what the concurrent job committed -- recovering instead of aborting the whole ingestion job. Added unit tests for the retry mechanism itself; genuinely reproducing the underlying race in an automated test would need real Postgres MVCC timing, which isn't something a deterministic test can force reliably. Co-Authored-By: Claude Sonnet 5 <[email protected]>	2026-09-05 02:27:07 -05:00
NixReaperandClaude Sonnet 5	64169c4aee	Fix test factory Provider collision from ingestion-cascade tests ... tests/integration/factories.py:make_provider did a raw INSERT assuming no "tmdb" Provider row existed yet. That held as long as every test used the db_session fixture's isolated, rolled-back transaction -- but the new ingestion-cascade tests call get_or_create_ingestion_job, which really commits via session_scope()'s own connection, outside that per-test isolation. Once that ran, every later test using the raw-insert factory hit "duplicate key value violates unique constraint uq_providers_slug". Fixed by using the app's own get_or_create_provider in the factory instead of a bare insert, which is also just correct: provider slugs are unique in production too, so the test helper should behave the same way real ingestion code does rather than assume a pristine table. Co-Authored-By: Claude Sonnet 5 <[email protected]>	2026-09-05 02:21:43 -05:00
NixReaperandClaude Sonnet 5	b4d1e8f755	Add opt-in season cascade to series ingestion ... Ingesting a series only ever produced season stubs -- getting actual episodes meant calling POST /admin/ingest/season once per season by hand, which is exactly what manual end-to-end testing kept requiring for every multi-season show. POST /api/v1/admin/ingest/series now takes include_seasons (default false, since a show can have dozens of seasons and that's dozens of TMDb calls fired from one request). When true, ingest_series_task fans out one ingest_season job per season once the series succeeds, using arq's own ctx['redis'] connection for the job (the same pattern arq's docs recommend for job chaining) rather than a second unrelated connection. Each fanned-out season job still goes through get_or_create_ingestion_job, so re-running a cascaded series ingestion reuses the same season job rows instead of creating duplicates. Co-Authored-By: Claude Sonnet 5 <[email protected]>	2026-09-05 02:03:25 -05:00
NixReaperandClaude Sonnet 5	ec6b5bdd7d	Surface series/episode credits in person filmography ... /people/{id}/credits only ever looked at entity_type == "movie", silently dropping the series-level cast/crew and episode-level guest-star Credit rows that series_ingestion.py has been writing all along -- a person ingested via a TV show would show an empty or movie-only filmography no matter how much TV work was actually in the database. Regular series credits map straight to a filmography row, same shape as movies. Guest-star episode credits are aggregated to one row per (series, department/job/character) with an episode_count, rather than one row per episode -- a prolific guest actor would otherwise flood the list with dozens of near-identical entries. Added tv_repository.get_series_by_ids/get_episodes_by_ids (bulk, mirroring the existing movie_repository pattern) to support it. Co-Authored-By: Claude Sonnet 5 <[email protected]>	2026-09-04 20:00:15 -05:00
NixReaperandClaude Sonnet 5	3acf00e9f0	Fix test bug: await inside a generator expression passed to all() ... Python turns a generator expression containing await into an async generator (since it's evaluated inside an async def), which the built-in all() can't consume -- it raised "TypeError: 'async_generator' object is not iterable" rather than testing anything. Replaced with a plain loop. Found by actually running the new integration tests. Co-Authored-By: Claude Sonnet 5 <[email protected]>	2026-09-04 19:45:29 -05:00
NixReaperandClaude Sonnet 5	3f6332559a	Add series/season/episode ingestion pipeline ... Movie and person ingestion existed, but the series/season/episode API endpoints had no way to actually populate them from TMDb -- the biggest remaining gap in Core 0.1's television scope. - normalize_series/normalize_season in the metadata normalizer, including two TMDb quirks that would otherwise silently produce empty data: /tv nests keywords and alternative_titles under "results" (not "keywords"/"titles" like /movie), and series-level creators come from a separate created_by array, not the credits cast/crew lists. - ingest_series upserts the Series row plus season *stubs* (number, title, air date, poster -- everything TMDb inlines on /tv/{id}); ingest_season fetches and upserts one season's full episode list, and requires the series to already exist (TMDb has no single endpoint for a whole series' episodes, so this has to be a separate per-season call). - Wired through the same job-lifecycle path as movie/person ingestion (runner.py, arq tasks, admin endpoints), with the shared boilerplate between them factored into runner._run_ingestion and admin.py:_enqueue rather than copy-pasted a third and fourth time. - Unit tests for both normalizers (runnable without a DB) and integration tests for the full ingest_series -> ingest_season flow, including the series-must-exist-first error and idempotency on re-ingestion. Co-Authored-By: Claude Sonnet 5 <[email protected]>	2026-09-04 19:43:56 -05:00
NixReaperandClaude Sonnet 5	c20df1c503	Initial Aurora Core 0.1 implementation ... Production-grade media metadata backend per docs/aurora-core-spec.md: Aurora-owned UUIDv7 identity system, normalized movie/TV/people/credits schema, provider abstraction with a TMDb adapter, an idempotent async ingestion pipeline (normalize -> match -> upsert -> provenance), a Postgres FTS/trigram search layer, a stateless FastAPI v1 API with Redis caching/rate-limiting, arq workers, dev/prod Docker Compose topologies, unit + integration + load tests, CI, and architecture/ database/API/ERD documentation. Co-Authored-By: Claude Sonnet 5 <[email protected]>	2026-09-04 17:34:03 -05:00