49 Commits

This Branch

This Branch

All Branches

Author	SHA1	Message	Date
NixReaperandClaude Sonnet 5	0ab31a314c	Add movie/series ratings and watchlist ... Builds on the accounts foundation: PUT/DELETE a rating (0.5-10.0) or a watchlist entry on any movie or series, all scoped to the bearer token's own user, plus GET /account/ratings|watchlist/{movies,series} to list them back. Both are polymorphic entity_type/entity_id tables, same shape as watch_provider_offerings and trending_entries, so movie and series share one repository/router each rather than duplicating per type. Co-Authored-By: Claude Sonnet 5 <[email protected]>	2026-09-06 10:47:41 -05:00
NixReaperandClaude Sonnet 5	a38c2074a6	Fix duplicate WatchProvider rows when the same provider appears twice ... Found by running the real test suite against Postgres: test_ingest_movie_dedupes_watch_provider_across_countries failed with 2 Netflix rows instead of 1 when a movie's watch/providers block lists the same provider (Netflix, external_id 8) in more than one country -- the first genuine case in the whole ingestion pipeline where the same external entity gets resolved via get_or_create twice within one ingest before anything commits (company/person/collection never have this shape in existing fixtures). sync_watch_providers now resolves each distinct provider (by external_id) at most once per ingest and reuses it for every offering that references it, rather than calling get_or_create_watch_provider once per offering regardless of repeats. Co-Authored-By: Claude Sonnet 5 <[email protected]>	2026-09-06 07:55:36 -05:00
NixReaperandClaude Sonnet 5	1cdfbc59ac	Fix test isolation: trending tests were leaking data into other tests ... Not a bug in production code -- refresh_trending()'s tests use session_scope() for real, permanent commits (necessary, since refresh_trending manages its own sessions), unlike every other integration test's db_session fixture, which rolls back after each test. FakeTrendingProvider reused FAKE_MOVIE/FAKE_SERIES unmodified except for id/title, so every synthetic trending movie/series carried the same embedded video id and imdb_id -- values meant to be globally unique per real TMDb entity. Those commits were never rolled back, so they permanently occupied uq_video_provider_source/uq_external_id_source in the shared test database for the rest of the run, causing test_watch_providers.py (which runs later alphabetically and ingests the same fixtures unmodified via the normal rolled-back db_session) to collide against leaked data with 14 failures that had nothing to do with watch providers. Found by actually running the full suite against real Postgres -- confirms it was worth doing before this deploys. Co-Authored-By: Claude Sonnet 5 <[email protected]>	2026-09-06 07:45:39 -05:00
NixReaperandClaude Sonnet 5	7bee596a0e	Add user accounts: registration, login, and JWT auth ... The first authenticated-user concept in Aurora Core -- everything else in this codebase is public, read-only catalog data. Foundation for ratings and a watchlist (not yet built): POST /auth/register, POST /auth/login, GET /auth/me. Security specifics worth knowing: - Passwords hashed with argon2 (via argon2-cffi), never stored plain or logged. - Login returns an identical 401 for a nonexistent email and a wrong password on a real one, including matching response timing (burn_time_like_a_password_check) -- otherwise either the status code or the timing alone lets a client enumerate registered emails. - AURORA_SECRET_KEY now actually signs JWTs (previously defined but unused anywhere) -- .env.example's existing token_urlsafe(48) guidance already produces a long enough key, just flagged its new stakes. - Email matched case-insensitively via explicit normalization in the service layer, not relied on from Pydantic's EmailStr (which does not lowercase the local part). 12 new integration tests (register/login/me over real HTTP via the `client` fixture) plus 10 unit tests for the hashing/JWT logic itself, following the same-day lesson that only unit tests actually execute locally without Postgres -- the auth core is tested for real here rather than only through integration tests that might not run before a deploy. Co-Authored-By: Claude Sonnet 5 <[email protected]>	2026-09-06 06:15:39 -05:00
NixReaperandClaude Sonnet 5	fde5e76bd7	Fix crash: watch_provider was never registered as an Aurora ID entity type ... app/services/common.py:build_watch_providers_out calls aurora_id("watch_provider", ...) for every provider entry -- this raised ValueError on any movie/series detail response (or the dedicated /watch-providers route) containing watch-provider data, since the entity type was never added to ENTITY_PREFIXES. Never caught locally because every test exercising this path is an integration test that skips without a reachable Postgres. Not yet deployed (redeploy has been on hold pending the bulk import), so this never hit production traffic, but confirms it's worth actually running the test suite against real Postgres before the next deploy rather than trusting lint/mypy + locally-skipped tests alone. Co-Authored-By: Claude Sonnet 5 <[email protected]>	2026-09-06 06:08:08 -05:00
NixReaperandClaude Sonnet 5	56a03f9cfe	Add a public API wiki at /wiki ... Builds docs/ into a static site via mkdocs + mkdocs-material in its own Docker build stage (docs-builder), so mkdocs itself never ends up in the runtime image -- only the rendered HTML does, mounted at /wiki alongside the existing /docs OpenAPI schema (unconditionally, unlike /docs, since this is curated content meant to be public regardless of AURORA_API_DOCS_ENABLED). docs/aurora-core-spec.md (the internal build spec, referenced by "section N" comments throughout the codebase) is excluded from the built site entirely via exclude_docs, not just left out of the nav -- it was never meant for public consumption. Co-Authored-By: Claude Sonnet 5 <[email protected]>	2026-09-06 06:04:59 -05:00
NixReaperandClaude Sonnet 5	4403305da6	Add trending movies/series, refreshed on a schedule ... Mirrors TMDb's own /trending/{movie|tv}/{day|week} ranking rather than Aurora computing trending itself from popularity deltas -- that would require re-syncing the entire catalog on a schedule, infeasible at any real scale, whereas TMDb already computes this well from signals (real view/search velocity) Aurora has no access to. New: TrendingEntry model + migration 0003, MetadataProvider.get_trending (concrete/optional, like aclose(), so existing test fakes don't all need updating), refresh_trending() which actually ingests each trending item (only ~20 per window, cheap enough to fully ingest rather than apply the never-fabricate-a-stub rule recommendations uses), a twice-daily arq cron job (WorkerSettings.cron_jobs, new to this codebase), a manual admin trigger for right after a deploy, and GET /movies/trending + GET /series/trending. Co-Authored-By: Claude Sonnet 5 <[email protected]>	2026-09-06 05:18:02 -05:00
NixReaperandClaude Sonnet 5	6c95e18aad	Add watch providers (where to stream/rent/buy) ... New TMDb integration end to end: append_to_response "watch/providers", normalizer parsing grouped by country/monetization type, a WatchProvider catalog entry (external-id keyed, same get_or_create pattern as Company/ Collection/Person) + polymorphic WatchProviderOffering table (new migration 0002), ingestion write (including provider logos via the existing Artwork table), and API exposure -- embedded on movie/series detail plus a dedicated /watch-providers sub-route, shaped to mirror TMDb's own response for easy porting. Note: this data comes from JustWatch via TMDb and carries its own attribution requirement separate from TMDb's general one -- flagged in docs/api.md, must be satisfied before showing this in the frontend. Co-Authored-By: Claude Sonnet 5 <[email protected]>	2026-09-06 04:59:02 -05:00
NixReaperandClaude Sonnet 5	74ebad379a	Add now-playing and upcoming movie endpoints ... Buildable now that release_dates exists: GET /movies/now-playing and /movies/upcoming filter on release_type="theatrical" per region, windowed around today. A movie can carry more than one theatrical-type row in the same region (limited release then wide release both map to "theatrical" per normalizer._RELEASE_TYPE_MAP) -- collapsed to one row per movie via a grouped subquery rather than joining ReleaseDate directly, which would duplicate the movie in the page and inflate the total count. Co-Authored-By: Claude Sonnet 5 <[email protected]>	2026-09-06 04:45:09 -05:00
NixReaperandClaude Sonnet 5	567b4ef068	Add keyword browsing/filtering and expose alternate titles ... Keywords were ingested and shown read-only on detail pages, but had no directory (GET /keywords) or filter param -- the same gap genres had before today. Mirrors the genre browse pattern exactly: repository, service, route, plus a `keyword` filter on GET /movies and GET /series. Alternate/translated titles ("Also Known As") were normalized down to bare title strings and never written to MovieAlternateTitle/ SeriesAlternateTitle at all. Normalizer now captures country_code + type per TMDb's actual shape; ingestion writes them; movie/series detail now expose an alternate_titles list. Co-Authored-By: Claude Sonnet 5 <[email protected]>	2026-09-06 04:40:32 -05:00
NixReaperandClaude Sonnet 5	e1a36ce227	Expose certifications and per-country release dates ... TMDb's release_dates/content_ratings blocks were already fetched via append_to_response but discarded at normalization -- every downstream layer (ingestion write, repository read, schema, service) was unbuilt. Adds the full pipeline: NormalizedReleaseDate + get_or_create_certification, a shared polymorphic sync/read (movie and series both use it), and MovieDetailOut/SeriesDetailOut certification + release_dates fields. Series content_ratings has no date of its own, so each rating is anchored to the show's first_air_date with release_type="tv". Co-Authored-By: Claude Sonnet 5 <[email protected]>	2026-09-06 04:07:18 -05:00
NixReaperandClaude Sonnet 5	340cadd99d	Add GET /genres directory endpoint ... GET /movies and GET /series accept a genre filter (an Aurora genre id), but nothing exposed what those ids actually are -- genres otherwise only ever appear embedded in a movie/series response's own genres list. This is the resolve-by-name-first-then-filter step the frontend needs to build real genre browsing. No filter, no sort_by -- always alphabetical by name, same pattern as collections/companies (genres have no categorical field or popularity/ date to sort by). page_size defaults to 50 (max 200) rather than the 20/50 used elsewhere, since genres are a small, fixed reference set (TMDb has ~19-20 total) a client typically wants in full for a filter dropdown, not paginated like a catalog of millions of movies. Also fixed two stale "behind Traefik" mentions in app/main.py's module docstring while touching this file, left over from the earlier cPanel/host-webserver deployment change. Co-Authored-By: Claude Sonnet 5 <[email protected]>	2026-09-06 00:05:10 -05:00
NixReaperandClaude Sonnet 5	71388e254e	Fix get_or_create_provider race crashing concurrent bulk import workers ... Found live: running bulk_import.py's fixed 50-worker pool against the real movie export, 49 of the first 50 concurrent workers crashed immediately with "duplicate key value violates unique constraint uq_providers_slug" -- all racing to create the one-and-only "tmdb" Provider row on first run. get_or_create_provider was a plain check-then-insert with no race recovery, unlike its sibling get_or_create_* helpers in reference_repository.py, which already handle exactly this class of race (arq's own concurrency and season- ingestion cascades already exercise it there). Extracted the SAVEPOINT-based create_or_recover helper out of reference_repository.py into app/repositories/concurrency.py so both modules share it, and applied it to get_or_create_provider and create_ingestion_job (the latter's idempotency-key race is less likely under bulk_import's own access pattern -- each of its workers processes a distinct external_id -- but a concurrent admin-triggered ingestion of the same id during the weeks this bulk import runs is a realistic way to hit it too). reference_repository.py re-exports the helper under its old private name so its 8 existing call sites and test_reference_repository.py need no changes. Co-Authored-By: Claude Sonnet 5 <[email protected]>	2026-09-05 23:21:47 -05:00
NixReaperandClaude Sonnet 5	b05d34db5f	Fix bulk_import.py stalling on real-scale exports (1M+ ids) ... asyncio.gather(*(_worker(id) for id in ids)) eagerly wraps every argument into a Task before awaiting any of them -- with a semaphore bounding concurrent *execution* but not concurrent Task *creation*, running this against the real movie export (1,240,787 ids) meant constructing 1.24 million live Task + coroutine objects up front, which looked indistinguishable from hanging for minutes before any work could start, confirmed live in production tonight. Replaced with a fixed pool of _CONCURRENCY (50) consumer tasks pulling from a plain asyncio.Queue pre-loaded with every id -- only 50 Task objects ever exist regardless of catalog size. Verified: 500K enqueues against no-op stubs now complete in 0.59s (previously: the setup loop alone dominated), and the real bottleneck is correctly back to the per-id DB round-trip + Redis push, gated by the pool of 50 -- not Python-level Task scheduling overhead. Co-Authored-By: Claude Sonnet 5 <[email protected]>	2026-09-05 23:08:37 -05:00
NixReaperandClaude Sonnet 5	a52ac676ff	Add full-catalog bulk import from TMDb's daily ID exports ... app/services/bulk_import.py fetches and parses TMDb's daily ID export files (published once a day specifically for full-catalog sync, unlike /discover which caps at 10,000 results per query) -- today's file with a fallback to yesterday's, since TMDb publishes at a fixed time and "today's" may not exist yet. app/scripts/bulk_import.py is a standalone ops script (not an arq task, since enqueueing millions of jobs would blow past arq's job_timeout) that fans every id from the export into the existing per-entity ingestion pipeline -- get_or_create_ingestion_job + the same arq task each admin endpoint already uses, so a single id's behavior is identical whether it came from the admin API or this script. Bounded concurrency (50) on the enqueue phase; idempotency-keyed jobs make interrupting and re-running the script safe. Real TMDb catalog size (checked live): ~1.24M movies, 231K series, 4.9M people, 9.8K collections -- ~6.4M entities. At default worker concurrency this is realistically weeks of continuous processing, not hours, documented as such in README.md. Also added 429 (rate-limited) handling to TMDbProvider before running this at volume: previously a 429 fell into the generic >=400 branch and failed the request outright with zero retry, since only httpx.TransportError triggered the existing retry decorator. Now a 429 raises ProviderRateLimitedError carrying the response's own Retry-After hint, and tenacity's wait strategy honors that hint directly instead of guessing via exponential backoff. Co-Authored-By: Claude Sonnet 5 <[email protected]>	2026-09-05 22:12:12 -05:00
NixReaperandClaude Sonnet 5	39b2d4bfcd	Fix pgbouncer scram-sha-256 auth and document Postgres prod env vars ... Two real bugs hit during tonight's first production deploy (found and fixed live on the server, folded back into the tracked config so the next deploy doesn't hit either wall): 1. pgbouncer defaults to md5 auth; postgres:17-alpine defaults password_encryption to scram-sha-256. Every connection failed with "wrong password type" until AUTH_TYPE=scram-sha-256 was set explicitly on the pgbouncer service. 2. POSTGRES_USER/PASSWORD/DB were never documented anywhere (not in .env.example, despite docker-compose.prod.yml requiring them) -- added, with a note against `openssl rand -base64` for the password specifically: its output can contain /+=, which breaks pgbouncer's (and any DATABASE_URL parser's) naive URL parsing. Pointed at token_urlsafe instead, matching the other generated secrets. Co-Authored-By: Claude Sonnet 5 <[email protected]>	2026-09-05 21:52:11 -05:00
NixReaperandClaude Sonnet 5	905cd202eb	Replace Traefik with host-webserver reverse proxy in prod deployment ... The production host runs cPanel, which already owns ports 80/443 for other sites on the box -- there's no Traefik there. docker-compose.prod.yml now binds the api container to 127.0.0.1:8000 only; the host's own Apache/LiteSpeed (via cPanel) reverse-proxies api.amdbs.net to it and terminates TLS itself (AutoSSL). Dropped the external traefik-public network requirement and Traefik labels entirely. Also corrected a separate latent inaccuracy while touching this: a bare `docker compose up` (not `docker stack deploy`) never actually honored deploy.replicas -- docs now say so and point at `--scale` instead of implying free multi-replica scaling that was never really happening. README's Production deployment section rewritten with concrete steps: Cloudflare DNS record, cPanel subdomain + reverse proxy, then compose up. Co-Authored-By: Claude Sonnet 5 <[email protected]>	2026-09-05 20:49:55 -05:00
NixReaperandClaude Sonnet 5	ec9b1c28e3	Move CI workflow from .github/workflows to .gitea/workflows ... auroradragon.dev is self-hosted Gitea (v1.27.3) -- confirmed via its public API (/api/v1/version, /api/v1/repos/.../actions/tasks showing zero workflow runs ever recorded). Gitea only reads .gitea/workflows/ by default, not .github/workflows/, so this CI pipeline (lint, typecheck, unit tests, integration tests against real Postgres/Redis, container builds, dependency audit) has never actually executed. Still needs confirming: whether an Actions runner is registered for this repo -- Actions being enabled doesn't guarantee a runner exists to pick up queued jobs. Co-Authored-By: Claude Sonnet 5 <[email protected]>	2026-09-05 20:44:08 -05:00
NixReaperandClaude Sonnet 5	a8cb807d56	Document production domain split in README ... api.amdbs.net (backend) / www.amdbs.net (frontend) -- records the required AURORA_API_HOST and CORS_ORIGINS values for the production .env, and the external-Traefik prerequisite docker-compose.prod.yml already assumed but never spelled out. Co-Authored-By: Claude Sonnet 5 <[email protected]>	2026-09-05 20:33:25 -05:00
NixReaperandClaude Sonnet 5	0a50e0ffb6	Fix two more test bugs surfaced by the full integration run ... 1. test_ingestion_cache_invalidation.py: the job was created through the test's db_session fixture, whose commit() only ends a savepoint that gets rolled back at teardown (see tests/conftest.py) -- it's never a real commit. _run_ingestion looks the job up through its own session_scope() (a genuinely separate connection), which could never see it, so the very first _require_job call raised "job vanished mid-run" every time. Worse, test_failed_ingestion_does_not_ invalidate_the_cache still passed, coincidentally -- it expected a RuntimeError and got one, just the wrong one, masking the bug. Fixed by having _make_job create the job through session_scope() itself, matching test_ingestion_retry.py's existing pattern. 2. Same class of bug as the video-collision fix in the last commit: FAKE_MOVIE's fixed imdb_id ("tt0133093"), not just its video, also collides across two different fake movies sharing it unmodified -- uq_external_id_source is a real unique constraint on (provider_id, entity_type, external_id). Fixed by also overriding imdb_id on the second movie in both affected tests. All caught by running the actual integration suite against Postgres -- these paths were previously only collect-only'd locally (no local DB). Co-Authored-By: Claude Sonnet 5 <[email protected]>	2026-09-05 20:17:02 -05:00
NixReaperandClaude Sonnet 5	29fffc8919	Fix test fixtures: two movies can't share FAKE_MOVIE's embedded video ... FAKE_MOVIE carries a fixed provider_video_id ("abc123"); two different fake movies both spreading it unmodified and getting ingested in the same test collide on the real uq_video_provider_source unique constraint (a video id belongs to exactly one entity). Found by actually running the tests against Postgres on the devbox -- these paths were only ever collect-only'd locally before now (no local DB). Affected: test_movie_recommendations_only_include_already_ingested_movies and test_ingest_collection_only_links_already_ingested_movies, both of which ingest two FAKE_MOVIE-derived movies in one test. Fixed by overriding videos to empty on the second movie in each. Co-Authored-By: Claude Sonnet 5 <[email protected]>	2026-09-05 20:12:03 -05:00
NixReaperandClaude Sonnet 5	f1ac072794	Add GET /collections browse endpoint; fix embedded collection poster_path ... Completes browse/discover across all five entity types: movies, series, people, companies, and now collections. No filter (no categorical field like company_type exists for a collection) -- just page/page_size, always alphabetical by name. While adding poster_path to CollectionBriefOut for the browse listing, found the same class of bug fixed earlier for company logo_path: movie_service's embedded `collection` field never looked up its poster artwork either, so it stayed null even after ingest_collection populated one. Fixed alongside since it's the same schema field this feature was already touching. Also removed PageMeta from schemas/common.py -- scaffolding from early spec work that was never actually wired into any endpoint; all five browse endpoints shipped this session use a flat page/page_size/ total_results/total_pages shape instead. Co-Authored-By: Claude Sonnet 5 <[email protected]>	2026-09-05 20:08:26 -05:00
NixReaperandClaude Sonnet 5	4cf3556433	Fix test fixture: seed countries before companies referencing them ... companies.country_code has a real FK to countries -- real ingestion always calls get_or_create_country first (see _replace_movie_companies), but the company browse test fixture called get_or_create_company directly with a raw country code, violating that FK. Found by actually running the tests against Postgres on the devbox. Co-Authored-By: Claude Sonnet 5 <[email protected]>	2026-09-05 19:15:09 -05:00
NixReaperandClaude Sonnet 5	92b0f4e077	Add GET /companies browse/discover endpoint ... Completes the browse/discover set across all four entity types. Companies have no popularity or date field, so unlike movies/series/ people there's no sort_by choice -- always alphabetical by name. Filters are company_type (validated against the enum) and country (a plain ISO 3166-1 code comparison, nothing to validate). build_company_summary refactored to share row-to-summary logic with the new list path, same pattern used for movies/series/people. Co-Authored-By: Claude Sonnet 5 <[email protected]>	2026-09-05 19:08:14 -05:00
NixReaperandClaude Sonnet 5	6aa9f0b840	Add GET /people browse/discover endpoint ... Extends the browse/discover pattern already built for movies/series to the fourth major entity type. No genre/year/status equivalent for people -- known_for_department is the one categorical filter that actually means something for a cast/crew list, and sort_by is just popularity (default) or name. No invalid-filter case either: unlike genre/status, known_for_department is a plain string compared directly, nothing to parse or validate. build_person_summary refactored to share row-to-summary logic with the new list path, same pattern used for movies/series/companies. Co-Authored-By: Claude Sonnet 5 <[email protected]>	2026-09-05 10:32:16 -05:00
NixReaperandClaude Sonnet 5	74adcf77bc	Add series endpoint parity: /credits, /images, /videos ... Movies have separate /credits, /images, /videos sub-endpoints; series only had /seasons. Adds the matching three for series, reusing the same cast/crew/creators three-way split series detail already does (extracted into _split_series_credits so both share it). Co-Authored-By: Claude Sonnet 5 <[email protected]>	2026-09-05 09:52:08 -05:00
NixReaperandClaude Sonnet 5	6956bf7406	Add movies/series filmography to company detail ... GET /companies/{id} showed only the company's own info. Adds movies and series lists (any role: production, distribution, network), ordered by release/first-air date, read straight off the existing movie_companies/series_companies junction rows -- no separate sync needed since those only ever exist as a side effect of ingesting the movie or series itself. Co-Authored-By: Claude Sonnet 5 <[email protected]>	2026-09-05 09:34:31 -05:00
NixReaperandClaude Sonnet 5	606d2532c4	Add per-episode breakdown to guest-star filmography credits ... FilmographyCreditOut already aggregated a guest star's episode appearances into one row per series with an episode_count, but never exposed which episodes those were even though the individual episode Credit rows exist in the database. Adds an `episodes` list (season/ episode number, title, air date, sorted chronologically) alongside the existing count -- closes a known limitation tracked in docs/api.md. Co-Authored-By: Claude Sonnet 5 <[email protected]>	2026-09-05 09:19:52 -05:00
NixReaperandClaude Sonnet 5	20f5e82d1b	Add GET /movies and GET /series browse/discover endpoints ... The complement to /search for browsing without a query: paginated, filterable (genre, year, status) and sortable (popularity, release/air date, rating, title) listings. Offset-based pagination rather than the Aurora ID cursor scheme, since sorting by popularity/rating/date doesn't line up with creation-order cursors. Total count is computed via a window function in the same query as the page of results, avoiding a second round trip per request. An invalid filter (malformed genre id, unknown status) is a 422 via a per-service InvalidMovieFilterError/InvalidSeriesFilterError, caught before any DB query runs. build_movie_summary/build_series_summary refactored to share their row-to-summary logic with the new list path, avoiding a redundant get_by_id lookup when the row is already in hand. Co-Authored-By: Claude Sonnet 5 <[email protected]>	2026-09-05 09:01:11 -05:00
NixReaperandClaude Sonnet 5	892f78016c	Add collection/franchise ingestion ... Collections were bare stubs (name only) lazily created the moment a movie's belongs_to_collection reference was seen. Adds a real ingest_collection pipeline: TMDb's /collection/{id} for overview and poster/backdrop artwork, plus ordered CollectionMovie membership -- linking only movies already ingested into Aurora, the same never-fabricate-a-stub rule movie recommendations already follow. get_or_create_collection now refreshes name/overview on an existing stub instead of only creating, guarded so a bare movie-stub call (overview=None) never clobbers real data a full ingestion already filled in. New POST /api/v1/admin/ingest/collection endpoint and ingest_collection_task; GET /api/v1/collections/{id} needed no changes since its read path (collection_service.py) already assembled the full detail shape. Co-Authored-By: Claude Sonnet 5 <[email protected]>	2026-09-05 08:37:27 -05:00
NixReaperandClaude Sonnet 5	4af4ba40ef	Add GET /movies/{id}/recommendations ... Ingestion already stored movie_recommendations edges from TMDb's recommendations list (linking only to movies already ingested into Aurora, per the never-fabricate-a-stub rule), but nothing exposed them. Adds the repository lookup, service assembly, and route to read them back in ranked order. Co-Authored-By: Claude Sonnet 5 <[email protected]>	2026-09-05 08:16:18 -05:00
NixReaperandClaude Sonnet 5	73488a0d80	Fix movie/series API responses never surfacing company logo_path ... movie_service.get_movie_detail() hardcoded companies[].logo_path = None instead of looking up the company's logo artwork, and tv_service's networks[]/companies[] never set logo_path at all -- both just built CompanyBriefOut without the artwork lookup that company_service.py (and _season_brief for posters) already does correctly. Ingestion and storage were working the whole time; the API layer just never read the result. Co-Authored-By: Claude Sonnet 5 <[email protected]>	2026-09-05 08:01:56 -05:00
NixReaperandClaude Sonnet 5	2138d4afc3	Invalidate movie/series/person detail cache on re-ingestion ... cached_response() had no write-side invalidation, so re-ingesting an entity kept serving its pre-ingestion cached response for up to CACHE_DEFAULT_TTL. Found live: re-ingesting a movie to test the new company-logo field still showed logo_path: None because the sliding 5-minute cache from earlier testing never expired. Co-Authored-By: Claude Sonnet 5 <[email protected]>	2026-09-05 07:56:00 -05:00
NixReaperandClaude Sonnet 5	c03a75989d	Ingest and self-host company/network logos ... CompanyBriefOut.logo_path and CompanyDetailOut.logo_path have existed in the API schemas since Core 0.1's first pass, but nothing ever actually populated them -- TMDb's production_companies/networks entries carry a logo_path field that the normalizer simply never read, so every response has silently returned null there. - NormalizedCompanyRef now captures logo_path from both movie production_companies and series networks/production_companies. - normalizer.company_logo_artwork() builds the single-image artwork entry for it (a company has exactly one logo ingested, so it's always is_primary) -- centralized there rather than duplicated in both ingestion modules, matching how movie/series artwork URLs are already built in one place. - movie_ingestion.py and series_ingestion.py sync it via the same sync_artwork() the rest of section 10's self-hosting already uses, so it gets the same re-ingestion-safe upsert behavior for free. - tasks.py cascades a download_artwork_task per referenced company after movie/series ingestion, same fan-out pattern as movie/series/season/person artwork. Co-Authored-By: Claude Sonnet 5 <[email protected]>	2026-09-05 07:46:30 -05:00
NixReaperandClaude Sonnet 5	69ce41d54f	Fix media volume permission denied for the non-root worker/api user ... Neither Dockerfile created /app/media before switching to the non-root `aurora` user, so a fresh named volume mounted there (docker-compose.yml's aurora_media_data) defaulted to root ownership -- Docker only auto-populates a volume's initial content/ownership from the image the first time it's mounted, and nothing in the image owned that path yet. The worker's first artwork download then failed with a permission error trying to mkdir a subdirectory under it. Creating and chowning /app/media to aurora before USER aurora fixes this for any volume created fresh after this build; an already-created volume from a prior `up --build` needs a one-time manual fix (`docker compose exec -u root worker chown -R aurora:aurora /app/media`) since Docker won't retroactively re-populate an existing volume. Co-Authored-By: Claude Sonnet 5 <[email protected]>	2026-09-05 07:35:50 -05:00
NixReaperandClaude Sonnet 5	7c4bfe6cf4	Fix sync_artwork clobbering is_primary via duplicate list entries ... TMDb's images.posters/backdrops alternates commonly include the *same* image as the top-level poster_path/backdrop_path -- so normalize_movie legitimately produces the same provider_asset_id twice in one artwork_list: once from the top-level field (is_primary=True), once from the general alternates loop (is_primary=False, the default). sync_artwork matched both occurrences to the same existing DB row by provider_asset_id and did a plain overwrite each time, so whichever occurrence it processed last won -- the non-primary duplicate always came second in list order, so it silently un-set is_primary right after the first occurrence had just set it. Caught by re-ingesting The Matrix on a devbox with the artwork download feature live: the movie ingested successfully, but zero download jobs were ever enqueued because get_primary_artwork_ids found nothing marked is_primary -- the flag had been clobbered back to false immediately after being set. Fixed by resetting is_primary on all existing rows up front (so a provider dropping its "primary" pick on an image actually clears, not just adds) and accumulating with OR within the batch instead of a plain overwrite, so any primary occurrence in one sync sticks regardless of processing order. Also dedupes brand-new rows within the same batch the same way, closing the equivalent bug for images that don't have an existing row yet. Co-Authored-By: Claude Sonnet 5 <[email protected]>	2026-09-05 07:30:40 -05:00
NixReaperandClaude Sonnet 5	fef914db36	Add artwork self-hosting: local-disk download worker + storage ... Section 10: every image API responses reference has pointed straight at TMDb's CDN since day one -- Aurora never actually downloaded or served artwork itself, despite storing dimensions/checksum/local_path columns for exactly that. Closes the gap: - is_primary was declared on the Artwork model but never actually set anywhere -- every row defaulted to False. Now the normalizer marks the one image each provider designates as *the* image for an entity (TMDb's poster_path/backdrop_path/profile_path) as primary, distinct from the dozens of alternates also ingested, including correctly marking an existing row primary (not inserting a duplicate) when that image already appears in the alternates list. - app/services/artwork/: an ArtworkStorage interface with a LocalDiskStorage implementation (checksum-named files, single-host deployment via a Docker volume shared between api and worker -- see docs/architecture.md for the S3/R2 upgrade path), and a download pipeline with content-type validation, a size cap, and path-traversal protection. - After movie/series/person ingestion succeeds, tasks.py cascades one download_artwork_task per primary artwork id via arq's ctx['redis'] (same fan-out pattern as the season cascade) -- fire-and-forget, not tracked in ingestion_jobs, since a failed image cache is a known-safe fallback to the provider URL, not a user-facing failure. - app/main.py serves MEDIA_ROOT at MEDIA_URL_PREFIX; local_path is preferred over original_url once set (already-existing fallback logic in app/services/common.py:artwork_url, now actually exercised). Also fixes a real interaction bug this surfaced: every ingestion pipeline deleted and recreated all of an entity's Artwork rows on each re-sync, which would have silently wiped local_path/checksum on every re-ingestion, forcing a full re-download every time and defeating the point of caching locally at all. app/repositories/polymorphic.py's new sync_artwork upserts by provider_asset_id (TMDb's stable per-image path) instead, preserving downloaded copies across re-sync, and replaces three near-identical delete+recreate helpers that used to live separately in movie_ingestion.py, series_ingestion.py, and person_ingestion.py. Co-Authored-By: Claude Sonnet 5 <[email protected]>	2026-09-05 03:26:28 -05:00
NixReaperandClaude Sonnet 5	cd4dd98adb	Make failed ingestion jobs actually retry, not just say "retrying" ... arq only auto-retries a job on its own Retry exception (or CancelledError) -- never on an arbitrary exception a task raises. _run_ingestion caught failures, recorded them via mark_job_failed (which sets ingestion_jobs.status to "retrying" when attempts hasn't hit max_attempts), and then just re-raised the original exception -- which arq treats as a terminal failure. The DB said "retrying" forever; nothing ever put the job back on the queue. Confirmed live: cascading Better Call Saul's 6 seasons hit the get_or_create race fixed in the previous commit on 4 of them, and the worker's own shutdown summary showed "4 failed, 0 retries" -- every one of those was permanently lost despite max_attempts=5 suggesting otherwise. _run_ingestion now raises arq.Retry (capped exponential backoff) when mark_job_failed left the job in RETRYING state, so arq actually requeues it; only once attempts are exhausted does the original exception propagate as a genuine terminal failure. Co-Authored-By: Claude Sonnet 5 <[email protected]>	2026-09-05 02:31:43 -05:00
NixReaperandClaude Sonnet 5	dd3ff9764b	Fix race condition in get_or_create_* under concurrent ingestion ... Every get_or_create_* helper (genre, keyword, language, country, company, collection, person) followed check-then-act: look up by natural key, insert if missing. That was safe as long as nothing ingested concurrently for overlapping data -- which stopped being true the moment ingest_series_task could cascade season ingestion (include_seasons), since arq runs multiple queued jobs concurrently. Two seasons of the same series sharing a guest star, crew member, or company can now race to create it: both see "not found", both try to INSERT, and the loser hits a unique-constraint violation instead of finding the winner's row. Reproduced live: cascading Better Call Saul (60059) with include_seasons crashed a season job with "duplicate key value violates unique constraint uq_external_id_source" on a recurring guest actor shared across two seasons ingesting at the same time. _create_or_recover wraps each insert in a SAVEPOINT and, on that specific failure, rolls back just the savepoint and re-queries for what the concurrent job committed -- recovering instead of aborting the whole ingestion job. Added unit tests for the retry mechanism itself; genuinely reproducing the underlying race in an automated test would need real Postgres MVCC timing, which isn't something a deterministic test can force reliably. Co-Authored-By: Claude Sonnet 5 <[email protected]>	2026-09-05 02:27:07 -05:00
NixReaperandClaude Sonnet 5	64169c4aee	Fix test factory Provider collision from ingestion-cascade tests ... tests/integration/factories.py:make_provider did a raw INSERT assuming no "tmdb" Provider row existed yet. That held as long as every test used the db_session fixture's isolated, rolled-back transaction -- but the new ingestion-cascade tests call get_or_create_ingestion_job, which really commits via session_scope()'s own connection, outside that per-test isolation. Once that ran, every later test using the raw-insert factory hit "duplicate key value violates unique constraint uq_providers_slug". Fixed by using the app's own get_or_create_provider in the factory instead of a bare insert, which is also just correct: provider slugs are unique in production too, so the test helper should behave the same way real ingestion code does rather than assume a pristine table. Co-Authored-By: Claude Sonnet 5 <[email protected]>	2026-09-05 02:21:43 -05:00
NixReaperandClaude Sonnet 5	b4d1e8f755	Add opt-in season cascade to series ingestion ... Ingesting a series only ever produced season stubs -- getting actual episodes meant calling POST /admin/ingest/season once per season by hand, which is exactly what manual end-to-end testing kept requiring for every multi-season show. POST /api/v1/admin/ingest/series now takes include_seasons (default false, since a show can have dozens of seasons and that's dozens of TMDb calls fired from one request). When true, ingest_series_task fans out one ingest_season job per season once the series succeeds, using arq's own ctx['redis'] connection for the job (the same pattern arq's docs recommend for job chaining) rather than a second unrelated connection. Each fanned-out season job still goes through get_or_create_ingestion_job, so re-running a cascaded series ingestion reuses the same season job rows instead of creating duplicates. Co-Authored-By: Claude Sonnet 5 <[email protected]>	2026-09-05 02:03:25 -05:00
NixReaperandClaude Sonnet 5	ec6b5bdd7d	Surface series/episode credits in person filmography ... /people/{id}/credits only ever looked at entity_type == "movie", silently dropping the series-level cast/crew and episode-level guest-star Credit rows that series_ingestion.py has been writing all along -- a person ingested via a TV show would show an empty or movie-only filmography no matter how much TV work was actually in the database. Regular series credits map straight to a filmography row, same shape as movies. Guest-star episode credits are aggregated to one row per (series, department/job/character) with an episode_count, rather than one row per episode -- a prolific guest actor would otherwise flood the list with dozens of near-identical entries. Added tv_repository.get_series_by_ids/get_episodes_by_ids (bulk, mirroring the existing movie_repository pattern) to support it. Co-Authored-By: Claude Sonnet 5 <[email protected]>	2026-09-04 20:00:15 -05:00
NixReaperandClaude Sonnet 5	e6841badf8	Fix worker never picking up code changes after restart ... docker compose restart worker looked like it should reflect new code (docker-compose.yml bind-mounts ./app:/app/app for dev), but the worker Dockerfile installed the app package non-editably (pip install .), which copies a frozen snapshot into site-packages at image build time. The bare `arq` console-script just imports whatever site-packages says `app` is -- unlike uvicorn --reload, which explicitly resolves its import string against the CWD -- so the worker silently kept running whatever code existed at the last `docker compose up --build`, no matter how many times it was restarted or how recent the git pull was. Confirmed live: after pulling the series/season ingestion pipeline and restarting, the worker log still read "Starting worker for 2 functions: ingest_movie_task, ingest_person_task" -- proof it never saw ingest_series_task/ingest_season_task at all. Switching to `pip install -e .` makes site-packages point at /app by absolute path instead of a frozen copy, so the same bind mount that already worked for the API now actually reaches the worker process too -- a plain restart after `git pull` is enough going forward. Applied to both Dockerfiles for consistency, since the API's working behavior was incidental to uvicorn, not something to rely on either. Co-Authored-By: Claude Sonnet 5 <[email protected]>	2026-09-04 19:51:52 -05:00
NixReaperandClaude Sonnet 5	3acf00e9f0	Fix test bug: await inside a generator expression passed to all() ... Python turns a generator expression containing await into an async generator (since it's evaluated inside an async def), which the built-in all() can't consume -- it raised "TypeError: 'async_generator' object is not iterable" rather than testing anything. Replaced with a plain loop. Found by actually running the new integration tests. Co-Authored-By: Claude Sonnet 5 <[email protected]>	2026-09-04 19:45:29 -05:00
NixReaperandClaude Sonnet 5	3f6332559a	Add series/season/episode ingestion pipeline ... Movie and person ingestion existed, but the series/season/episode API endpoints had no way to actually populate them from TMDb -- the biggest remaining gap in Core 0.1's television scope. - normalize_series/normalize_season in the metadata normalizer, including two TMDb quirks that would otherwise silently produce empty data: /tv nests keywords and alternative_titles under "results" (not "keywords"/"titles" like /movie), and series-level creators come from a separate created_by array, not the credits cast/crew lists. - ingest_series upserts the Series row plus season *stubs* (number, title, air date, poster -- everything TMDb inlines on /tv/{id}); ingest_season fetches and upserts one season's full episode list, and requires the series to already exist (TMDb has no single endpoint for a whole series' episodes, so this has to be a separate per-season call). - Wired through the same job-lifecycle path as movie/person ingestion (runner.py, arq tasks, admin endpoints), with the shared boilerplate between them factored into runner._run_ingestion and admin.py:_enqueue rather than copy-pasted a third and fourth time. - Unit tests for both normalizers (runnable without a DB) and integration tests for the full ingest_series -> ingest_season flow, including the series-must-exist-first error and idempotency on re-ingestion. Co-Authored-By: Claude Sonnet 5 <[email protected]>	2026-09-04 19:43:56 -05:00
NixReaperandClaude Sonnet 5	669034896a	Fix job_id mismatch between ingestion enqueue and status endpoints ... POST /api/v1/admin/ingest/movie returned arq's own internal queue job id, but GET /api/v1/admin/ingest/jobs/{job_id} looked up our IngestionJob DB row by its own UUID -- a completely different id that didn't even exist yet at enqueue time (the row was only created once the worker picked up the task). Every status poll 404'd. Fix: create the IngestionJob row synchronously in the enqueue endpoint (reusing the existing idempotency-key logic, so repeat enqueues for the same tmdb_id reuse the same row rather than creating duplicates), pass its real id into the arq task, and have the worker update that row instead of creating a new one. Found by actually running the milestone end-to-end: enqueue a real TMDb ingestion, then poll its status. Co-Authored-By: Claude Sonnet 5 <[email protected]>	2026-09-04 19:28:31 -05:00
NixReaperandClaude Sonnet 5	dd6ecce78d	Fix asyncpg cross-event-loop errors in the integration test suite ... tests/conftest.py's `engine` fixture is session-scoped (one asyncpg engine/schema for the whole run, for speed), but pytest-asyncio's default is a fresh event loop per test function. asyncpg connections are bound to the event loop that created them, so every test after the first failed with "cannot perform operation: another operation is in progress" or "attached to a different loop" the moment it touched the database. Pinning both the fixture and test loop scope to "session" in pyproject.toml keeps everything on one loop, matching the session-scoped engine. Found by actually running the integration suite for the first time, against real PostgreSQL + Redis on a devbox. Co-Authored-By: Claude Sonnet 5 <[email protected]>	2026-09-04 19:23:18 -05:00
NixReaperandClaude Sonnet 5	5b38226818	Fix duplicate enum type creation in initial migration ... Postgres ENUM columns auto-create their type as a side effect of the first CREATE TABLE that uses them; since the migration also explicitly pre-creates all 7 enum types up front (to group and control that DDL alongside their explicit drop in downgrade()), the two collided with "type already exists" on the first table past the first enum's owner. create_type=False on each ENUM tells SQLAlchemy the migration manages that DDL itself. Found by running the migration against a real PostgreSQL for the first time (docker compose up on a devbox) — this repo had no Docker/Postgres available to test against when the migration was originally written. Co-Authored-By: Claude Sonnet 5 <[email protected]>	2026-09-04 18:48:44 -05:00
NixReaperandClaude Sonnet 5	c20df1c503	Initial Aurora Core 0.1 implementation ... Production-grade media metadata backend per docs/aurora-core-spec.md: Aurora-owned UUIDv7 identity system, normalized movie/TV/people/credits schema, provider abstraction with a TMDb adapter, an idempotent async ingestion pipeline (normalize -> match -> upsert -> provenance), a Postgres FTS/trigram search layer, a stateless FastAPI v1 API with Redis caching/rate-limiting, arq workers, dev/prod Docker Compose topologies, unit + integration + load tests, CI, and architecture/ database/API/ERD documentation. Co-Authored-By: Claude Sonnet 5 <[email protected]>	2026-09-04 17:34:03 -05:00