Builds on the accounts foundation: PUT/DELETE a rating (0.5-10.0) or a
watchlist entry on any movie or series, all scoped to the bearer token's
own user, plus GET /account/ratings|watchlist/{movies,series} to list
them back. Both are polymorphic entity_type/entity_id tables, same shape
as watch_provider_offerings and trending_entries, so movie and series
share one repository/router each rather than duplicating per type.
Co-Authored-By: Claude Sonnet 5 <[email protected]>
The first authenticated-user concept in Aurora Core -- everything else in
this codebase is public, read-only catalog data. Foundation for ratings
and a watchlist (not yet built): POST /auth/register, POST /auth/login,
GET /auth/me.
Security specifics worth knowing:
- Passwords hashed with argon2 (via argon2-cffi), never stored plain or
logged.
- Login returns an identical 401 for a nonexistent email and a wrong
password on a real one, including matching response timing
(burn_time_like_a_password_check) -- otherwise either the status code
or the timing alone lets a client enumerate registered emails.
- AURORA_SECRET_KEY now actually signs JWTs (previously defined but
unused anywhere) -- .env.example's existing token_urlsafe(48) guidance
already produces a long enough key, just flagged its new stakes.
- Email matched case-insensitively via explicit normalization in the
service layer, not relied on from Pydantic's EmailStr (which does not
lowercase the local part).
12 new integration tests (register/login/me over real HTTP via the
`client` fixture) plus 10 unit tests for the hashing/JWT logic itself,
following the same-day lesson that only unit tests actually execute
locally without Postgres -- the auth core is tested for real here rather
than only through integration tests that might not run before a deploy.
Co-Authored-By: Claude Sonnet 5 <[email protected]>
Mirrors TMDb's own /trending/{movie|tv}/{day|week} ranking rather than
Aurora computing trending itself from popularity deltas -- that would
require re-syncing the entire catalog on a schedule, infeasible at any
real scale, whereas TMDb already computes this well from signals (real
view/search velocity) Aurora has no access to.
New: TrendingEntry model + migration 0003, MetadataProvider.get_trending
(concrete/optional, like aclose(), so existing test fakes don't all need
updating), refresh_trending() which actually ingests each trending item
(only ~20 per window, cheap enough to fully ingest rather than apply the
never-fabricate-a-stub rule recommendations uses), a twice-daily arq cron
job (WorkerSettings.cron_jobs, new to this codebase), a manual admin
trigger for right after a deploy, and GET /movies/trending + GET
/series/trending.
Co-Authored-By: Claude Sonnet 5 <[email protected]>
New TMDb integration end to end: append_to_response "watch/providers",
normalizer parsing grouped by country/monetization type, a WatchProvider
catalog entry (external-id keyed, same get_or_create pattern as Company/
Collection/Person) + polymorphic WatchProviderOffering table (new
migration 0002), ingestion write (including provider logos via the
existing Artwork table), and API exposure -- embedded on movie/series
detail plus a dedicated /watch-providers sub-route, shaped to mirror
TMDb's own response for easy porting.
Note: this data comes from JustWatch via TMDb and carries its own
attribution requirement separate from TMDb's general one -- flagged in
docs/api.md, must be satisfied before showing this in the frontend.
Co-Authored-By: Claude Sonnet 5 <[email protected]>
Buildable now that release_dates exists: GET /movies/now-playing and
/movies/upcoming filter on release_type="theatrical" per region, windowed
around today. A movie can carry more than one theatrical-type row in the
same region (limited release then wide release both map to "theatrical"
per normalizer._RELEASE_TYPE_MAP) -- collapsed to one row per movie via a
grouped subquery rather than joining ReleaseDate directly, which would
duplicate the movie in the page and inflate the total count.
Co-Authored-By: Claude Sonnet 5 <[email protected]>
Keywords were ingested and shown read-only on detail pages, but had no
directory (GET /keywords) or filter param -- the same gap genres had
before today. Mirrors the genre browse pattern exactly: repository,
service, route, plus a `keyword` filter on GET /movies and GET /series.
Alternate/translated titles ("Also Known As") were normalized down to
bare title strings and never written to MovieAlternateTitle/
SeriesAlternateTitle at all. Normalizer now captures country_code + type
per TMDb's actual shape; ingestion writes them; movie/series detail now
expose an alternate_titles list.
Co-Authored-By: Claude Sonnet 5 <[email protected]>
TMDb's release_dates/content_ratings blocks were already fetched via
append_to_response but discarded at normalization -- every downstream
layer (ingestion write, repository read, schema, service) was unbuilt.
Adds the full pipeline: NormalizedReleaseDate + get_or_create_certification,
a shared polymorphic sync/read (movie and series both use it), and
MovieDetailOut/SeriesDetailOut certification + release_dates fields.
Series content_ratings has no date of its own, so each rating is anchored
to the show's first_air_date with release_type="tv".
Co-Authored-By: Claude Sonnet 5 <[email protected]>
GET /movies and GET /series accept a genre filter (an Aurora genre id),
but nothing exposed what those ids actually are -- genres otherwise
only ever appear embedded in a movie/series response's own genres
list. This is the resolve-by-name-first-then-filter step the frontend
needs to build real genre browsing.
No filter, no sort_by -- always alphabetical by name, same pattern as
collections/companies (genres have no categorical field or popularity/
date to sort by). page_size defaults to 50 (max 200) rather than the
20/50 used elsewhere, since genres are a small, fixed reference set
(TMDb has ~19-20 total) a client typically wants in full for a filter
dropdown, not paginated like a catalog of millions of movies.
Also fixed two stale "behind Traefik" mentions in app/main.py's module
docstring while touching this file, left over from the earlier
cPanel/host-webserver deployment change.
Co-Authored-By: Claude Sonnet 5 <[email protected]>
app/services/bulk_import.py fetches and parses TMDb's daily ID export
files (published once a day specifically for full-catalog sync, unlike
/discover which caps at 10,000 results per query) -- today's file with
a fallback to yesterday's, since TMDb publishes at a fixed time and
"today's" may not exist yet.
app/scripts/bulk_import.py is a standalone ops script (not an arq task,
since enqueueing millions of jobs would blow past arq's job_timeout)
that fans every id from the export into the existing per-entity
ingestion pipeline -- get_or_create_ingestion_job + the same arq task
each admin endpoint already uses, so a single id's behavior is
identical whether it came from the admin API or this script. Bounded
concurrency (50) on the enqueue phase; idempotency-keyed jobs make
interrupting and re-running the script safe.
Real TMDb catalog size (checked live): ~1.24M movies, 231K series,
4.9M people, 9.8K collections -- ~6.4M entities. At default worker
concurrency this is realistically weeks of continuous processing, not
hours, documented as such in README.md.
Also added 429 (rate-limited) handling to TMDbProvider before running
this at volume: previously a 429 fell into the generic >=400 branch and
failed the request outright with zero retry, since only httpx.TransportError
triggered the existing retry decorator. Now a 429 raises
ProviderRateLimitedError carrying the response's own Retry-After hint,
and tenacity's wait strategy honors that hint directly instead of
guessing via exponential backoff.
Co-Authored-By: Claude Sonnet 5 <[email protected]>
Completes browse/discover across all five entity types: movies, series,
people, companies, and now collections. No filter (no categorical field
like company_type exists for a collection) -- just page/page_size,
always alphabetical by name.
While adding poster_path to CollectionBriefOut for the browse listing,
found the same class of bug fixed earlier for company logo_path:
movie_service's embedded `collection` field never looked up its poster
artwork either, so it stayed null even after ingest_collection populated
one. Fixed alongside since it's the same schema field this feature was
already touching.
Also removed PageMeta from schemas/common.py -- scaffolding from early
spec work that was never actually wired into any endpoint; all five
browse endpoints shipped this session use a flat page/page_size/
total_results/total_pages shape instead.
Co-Authored-By: Claude Sonnet 5 <[email protected]>
Completes the browse/discover set across all four entity types.
Companies have no popularity or date field, so unlike movies/series/
people there's no sort_by choice -- always alphabetical by name.
Filters are company_type (validated against the enum) and country
(a plain ISO 3166-1 code comparison, nothing to validate).
build_company_summary refactored to share row-to-summary logic with
the new list path, same pattern used for movies/series/people.
Co-Authored-By: Claude Sonnet 5 <[email protected]>
Extends the browse/discover pattern already built for movies/series
to the fourth major entity type. No genre/year/status equivalent for
people -- known_for_department is the one categorical filter that
actually means something for a cast/crew list, and sort_by is just
popularity (default) or name. No invalid-filter case either: unlike
genre/status, known_for_department is a plain string compared
directly, nothing to parse or validate.
build_person_summary refactored to share row-to-summary logic with
the new list path, same pattern used for movies/series/companies.
Co-Authored-By: Claude Sonnet 5 <[email protected]>
Movies have separate /credits, /images, /videos sub-endpoints; series
only had /seasons. Adds the matching three for series, reusing the
same cast/crew/creators three-way split series detail already does
(extracted into _split_series_credits so both share it).
Co-Authored-By: Claude Sonnet 5 <[email protected]>
GET /companies/{id} showed only the company's own info. Adds movies
and series lists (any role: production, distribution, network),
ordered by release/first-air date, read straight off the existing
movie_companies/series_companies junction rows -- no separate sync
needed since those only ever exist as a side effect of ingesting the
movie or series itself.
Co-Authored-By: Claude Sonnet 5 <[email protected]>
FilmographyCreditOut already aggregated a guest star's episode
appearances into one row per series with an episode_count, but never
exposed which episodes those were even though the individual episode
Credit rows exist in the database. Adds an `episodes` list (season/
episode number, title, air date, sorted chronologically) alongside the
existing count -- closes a known limitation tracked in docs/api.md.
Co-Authored-By: Claude Sonnet 5 <[email protected]>
The complement to /search for browsing without a query: paginated,
filterable (genre, year, status) and sortable (popularity,
release/air date, rating, title) listings. Offset-based pagination
rather than the Aurora ID cursor scheme, since sorting by
popularity/rating/date doesn't line up with creation-order cursors.
Total count is computed via a window function in the same query as
the page of results, avoiding a second round trip per request. An
invalid filter (malformed genre id, unknown status) is a 422 via a
per-service InvalidMovieFilterError/InvalidSeriesFilterError, caught
before any DB query runs.
build_movie_summary/build_series_summary refactored to share their
row-to-summary logic with the new list path, avoiding a redundant
get_by_id lookup when the row is already in hand.
Co-Authored-By: Claude Sonnet 5 <[email protected]>
Collections were bare stubs (name only) lazily created the moment a
movie's belongs_to_collection reference was seen. Adds a real
ingest_collection pipeline: TMDb's /collection/{id} for overview and
poster/backdrop artwork, plus ordered CollectionMovie membership --
linking only movies already ingested into Aurora, the same
never-fabricate-a-stub rule movie recommendations already follow.
get_or_create_collection now refreshes name/overview on an existing
stub instead of only creating, guarded so a bare movie-stub call
(overview=None) never clobbers real data a full ingestion already
filled in.
New POST /api/v1/admin/ingest/collection endpoint and
ingest_collection_task; GET /api/v1/collections/{id} needed no changes
since its read path (collection_service.py) already assembled the full
detail shape.
Co-Authored-By: Claude Sonnet 5 <[email protected]>
Ingestion already stored movie_recommendations edges from TMDb's
recommendations list (linking only to movies already ingested into
Aurora, per the never-fabricate-a-stub rule), but nothing exposed them.
Adds the repository lookup, service assembly, and route to read them
back in ranked order.
Co-Authored-By: Claude Sonnet 5 <[email protected]>
movie_service.get_movie_detail() hardcoded companies[].logo_path = None
instead of looking up the company's logo artwork, and tv_service's
networks[]/companies[] never set logo_path at all -- both just built
CompanyBriefOut without the artwork lookup that company_service.py (and
_season_brief for posters) already does correctly. Ingestion and storage
were working the whole time; the API layer just never read the result.
Co-Authored-By: Claude Sonnet 5 <[email protected]>
cached_response() had no write-side invalidation, so re-ingesting an
entity kept serving its pre-ingestion cached response for up to
CACHE_DEFAULT_TTL. Found live: re-ingesting a movie to test the new
company-logo field still showed logo_path: None because the sliding
5-minute cache from earlier testing never expired.
Co-Authored-By: Claude Sonnet 5 <[email protected]>
CompanyBriefOut.logo_path and CompanyDetailOut.logo_path have existed
in the API schemas since Core 0.1's first pass, but nothing ever
actually populated them -- TMDb's production_companies/networks
entries carry a logo_path field that the normalizer simply never read,
so every response has silently returned null there.
- NormalizedCompanyRef now captures logo_path from both movie
production_companies and series networks/production_companies.
- normalizer.company_logo_artwork() builds the single-image artwork
entry for it (a company has exactly one logo ingested, so it's
always is_primary) -- centralized there rather than duplicated in
both ingestion modules, matching how movie/series artwork URLs are
already built in one place.
- movie_ingestion.py and series_ingestion.py sync it via the same
sync_artwork() the rest of section 10's self-hosting already uses,
so it gets the same re-ingestion-safe upsert behavior for free.
- tasks.py cascades a download_artwork_task per referenced company
after movie/series ingestion, same fan-out pattern as
movie/series/season/person artwork.
Co-Authored-By: Claude Sonnet 5 <[email protected]>
Section 10: every image API responses reference has pointed straight
at TMDb's CDN since day one -- Aurora never actually downloaded or
served artwork itself, despite storing dimensions/checksum/local_path
columns for exactly that. Closes the gap:
- is_primary was declared on the Artwork model but never actually set
anywhere -- every row defaulted to False. Now the normalizer marks
the one image each provider designates as *the* image for an entity
(TMDb's poster_path/backdrop_path/profile_path) as primary,
distinct from the dozens of alternates also ingested, including
correctly marking an existing row primary (not inserting a
duplicate) when that image already appears in the alternates list.
- app/services/artwork/: an ArtworkStorage interface with a
LocalDiskStorage implementation (checksum-named files, single-host
deployment via a Docker volume shared between api and worker -- see
docs/architecture.md for the S3/R2 upgrade path), and a download
pipeline with content-type validation, a size cap, and path-traversal
protection.
- After movie/series/person ingestion succeeds, tasks.py cascades one
download_artwork_task per primary artwork id via arq's ctx['redis']
(same fan-out pattern as the season cascade) -- fire-and-forget, not
tracked in ingestion_jobs, since a failed image cache is a
known-safe fallback to the provider URL, not a user-facing failure.
- app/main.py serves MEDIA_ROOT at MEDIA_URL_PREFIX; local_path is
preferred over original_url once set (already-existing fallback
logic in app/services/common.py:artwork_url, now actually exercised).
Also fixes a real interaction bug this surfaced: every ingestion
pipeline deleted and recreated all of an entity's Artwork rows on each
re-sync, which would have silently wiped local_path/checksum on every
re-ingestion, forcing a full re-download every time and defeating the
point of caching locally at all. app/repositories/polymorphic.py's new
sync_artwork upserts by provider_asset_id (TMDb's stable per-image
path) instead, preserving downloaded copies across re-sync, and
replaces three near-identical delete+recreate helpers that used to
live separately in movie_ingestion.py, series_ingestion.py, and
person_ingestion.py.
Co-Authored-By: Claude Sonnet 5 <[email protected]>
arq only auto-retries a job on its own Retry exception (or
CancelledError) -- never on an arbitrary exception a task raises.
_run_ingestion caught failures, recorded them via mark_job_failed
(which sets ingestion_jobs.status to "retrying" when attempts hasn't
hit max_attempts), and then just re-raised the original exception --
which arq treats as a terminal failure. The DB said "retrying"
forever; nothing ever put the job back on the queue.
Confirmed live: cascading Better Call Saul's 6 seasons hit the
get_or_create race fixed in the previous commit on 4 of them, and the
worker's own shutdown summary showed "4 failed, 0 retries" -- every
one of those was permanently lost despite max_attempts=5 suggesting
otherwise.
_run_ingestion now raises arq.Retry (capped exponential backoff) when
mark_job_failed left the job in RETRYING state, so arq actually
requeues it; only once attempts are exhausted does the original
exception propagate as a genuine terminal failure.
Co-Authored-By: Claude Sonnet 5 <[email protected]>
Ingesting a series only ever produced season stubs -- getting actual
episodes meant calling POST /admin/ingest/season once per season by
hand, which is exactly what manual end-to-end testing kept requiring
for every multi-season show.
POST /api/v1/admin/ingest/series now takes include_seasons (default
false, since a show can have dozens of seasons and that's dozens of
TMDb calls fired from one request). When true, ingest_series_task
fans out one ingest_season job per season once the series succeeds,
using arq's own ctx['redis'] connection for the job (the same pattern
arq's docs recommend for job chaining) rather than a second unrelated
connection. Each fanned-out season job still goes through
get_or_create_ingestion_job, so re-running a cascaded series ingestion
reuses the same season job rows instead of creating duplicates.
Co-Authored-By: Claude Sonnet 5 <[email protected]>
/people/{id}/credits only ever looked at entity_type == "movie",
silently dropping the series-level cast/crew and episode-level
guest-star Credit rows that series_ingestion.py has been writing all
along -- a person ingested via a TV show would show an empty or
movie-only filmography no matter how much TV work was actually in the
database.
Regular series credits map straight to a filmography row, same shape
as movies. Guest-star episode credits are aggregated to one row per
(series, department/job/character) with an episode_count, rather than
one row per episode -- a prolific guest actor would otherwise flood
the list with dozens of near-identical entries. Added
tv_repository.get_series_by_ids/get_episodes_by_ids (bulk, mirroring
the existing movie_repository pattern) to support it.
Co-Authored-By: Claude Sonnet 5 <[email protected]>
Movie and person ingestion existed, but the series/season/episode API
endpoints had no way to actually populate them from TMDb -- the
biggest remaining gap in Core 0.1's television scope.
- normalize_series/normalize_season in the metadata normalizer,
including two TMDb quirks that would otherwise silently produce
empty data: /tv nests keywords and alternative_titles under
"results" (not "keywords"/"titles" like /movie), and series-level
creators come from a separate created_by array, not the credits
cast/crew lists.
- ingest_series upserts the Series row plus season *stubs* (number,
title, air date, poster -- everything TMDb inlines on /tv/{id});
ingest_season fetches and upserts one season's full episode list,
and requires the series to already exist (TMDb has no single
endpoint for a whole series' episodes, so this has to be a separate
per-season call).
- Wired through the same job-lifecycle path as movie/person ingestion
(runner.py, arq tasks, admin endpoints), with the shared boilerplate
between them factored into runner._run_ingestion and
admin.py:_enqueue rather than copy-pasted a third and fourth time.
- Unit tests for both normalizers (runnable without a DB) and
integration tests for the full ingest_series -> ingest_season flow,
including the series-must-exist-first error and idempotency on
re-ingestion.
Co-Authored-By: Claude Sonnet 5 <[email protected]>
POST /api/v1/admin/ingest/movie returned arq's own internal queue
job id, but GET /api/v1/admin/ingest/jobs/{job_id} looked up our
IngestionJob DB row by its own UUID -- a completely different id that
didn't even exist yet at enqueue time (the row was only created once
the worker picked up the task). Every status poll 404'd.
Fix: create the IngestionJob row synchronously in the enqueue endpoint
(reusing the existing idempotency-key logic, so repeat enqueues for
the same tmdb_id reuse the same row rather than creating duplicates),
pass its real id into the arq task, and have the worker update that
row instead of creating a new one.
Found by actually running the milestone end-to-end: enqueue a real
TMDb ingestion, then poll its status.
Co-Authored-By: Claude Sonnet 5 <[email protected]>
