Add artwork self-hosting: local-disk download worker + storage
CI / lint (push) Canceled after 0s
CI / typecheck (push) Canceled after 0s
CI / unit-tests (push) Canceled after 0s
CI / integration-tests (push) Canceled after 0s
CI / container-build (push) Canceled after 0s
CI / dependency-audit (push) Canceled after 0s

Section 10: every image API responses reference has pointed straight
at TMDb's CDN since day one -- Aurora never actually downloaded or
served artwork itself, despite storing dimensions/checksum/local_path
columns for exactly that. Closes the gap:

- is_primary was declared on the Artwork model but never actually set
  anywhere -- every row defaulted to False. Now the normalizer marks
  the one image each provider designates as *the* image for an entity
  (TMDb's poster_path/backdrop_path/profile_path) as primary,
  distinct from the dozens of alternates also ingested, including
  correctly marking an existing row primary (not inserting a
  duplicate) when that image already appears in the alternates list.
- app/services/artwork/: an ArtworkStorage interface with a
  LocalDiskStorage implementation (checksum-named files, single-host
  deployment via a Docker volume shared between api and worker -- see
  docs/architecture.md for the S3/R2 upgrade path), and a download
  pipeline with content-type validation, a size cap, and path-traversal
  protection.
- After movie/series/person ingestion succeeds, tasks.py cascades one
  download_artwork_task per primary artwork id via arq's ctx['redis']
  (same fan-out pattern as the season cascade) -- fire-and-forget, not
  tracked in ingestion_jobs, since a failed image cache is a
  known-safe fallback to the provider URL, not a user-facing failure.
- app/main.py serves MEDIA_ROOT at MEDIA_URL_PREFIX; local_path is
  preferred over original_url once set (already-existing fallback
  logic in app/services/common.py:artwork_url, now actually exercised).

Also fixes a real interaction bug this surfaced: every ingestion
pipeline deleted and recreated all of an entity's Artwork rows on each
re-sync, which would have silently wiped local_path/checksum on every
re-ingestion, forcing a full re-download every time and defeating the
point of caching locally at all. app/repositories/polymorphic.py's new
sync_artwork upserts by provider_asset_id (TMDb's stable per-image
path) instead, preserving downloaded copies across re-sync, and
replaces three near-identical delete+recreate helpers that used to
live separately in movie_ingestion.py, series_ingestion.py, and
person_ingestion.py.

Co-Authored-By: Claude Sonnet 5 <[email protected]>
This commit is contained in:
NixReaper
2026-09-05 03:26:28 -05:00
co-authored by Claude Sonnet 5
parent cd4dd98adb
commit fef914db36
24 changed files with 980 additions and 129 deletions
+34
View File
@@ -90,6 +90,40 @@ star can race to create the same `Person` row (see
recovers from that specific race directly; the arq retry here is the
backstop for whatever it doesn't).
## Artwork self-hosting (section 10)
```
ingest_movie/ingest_series/ingest_person succeeds
-> tasks.py: get_primary_artwork_for_download(entity_type, [entity_id, ...])
-> one download_artwork_task enqueued per primary artwork id
-> download_artwork(): fetch original_url, verify it's an image, cap
size, sha256 it, storage.save(), set Artwork.local_path + checksum
```
Every image API responses reference is stored with its provider CDN URL
(`original_url`) from the moment it's ingested; `local_path` starts empty
and is filled in by this pipeline running asynchronously afterward, so
responses transparently point at TMDb until a local copy exists, then
switch over. Only artwork marked `is_primary` (the one image TMDb itself
designates for an entity, not the dozens of alternates also ingested) gets
downloaded automatically — see `docs/database.md#artwork-storage-self-hosting`.
Storage is behind an `ArtworkStorage` interface
(`app/services/artwork/storage.py`) with one implementation,
`LocalDiskStorage` — a shared Docker volume between the `api` and `worker`
containers (`docker-compose.yml`'s `aurora_media_data`), which is enough
for this project's actual deployment shape (one Docker host, principle 9:
avoid premature complexity) but assumes exactly that. A deployment that
outgrew a single host would implement the same interface against S3/R2
instead, without the download pipeline or callers changing.
This is also why `app/repositories/polymorphic.py:sync_artwork` upserts
artwork by `provider_asset_id` instead of the delete-then-recreate pattern
every other satellite table uses (movie/series/person credits, videos,
genre/keyword/company links): a delete-then-recreate would generate a fresh
`Artwork.id` with `local_path=None` on every re-ingestion, silently
discarding every downloaded copy and forcing a full re-download each sync.
## Search (section 16)
`app/services/search/base.py:SearchProvider` is the abstraction; API