Add artwork self-hosting: local-disk download worker + storage
CI / lint (push) Canceled after 0s
CI / typecheck (push) Canceled after 0s
CI / unit-tests (push) Canceled after 0s
CI / integration-tests (push) Canceled after 0s
CI / container-build (push) Canceled after 0s
CI / dependency-audit (push) Canceled after 0s
CI / lint (push) Canceled after 0s
CI / typecheck (push) Canceled after 0s
CI / unit-tests (push) Canceled after 0s
CI / integration-tests (push) Canceled after 0s
CI / container-build (push) Canceled after 0s
CI / dependency-audit (push) Canceled after 0s
Section 10: every image API responses reference has pointed straight at TMDb's CDN since day one -- Aurora never actually downloaded or served artwork itself, despite storing dimensions/checksum/local_path columns for exactly that. Closes the gap: - is_primary was declared on the Artwork model but never actually set anywhere -- every row defaulted to False. Now the normalizer marks the one image each provider designates as *the* image for an entity (TMDb's poster_path/backdrop_path/profile_path) as primary, distinct from the dozens of alternates also ingested, including correctly marking an existing row primary (not inserting a duplicate) when that image already appears in the alternates list. - app/services/artwork/: an ArtworkStorage interface with a LocalDiskStorage implementation (checksum-named files, single-host deployment via a Docker volume shared between api and worker -- see docs/architecture.md for the S3/R2 upgrade path), and a download pipeline with content-type validation, a size cap, and path-traversal protection. - After movie/series/person ingestion succeeds, tasks.py cascades one download_artwork_task per primary artwork id via arq's ctx['redis'] (same fan-out pattern as the season cascade) -- fire-and-forget, not tracked in ingestion_jobs, since a failed image cache is a known-safe fallback to the provider URL, not a user-facing failure. - app/main.py serves MEDIA_ROOT at MEDIA_URL_PREFIX; local_path is preferred over original_url once set (already-existing fallback logic in app/services/common.py:artwork_url, now actually exercised). Also fixes a real interaction bug this surfaced: every ingestion pipeline deleted and recreated all of an entity's Artwork rows on each re-sync, which would have silently wiped local_path/checksum on every re-ingestion, forcing a full re-download every time and defeating the point of caching locally at all. app/repositories/polymorphic.py's new sync_artwork upserts by provider_asset_id (TMDb's stable per-image path) instead, preserving downloaded copies across re-sync, and replaces three near-identical delete+recreate helpers that used to live separately in movie_ingestion.py, series_ingestion.py, and person_ingestion.py. Co-Authored-By: Claude Sonnet 5 <[email protected]>
This commit is contained in:
co-authored by
Claude Sonnet 5
parent
cd4dd98adb
commit
fef914db36
@@ -103,6 +103,22 @@ curl http://localhost:8000/api/v1/admin/ingest/jobs/<job_id> \
|
||||
curl "http://localhost:8000/api/v1/search?q=the%20matrix"
|
||||
```
|
||||
|
||||
Ingesting a series also queues its posters/backdrops/profile images for
|
||||
local download automatically (only the provider's *primary* image per
|
||||
entity, not every alternate) — see the next section.
|
||||
|
||||
## Artwork
|
||||
|
||||
Images are referenced by their provider CDN URL (TMDb) from the moment
|
||||
they're ingested, and cached locally in the background afterward — API
|
||||
responses point at the provider until a local copy exists, then switch over
|
||||
automatically, with no client-visible change. Cached files live under
|
||||
`MEDIA_ROOT` (a Docker volume shared between `api` and `worker`, see
|
||||
`docker-compose.yml`) and are served at `MEDIA_URL_PREFIX` (default
|
||||
`/media`). See [`docs/architecture.md`](docs/architecture.md#artwork-self-hosting-section-10)
|
||||
and [`docs/database.md`](docs/database.md#artwork-storage-self-hosting) for
|
||||
how it works and its current single-host-deployment assumption.
|
||||
|
||||
## Tests
|
||||
|
||||
```bash
|
||||
@@ -170,14 +186,8 @@ See [`docs/database.md#known-limitations`](docs/database.md#known-limitations)
|
||||
and [`docs/api.md#known-limitations`](docs/api.md#known-limitations) for the
|
||||
current, tracked list (no curated language/country seed data, no fuzzy
|
||||
entity matching, no field-level provenance, no pagination beyond search,
|
||||
person credits limited to movies). None of these block the Core 0.1
|
||||
milestone (section 34): search → movie/person detail works end-to-end.
|
||||
|
||||
This repository was built in an environment without Docker or a
|
||||
pre-existing PostgreSQL instance; the initial Alembic migration was
|
||||
hand-written from the models (not autogenerated against a live database)
|
||||
and validated with `alembic history`, but has not yet been executed against
|
||||
a running PostgreSQL instance. **Run `docker compose up --build` and confirm
|
||||
`alembic upgrade head` succeeds against a genuinely empty database before
|
||||
treating the schema as final** — see
|
||||
[`docs/database.md#migrations`](docs/database.md#migrations).
|
||||
person credits' episode-level guest appearances rolled up to a per-series
|
||||
count rather than broken out individually). None of these block the Core
|
||||
0.1 milestone (section 34): search → movie/series/person detail works
|
||||
end-to-end against real TMDb data, verified on a real PostgreSQL + Redis +
|
||||
Docker deployment (not just this repo's own dev machine).
|
||||
|
||||
Reference in New Issue
Block a user