Add artwork self-hosting: local-disk download worker + storage
CI / lint (push) Canceled after 0s
CI / typecheck (push) Canceled after 0s
CI / unit-tests (push) Canceled after 0s
CI / integration-tests (push) Canceled after 0s
CI / container-build (push) Canceled after 0s
CI / dependency-audit (push) Canceled after 0s
CI / lint (push) Canceled after 0s
CI / typecheck (push) Canceled after 0s
CI / unit-tests (push) Canceled after 0s
CI / integration-tests (push) Canceled after 0s
CI / container-build (push) Canceled after 0s
CI / dependency-audit (push) Canceled after 0s
Section 10: every image API responses reference has pointed straight at TMDb's CDN since day one -- Aurora never actually downloaded or served artwork itself, despite storing dimensions/checksum/local_path columns for exactly that. Closes the gap: - is_primary was declared on the Artwork model but never actually set anywhere -- every row defaulted to False. Now the normalizer marks the one image each provider designates as *the* image for an entity (TMDb's poster_path/backdrop_path/profile_path) as primary, distinct from the dozens of alternates also ingested, including correctly marking an existing row primary (not inserting a duplicate) when that image already appears in the alternates list. - app/services/artwork/: an ArtworkStorage interface with a LocalDiskStorage implementation (checksum-named files, single-host deployment via a Docker volume shared between api and worker -- see docs/architecture.md for the S3/R2 upgrade path), and a download pipeline with content-type validation, a size cap, and path-traversal protection. - After movie/series/person ingestion succeeds, tasks.py cascades one download_artwork_task per primary artwork id via arq's ctx['redis'] (same fan-out pattern as the season cascade) -- fire-and-forget, not tracked in ingestion_jobs, since a failed image cache is a known-safe fallback to the provider URL, not a user-facing failure. - app/main.py serves MEDIA_ROOT at MEDIA_URL_PREFIX; local_path is preferred over original_url once set (already-existing fallback logic in app/services/common.py:artwork_url, now actually exercised). Also fixes a real interaction bug this surfaced: every ingestion pipeline deleted and recreated all of an entity's Artwork rows on each re-sync, which would have silently wiped local_path/checksum on every re-ingestion, forcing a full re-download every time and defeating the point of caching locally at all. app/repositories/polymorphic.py's new sync_artwork upserts by provider_asset_id (TMDb's stable per-image path) instead, preserving downloaded copies across re-sync, and replaces three near-identical delete+recreate helpers that used to live separately in movie_ingestion.py, series_ingestion.py, and person_ingestion.py. Co-Authored-By: Claude Sonnet 5 <[email protected]>
This commit is contained in:
co-authored by
Claude Sonnet 5
parent
cd4dd98adb
commit
fef914db36
@@ -66,6 +66,12 @@ class NormalizedArtwork:
|
||||
aspect_ratio: float | None
|
||||
language: str | None
|
||||
vote_average: float | None = None
|
||||
# True for the single image the provider designates as *the* image for
|
||||
# this entity (TMDb's top-level poster_path/backdrop_path/profile_path),
|
||||
# as opposed to the full alternates list — this is what gets
|
||||
# auto-downloaded for local hosting (app/services/artwork/), so it's
|
||||
# deliberately narrow rather than "the first one in the list".
|
||||
is_primary: bool = False
|
||||
|
||||
|
||||
@dataclass
|
||||
@@ -182,6 +188,7 @@ def normalize_movie(raw: dict) -> NormalizedMovie:
|
||||
None,
|
||||
None,
|
||||
None,
|
||||
is_primary=True,
|
||||
)
|
||||
)
|
||||
if raw.get("backdrop_path"):
|
||||
@@ -194,6 +201,7 @@ def normalize_movie(raw: dict) -> NormalizedMovie:
|
||||
None,
|
||||
None,
|
||||
None,
|
||||
is_primary=True,
|
||||
)
|
||||
)
|
||||
for kind, tmdb_key in (("poster", "posters"), ("backdrop", "backdrops"), ("logo", "logos")):
|
||||
@@ -309,19 +317,26 @@ def normalize_person(raw: dict) -> NormalizedPerson:
|
||||
)
|
||||
for img in images
|
||||
]
|
||||
if raw.get("profile_path") and not any(a.provider_asset_id == raw["profile_path"] for a in artwork):
|
||||
artwork.insert(
|
||||
0,
|
||||
NormalizedArtwork(
|
||||
"profile",
|
||||
_image_url_required(raw["profile_path"]),
|
||||
raw["profile_path"],
|
||||
None,
|
||||
None,
|
||||
None,
|
||||
None,
|
||||
),
|
||||
if raw.get("profile_path"):
|
||||
existing = next(
|
||||
(a for a in artwork if a.provider_asset_id == raw["profile_path"]), None
|
||||
)
|
||||
if existing is not None:
|
||||
existing.is_primary = True
|
||||
else:
|
||||
artwork.insert(
|
||||
0,
|
||||
NormalizedArtwork(
|
||||
"profile",
|
||||
_image_url_required(raw["profile_path"]),
|
||||
raw["profile_path"],
|
||||
None,
|
||||
None,
|
||||
None,
|
||||
None,
|
||||
is_primary=True,
|
||||
),
|
||||
)
|
||||
|
||||
also_known_as = raw.get("also_known_as") or []
|
||||
|
||||
@@ -473,6 +488,7 @@ def normalize_series(raw: dict) -> NormalizedSeries:
|
||||
None,
|
||||
None,
|
||||
None,
|
||||
is_primary=True,
|
||||
)
|
||||
)
|
||||
if raw.get("backdrop_path"):
|
||||
@@ -485,6 +501,7 @@ def normalize_series(raw: dict) -> NormalizedSeries:
|
||||
None,
|
||||
None,
|
||||
None,
|
||||
is_primary=True,
|
||||
)
|
||||
)
|
||||
for kind, tmdb_key in (("poster", "posters"), ("backdrop", "backdrops"), ("logo", "logos")):
|
||||
@@ -526,7 +543,7 @@ def normalize_series(raw: dict) -> NormalizedSeries:
|
||||
poster=(
|
||||
NormalizedArtwork(
|
||||
"poster", _image_url_required(s["poster_path"]), s["poster_path"],
|
||||
None, None, None, None,
|
||||
None, None, None, None, is_primary=True,
|
||||
)
|
||||
if s.get("poster_path")
|
||||
else None
|
||||
@@ -621,7 +638,7 @@ def normalize_season(raw: dict) -> NormalizedSeason:
|
||||
artwork.append(
|
||||
NormalizedArtwork(
|
||||
"poster", _image_url_required(raw["poster_path"]), raw["poster_path"],
|
||||
None, None, None, None,
|
||||
None, None, None, None, is_primary=True,
|
||||
)
|
||||
)
|
||||
for img in images.get("posters", []):
|
||||
@@ -656,7 +673,7 @@ def normalize_season(raw: dict) -> NormalizedSeason:
|
||||
stills.append(
|
||||
NormalizedArtwork(
|
||||
"still", _image_url_required(ep["still_path"]), ep["still_path"],
|
||||
None, None, None, None,
|
||||
None, None, None, None, is_primary=True,
|
||||
)
|
||||
)
|
||||
episodes.append(
|
||||
|
||||
Reference in New Issue
Block a user