NixReaperandClaude Sonnet 5
fef914db36
CI / lint (push) Canceled after 0s
CI / typecheck (push) Canceled after 0s
CI / unit-tests (push) Canceled after 0s
CI / integration-tests (push) Canceled after 0s
CI / container-build (push) Canceled after 0s
CI / dependency-audit (push) Canceled after 0s
Section 10: every image API responses reference has pointed straight at TMDb's CDN since day one -- Aurora never actually downloaded or served artwork itself, despite storing dimensions/checksum/local_path columns for exactly that. Closes the gap: - is_primary was declared on the Artwork model but never actually set anywhere -- every row defaulted to False. Now the normalizer marks the one image each provider designates as *the* image for an entity (TMDb's poster_path/backdrop_path/profile_path) as primary, distinct from the dozens of alternates also ingested, including correctly marking an existing row primary (not inserting a duplicate) when that image already appears in the alternates list. - app/services/artwork/: an ArtworkStorage interface with a LocalDiskStorage implementation (checksum-named files, single-host deployment via a Docker volume shared between api and worker -- see docs/architecture.md for the S3/R2 upgrade path), and a download pipeline with content-type validation, a size cap, and path-traversal protection. - After movie/series/person ingestion succeeds, tasks.py cascades one download_artwork_task per primary artwork id via arq's ctx['redis'] (same fan-out pattern as the season cascade) -- fire-and-forget, not tracked in ingestion_jobs, since a failed image cache is a known-safe fallback to the provider URL, not a user-facing failure. - app/main.py serves MEDIA_ROOT at MEDIA_URL_PREFIX; local_path is preferred over original_url once set (already-existing fallback logic in app/services/common.py:artwork_url, now actually exercised). Also fixes a real interaction bug this surfaced: every ingestion pipeline deleted and recreated all of an entity's Artwork rows on each re-sync, which would have silently wiped local_path/checksum on every re-ingestion, forcing a full re-download every time and defeating the point of caching locally at all. app/repositories/polymorphic.py's new sync_artwork upserts by provider_asset_id (TMDb's stable per-image path) instead, preserving downloaded copies across re-sync, and replaces three near-identical delete+recreate helpers that used to live separately in movie_ingestion.py, series_ingestion.py, and person_ingestion.py. Co-Authored-By: Claude Sonnet 5 <[email protected]>
50 lines
1.2 KiB
Python
50 lines
1.2 KiB
Python
"""arq worker process entrypoint.
|
|
|
|
Run with: `arq app.workers.worker.WorkerSettings`
|
|
|
|
Workers scale independently of the API (section 1) — run as many worker
|
|
processes/containers as needed; arq's Redis-backed queue coordinates them so
|
|
each job is picked up by exactly one worker.
|
|
"""
|
|
|
|
from __future__ import annotations
|
|
|
|
from arq.connections import RedisSettings
|
|
|
|
from app.core.config import get_settings
|
|
from app.core.logging import configure_logging, get_logger
|
|
from app.workers.tasks import (
|
|
download_artwork_task,
|
|
ingest_movie_task,
|
|
ingest_person_task,
|
|
ingest_season_task,
|
|
ingest_series_task,
|
|
)
|
|
|
|
logger = get_logger(__name__)
|
|
|
|
|
|
async def startup(ctx: dict) -> None:
|
|
configure_logging()
|
|
logger.info("worker_started")
|
|
|
|
|
|
async def shutdown(ctx: dict) -> None:
|
|
logger.info("worker_stopped")
|
|
|
|
|
|
class WorkerSettings:
|
|
functions = [
|
|
ingest_movie_task,
|
|
ingest_person_task,
|
|
ingest_series_task,
|
|
ingest_season_task,
|
|
download_artwork_task,
|
|
]
|
|
on_startup = startup
|
|
on_shutdown = shutdown
|
|
redis_settings = RedisSettings.from_dsn(get_settings().REDIS_URL)
|
|
max_jobs = get_settings().WORKER_CONCURRENCY
|
|
job_timeout = 300
|
|
max_tries = 5
|