NixReaperandClaude Sonnet 5
4403305da6
CI / lint (push) Canceled after 0s
CI / typecheck (push) Canceled after 0s
CI / unit-tests (push) Canceled after 0s
CI / integration-tests (push) Canceled after 0s
CI / container-build (push) Canceled after 0s
CI / dependency-audit (push) Canceled after 0s
Mirrors TMDb's own /trending/{movie|tv}/{day|week} ranking rather than
Aurora computing trending itself from popularity deltas -- that would
require re-syncing the entire catalog on a schedule, infeasible at any
real scale, whereas TMDb already computes this well from signals (real
view/search velocity) Aurora has no access to.
New: TrendingEntry model + migration 0003, MetadataProvider.get_trending
(concrete/optional, like aclose(), so existing test fakes don't all need
updating), refresh_trending() which actually ingests each trending item
(only ~20 per window, cheap enough to fully ingest rather than apply the
never-fabricate-a-stub rule recommendations uses), a twice-daily arq cron
job (WorkerSettings.cron_jobs, new to this codebase), a manual admin
trigger for right after a deploy, and GET /movies/trending + GET
/series/trending.
Co-Authored-By: Claude Sonnet 5 <[email protected]>
58 lines
1.6 KiB
Python
58 lines
1.6 KiB
Python
"""arq worker process entrypoint.
|
|
|
|
Run with: `arq app.workers.worker.WorkerSettings`
|
|
|
|
Workers scale independently of the API (section 1) — run as many worker
|
|
processes/containers as needed; arq's Redis-backed queue coordinates them so
|
|
each job is picked up by exactly one worker.
|
|
"""
|
|
|
|
from __future__ import annotations
|
|
|
|
from arq.connections import RedisSettings
|
|
from arq.cron import cron
|
|
|
|
from app.core.config import get_settings
|
|
from app.core.logging import configure_logging, get_logger
|
|
from app.workers.tasks import (
|
|
download_artwork_task,
|
|
ingest_collection_task,
|
|
ingest_movie_task,
|
|
ingest_person_task,
|
|
ingest_season_task,
|
|
ingest_series_task,
|
|
refresh_trending_task,
|
|
)
|
|
|
|
logger = get_logger(__name__)
|
|
|
|
|
|
async def startup(ctx: dict) -> None:
|
|
configure_logging()
|
|
logger.info("worker_started")
|
|
|
|
|
|
async def shutdown(ctx: dict) -> None:
|
|
logger.info("worker_stopped")
|
|
|
|
|
|
class WorkerSettings:
|
|
functions = [
|
|
ingest_movie_task,
|
|
ingest_person_task,
|
|
ingest_series_task,
|
|
ingest_season_task,
|
|
ingest_collection_task,
|
|
download_artwork_task,
|
|
]
|
|
# Twice a day (UTC) rather than run_at_startup=True -- a worker
|
|
# container restart (common during deploys) must not hammer TMDb with
|
|
# a full trending refresh every time it comes up.
|
|
cron_jobs = [cron(refresh_trending_task, hour={3, 15}, minute=0)]
|
|
on_startup = startup
|
|
on_shutdown = shutdown
|
|
redis_settings = RedisSettings.from_dsn(get_settings().REDIS_URL)
|
|
max_jobs = get_settings().WORKER_CONCURRENCY
|
|
job_timeout = 300
|
|
max_tries = 5
|