NixReaperandClaude Sonnet 5
3f6332559a
CI / lint (push) Canceled after 0s
CI / typecheck (push) Canceled after 0s
CI / unit-tests (push) Canceled after 0s
CI / integration-tests (push) Canceled after 0s
CI / container-build (push) Canceled after 0s
CI / dependency-audit (push) Canceled after 0s
Movie and person ingestion existed, but the series/season/episode API
endpoints had no way to actually populate them from TMDb -- the
biggest remaining gap in Core 0.1's television scope.
- normalize_series/normalize_season in the metadata normalizer,
including two TMDb quirks that would otherwise silently produce
empty data: /tv nests keywords and alternative_titles under
"results" (not "keywords"/"titles" like /movie), and series-level
creators come from a separate created_by array, not the credits
cast/crew lists.
- ingest_series upserts the Series row plus season *stubs* (number,
title, air date, poster -- everything TMDb inlines on /tv/{id});
ingest_season fetches and upserts one season's full episode list,
and requires the series to already exist (TMDb has no single
endpoint for a whole series' episodes, so this has to be a separate
per-season call).
- Wired through the same job-lifecycle path as movie/person ingestion
(runner.py, arq tasks, admin endpoints), with the shared boilerplate
between them factored into runner._run_ingestion and
admin.py:_enqueue rather than copy-pasted a third and fourth time.
- Unit tests for both normalizers (runnable without a DB) and
integration tests for the full ingest_series -> ingest_season flow,
including the series-must-exist-first error and idempotency on
re-ingestion.
Co-Authored-By: Claude Sonnet 5 <[email protected]>
43 lines
1.1 KiB
Python
43 lines
1.1 KiB
Python
"""arq worker process entrypoint.
|
|
|
|
Run with: `arq app.workers.worker.WorkerSettings`
|
|
|
|
Workers scale independently of the API (section 1) — run as many worker
|
|
processes/containers as needed; arq's Redis-backed queue coordinates them so
|
|
each job is picked up by exactly one worker.
|
|
"""
|
|
|
|
from __future__ import annotations
|
|
|
|
from arq.connections import RedisSettings
|
|
|
|
from app.core.config import get_settings
|
|
from app.core.logging import configure_logging, get_logger
|
|
from app.workers.tasks import (
|
|
ingest_movie_task,
|
|
ingest_person_task,
|
|
ingest_season_task,
|
|
ingest_series_task,
|
|
)
|
|
|
|
logger = get_logger(__name__)
|
|
|
|
|
|
async def startup(ctx: dict) -> None:
|
|
configure_logging()
|
|
logger.info("worker_started")
|
|
|
|
|
|
async def shutdown(ctx: dict) -> None:
|
|
logger.info("worker_stopped")
|
|
|
|
|
|
class WorkerSettings:
|
|
functions = [ingest_movie_task, ingest_person_task, ingest_series_task, ingest_season_task]
|
|
on_startup = startup
|
|
on_shutdown = shutdown
|
|
redis_settings = RedisSettings.from_dsn(get_settings().REDIS_URL)
|
|
max_jobs = get_settings().WORKER_CONCURRENCY
|
|
job_timeout = 300
|
|
max_tries = 5
|