Ingesting a series only ever produced season stubs -- getting actual
episodes meant calling POST /admin/ingest/season once per season by
hand, which is exactly what manual end-to-end testing kept requiring
for every multi-season show.
POST /api/v1/admin/ingest/series now takes include_seasons (default
false, since a show can have dozens of seasons and that's dozens of
TMDb calls fired from one request). When true, ingest_series_task
fans out one ingest_season job per season once the series succeeds,
using arq's own ctx['redis'] connection for the job (the same pattern
arq's docs recommend for job chaining) rather than a second unrelated
connection. Each fanned-out season job still goes through
get_or_create_ingestion_job, so re-running a cascaded series ingestion
reuses the same season job rows instead of creating duplicates.
Co-Authored-By: Claude Sonnet 5 <[email protected]>
/people/{id}/credits only ever looked at entity_type == "movie",
silently dropping the series-level cast/crew and episode-level
guest-star Credit rows that series_ingestion.py has been writing all
along -- a person ingested via a TV show would show an empty or
movie-only filmography no matter how much TV work was actually in the
database.
Regular series credits map straight to a filmography row, same shape
as movies. Guest-star episode credits are aggregated to one row per
(series, department/job/character) with an episode_count, rather than
one row per episode -- a prolific guest actor would otherwise flood
the list with dozens of near-identical entries. Added
tv_repository.get_series_by_ids/get_episodes_by_ids (bulk, mirroring
the existing movie_repository pattern) to support it.
Co-Authored-By: Claude Sonnet 5 <[email protected]>
docker compose restart worker looked like it should reflect new code
(docker-compose.yml bind-mounts ./app:/app/app for dev), but the
worker Dockerfile installed the app package non-editably (pip install
.), which copies a frozen snapshot into site-packages at image build
time. The bare `arq` console-script just imports whatever
site-packages says `app` is -- unlike uvicorn --reload, which
explicitly resolves its import string against the CWD -- so the
worker silently kept running whatever code existed at the last
`docker compose up --build`, no matter how many times it was
restarted or how recent the git pull was.
Confirmed live: after pulling the series/season ingestion pipeline
and restarting, the worker log still read "Starting worker for 2
functions: ingest_movie_task, ingest_person_task" -- proof it never
saw ingest_series_task/ingest_season_task at all.
Switching to `pip install -e .` makes site-packages point at /app by
absolute path instead of a frozen copy, so the same bind mount that
already worked for the API now actually reaches the worker process
too -- a plain restart after `git pull` is enough going forward.
Applied to both Dockerfiles for consistency, since the API's working
behavior was incidental to uvicorn, not something to rely on either.
Co-Authored-By: Claude Sonnet 5 <[email protected]>
Python turns a generator expression containing await into an async
generator (since it's evaluated inside an async def), which the
built-in all() can't consume -- it raised "TypeError: 'async_generator'
object is not iterable" rather than testing anything. Replaced with a
plain loop. Found by actually running the new integration tests.
Co-Authored-By: Claude Sonnet 5 <[email protected]>
Movie and person ingestion existed, but the series/season/episode API
endpoints had no way to actually populate them from TMDb -- the
biggest remaining gap in Core 0.1's television scope.
- normalize_series/normalize_season in the metadata normalizer,
including two TMDb quirks that would otherwise silently produce
empty data: /tv nests keywords and alternative_titles under
"results" (not "keywords"/"titles" like /movie), and series-level
creators come from a separate created_by array, not the credits
cast/crew lists.
- ingest_series upserts the Series row plus season *stubs* (number,
title, air date, poster -- everything TMDb inlines on /tv/{id});
ingest_season fetches and upserts one season's full episode list,
and requires the series to already exist (TMDb has no single
endpoint for a whole series' episodes, so this has to be a separate
per-season call).
- Wired through the same job-lifecycle path as movie/person ingestion
(runner.py, arq tasks, admin endpoints), with the shared boilerplate
between them factored into runner._run_ingestion and
admin.py:_enqueue rather than copy-pasted a third and fourth time.
- Unit tests for both normalizers (runnable without a DB) and
integration tests for the full ingest_series -> ingest_season flow,
including the series-must-exist-first error and idempotency on
re-ingestion.
Co-Authored-By: Claude Sonnet 5 <[email protected]>
POST /api/v1/admin/ingest/movie returned arq's own internal queue
job id, but GET /api/v1/admin/ingest/jobs/{job_id} looked up our
IngestionJob DB row by its own UUID -- a completely different id that
didn't even exist yet at enqueue time (the row was only created once
the worker picked up the task). Every status poll 404'd.
Fix: create the IngestionJob row synchronously in the enqueue endpoint
(reusing the existing idempotency-key logic, so repeat enqueues for
the same tmdb_id reuse the same row rather than creating duplicates),
pass its real id into the arq task, and have the worker update that
row instead of creating a new one.
Found by actually running the milestone end-to-end: enqueue a real
TMDb ingestion, then poll its status.
Co-Authored-By: Claude Sonnet 5 <[email protected]>
tests/conftest.py's `engine` fixture is session-scoped (one asyncpg
engine/schema for the whole run, for speed), but pytest-asyncio's
default is a fresh event loop per test function. asyncpg connections
are bound to the event loop that created them, so every test after the
first failed with "cannot perform operation: another operation is in
progress" or "attached to a different loop" the moment it touched the
database. Pinning both the fixture and test loop scope to "session"
in pyproject.toml keeps everything on one loop, matching the
session-scoped engine.
Found by actually running the integration suite for the first time,
against real PostgreSQL + Redis on a devbox.
Co-Authored-By: Claude Sonnet 5 <[email protected]>
Postgres ENUM columns auto-create their type as a side effect of the
first CREATE TABLE that uses them; since the migration also explicitly
pre-creates all 7 enum types up front (to group and control that DDL
alongside their explicit drop in downgrade()), the two collided with
"type already exists" on the first table past the first enum's owner.
create_type=False on each ENUM tells SQLAlchemy the migration manages
that DDL itself.
Found by running the migration against a real PostgreSQL for the first
time (docker compose up on a devbox) — this repo had no Docker/Postgres
available to test against when the migration was originally written.
Co-Authored-By: Claude Sonnet 5 <[email protected]>
