Files
Aurora-Core/docker-compose.prod.yml
T
NixReaperandClaude Sonnet 5 fef914db36
CI / lint (push) Canceled after 0s
CI / typecheck (push) Canceled after 0s
CI / unit-tests (push) Canceled after 0s
CI / integration-tests (push) Canceled after 0s
CI / container-build (push) Canceled after 0s
CI / dependency-audit (push) Canceled after 0s
Add artwork self-hosting: local-disk download worker + storage
Section 10: every image API responses reference has pointed straight
at TMDb's CDN since day one -- Aurora never actually downloaded or
served artwork itself, despite storing dimensions/checksum/local_path
columns for exactly that. Closes the gap:

- is_primary was declared on the Artwork model but never actually set
  anywhere -- every row defaulted to False. Now the normalizer marks
  the one image each provider designates as *the* image for an entity
  (TMDb's poster_path/backdrop_path/profile_path) as primary,
  distinct from the dozens of alternates also ingested, including
  correctly marking an existing row primary (not inserting a
  duplicate) when that image already appears in the alternates list.
- app/services/artwork/: an ArtworkStorage interface with a
  LocalDiskStorage implementation (checksum-named files, single-host
  deployment via a Docker volume shared between api and worker -- see
  docs/architecture.md for the S3/R2 upgrade path), and a download
  pipeline with content-type validation, a size cap, and path-traversal
  protection.
- After movie/series/person ingestion succeeds, tasks.py cascades one
  download_artwork_task per primary artwork id via arq's ctx['redis']
  (same fan-out pattern as the season cascade) -- fire-and-forget, not
  tracked in ingestion_jobs, since a failed image cache is a
  known-safe fallback to the provider URL, not a user-facing failure.
- app/main.py serves MEDIA_ROOT at MEDIA_URL_PREFIX; local_path is
  preferred over original_url once set (already-existing fallback
  logic in app/services/common.py:artwork_url, now actually exercised).

Also fixes a real interaction bug this surfaced: every ingestion
pipeline deleted and recreated all of an entity's Artwork rows on each
re-sync, which would have silently wiped local_path/checksum on every
re-ingestion, forcing a full re-download every time and defeating the
point of caching locally at all. app/repositories/polymorphic.py's new
sync_artwork upserts by provider_asset_id (TMDb's stable per-image
path) instead, preserving downloaded copies across re-sync, and
replaces three near-identical delete+recreate helpers that used to
live separately in movie_ingestion.py, series_ingestion.py, and
person_ingestion.py.

Co-Authored-By: Claude Sonnet 5 <[email protected]>
2026-09-05 03:26:28 -05:00

145 lines
4.3 KiB
YAML

# Production-oriented topology (section 1/28): Traefik -> N stateless API
# replicas -> PgBouncer -> PostgreSQL, plus Redis and independently-scaled
# workers. Requires a pre-existing external Traefik network named
# `traefik-public` and a populated `.env` (never commit real secrets).
name: aurora-core
networks:
traefik-public:
external: true
aurora-internal:
driver: bridge
services:
postgres:
image: postgres:17-alpine
restart: unless-stopped
env_file: .env
environment:
POSTGRES_USER: ${POSTGRES_USER:-aurora}
POSTGRES_PASSWORD: ${POSTGRES_PASSWORD}
POSTGRES_DB: ${POSTGRES_DB:-aurora_core}
volumes:
- aurora_postgres_data:/var/lib/postgresql/data
networks: [aurora-internal]
healthcheck:
test: ["CMD-SHELL", "pg_isready -U ${POSTGRES_USER:-aurora}"]
interval: 5s
timeout: 5s
retries: 10
deploy:
resources:
limits: { cpus: "2", memory: 2G }
pgbouncer:
image: edoburu/pgbouncer:latest
restart: unless-stopped
environment:
DATABASE_URL: postgres://${POSTGRES_USER:-aurora}:${POSTGRES_PASSWORD}@postgres:5432/${POSTGRES_DB:-aurora_core}
POOL_MODE: transaction
MAX_CLIENT_CONN: "1000"
DEFAULT_POOL_SIZE: "25"
depends_on:
postgres:
condition: service_healthy
networks: [aurora-internal]
healthcheck:
test: ["CMD", "pg_isready", "-h", "localhost", "-p", "5432"]
interval: 10s
timeout: 5s
retries: 10
redis:
image: redis:7-alpine
restart: unless-stopped
command: ["redis-server", "--maxmemory", "512mb", "--maxmemory-policy", "allkeys-lru"]
volumes:
- aurora_redis_data:/data
networks: [aurora-internal]
healthcheck:
test: ["CMD", "redis-cli", "ping"]
interval: 5s
timeout: 5s
retries: 10
migrate:
build:
context: .
dockerfile: docker/api.Dockerfile
env_file: .env
environment:
DATABASE_URL_SYNC: postgresql+psycopg2://${POSTGRES_USER:-aurora}:${POSTGRES_PASSWORD}@postgres:5432/${POSTGRES_DB:-aurora_core}
depends_on:
postgres:
condition: service_healthy
networks: [aurora-internal]
entrypoint: ["alembic"]
command: ["upgrade", "head"]
api:
build:
context: .
dockerfile: docker/api.Dockerfile
restart: unless-stopped
env_file: .env
environment:
AURORA_ENV: production
DATABASE_BEHIND_TRANSACTION_POOLER: "true"
DATABASE_URL: postgresql+asyncpg://${POSTGRES_USER:-aurora}:${POSTGRES_PASSWORD}@pgbouncer:5432/${POSTGRES_DB:-aurora_core}
REDIS_URL: redis://redis:6379/0
MEDIA_ROOT: /app/media
volumes:
- aurora_media_data:/app/media
depends_on:
pgbouncer:
condition: service_healthy
redis:
condition: service_healthy
migrate:
condition: service_completed_successfully
networks: [traefik-public, aurora-internal]
deploy:
replicas: 3
resources:
limits: { cpus: "1", memory: 1G }
labels:
- "traefik.enable=true"
- "traefik.docker.network=traefik-public"
- "traefik.http.routers.aurora-core-api.rule=Host(`${AURORA_API_HOST:-api.aurora.local}`)"
- "traefik.http.routers.aurora-core-api.entrypoints=websecure"
- "traefik.http.routers.aurora-core-api.tls.certresolver=letsencrypt"
- "traefik.http.services.aurora-core-api.loadbalancer.server.port=8000"
- "traefik.http.services.aurora-core-api.loadbalancer.healthcheck.path=/health"
worker:
build:
context: .
dockerfile: docker/worker.Dockerfile
restart: unless-stopped
env_file: .env
environment:
AURORA_ENV: production
DATABASE_BEHIND_TRANSACTION_POOLER: "true"
DATABASE_URL: postgresql+asyncpg://${POSTGRES_USER:-aurora}:${POSTGRES_PASSWORD}@pgbouncer:5432/${POSTGRES_DB:-aurora_core}
REDIS_URL: redis://redis:6379/0
MEDIA_ROOT: /app/media
volumes:
- aurora_media_data:/app/media
depends_on:
pgbouncer:
condition: service_healthy
redis:
condition: service_healthy
migrate:
condition: service_completed_successfully
networks: [aurora-internal]
deploy:
replicas: 2
resources:
limits: { cpus: "1", memory: 1G }
volumes:
aurora_postgres_data:
aurora_redis_data:
aurora_media_data: