Fix duplicate WatchProvider rows when the same provider appears twice
CI / lint (push) Canceled after 0s
CI / typecheck (push) Canceled after 0s
CI / unit-tests (push) Canceled after 0s
CI / integration-tests (push) Canceled after 0s
CI / container-build (push) Canceled after 0s
CI / dependency-audit (push) Canceled after 0s

Found by running the real test suite against Postgres:
test_ingest_movie_dedupes_watch_provider_across_countries failed with 2
Netflix rows instead of 1 when a movie's watch/providers block lists the
same provider (Netflix, external_id 8) in more than one country -- the
first genuine case in the whole ingestion pipeline where the same
external entity gets resolved via get_or_create twice within one ingest
before anything commits (company/person/collection never have this
shape in existing fixtures).

sync_watch_providers now resolves each distinct provider (by
external_id) at most once per ingest and reuses it for every offering
that references it, rather than calling get_or_create_watch_provider
once per offering regardless of repeats.

Co-Authored-By: Claude Sonnet 5 <[email protected]>
This commit is contained in:
NixReaper
2026-09-06 07:55:36 -05:00
co-authored by Claude Sonnet 5
parent 1cdfbc59ac
commit a38c2074a6
+22 -7
View File
@@ -352,7 +352,17 @@ async def sync_watch_providers(
distinct WatchProvider touched paired with its current logo_path so the
caller can sync its logo artwork via company_logo_artwork + sync_artwork
-- this repository layer doesn't import the services layer to build a
NormalizedArtwork itself (see _NormalizedArtworkLike above)."""
NormalizedArtwork itself (see _NormalizedArtworkLike above).
Resolves each distinct provider (by external_id) at most once, even
when it appears in several countries/monetization types for the same
entity (e.g. Netflix offering both a US and a GB flatrate row) --
calling get_or_create_watch_provider a second time for the same
external_id later in the same uncommitted session produced a second,
duplicate WatchProvider row instead of finding the first (caught
against real Postgres; watch providers was the first place in the
ingestion pipeline where the same external entity is ever resolved
twice within one ingest before anything commits)."""
await session.execute(
delete(WatchProviderOffering).where(
@@ -361,14 +371,19 @@ async def sync_watch_providers(
)
)
touched: dict[uuid.UUID, tuple[WatchProvider, str | None]] = {}
provider_by_external_id: dict[str, WatchProvider] = {}
for offering in offerings:
await refs.get_or_create_country(session, offering.country_code, offering.country_code)
watch_provider = await refs.get_or_create_watch_provider(
session,
provider_id=tmdb_provider_id,
external_id=offering.provider.external_id,
name=offering.provider.name,
)
ext_id = offering.provider.external_id
watch_provider = provider_by_external_id.get(ext_id)
if watch_provider is None:
watch_provider = await refs.get_or_create_watch_provider(
session,
provider_id=tmdb_provider_id,
external_id=ext_id,
name=offering.provider.name,
)
provider_by_external_id[ext_id] = watch_provider
touched[watch_provider.id] = (watch_provider, offering.provider.logo_path)
session.add(
WatchProviderOffering(