Fix duplicate WatchProvider rows when the same provider appears twice
CI / lint (push) Canceled after 0s
CI / typecheck (push) Canceled after 0s
CI / unit-tests (push) Canceled after 0s
CI / integration-tests (push) Canceled after 0s
CI / container-build (push) Canceled after 0s
CI / dependency-audit (push) Canceled after 0s

Found by running the real test suite against Postgres:
test_ingest_movie_dedupes_watch_provider_across_countries failed with 2
Netflix rows instead of 1 when a movie's watch/providers block lists the
same provider (Netflix, external_id 8) in more than one country -- the
first genuine case in the whole ingestion pipeline where the same
external entity gets resolved via get_or_create twice within one ingest
before anything commits (company/person/collection never have this
shape in existing fixtures).

sync_watch_providers now resolves each distinct provider (by
external_id) at most once per ingest and reuses it for every offering
that references it, rather than calling get_or_create_watch_provider
once per offering regardless of repeats.

Co-Authored-By: Claude Sonnet 5 <[email protected]>
This commit is contained in:
NixReaper
2026-09-06 07:55:36 -05:00
co-authored by Claude Sonnet 5
parent 1cdfbc59ac
commit a38c2074a6
+17 -2
View File
@@ -352,7 +352,17 @@ async def sync_watch_providers(
distinct WatchProvider touched paired with its current logo_path so the distinct WatchProvider touched paired with its current logo_path so the
caller can sync its logo artwork via company_logo_artwork + sync_artwork caller can sync its logo artwork via company_logo_artwork + sync_artwork
-- this repository layer doesn't import the services layer to build a -- this repository layer doesn't import the services layer to build a
NormalizedArtwork itself (see _NormalizedArtworkLike above).""" NormalizedArtwork itself (see _NormalizedArtworkLike above).
Resolves each distinct provider (by external_id) at most once, even
when it appears in several countries/monetization types for the same
entity (e.g. Netflix offering both a US and a GB flatrate row) --
calling get_or_create_watch_provider a second time for the same
external_id later in the same uncommitted session produced a second,
duplicate WatchProvider row instead of finding the first (caught
against real Postgres; watch providers was the first place in the
ingestion pipeline where the same external entity is ever resolved
twice within one ingest before anything commits)."""
await session.execute( await session.execute(
delete(WatchProviderOffering).where( delete(WatchProviderOffering).where(
@@ -361,14 +371,19 @@ async def sync_watch_providers(
) )
) )
touched: dict[uuid.UUID, tuple[WatchProvider, str | None]] = {} touched: dict[uuid.UUID, tuple[WatchProvider, str | None]] = {}
provider_by_external_id: dict[str, WatchProvider] = {}
for offering in offerings: for offering in offerings:
await refs.get_or_create_country(session, offering.country_code, offering.country_code) await refs.get_or_create_country(session, offering.country_code, offering.country_code)
ext_id = offering.provider.external_id
watch_provider = provider_by_external_id.get(ext_id)
if watch_provider is None:
watch_provider = await refs.get_or_create_watch_provider( watch_provider = await refs.get_or_create_watch_provider(
session, session,
provider_id=tmdb_provider_id, provider_id=tmdb_provider_id,
external_id=offering.provider.external_id, external_id=ext_id,
name=offering.provider.name, name=offering.provider.name,
) )
provider_by_external_id[ext_id] = watch_provider
touched[watch_provider.id] = (watch_provider, offering.provider.logo_path) touched[watch_provider.id] = (watch_provider, offering.provider.logo_path)
session.add( session.add(
WatchProviderOffering( WatchProviderOffering(