Fix duplicate WatchProvider rows when the same provider appears twice
CI / lint (push) Canceled after 0s
CI / typecheck (push) Canceled after 0s
CI / unit-tests (push) Canceled after 0s
CI / integration-tests (push) Canceled after 0s
CI / container-build (push) Canceled after 0s
CI / dependency-audit (push) Canceled after 0s
CI / lint (push) Canceled after 0s
CI / typecheck (push) Canceled after 0s
CI / unit-tests (push) Canceled after 0s
CI / integration-tests (push) Canceled after 0s
CI / container-build (push) Canceled after 0s
CI / dependency-audit (push) Canceled after 0s
Found by running the real test suite against Postgres: test_ingest_movie_dedupes_watch_provider_across_countries failed with 2 Netflix rows instead of 1 when a movie's watch/providers block lists the same provider (Netflix, external_id 8) in more than one country -- the first genuine case in the whole ingestion pipeline where the same external entity gets resolved via get_or_create twice within one ingest before anything commits (company/person/collection never have this shape in existing fixtures). sync_watch_providers now resolves each distinct provider (by external_id) at most once per ingest and reuses it for every offering that references it, rather than calling get_or_create_watch_provider once per offering regardless of repeats. Co-Authored-By: Claude Sonnet 5 <[email protected]>
This commit is contained in:
co-authored by
Claude Sonnet 5
parent
1cdfbc59ac
commit
a38c2074a6
@@ -352,7 +352,17 @@ async def sync_watch_providers(
|
||||
distinct WatchProvider touched paired with its current logo_path so the
|
||||
caller can sync its logo artwork via company_logo_artwork + sync_artwork
|
||||
-- this repository layer doesn't import the services layer to build a
|
||||
NormalizedArtwork itself (see _NormalizedArtworkLike above)."""
|
||||
NormalizedArtwork itself (see _NormalizedArtworkLike above).
|
||||
|
||||
Resolves each distinct provider (by external_id) at most once, even
|
||||
when it appears in several countries/monetization types for the same
|
||||
entity (e.g. Netflix offering both a US and a GB flatrate row) --
|
||||
calling get_or_create_watch_provider a second time for the same
|
||||
external_id later in the same uncommitted session produced a second,
|
||||
duplicate WatchProvider row instead of finding the first (caught
|
||||
against real Postgres; watch providers was the first place in the
|
||||
ingestion pipeline where the same external entity is ever resolved
|
||||
twice within one ingest before anything commits)."""
|
||||
|
||||
await session.execute(
|
||||
delete(WatchProviderOffering).where(
|
||||
@@ -361,14 +371,19 @@ async def sync_watch_providers(
|
||||
)
|
||||
)
|
||||
touched: dict[uuid.UUID, tuple[WatchProvider, str | None]] = {}
|
||||
provider_by_external_id: dict[str, WatchProvider] = {}
|
||||
for offering in offerings:
|
||||
await refs.get_or_create_country(session, offering.country_code, offering.country_code)
|
||||
watch_provider = await refs.get_or_create_watch_provider(
|
||||
session,
|
||||
provider_id=tmdb_provider_id,
|
||||
external_id=offering.provider.external_id,
|
||||
name=offering.provider.name,
|
||||
)
|
||||
ext_id = offering.provider.external_id
|
||||
watch_provider = provider_by_external_id.get(ext_id)
|
||||
if watch_provider is None:
|
||||
watch_provider = await refs.get_or_create_watch_provider(
|
||||
session,
|
||||
provider_id=tmdb_provider_id,
|
||||
external_id=ext_id,
|
||||
name=offering.provider.name,
|
||||
)
|
||||
provider_by_external_id[ext_id] = watch_provider
|
||||
touched[watch_provider.id] = (watch_provider, offering.provider.logo_path)
|
||||
session.add(
|
||||
WatchProviderOffering(
|
||||
|
||||
Reference in New Issue
Block a user