Fix duplicate WatchProvider rows when the same provider appears twice
CI / lint (push) Canceled after 0s
CI / typecheck (push) Canceled after 0s
CI / unit-tests (push) Canceled after 0s
CI / integration-tests (push) Canceled after 0s
CI / container-build (push) Canceled after 0s
CI / dependency-audit (push) Canceled after 0s
CI / lint (push) Canceled after 0s
CI / typecheck (push) Canceled after 0s
CI / unit-tests (push) Canceled after 0s
CI / integration-tests (push) Canceled after 0s
CI / container-build (push) Canceled after 0s
CI / dependency-audit (push) Canceled after 0s
Found by running the real test suite against Postgres: test_ingest_movie_dedupes_watch_provider_across_countries failed with 2 Netflix rows instead of 1 when a movie's watch/providers block lists the same provider (Netflix, external_id 8) in more than one country -- the first genuine case in the whole ingestion pipeline where the same external entity gets resolved via get_or_create twice within one ingest before anything commits (company/person/collection never have this shape in existing fixtures). sync_watch_providers now resolves each distinct provider (by external_id) at most once per ingest and reuses it for every offering that references it, rather than calling get_or_create_watch_provider once per offering regardless of repeats. Co-Authored-By: Claude Sonnet 5 <[email protected]>
This commit is contained in:
co-authored by
Claude Sonnet 5
parent
1cdfbc59ac
commit
a38c2074a6
@@ -352,7 +352,17 @@ async def sync_watch_providers(
|
|||||||
distinct WatchProvider touched paired with its current logo_path so the
|
distinct WatchProvider touched paired with its current logo_path so the
|
||||||
caller can sync its logo artwork via company_logo_artwork + sync_artwork
|
caller can sync its logo artwork via company_logo_artwork + sync_artwork
|
||||||
-- this repository layer doesn't import the services layer to build a
|
-- this repository layer doesn't import the services layer to build a
|
||||||
NormalizedArtwork itself (see _NormalizedArtworkLike above)."""
|
NormalizedArtwork itself (see _NormalizedArtworkLike above).
|
||||||
|
|
||||||
|
Resolves each distinct provider (by external_id) at most once, even
|
||||||
|
when it appears in several countries/monetization types for the same
|
||||||
|
entity (e.g. Netflix offering both a US and a GB flatrate row) --
|
||||||
|
calling get_or_create_watch_provider a second time for the same
|
||||||
|
external_id later in the same uncommitted session produced a second,
|
||||||
|
duplicate WatchProvider row instead of finding the first (caught
|
||||||
|
against real Postgres; watch providers was the first place in the
|
||||||
|
ingestion pipeline where the same external entity is ever resolved
|
||||||
|
twice within one ingest before anything commits)."""
|
||||||
|
|
||||||
await session.execute(
|
await session.execute(
|
||||||
delete(WatchProviderOffering).where(
|
delete(WatchProviderOffering).where(
|
||||||
@@ -361,14 +371,19 @@ async def sync_watch_providers(
|
|||||||
)
|
)
|
||||||
)
|
)
|
||||||
touched: dict[uuid.UUID, tuple[WatchProvider, str | None]] = {}
|
touched: dict[uuid.UUID, tuple[WatchProvider, str | None]] = {}
|
||||||
|
provider_by_external_id: dict[str, WatchProvider] = {}
|
||||||
for offering in offerings:
|
for offering in offerings:
|
||||||
await refs.get_or_create_country(session, offering.country_code, offering.country_code)
|
await refs.get_or_create_country(session, offering.country_code, offering.country_code)
|
||||||
|
ext_id = offering.provider.external_id
|
||||||
|
watch_provider = provider_by_external_id.get(ext_id)
|
||||||
|
if watch_provider is None:
|
||||||
watch_provider = await refs.get_or_create_watch_provider(
|
watch_provider = await refs.get_or_create_watch_provider(
|
||||||
session,
|
session,
|
||||||
provider_id=tmdb_provider_id,
|
provider_id=tmdb_provider_id,
|
||||||
external_id=offering.provider.external_id,
|
external_id=ext_id,
|
||||||
name=offering.provider.name,
|
name=offering.provider.name,
|
||||||
)
|
)
|
||||||
|
provider_by_external_id[ext_id] = watch_provider
|
||||||
touched[watch_provider.id] = (watch_provider, offering.provider.logo_path)
|
touched[watch_provider.id] = (watch_provider, offering.provider.logo_path)
|
||||||
session.add(
|
session.add(
|
||||||
WatchProviderOffering(
|
WatchProviderOffering(
|
||||||
|
|||||||
Reference in New Issue
Block a user