Skip to main content

origo.scraper

What this package owns​

  • ETF and other scraper-oriented acquisition helpers including fetch, parse, normalization, retry, staging, audit, and rights helpers.
  • Adapter-boundary-facing ingestion helpers for scraper-backed sources.

What this package does not own​

  • Generic query execution across all Origo datasets.
  • Canonical event runtime authority for the whole platform.
  • FastAPI endpoints.

Key entry points​

  • pipeline.py
  • etf_adapters.py
  • etf_canonical_event_ingest.py
  • parse_dispatch.py
  • browser_fetch.py
  • http_fetch.py
  • retry.py
  • rights.py

Major dependencies and adjacent modules​

  • Adjacent modules:
    • origo/events
    • api/origo_api
    • control-plane/origo_control_plane/jobs

Canonical docs​

  • ../../docs/Reference/README.md
  • ../../docs/etf-reference.md
  • ../../docs/scraper-reference.md
  • ../../docs/adapter-boundary-reference.md