Skip to main content

origo.data

What this package owns​

  • Historical data-access helpers for Binance, OKX, Bybit, ETF, FRED, and Bitcoin dataset retrieval used by local and API-facing flows.
  • Data-frame shaping helpers such as bar computation and split utilities.
  • Internal fetch helpers that bridge file-backed and database-backed historical access.

What this package does not own​

  • HTTP request and response handling for Origo APIs.
  • Canonical event storage, projection, or serving guarantees.
  • Governance authority or proof policy.

Key entry points​

  • historical_data.py
    • HistoricalData
  • utils/compute_data_bars.py
    • compute_data_bars
  • utils/splits.py
    • split_sequential
    • split_random
    • split_data_to_prep_output
  • _internal/generic_endpoints.py
    • query helpers used by higher-level surfaces

Major dependencies and adjacent modules​

  • External dependency:
    • polars
  • Adjacent modules:
    • origo/query
    • api/origo_api
    • origo/events

Notes​

  • origo.data.bars intentionally exposes unavailable adaptive bar builders in this build; only the base pass-through path is usable unless those builders are restored.
  • HistoricalData is stateful and updates .data and .data_columns on each get_* call.

Operational notes​

  • HistoricalData._get_data_for_test() reads from <repo>/datasets/klines_2h_2020_2025.csv; that file must exist in the repository root.
  • get_binance_file() normalizes millisecond timestamps to seconds automatically if the value exceeds 10^13.
  • Historical spot interfaces use strict window selection: exactly one of date-window(start_date/end_date), n_latest_rows, or n_random_rows.
  • Date-window inputs are strict UTC day strings (YYYY-MM-DD) and apply start inclusive plus end-day inclusive, implemented as next-day exclusive.
  • The auth_token parameter is forwarded to generic_endpoints for authenticated database queries; leave it None for public Binance file access.
  • generic_endpoints is fail-loud for env config: set CLICKHOUSE_HOST, CLICKHOUSE_HTTP_PORT, CLICKHOUSE_USER, CLICKHOUSE_PASSWORD, and CLICKHOUSE_DATABASE before calling database query helpers.
  • split_data_to_prep_output expects all splits to share the same column schema.

Canonical docs​

  • ../../docs/Reference/README.md
  • ../../docs/raw-query-reference.md
  • ../../docs/historical-reference.md