limen.data

Canonical docs

In the research projects

The Python project checks recorded input digests before reading bounded populations. The Compiled project reads the literal inputs/ source path declared in YAML from the bundle root. The data chapter explains both acquisition paths and the difference between a fixed file and a provider’s current snapshot.

What this package owns

Owns raw data access, optional threshold-bar formation, train/validation/test splitting, and the helpers that turn raw frames into the data_dict schema used by model code. Does not own indicators, higher-level features, manifests, or model training.

Key entry points

Entry point	Use case	Notes
`HistoricalData`	File-backed BTCUSDT spot klines or raw file ingestion	The main public class exported by `limen.data`
`compute_data_bars()`	Aggregate kline rows into threshold bars before feature engineering	Used by manifests through `set_bar_formation()`
`split_walk_forward()`	Expanding or rolling chronological folds	Explicit train/validation/test windows and step sizes
`split_sequential()`	Ordered train/validation/test windows	Used by manifest-driven prep
`split_data_to_prep_output()`	Standard `data_dict` structure	Converts split frames into model-ready keys like `x_train` and `y_test` without mutating the passed split list or column list

Adjacent modules

limen.experiment consumes this package through manifests and the Universal Experiment Loop.
limen.indicators and limen.features run after data retrieval and optional bar formation.
limen.utils.data_dict_to_numpy is commonly used one layer downstream inside model functions.

Quick orientation

Module or family	Responsibility
`historical_data.py`	HistoricalData class
`_internal/binance_file_to_polars.py`	Binance archive download and parsing
`bars/standard_bars.py`	Threshold bar implementation
`utils/compute_data_bars.py`	Public bar-formation entry point
`utils/splits.py`	Train/validation/test split helpers
`utils/random_slice.py`	Random window slicing helper

text · README.md

1data/2├── README.md3├── __init__.py4├── _internal/5├── bars/6├── historical_data.py7└── utils/

Merged source PR · 87894eb8Package layout · Limen 6.0.1 · current package validation · package source

Things to know

HistoricalData is stateful. Each get_* call mutates self.data and self.data_columns, and also returns the resulting pl.DataFrame.
get_spot_klines() reads the Hugging Face BTCUSDT 1-minute dataset by default.
get_binance_file() normalizes millisecond timestamps automatically when the source file stores them as large integers.
get_any_file() is the generic loader for local paths and URLs.