Define and run the experiment

The Python project and Compiled project are independent starting points. Their models, units and assessment populations differ. Running both does not turn their scores into a controlled comparison.

Prerequisites

Use the matching Limen wheel, its recorded input bundle and the extras stated on each project page. Run Compiled commands from the bundle root. Python Research supplies its own data and functions; its complete project gives the corresponding entry point.

The declaration connects the research decisions

The feature, target, learner, and execution policy jointly define an experiment. A feature expresses information available at a decision time. A target supplies the observed outcome. The learner fits their relationship, and the execution policy determines how its output requests a position.

The preceding chapters develop features, targets, model families, and rule strategies. This chapter connects those decisions in the complete research definition. The workflow table identifies additional requirements for recovery, workers, folds, and promotion.

The three chronological populations serve different roles:

A reserved final assessment adds a separately protected population after those research choices. Its reservation precedes research, as developed in final holdout.

Own the Python experiment

The Python project supplies data, a parameter space and ordinary preparation and model functions to UEL. Its model function fits one candidate, scores validation and retains that fitted object. Selection wraps those same evaluated objects in Sensors and combines compatible forecasts in a Cohort.

The complete selection function keeps each of those decisions visible. The SFD reference describes its function signatures; the UEL reference describes execution, recording and recovery.

1def run_research(bars, experiment_name):2    research = bars[:1440]3    uel, retained = run_trials(research, experiment_name)4    selected = choose_parameters(uel)5    sensors = selected_sensors(bars, selected, retained)6    cohort = Cohort.from_python(sensors)7    return cohort, uel, selected, retained

Declare the Compiled experiment

compiled-research/research.yaml is the complete, composable worked declaration. It compares regularization, feature ablation and optional PCA over eight candidates. Incompatible decisions have separate complete YAML files: rule-based models, continuous targets, alternate bars, walk-forward folds and guarded final assessment.

Run from the recorded bundle root with its matching wheel and data, stats and boosting extras installed. The source paths below refer to the bundle's pinned inputs/ directory.

Fix the source and partitions

1    data_source:2      method: limen.data.HistoricalData.get_any_file3      params:4        file_path_or_url: inputs/spot_15m_20250101_20250531.parquet

1    split_dates:2      train_start: "2025-01-01"3      train_end: "2025-01-15"4      val_start: "2025-01-15"5      val_end: "2025-01-18"6      test_start: "2025-01-18"7      test_end: "2025-01-22"8      val_predict_guard: true9      test_predict_guard: true

Dates are half-open UTC windows. Fit target and transform state on training; validation chooses among candidates and tunes the declared threshold. The test partition contributes neither choice. Reusing validation for calibration and selection consumes that evidence; it is not a fresh assessment.

Define the information the model receives

1    features:2      - func: limen.features.lag_range3        params:4          col: roc_15          start: 16          end: 37          group: momentum8      - func: limen.features.fractional_diff9        params:10          d: 0.411          cols: [close]12          threshold: 0.0513          group: price

Indicators add three past-return horizons; the features above add past-return lags and fixed-order fractional differentiation. Feature groups identify which added columns enter the model. The target shifts the next return into a train-fitted binary label. Its fitted cutoff is reused on later partitions.

1    scaler:2      class: limen.scalers.RobustScaler3    feature_ablation:4      drop_count_key: feature_drop_count5      seed_key: feature_drop_seed6    pca_compression:7      enabled_param: auto_pca8      n_components_param: pca_k

Robust statistics and PCA fit on training. Ablation has its own explicit seed. PCA and feature dropping are candidate choices, not claims that either improves future performance. Targets, features, scalers and PCA give the parameter and boundary details.

Declare calibration and the objective

1    calibration:2      probability_calibration:3        func: limen.calibration.sklearn_probability_calibrator4        params:5          method: sigmoid6      threshold_function:7        func: limen.calibration.grid_threshold_optimizer8        params: {}

1    objective:2      metric: backtest_total_return3      direction: maximize

The classifier fits on training. Probability calibration and threshold choice use validation. The objective names validation event-ledger return explicitly; the same field later ranks candidates. Calibration explains what the four on/off modes measure.

Measure the decision under an execution policy

1    backtest:2      source_interval_seconds: 9003      fee_bps: 7.04      slip_bps: 3.05      notional_rate: 0.56      execution_lag_seconds: 0.07      product:8        kind: cash_spot9        instrument: BTCUSDT10        base_currency: BTC11        quote_currency: USDT12        quantity_step: 1.0e-913        min_notional: 0.0

The declared product, source clock, fees, slippage and exposure determine the ledger. They are scenario assumptions. Backtest develops their timing and arithmetic, including separate recorded-funding and allocation variants.

Bound and record the search

1  params:2    C: [0.5, 1.0]3    random_state: [42]4    feature_groups: [momentum|price]5    feature_drop_count: [0, 1]6    feature_drop_seed: [7]7    auto_pca: [false, true]8    pca_k: [2]9    use_calibration: [true]10    use_threshold: [true]

1uel:2  output_path: compiled-research3  n_permutations: 84  workers: 15  search_strategy:6    type: grid7    shuffle: true8    seed: 179  prep_each_round: true10  checkpoint_interval: 111  feedback_interval: 112  output_format: parquet13  recording:14    predictions: true15    alignment: true16    trade_ledger: true17    fold_details: true

Eight candidates exhaust this finite grid. The search seed controls traversal; it does not replace a model seed. All recording groups are explicit so the artifacts retain predictions, alignment, ledgers and fold detail. Checkpoints preserve search state for CLI recovery.

1limen run --no-progress-bar compiled-research/research.yaml

The CLI writes the unchanged definition, metadata.json, results.csv, results.parquet, compressed round records and checkpoint/audit state under results/dev/compiled-research. A round with a pipeline error does not count as a successful research result.

Carry the selected behavior into inference

1assessment:2  metric: val_backtest_total_return3  direction: maximize4  top_k: 25  predict: true

1limen assess compiled-research/research.yaml --results results/dev/compiled-research

Assessment selects two finite validation results, breaking metric ties by recorded identity. Trainer reconstructs those rounds under their original declaration and checks replayed metrics. The resulting Sensors retain fitted preprocessing and calibration; the Cohort combines their compatible outputs. assessment.json retains membership and predictions.parquet retains decisions, unavailable reasons and native availability values where supplied.

This reconstruction is a checked refit, rather than restoration of arbitrary saved weights. Trainer and Cohort document the artifact and inference contracts. Their optional Python APIs are reference material; the Compiled workflow remains YAML and CLI.

Validation establishes the declared contract

Schema version 1.0 identifies the configuration language independently of the package version. Structural validation rejects unknown fields, incompatible types, broken references, and unsupported combinations. Native validation also checks callable binding, consumed search parameters, chronology, and component contracts. An editor schema supplies structural assistance while native validation establishes those additional checks.

Profiling loads real observations and samples preparation and fitting. Its completed and failed sample counts determine whether the estimate contains successful work. A zero-error process exit alone can accompany a profile with no successful samples. Dry-run checks compilation and setup without executing the candidate campaign.

Changing source, parameters, target geometry, or execution economics creates a changed research definition. The CLI retains the exact declaration used by the experiment and requires that declaration for assessment. Project storage and lineage preserve the relationship between successive definitions, as developed in the complete workflow.

Continue the question

Continue to controlled perturbations to compare changes to the declared population, features, representation, target, and learner.

Use search to change traversal, feedback to bound the remaining work, and walk-forward to repeat the decision over later windows. Reserve final observations before research with the separate guarded YAML project.

The Manifest appendix keeps complete field and component API detail. The complete workflow connects each declaration to its native result files and current execution evidence.