Continue with the evaluated objects returned by the Python project, or the completed result directory of the Compiled project. The handoff preserves different kinds of state.
Prerequisites
Retain the completed research definition and its selection rule. Python needs the evaluated fitted objects; Compiled Research needs its original YAML and result directory. Calibration also needs the validation population it consumes.
Python Research
The project selects the best validation return separately for each momentum window, breaking ties by alpha. It retains every fitted pipeline and wraps the selected objects in Sensor.from_model; Cohort.from_python averages their compatible next-bar return forecasts. No fit is performed after selection. The complete Python project returns that Cohort, its loop, selected parameters and retained objects in the same process.
The calibration variant uses an arbitrary fitted classifier and selected threshold through Sensor.from_python. The preparation and output adapters remain caller-owned. A probability, a binary decision and a return forecast have different meanings; an aggregator must preserve or explicitly map them.
- selected
[ { "window": 4, "alpha": 0.1 }, { "window": 12, "alpha": 0.1 } ]- retained models
4
- inference rows
480
- fitting cutoff
"2025-01-11T00:00:00+00:00"
- selection cutoff
"2025-01-16T00:00:00+00:00"
Limen 6.0.1 · 87894eb8 · Merged source PR · Validation and result digest · Native records
Compiled Research
The research declaration fixes selection before assess reads the finished run. Run the declaration and apply its validation ranking through the CLI:
limen run --no-progress-bar compiled-research/preparation.yamllimen run --no-progress-bar compiled-research/research.yamllimen assess compiled-research/research.yaml --results results/dev/compiled-researchassessment.json names the selected recorded rounds and bound members. predictions.parquet retains their Cohort decisions, probabilities where available, timestamps, reasons and actual availability. Reconstruction refits the recorded training definition and checks replayed metrics; it does not restore arbitrary saved weights.
- rounds
8
- selected members
2
Limen 6.0.1 · 42593d34 · Merged source PR · Validation and result digest · Native records
Output meaning determines compatible composition
A Sensor carries a fitted prediction mechanism, preparation, output meaning, and relevant cutoffs. Fitting cutoff identifies observations that fitted state. Selection cutoff also includes observations used to choose that state. Actual availability then describes when each later decision can be acted upon.
| Composition path | Admitted inputs | Aggregation and identity |
|---|---|---|
| Python mean | Compatible continuous, signed, or binary outputs | Mean preserves units and signed magnitude under a common output meaning. |
| Python vote | Exact binary decisions | Majority vote resolves ties to zero and carries no calibrated probability. |
| Python custom | Explicitly mapped compatible outputs | The supplied mapper and aggregate define meaning and retain strict decision alignment. |
| Native homogeneous ordinary Cohort | Compatible Sensors from one architecture with retained artifact identity | Architecture hints select probability weighting or majority vote; ordinary regressor predictions vote up above zero. |
| Native mixed or fold-containing Cohort | Admitted Sensors declaring compatible native binary outputs | Probability weighting requires every member's probability declaration; other admitted sets vote on binary decisions. |
| Native explicit membership | Admitted rounds across experiments or qualified folds | Original experiment, round, and fold identities remain bound to each member. |
Timestamp cardinality and alignment are part of compatibility. Python composition preserves unavailable-member blocking and the latest known member availability. Native composition follows its separately documented failure and probability policy. Cohort owns those exact behaviors.
Reliability weighting consumes observed correctness evidence
The Bayesian Python Cohort uses a Beta correctness prior and supplied observed binary outcomes to fix each member's historical reliability weight. Evidence names the member, prediction timestamp, later outcome availability, prediction, observed outcome, and source. Generated labels do not supply observed correctness evidence.
The evidence availability extends the Cohort's selection cutoff. Weights remain fixed until a new explicit construction admits additional evidence. Correlated errors and changing regimes affect interpretation of historical correctness. The aggregate is a weighted vote and reports no calibrated probability. Bayesian composition defines the evidence schema and posterior calculation.
Reconstruction checks the evidence actually present
Trainer reconstructs native YAML rounds from their original definition and recorded parameters. With results.csv, it compares matching eligible numeric replay metrics under the architecture's tolerances. Metrics outside that intersection remain outside the comparison. Without the CSV, the component API skips metric validation and creates Sensors from the admitted reconstruction artifacts.
The complete CLI selection workflow retains the CSV and invokes the metric comparison. Qualified fold promotion additionally needs its original witnesses. Reconstruction, live caller-owned state, and protected final assessment therefore retain distinct evidence requirements. The workflow table connects those requirements to each mode.
Calibration consumes evidence
Probability calibration and threshold selection use the declared validation population. Comparing their four on/off modes explains what state was learned and retained; it does not create four untouched assessments:
Python Research calibration
- decisions
[ { "calibrated": false, "tuned": false, "threshold": 0.5, "inference_rows": 480, "positive_predictions": 223 }, { "calibrated": false, "tuned": true, "threshold": 0.2, "inference_rows": 480, "positive_predictions": 478 }, { "calibrated": true, "tuned": false, "threshold": 0.5, "inference_rows": 480, "positive_predictions": 166 }, { "calibrated": true, "tuned": true, "threshold": 0.2, "inference_rows": 480, "positive_predictions": 480 } ]- validation reused
true
Limen 6.0.1 · 42593d34 · Merged source PR · Validation and result digest · Native records
Compiled Research calibration
- modes
[ { "use_calibration": false, "use_threshold": false, "optimal_threshold": null }, { "use_calibration": true, "use_threshold": false, "optimal_threshold": 0.5 }, { "use_calibration": false, "use_threshold": true, "optimal_threshold": 0.2 }, { "use_calibration": true, "use_threshold": true, "optimal_threshold": 0.49999999999999994 } ]
Limen 6.0.1 · 42593d34 · Merged source PR · Validation and result digest · Native records
The calibration reference explains fitted-state reuse and each supported method. Sensor defines construction, output shape, guards and failure behavior. Cohort distinguishes Python aggregation, YAML assessment and optional native component selectors. Those optional Python component APIs are not part of Compiled authoring.
Continue to final assessment with selection already frozen. The protected branch must have been reserved before its research began.