Single-File Decoder

limen run compiles YAML into a manifest-backed SFD. Python callers supply an SFD and search strategy directly. Both routes execute through UEL with MSQ; author a custom SFD when preparation or model code needs arbitrary Python logic.

Prerequisites

• familiarity with the Experiment Manifest and Universal Experiment Loop
• a parameter space whose values are lists
• either a concrete manifest or custom prep() and model() functions

Component and Python SFD styles

This reference describes the two internal SFD shapes. Python Research authors ordinary params/prep/model functions. Compiled Research authors YAML; the CLI builds its internal SFD without requiring Python SFD authoring.

Choose by methodological ownership. The Python project supplies ordinary functions and retains its evaluated fitted models. The Compiled project adopts the Manifest contract and reconstructs selected recorded rounds. Neither requires the other interface as a preliminary step.

Python SFD parameter contract

Every SFD must expose params().

The Python project's run_research function below supplies a four-candidate space through params=lambda: space. A normal module-level params() can return the same dictionary; the callable boundary is unchanged.

params() returns the search space dictionary. Each key is a parameter name and each value must be a list, even when only one value is present.

params() rules

• the return value must be a dictionary
• every value must be a list
• individual values are scalars such as ints, floats, strings, booleans, or callables
• structured values are possible when a manifest step explicitly expects them; keep them deterministic and inspectable

Ordinary single-process searches can use callable values without an experiment_dir. Search identities and CSV cells use the callable's str() representation, so distinct candidates need distinct representations within the process; these are not portable checkpoint identities. The live experiment_log retains callables in an Object column, which is not a numeric parameter-correlation input. Setting experiment_dir still requires JSON-serializable SFD params().

Optional Manifest component API

Manifest-driven SFDs expose manifest() instead of custom prep() and model() functions.

1limen validate compiled-research/selection.yaml2limen profile compiled-research/selection.yaml3limen run --dry-run compiled-research/selection.yaml4limen run --no-progress-bar compiled-research/selection.yaml5limen assess compiled-research/selection.yaml --results results/dev/compiled-cli

This project uses CompiledSFD to translate its YAML definition into a Manifest-based SFD. Packaged foundational SFDs expose the same boundary through native params() and manifest() functions.

What the manifest path provides

• declarative data fetching
• split-first preparation with train-only fitting
• automatic preparation/model wiring inside UEL
• an inspectable definition for reconstruction and collaboration

Runtime rules for manifest-driven SFDs

The ordinary MSQ path prepares data for every candidate using the Manifest's preparation and model functions. The search strategy owns the parameter domain; the Manifest owns the pipeline. When data= is omitted, Limen fetches data from the Manifest. The legacy execution reference records the older prep_each_round and callable-override rules.

Custom SFDs

Python Research supplies prep() and model() directly. The complete project constructs its own parameter domain, evaluates caller-owned Ridge pipelines and retains those same fitted objects for inference:

1def run_trials(research, experiment_name):2    space = {"window": [4, 12], "alpha": [0.1, 1.0]}3    retained = {}4 5    def model(data, round_params):6        fitted = fit(data["train"], round_params)7        forecasts = fitted.predict(data["validation"].select("momentum"))8        score = backtest(data["validation"], forecasts)9        retained[(round_params["window"], round_params["alpha"])] = fitted10        return {"validation_net_return": score}11 12    sfd = SimpleNamespace(params=lambda: space, prep=prep, model=model)13    uel = UniversalExperimentLoop(14        data=research, sfd=sfd, search_strategy=GridStrategy(ParamDomain(space))15    )16    # An absolute filename stem writes only this CSV; no persisted round artifacts.17    uel.run(experiment_name, n_permutations=4, workers=1,18            post_processing=False, progress_bar=False)19    return uel, retained

The function supplies caller-owned prep and model callables through a namespace. Its model returns an explicit validation net return and retains the exact fitted pipeline later wrapped in a Sensor. Use the complete project for its imports, feature/target preparation, fitting and input checks.

Function Contracts

prep(data, round_params)

Custom prep() receives the experiment dataframe and the round-specific parameters. It must return a data_dict that the model function can consume.

Defaults:

• keep datetime in the dataframe until just before split_data_to_prep_output()
• pass split lists and column lists directly to split_data_to_prep_output(); it copies both inputs before dropping datetime, so caller-owned values are unchanged
• capture all_datetimes = data['datetime'].to_list() before dropping rows when alignment metadata is required
• make the function deterministic with respect to round_params

model(data, round_params)

Custom model() receives whatever prepared value the caller's prep() returns, plus the round parameters. It returns a results dictionary. The Python project supplies validation_net_return; callers may use Limen's binary, multiclass or continuous metrics when those measures fit their model outputs.

Include _preds when native prediction recording or downstream analysis needs predictions. Include fitted preprocessing objects under the supported data keys, such as _scaler, when using helpers that consume that state. Neither an arbitrary score dictionary nor a retained Python object creates a reconstructible Compiled experiment. The project retains its evaluated scaler inside the same sklearn pipeline as its fitted Ridge model.

Foundational versus custom

Limen ships foundational SFDs under limen.sfd.foundational_sfd. These are reference implementations that show the preferred style for shared Limen workflows.

Custom SFDs are project experiment modules. They can use the same manifest system or the custom prep() and model() path.

Foundational SFDs versus reference architecture

There is one more design split that matters in practice:

For example:

• limen.sfd.foundational_sfd.logreg_binary packages the experiment
• limen.sfd.reference_architecture.logreg_binary owns the model implementation

For reconstructible Compiled runs, Trainer replays a selected recorded round with its declared architecture and preprocessing. Python Research instead wraps its retained evaluated fitted state directly.

Execution checks optional dependencies at the model boundary. Importing the SFD catalog does not require every model extra.

Use Built-In SFDs for the current shipped catalog and Reference Architecture for the model layer underneath it.

Read next

• Continue to Command-Line Interface for the normal YAML run path.
• Continue to Experiment Manifest for the full declarative pipeline used by standard SFDs.
• Continue to Built-In SFDs for the shipped foundational decoder catalog.
• Continue to Universal Experiment Loop for direct Python execution.
• Continue to Reference Architecture for model implementation authoring beyond packaged decoders.
• Use Indicators, Features, Transforms, and Scalers as the reference layer while authoring new SFDs.