Choose which trials run

The Python project supplies its search functions. The Compiled project supplies complete YAML declarations and CLI commands. Each example retains its own population and execution assumptions.

Prerequisites

• the matching package and recorded input bundle described in each project
• a bounded parameter domain and an explicit research objective
• the statistics extra for Sobol sampling
• an experiment directory and the required recording groups when continuing an interrupted run

The search components retain distinct state

A candidate passes through four stages:

1. SearchStrategy proposes combinations
2. MSQ decides what is still legal
3. feedback mutates MSQ and sometimes the underlying ParamDomain
4. checkpoints and full recording preserve enough state to continue later with resume=True

Choose a built-in strategy

CLI defaults to random search. Grid search is unshuffled by default. From 5.31.0, YAML and Python accept a seeded finite shuffle; YAML shuffle: true requires an integer seed. A shuffle declaration on other strategies is rejected. The two project functions make those choices explicit:

1def search(data_dir, output):2    bars = hourly(data_dir, 1440)3    space = {'window': [4, 12], 'alpha': [0.1, 1.0]}4    runs = {}5    for name in ('grid', 'random', 'sobol', 'shuffled'):6        domain = ParamDomain(space)7        strategy = (GridStrategy(domain, shuffle=True, seed=42) if name == 'shuffled'8                    else {'grid': GridStrategy, 'random': RandomStrategy, 'sobol': SobolStrategy}[name](domain, **({} if name == 'grid' else {'seed': 42})))9 10        def model(data, round_params):11            fitted = project.fit(data['train'], round_params)12            return {'validation_net_return': project.backtest(data['validation'], fitted.predict(data['validation'].select('momentum')))}13 14        loop = UniversalExperimentLoop(data=bars, sfd=SimpleNamespace(params=lambda: space, prep=project.prep, model=model), search_strategy=strategy)15        loop.run(str(output / name), n_permutations=4, workers=1, progress_bar=False)16        runs[name] = project.completed_log(loop).select('window', 'alpha', 'validation_net_return').to_dicts()17    return {'runs': runs, 'common_baselines': common_baselines(data_dir, output / 'common-baselines'), 'preparation': preparation(data_dir, output / 'preparation'), 'seed': 42, 'adaptive': adaptive_search(data_dir, output / 'adaptive'),18            'feedback': mixed_feedback(data_dir, output / 'feedback'), 'helper_diagnostics': preparation_helpers(bars, data_dir, output / 'helpers'), 'target_diagnostics': label_diagnostics(bars, data_dir, output / 'targets'), 'analysis': analyze_candidates(bars, output / 'analysis'), 'input_formats': read_source_formats(data_dir), 'multiclass': evaluate_three_classes(bars), 'rules': evaluate_rules(bars), 'linked': residual_research(bars, output / 'linked')}

1limen run --no-progress-bar \2  compiled-research/search.yaml \3  compiled-research/search_random.yaml \4  compiled-research/search_sobol.yaml \5  compiled-research/factorized.yaml \6  compiled-research/adaptive.yaml

The Python case compares grid, random, Sobol and shuffled traversal over two windows and two Ridge penalties. The Compiled case declares its traversal campaigns, a separately admitted factorized grid, and model-family experiments. Their budgets finish the bounded comparison used in this proof; they do not establish universal finite-domain coverage for a sampler. A search seed does not control model randomness.

Strategy choice follows the research budget

Grid search enumerates the finite domain. A seeded shuffle changes joint candidate order while preserving a reproducible prefix. Increasing the permutation budget extends that prefix when the ordered domain and traversal identity remain fixed. Grid is the appropriate choice when completion must establish exhaustive coverage.

Random and Sobol strategies sample unseen discrete combinations under their respective contracts. Their seeds control candidate generation. Models, source snapshots, and fitted preprocessing retain separate sources of variation. Sampling, projection, deduplication, and feedback also affect which alternatives actually execute.

The four-candidate Python comparison completes grid, random, Sobol, and shuffled traversal over its bounded domain. Compiled YAML separately records grid/random/Sobol campaigns, the shuffled composable experiment, and a factorized shuffled grid. Their native artifacts identify the actual candidate identities and scores.

Sobol projects a seeded sequence onto declared values

Sobol sampling uses scrambled sequence coordinates to select from each parameter's ordered values. Multiple sequence points can map to one discrete candidate. Deduplication and bounded retries preserve the sampler's declared behavior, while grid traversal provides exhaustive finite-domain completion. The Sobol implementation contract states the seed, dependency, projection, and recovery requirements.

Candidate identity preserves declared alternatives

Parameter identity includes the sorted parameter keys and each declared choice. Distinct execution identities are required before search begins. Exact-choice mutations and priority injections use that identity, while occurrence identity distinguishes repeated executions of a configuration.

A restored strategy validates its saved domain and traversal state before accepting work. Changing the ordered choices or seed changes the continuation contract. The finite traversal reference owns exact identity rules, traversal versions, recovery compatibility, and prefix measurements. The Sobol reference defines projection, bounded retries, sequence limits, and dependency compatibility.

Factorization reuses independent preparation and signals

The Compiled factorized case crosses two LightGBM tree counts with two fee assumptions. Four economic results require only the admitted fitting work for each model choice. Replaying fees can reuse predictions because the fee axis is independent of the configured preparation and model signal:

1schema_version: '1.0'2metadata:3  name: book_compiled_factorized4  mode: development5sfd:6  manifest:7    type: ml8    data_source:9      method: limen.data.HistoricalData.get_any_file10      params:11        file_path_or_url: inputs/spot_1h_20240101_20241231.parquet12    split_dates:13      train_start: '2024-01-01'14      train_end: '2024-01-15'15      val_start: '2024-01-15'16      val_end: '2024-01-18'17      test_start: '2024-01-18'18      test_end: '2024-01-22'19    indicators:20    - func: limen.indicators.window_return21      params:22        period: 123    target:24      name: quantile_flag25      class: limen.targets.QuantileBinaryTarget26      fit_params:27        source_column: ret_128        quantile: 0.529      transform_params:30        shift: -131    backtest:32      fee_bps: '{fee}'33      slip_bps: 3.034      notional_rate: 0.535    scaler:36      class: limen.scalers.RobustScaler37    reference_architecture: limen.sfd.reference_architecture.lightgbm_binary38  params:39    fee:40    - 7.041    - 9.042    n_estimators:43    - 444    - 845    n_jobs:46    - 147    random_state:48    - 4249    deterministic:50    - true51    force_row_wise:52    - true53    early_stopping_rounds:54    - 055    subsample_freq:56    - 057    subsample:58    - 1.059    colsample_bytree:60    - 1.061uel:62  output_path: compiled-factorized63  factorize: true64  n_permutations: 465  search_strategy:66    type: grid67    shuffle: true68    seed: 1369  output_format: csv70  checkpoint_interval: 1

Admission requires a deterministic reuse declaration for every relevant stage and an immutable compatible grid. Preparation and signal caches are bounded separately at 128 entries. Supported training preparation reuse has a narrower target, scaler, and feature contract than signal reuse. Cache eviction affects repeated work while preserving the required research result.

Factorization uses snapshot execution. Walk-forward, multiple native workers, active feedback, allocation, explicit exits, and validation objectives have their own execution paths. A resumed factorized run validates its prefix and starts reuse caches empty. The factorization contract gives exact admission and comparison requirements.

Workers parallelize admitted candidate fits

Native workers reconstruct supported components from unchanged compiled YAML and frozen hashed input frames. The parent selects candidates, applies feedback, commits completed results in order, and writes artifacts. Interrupted execution preserves the committed contiguous prefix and its matching checkpoint.

Arbitrary caller-owned Python functions execute with one worker. Native worker admission includes per-round preparation, thread controls, source identity, and recording. The workflow table connects these requirements to folds, recovery, and protected research. The complete workflow demonstrates the installed CLI worker and resume commands.

Keep the completed search identifiable

The candidate domain defines which alternatives were eligible. The completed log shows which actually ran; reducers can change that population. Keep those distinctions when reading the best result. Queue and recovery reference defines persisted state, stopping, resumed execution and optional search-strategy extension.

Read the completed campaign

The Python campaign reports its explicit seed and the helper families actually exercised. The Compiled campaign records traversal, the reduced budget and its individual YAML helper campaigns. Full per-candidate rows remain in the native records; these are population and execution counts, not a significance test.

Continue the search decision

Continue to reducers and feedback to see an intervention change the remaining work. Then retain and recover the experiment and its unfinished queue. The CLI and UEL appendices provide the complete command and Python contracts.