Controlled perturbations test dependence on research choices

The Python project chooses columns and fitted transforms directly. The Compiled project declares preparation choices as candidate parameters. Their worked cases retain the prepared observations needed to inspect those changes.

Prerequisites

Both projects require the matching wheel, recorded-input bundle, and data and stats extras. The source population, target, model, and execution policy follow the experiment definition. A comparison needs an explicit baseline before perturbations are selected.

The varied axis determines the comparison

Population changes can also change warm-up and eligible labels. Their resulting scores require that changed denominator. A representation change can preserve rows while changing every model input. The perturbation reference owns the exact primitives and parameter syntax.

Python retains the fitted representation

The Python preparation case divides 1,440 recorded hourly observations into raw partitions of 864, 288, and 288 rows. It builds ROC and fractional features independently within each partition, applies Triple Barrier labels, and fits robust scaling on the training features.

The baseline retains four features. The compressed case removes roc_12 and fits two PCA components. Both cases retain transformed partitions, fitted scaler statistics, PCA state where used, predictions, and regression error:

1def preparation(data_dir, output):2    from sklearn.decomposition import PCA3    from sklearn.linear_model import Ridge4    from limen.features import fractional_diff, find_min_d5    from limen.indicators import roc6    from limen.scalers import RobustScaler7    from limen.targets import TripleBarrierTarget8    from limen.utils import adf_test9 10    raw = hourly(data_dir, 1440)11    partitions = [raw[:864], raw[864:1152], raw[1152:1440]]12    d = find_min_d(partitions[0], 'close', d_end=.4, step=.2, threshold=.05)13    stationarity = adf_test(fractional_diff(partitions[0], d=d, cols=['close'], threshold=.05)['close_fracdiff'])14    cases = []15    for compressed in (False, True):16        name = 'compressed' if compressed else 'baseline'17        directory = output / name18        directory.mkdir(parents=True)19        target = TripleBarrierTarget(partitions[0], 'label', upper_multiple=1.5,20                                     lower_multiple=1., max_horizon=3, span=8, min_periods=8)21        columns = ['close', 'roc_3', 'roc_12', 'close_fracdiff']22        dropped = ['roc_12'] if compressed else []23        features = [column for column in columns if column not in dropped]24        prepared = []25        for partition in partitions:26            frame = fractional_diff(roc(roc(partition, period=3), period=12),27                                    d=.4, cols=['close'], threshold=.05)28            prepared.append(target.transform(frame).select('datetime', *features, 'label').drop_nulls())29        scaler = RobustScaler(prepared[0].select(features))30        scaled = [scaler.transform(frame.select(features)) for frame in prepared]31        pca = PCA(n_components=2, svd_solver='full') if compressed else None32        matrices = ([pca.fit_transform(scaled[0].to_numpy()),33                     *[pca.transform(frame.to_numpy()) for frame in scaled[1:]]]34                    if pca is not None else [frame.to_numpy() for frame in scaled])35        names = ['pc_0', 'pc_1'] if compressed else features36        model = Ridge(alpha=1., solver='svd').fit(matrices[0], prepared[0]['label'].to_numpy())37        predictions = model.predict(matrices[2])38        for split, frame, matrix in zip(('train', 'val', 'test'), prepared, matrices, strict=True):39            pl.DataFrame(matrix, schema=names, orient='row').with_columns(40                frame['datetime'], frame['label']).write_parquet(directory / (split + '.parquet'))41        cases.append({'name': name, 'features': features, 'dropped': dropped,42                      'rows': [frame.height for frame in prepared],43                      'medians': scaler.medians, 'iqrs': scaler.iqrs,44                      'pca_mean': None if pca is None else pca.mean_.tolist(),45                      'components': None if pca is None else pca.components_.tolist(),46                      'predictions': predictions.tolist(),47                      'rmse': float(np.sqrt(np.mean((predictions-prepared[2]['label'].to_numpy())**2)))})48    return {'cases': cases, 'partition_rows': [864, 288, 288], 'carried_context': False,49            'fixed_d': .4, 'weight_threshold': .05, 'adf_grid_result': d,50            'adf_stationary': bool(stationarity.stationary), 'adf_p_value': float(stationarity.p_value)}

This comparison changes feature ablation and PCA together. Its result measures that combined change. A separate comparison of each axis requires candidates that vary one choice at a time while preserving the others.

The training-only stationarity diagnostic tests a bounded fractional-order grid. Feature construction explicitly uses order 0.4; the diagnostic's selected order is recorded separately. Its result describes the supplied training population and statistical test.

Compiled preparation declares the candidate axes

The composable YAML experiment crosses two regularization values, two ablation counts, and two PCA states. Group selection and the ablation seed remain fixed:

1  params:2    C: [0.5, 1.0]3    random_state: [42]4    feature_groups: [momentum|price]5    feature_drop_count: [0, 1]6    feature_drop_seed: [7]7    auto_pca: [false, true]8    pca_k: [2]9    use_calibration: [true]10    use_threshold: [true]

The eight combinations include the uncompressed and compressed alternatives at each regularization value. Recorded round parameters identify which representation produced each score. Training fits the target, scaler, and PCA state before their application to later splits.

1limen run --no-progress-bar compiled-research/preparation.yaml2limen run --no-progress-bar compiled-research/research.yaml3limen assess compiled-research/research.yaml --results results/dev/compiled-research

The complete result directory retains candidate-level parameters and preparation witnesses. The two selected members reflect the declared validation ranking. Their selection does not isolate the causal contribution of each preparation choice.

Fitted scalers and per-frame transforms have different state

The five registered scaler types provide linear column policies, logistic-workflow policies, robust training statistics, rank-Gaussian mapping, and causal rolling robust scaling. Their reference defines training state and context requirements. Causal rolling scaling also requires sufficient preceding history at inference.

Stateless Z-score, MAD, winsorization, trimming, and shifting functions operate on the supplied frame. Their per-call statistics do not provide a training-fit/apply-later contract. The transform reference states those boundaries and the consequences of removing rows.

Target perturbations preserve outcome provenance

Candidate target jitter uses deterministic candidate identity and its declared seed. Label smoothing applies to training labels. Observed evaluation outcomes retain their own meaning and availability. A target perturbation therefore belongs in the experiment definition and reconstruction evidence.

The label and feature choices used to select a configuration consume selection evidence. Stability across declared perturbations is evidence about those comparisons. Assessment on successive windows and reserved final observations answers additional questions about later populations.

Continue to search to choose how the declared alternatives are traversed. Research evidence develops correlation, importance, and family reports after completed candidates are available.