Skip to main content

Projection Rebuild Reference

Metadata​

  • Owner: Origo Engineering
  • Last updated: 2026-04-15
  • Slice/version reference: S52 + S56

Purpose and scope​

  • Runtime reference for the sanctioned projection rebuild path.
  • Scope is the operator-visible Dagster rebuild entrypoint, reset/cleanup boundary, and rebuild proof artifact contract for native and aligned serving surfaces.

Sanctioned operator entrypoint​

  • Dagster job: origo_projection_rebuild_job
  • Input contract:
    • rebuild_unit_id
    • partition_ids_csv
    • optional batch_size
  • Supported rebuild units match the currently exposed historical datasets in docs/data-taxonomy.md.

Reset and replay boundary​

  • The sanctioned rebuild path owns reset of:
    • native serving rows
    • aligned aggregate rows
    • projector checkpoints
    • projector watermarks
  • Rebuild replay consumes current-truth canonical events from canonical_event_log_active_v1.
  • Raw append-only history in canonical_event_log_history_v1 remains forensic/event-sourcing truth, not the operator rebuild input for this slice.

Completion boundary​

  • rebuild complete means all required surfaces are complete in the declared priority order:
    • canonical_event_log_active_v1
    • native serving tables
    • aligned_1s serving tables
    • required proof and promotion surfaces
  • canonical-only replay does not qualify as rebuild complete.

Deterministic equality​

  • Rebuild equivalence is defined on:
    • projector identity
    • governed aligned view identity
    • covered partition set
    • serving-visible content
  • Excluded volatile fields:
    • projected_at_utc
    • checkpointed_at_utc
    • checkpoint run metadata
  • Served timestamps that already participate in serving semantics remain part of rebuild equality.

Proof artifacts​

  • Every sanctioned rebuild writes a proof artifact with:
    • normalized content hashes
    • typed row-level diff explainers
  • Artifact root:
    • ORIGO_PROJECTION_REBUILD_ARTIFACT_ROOT_DIR when set
    • otherwise /tmp/origo_projection_rebuild
  • Diff vocabulary:
    • bucket_mismatch
    • event_within_bucket_mismatch
    • metric_observation_mismatch
    • validity_interval_mismatch
    • coverage_mismatch
  • Proof artifacts are Dagster-visible operator outputs; the exact long-term persistence backend remains implementation-owned.
  • Proof hashing is multiset-stable and row-order-independent; mismatch samples are bounded and are not allowed to rely on SELECT * row order.

Compatibility boundaries​

  • Historically valid pre-S39 rows with null envelope metadata must remain rebuild-compatible.
  • Bypass-writer canonical rows must remain rebuild-compatible with writer-path rows.