Canonical source-pressure law is enforced separately from downstream store/client limits:
central registry: contracts/source-pressure-v1.json
registry scope is declared source-pressure subjects only
current live subjects are all pull
fixed controls cover source timeouts, source-safe concurrency, retry policy, worker-pool size, request window size, and source-safe pacing where declared
operator-visible distress surfaces are Dagster run failure, Dagster sensor/alert, runtime-audit evidence, and proof/quarantine state
push-source admission stays blocked until a durable platform-owned accept-then-drain buffer contract exists
retained raw artifacts are not sufficient to qualify as a push buffer
Pre-S39 historical rows can surface null metadata values until replay/rewrite completes; this is intentional so old thin-envelope rows do not pretend to carry valid provenance.
Canonical domain_event_type is a source/domain taxonomy. Runtime-audit event_type remains a separate system-operation taxonomy.
payload_raw and payload_sha256_raw remain the byte-level source provenance anchor.
Canonical storage ordering is per stream key (source_id, stream_id, partition_id) via stream_sequence, assigned at the store boundary.
Authoritative append concurrency is ClickHouse-only and optimistic. Origo does not use a second stream-head store or an external lease service.
The authoritative head components are:
canonical_stream_sequence_heads
canonical_event_log_active_v1
canonical_partition_reset_boundaries
Every authoritative append must target exactly one canonical stream key.
Same-stream multi-row append batches are allowed.
Mixed-stream authoritative append batches are rejected.
Cross-stream coordination is choreography only.
Cross-stream choreography uses separate stream-local canonical appends linked by explicit correlation_id; it does not use a shared commit boundary.
run_id remains execution identity and must not be treated as choreography identity.
The writer/input seam supports explicit correlation_id so choreography identity does not need to piggyback on run_id.
Compensation is represented as an ordinary stream-local canonical event that points to exactly one prior parent event via causation_id, with wider grouping on correlation_id.
Expected-head semantics are lineage-aware and include the current stream key, latest reset-boundary lineage, and current-truth last_stream_sequence.
Unreplayed pre-S42 rows may still keep null stream_sequence until rewrite cutover completes.
Any incremental write or idempotent dedup path that would match against those pre-S42 null-sequence rows must fail closed and requires replay/rewrite first; the system must not pretend those rows are safe incremental dedup targets.
Gap detection and continuity truth come from the raw append-only log, not from the current-truth active view.
Logical reset boundaries explicitly rebase current-truth head lineage for append concurrency.
Projector fetch/resume order and checkpoint/watermark progression must therefore follow stream_sequence.
Default current-truth reads that need canonical event-log truth must route through canonical_event_log_active_v1.
Raw append-only event-log history is forensic/proof-only and must route explicitly through canonical_event_log_history_v1.
Freshness semantics remain source-driven and warning-aware per endpoint docs, with ETF/FRED auxiliary freshness rules explicitly separate from canonical event time.
Projection rebuild and serving promotion are gated on terminal proof coverage during Slice 34 backfill.
Historical availability claims therefore mean terminally proved history, not merely source history that exists upstream.
Canonical append is the only business-truth write boundary after acquisition/normalization.
Native and aligned tables are read models only.
Canonical truth and first-class serving truth stay forever hot in ClickHouse:
canonical_event_log
canonical_event_log_active_v1
native historical serving tables
aligned_1s serving tables
Hot/warm/cold tiering is not part of the current Origo design.
Retained source files are rebuild substrate only where a governed substrate exists; they are not a consumer query tier.
Supported projection modes are:
projector: dedicated post-append projection path
deferred: append and proof now, projection or rebuild later
inline is not a supported runtime mode.
Authoritative partition completion is separate from proof and separate from raw/native/aligned row existence:
only completion_state=complete may mean partition green/completed
terminal proof remains supporting evidence, not final success by itself
ordinary rerun refusal is allowed only from authoritative completion complete
User-visible native and aligned historical serving must remain dark until the same completion authority says complete.
Native/aligned rows written before completion promotion are provisional residue, not served truth.
Operational/proof control stores such as partition proofs, quarantines, checkpoints, and reset boundaries are not CQRS debt.
Any temporary non-serving legacy residue must be statically excluded from supported runtime surfaces and emit runtime-audit visibility on invocation.
The only sanctioned operator rebuild path for canonical projections is Dagster job origo_projection_rebuild_job.
The sanctioned rebuild path:
resets native rows, aligned aggregate rows, projector checkpoints, and projector watermarks together under one authority
replays current-truth canonical events from canonical_event_log_active_v1
treats append-only raw history as forensic/event-sourcing truth rather than the operator rebuild input for this slice
rebuild complete requires the full end-to-end hot truth chain:
canonical_event_log_active_v1
native historical serving tables
aligned_1s serving tables
required proof and promotion surfaces
canonical-only replay is not enough to claim rebuild completion.
Rebuild proof must include:
normalized content hashes
typed row-level diff explainers
Typed rebuild diff vocabulary:
bucket_mismatch
event_within_bucket_mismatch
metric_observation_mismatch
validity_interval_mismatch
coverage_mismatch
Volatile rebuild bookkeeping such as projected_at_utc, checkpointed_at_utc, and checkpoint run metadata is excluded from deterministic equality.
Served timestamps that already participate in serving semantics remain inside rebuild equality.
Historically valid pre-S39 null envelope metadata rows and bypass-writer canonical rows must remain rebuild-compatible; sanctioned rebuild must not pretend they disappeared.
Authoritative-surface disagreement is itself a hard failure:
Dagster green with missing or non-complete authoritative completion
Dagster red with authoritative complete
proof terminal while authoritative completion is not complete
Completion transitions and authoritative-surface disagreement must surface through runtime-audit; they may not remain manual forensic interpretation only.
Missing rights metadata in export tags is fail-loud (EXPORT_STATUS_METADATA_ERROR path).
Missing rights metadata in query response contract is fail-loud (response validation error).
Unsupported/missing view metadata combinations are rejected by request contract validation.