How to wire point-in-time reference data (M4)¶

When you need this: A reference table (e.g. customer) carries a history of attribute changes (risk ratings, KYC tier, segment). You want an aggregation_window rule to evaluate each transaction against the reference value in force at that transaction's booking date — not the latest row. Without this, a customer whose risk rating went low → high mid-window is scored on the latest value for all their transactions, which overstates the historical alert count.

Prereqs: A reference contract with valid_from / valid_to timestamp columns. aml validate succeeds on the base spec.

Time: ~15 min to declare and verify. Closes the Pillar-3 SCD-2 gap.

The M4 point-in-time mechanism works in two steps: (1) declare the reference DataContract as effective_dated; (2) add an enrich block to the aggregation_window rule. The SQL generator joins the reference contract against the pre-filtered source subquery (aliased src) on the key AND the validity window:

-- auto-generated by the SQL generator (joined against the
-- window/filter-narrowed source subquery, aliased `src`):
JOIN customer
  ON src.customer_id = customer.customer_id
  AND customer.valid_from <= src.booked_at
  AND (customer.valid_to IS NULL OR src.booked_at < customer.valid_to)
  AND (customer.risk_rating = 'high')   -- each enrich.where predicate, parenthesized

The reference is joined by its own contract name (customer), not aliased ref; the source side is the already-narrowed subquery alias src. So a transaction booked at 2026-03-01 joins the customer row whose [valid_from, valid_to) interval covers 2026-03-01 — not the latest row.

Steps¶

1 · Declare the reference contract as `effective_dated`¶

data_contracts:
  - id: customer
    source: raw.customers_history        # SCD-2 table with valid_from / valid_to
    effective_dated:
      valid_from: valid_from             # column name in the source table
      valid_to:   valid_to               # nullable — NULL means "current"
    columns:
      - { name: customer_id,   type: string,    nullable: false }
      - { name: risk_rating,   type: string }
      - { name: valid_from,    type: timestamp, nullable: false }
      - { name: valid_to,      type: timestamp, nullable: true  }

valid_to: null on a row means "this row is current and has no end date." The engine treats NULL valid_to as open-ended — booked_at < valid_to becomes valid_to IS NULL OR booked_at < valid_to.

2 · Add the `enrich` block to the rule¶

rules:
  - id: high_risk_burst
    name: High-risk customer burst
    severity: high
    regulation_refs:
      - citation: FinCEN SAR Advisory 2020-A001
        description: SAR-filing obligations for high-risk customers
    logic:
      type: aggregation_window
      source: txn
      group_by: [customer_id]
      window: 30d
      having:
        count: { gte: 2 }
      enrich:
        contract: customer                       # must be declared effective_dated above
        key: customer_id                         # join key — use `key`, NOT `on` (YAML 1.1 coerces `on:` to boolean true)
        where:
          - "customer.risk_rating = 'high'"      # raw SQL predicates on the joined contract columns
    escalate_to: aml_queue
    evidence:
      - alert_amount_breakdown

Important: use enrich.key, not enrich.on. YAML 1.1 coerces a bare on: key to boolean true, which the spec parser rejects.

3 · Validate¶

aml validate my_aml.yaml

Expected: ✓ Spec is valid. If the enrich points at a contract that lacks an effective_dated block, validation fails with rule 'high_risk_burst' enriches 'customer' which is not effective_dated (declare valid_from/valid_to on it) — add the effective_dated block to the data contract.

4 · Run against effective-dated reference rows and verify the count¶

The as-of join only does anything when the reference contract actually carries valid_from / valid_to history. The built-in synthetic generator does not emit effective-dated customer rows (each synthetic customer has a single risk_rating, no validity window), so to prove point-in-time behaviour you supply an SCD-2 reference table yourself.

The canonical worked example is tests/test_point_in_time.py::test_point_in_time_join_resolves_contemporaneous_row: it builds a small in-memory customer table with two rows for C0001 (risk_rating: low valid until 2026-06-05, then risk_rating: high open-ended) plus four transactions — two booked while low, two while high. The rule fires only on the 2 high-period transactions, not all four:

assert fired["C0001"]["count"] == 2   # high-period txns only, NOT 4 (latest-row)

That count == 2 (instead of 4) is the proof the as-of join resolved the contemporaneous row rather than the latest one. To reproduce outside the test, point your spec's customer contract at an SCD-2 source (CSV / Parquet / DuckDB) whose rows carry valid_from / valid_to, e.g.:

customer_id,risk_rating,valid_from,valid_to
C0001,low,2026-01-01 00:00:00,2026-06-05 00:00:00
C0001,high,2026-06-05 00:00:00,

then aml run my_aml.yaml --data-source csv --data-dir <dir> and confirm the alert's count reflects only the in-window high-risk transactions.

5 · Check the North Star Coverage page¶

Navigate to North Star Coverage (page 43) in the dashboard. Pillar 3 (Point-in-time correctness) reads COVERED and its evidence panel describes the M4 SCD-2 effective_dated + enrich mechanism you just used. (This is a static framework-capability assessment — the status is not recomputed per spec, so it does not change based on whether your spec declares an enrich block.)

Verify it worked¶

Two checks, in order of strength.

(a) Inspect the generated SQL — this is the most direct proof the as-of join was wired:

aml run my_aml.yaml --data-source csv --data-dir <dir>
grep -A3 "JOIN customer" .artifacts/run-*/rules/high_risk_burst.sql

You should see the join against the source alias src on customer.valid_from <= src.booked_at and the valid_to window — that clause is emitted only when the enrich block resolved.

(b) Check the alert count — the as-of join changes which rows aggregate, so the alert's count is the as-of count, not the latest-row count:

jq '.count' .artifacts/run-*/alerts/high_risk_burst.jsonl

With the SCD-2 fixture above, count is 2 (high-period transactions only); a latest-row join would report 4. The difference is the behaviour you're verifying.

Note: do not use the alert's matched_row_ids as proof of the as-of join. Those row ids are looked up from the source transaction table alone (engine/runner.py re-queries SELECT rowid FROM <source> WHERE customer_id = ? AND booked_at BETWEEN window_start AND window_end) — they reflect the window, not the reference join, so they can be non-empty even if the enrichment did nothing. Likewise, the Lineage Explorer (page 32) renders a generic source → contract → table → rule → alert → case Mermaid graph; it does not show per-reference as-of intervals.

Common problems¶

Symptom	Cause	Fix
`rule '<id>' enriches '<contract>' which is not effective_dated (declare valid_from/valid_to on it)`	Missing `effective_dated` block on the contract	Add `effective_dated: { valid_from: <col>, valid_to: <col> }` to the data_contract
`rule '<id>' enrich key '<col>' is not a column of '<contract>'` (or `... of source '<src>'`)	`enrich.key` names a column absent from the ref or source contract	Use a join key declared on both the source and reference contracts
`KeyError: 'on'` or `enrich.on: true` in YAML	Used `on:` instead of `key:`	Change to `enrich.key: <column_name>` — YAML 1.1 coerces bare `on:` to boolean `true`
Alert count inflated (same as without enrich)	`where` predicate not filtering correctly	Check the predicate uses the contract name: `"customer.risk_rating = 'high'"`, not `"risk_rating = 'high'"`
Transactions silently dropped from the count	SCD-2 table has gaps — some booking dates have no covering reference row	The as-of join is an INNER join: a transaction with no `[valid_from, valid_to)` row matching its `booked_at` is excluded (no error is raised). Ensure the SCD-2 table has gap-free coverage, with `valid_to: null` on the current row

Next steps¶

The as-of join also works with non-customer reference tables (e.g. product codes, country lists). Any effective_dated contract can be enriched into any aggregation_window rule — enrich is defined only on aggregation_window logic, not on custom_sql, python_ref, list_match, or network_pattern rules.
See add-a-rule.md for the full rule authoring flow.
See walk-lineage.md to trace an alert back to its source rows (note: lineage shows the source→rule→alert→case chain, not the reference as-of interval).