# BFM apps2 — v2 Architecture

> The decoupling / cross-asset crystal-ball engine. Drafted 2026-06-25.
> Supersedes the ~40-app v1 sprawl (`apps/`). Read this before building anything here.

---

## Mission

Surface the **hottest alpha** by detecting **relationships breaking** — not levels moving.
The edge is in **extremes, decouplings, and decorrelations** across assets and across
timescales (micro → macro), and in projecting the **chain reactions** one unusual event
sets off. v1 computed levels and threw away time; v2 records everything, computes the
leading-edge transforms centrally, and ranks *what's turning right now*.

The north star example (the gold thesis): gold rose while real yields rose — the normal
gold↔real-yield link **broke**, revealing a price-insensitive official-sector bid. The
**volume** on that decoupled move scaled the coming USD suppression, which projected the
NAS100 repricing. v2 must catch that *as it begins*, at the smallest timescale it first
appears, and project the snowball with honest, widening uncertainty.

---

## 0. What changed from v1 (the whole point)

v1 = ~40 apps that each re-fetch the same chain/bars, compute one **level**, render a
**snapshot**, and forget it. No history → no velocity, no z-score, no divergence. The
fusion layer recorded a goldmine but only ever showed the latest snapshot; the real
early-warning engines (scanner/harbinger/forecast) were siloed from the ensemble.

v2 inverts it: **fetch once → record every signal → compute leading-edge transforms in
one shared library → fuse → rank heat.** Fewer screens, strictly more alpha, because the
leading edge is structural.

---

## 1. Design laws (the constitution)

1. **Record-first. No ephemeral path.** If a number is computed, it's written the same call. Un-recorded data is lost alpha — v1's #1 failure.
2. **Compute the chain/bars once per tick.** One snapshot per symbol feeds *all* signals → cross-signal consistency (same instant) + ~10× less I/O.
3. **Decoupling-first.** The primary object of study is the **relationship**, not the level. Every asset and pair is monitored for EXTREME / DECOUPLE / DECORRELATE at multiple timescales.
4. **Everything is a voter.** Every predictor emits `{dir, conf, target}` × 9 horizons. No siloed detectors; the chain projection and the radar are voters too.
5. **The leading edge is a transform, not an app.** velocity / accel / z-score / percentile / residual / corr-delta / regime-flip-distance is ONE shared library, parameterized by window.
6. **Multi-scale by default.** Every transform runs across a window-set (micro→macro). A signal climbing the timescales (micro decoupling → macro regime break) is the earliest tell.
7. **Default screen ranks heat, not a catalog.** You open to "what's turning right now," not 40 cards.
8. **Hazy by design.** Outputs are probabilistic with sample sizes and widening cones. Honest uncertainty beats false precision.
9. **The UI flashes state-changes and is fully tunable.** Breaks (decorrelation incl. inverse, decoupling, unusual trend, accel/decel) actively *flash the moment they fire* — not just sit in a heatmap. And every detector parameter (date range, candle/timeframe, algo-curve windows/thresholds) is live-adjustable so the user can tune the radar to match their own eyeball read of a turn (§9b). *(Amendment 2026-07-04, R-UI-23: the DETECTION stays, but EXTREME/DECOUPLE/ACCEL/DECEL word-labels are banned from the UI as eyeballable noise — surface only computed non-obvious reads, e.g. RSI divergences. See §1.10.)*
10. **Tape + line are a mandatory PAIR (never ship one alone).** Any view of price action relative to SPX/NAS100 (or any many-series comparison) MUST present BOTH forms, side by side, on the same data: the **stretch tape** (heatmap, rows×time, each row vs its own history on one diverging ramp) is *visually clear* — glanceable correlation/anticorrelation across the whole set and sortable by date column; the **normalized line chart** (all tickers, z-score default + fit-to-axis mode, SPX/NAS baselines, single-line click tooltip) is *precise but chaotic* — exact trajectories. Neither alone is sufficient: the tape shows the shape of the relationship at a glance, the line chart lets you read the exact number. Both are necessary to give the user multiple complementary perspectives on the same price action. Presence of either REQUIRES the other. (2026-07-04; refs R-UI-22 / R-UI-25; ref impl `lenses`/cohorts.)

---

## 2. The pipeline (strict one-direction flow)

```
 INGEST           STORE             DETECT / SIGNAL          FUSE             SURFACE
 ┌────────┐  ┌──────────────┐  ┌────────────────────┐  ┌──────────┐  ┌──────────────────┐
 │snapshot│─▶│ series store │─▶│ radar (decoupling) │─▶│ ensemble │─▶│ heat board +     │
 │ once   │  │ append-only  │  │ + voters emit      │  │ + calib  │  │ radar/chain/...  │
 │ /tick  │  │ one schema   │  │   dir/conf/target  │  │ + chain  │  │ lenses (render-  │
 └────────┘  └──────────────┘  └────────────────────┘  │ projection│  │  only, 0 compute)│
  chain+bars   every metric,     transforms READ the     └──────────┘  └──────────────────┘
  +fut+fx+      every tick,        store (not re-fetch);   P(up), flip,
  yields+perp   one flat schema    radar = the funnel      dispersion,
                                   front                   snowball cones
```

Key change vs v1: **signals read the store, not the API** — so every signal automatically
has history → velocity/z/residual/corr-delta are free.

---

## 3. Data sources (NO new paid subscriptions; add FRED, free)

| Need | Source | Status |
|---|---|---|
| Futures price **+ real volume** (GC, CL, NQ, ZN/ZB, DX) | **Databento GLBX** (daily/hourly batch; cPanel can't run live ws, and we don't need it — chain is multi-day) | ✅ have key; budget a small one-time daily backfill |
| Options chains+greeks+OI+IV (the options voters) | **Massive/Polygon** (`_common/massive.php`) | ✅ have key |
| Indices (^NDX, ^HSI), FX (DXY, USDCNH, EURUSD), continuous-future prices | **Yahoo** (free) | ✅ |
| Near-real-time US equity bars | **Alpaca** | ✅ have creds |
| **Real yields / breakevens** (`DFII10`, `T10YIE`, `DGS10`) — the gold-decoupling anchor | **FRED** (free API key, daily, back to ~2003) | ❌ **ADD — free, do first** |
| Crypto perp funding / OI / basis | **Hyperliquid** (free, unauthenticated) | ✅ free; currently unused |

Volume must come from the **futures** (Databento), never ETF proxies (GC volume ≠ GLD).
History depth: Yahoo prices + FRED yields go back decades; GLBX volume ~2010+ (covers the
key analogs: 2011 gold, 2015 yuan deval, 2018–19 trade war, 2020, 2022, 2024–25).

---

## 4. Schemas (three canonical stores)

**1. Snapshot (raw, per tick)** — one pull, stored verbatim so any signal is re-derivable/backtestable:
```
data/snapshots/<sym>/YYYY-MM-DD.jsonl
  {ts, spot, volume, atm_iv?, chain?:[{k,exp,type,iv,delta,gamma,vega,oi,vol,bid,ask}], bars:{1m,1h,1d}}
```

**2. Series (the product)** — every signal value, one flat schema; the thing every leading view reads:
```
data/series/<sym>/<signal>.jsonl
  {ts, h:"1h", dir:0.6, conf:0.55, value:1.1e9, target:750}
```

**3. Radar (the decoupling tensor, per tick)** — extreme/decouple/decorrelate × scale:
```
data/radar/<sym>.jsonl
  {ts, scale:"meso", extreme_z:2.4, residual_z:3.1, decorr:[{pair:"gold:real_yield", corr:0.05, corr_norm:-0.62, break:0.67}], heat:0.88}
```

**4. Fusion (the recorded ensemble, per tick)** — so the fusion itself has history:
```
data/fusion/<sym>.jsonl
  {ts, spot, horizons:[{h,p_up,dir,target,lo,hi,conf,contrib:{...}}], family:{hedging,vol,flow,chain}}
```

---

## 5. The transform library (`core/transform.php`) — multi-scale leading edge

Every transform takes a recorded series + a **window** and returns a comparable value.
The radar and fusion call these; no app re-implements them.

| Transform | What it surfaces |
|---|---|
| `level` | the raw value |
| `velocity` / `accel` | 1st / 2nd derivative — momentum & its change (the "pulse", now free for every signal) |
| `zscore(window)` / `percentile` | how extreme vs its own history |
| `residual(basket, window)` | actual − expected-from-drivers (rolling multi-regression) = **decoupling** |
| `corr(b, window)` / `corr_delta` | pairwise correlation and its **change** = **decorrelation** |
| `regime_flip_distance` | signed distance to a sign-flip line (zero-gamma, delta-flip, max-pain, corr=0) |
| `divergence(price)` | signal vs price disagreement |

**Window-set (the micro→macro spine):**
`micro = {1h, 4h}` · `meso = {1d, 5d, 20d}` · `macro = {60d, 120d, 252d}`.
Every transform runs across the set so the same break can be flagged at multiple scales.

---

## 6. The Decoupling Radar (`core/radar.php` + `lenses/radar`) — the universal anomaly front

Generalizes the gold insight into a radar over **every asset and every pair**, monitoring
three break-types at three scales. This is the funnel that feeds the chain engine and the
heat board.

**The three break-types**
- **EXTREME** — one series at an unusual level/velocity vs its own history (`zscore`, `velocity`, `accel`). *"How unusual is this asset right now?"*
- **DECOUPLE** — one asset moving against what its drivers predict (`residual` from the rolling cross-asset regression). *"It's doing something its drivers can't explain."* (gold↑ while real-yield↑)
- **DECORRELATE** — a pair's rolling correlation breaking from its norm (`corr_delta`, corr regime flip). *"Two things that always move together just stopped."* (gold/real-yield −0.7 → 0)

**The radar tensor:** `[asset | pair] × [extreme | decouple | decorrelate] × [micro | meso | macro]`.
Any hot cell = a candidate trigger. `heat = |residual_z| · |corr_break| · velocity_z` (per cell).

**Cross-timescale climbing (the earliest tell):** the radar shows the *same* relationship
across micro/meso/macro side by side. A break that appears at **micro first and propagates
up** to meso then macro is the earliest possible warning that a structural regime change is
forming — catch it on the 1h scale before it's a 20d trend. The radar flags "climbing"
cells explicitly.

**Radar lens UI:** a tensor heatmap (rows = assets/pairs, columns = scale, cell = break-type
heat, with a small EXTREME/DECOUPLE/DECORR glyph), a "climbing now" rail (breaks ascending
the timescales), and click → the asset's multi-scale residual + corr-delta time series.

---

## 7. The Cross-Asset Chain crystal ball (`core/{graph,propagate,analog}.php` + `data/graph.json` + `lenses/chain`)

When the radar fires a trigger, the chain engine answers *"what snowball does this start, how
big, and when?"* Four parts:

1. **Causal graph** (`data/graph.json`) — editable map of cross-asset edges `{from, to, sign, lag, mechanism, trigger_condition, beta, calibrate}`. Domain knowledge as data. The gold→USD→NAS100 chain is encoded there.
2. **Anomaly/Residual** = the radar (§6) — supplies the trigger (which asset, direction, magnitude, which relationship broke).
3. **Propagation** (`propagate.php`) — walk the graph forward from the trigger: projected downstream move = `trigger_magnitude × calibrated_edge_beta`, lagged by `edge.lag`, with **confidence decaying per hop** (the haze). Only edges whose recent calibrated r² is strong are "armed"; weak edges grey out.
4. **Analog** (`analog.php`) — the honest crystal ball. Find historical days where this *same* anomaly+decoupling occurred (same asset, same broken relationship, similar magnitude); aggregate the realized forward paths of every downstream asset → an **empirical forward cone** (median + quantiles + sample size + hit-rate). *"Last 8 times gold spiked on volume while decoupling from real yields: DXY −3% med /3wk, NDX +6% med /6wk, n=8, 6/8."*

**Chain lens UI (the crystal ball screen):** (1) normalized z-scored overlay of all chain
assets on one axis so 0.3σ divergences show, trigger marked, projected cones drawn; (2) the
rolling **correlation-delta matrix** (earliest tell); (3) the **residual heatmap** (trigger
detector); (4) the **snowball panel** — trigger node → downstream nodes with projected
magnitude/lag (propagate) + analog cone overlaid, confidence widening per hop.

**It's also a voter:** `signals/chain/chain_projection.php` emits the downstream call (e.g.
NAS100 `{dir:+0.7, conf, target}`) into the fusion engine — so the macro snowball shows up on
the heat board and votes in Oracle alongside the options engine, weighted by its calibrated
hit-rate.

---

## 8. Signal families (replaces ~40 apps with ~20 small files)

Each signal = one file: read snapshot → compute → emit `{dir,conf,target}×9` + record its series.

- **`signals/hedging/`** — gex, dex, gamma_term, vanna_charm, oi_delta (per-strike ΔOI), futures_basis, perp_funding (Hyperliquid). *Dealer/bank forced flows.*
- **`signals/vol/`** — iv_surface, skew, vrp, term_structure, zerodte. *Fear pricing / protection demand.*
- **`signals/flow/`** — premium_cohort (retail vs inst), breadth, cross_asset, ratios, momentum, price_structure (squeeze/divergence). *Where capital actually is.*
- **`signals/chain/`** — cross_asset_resid (per-asset residual + decoupling as a voter), chain_projection (the snowball voter). *The macro crystal ball.*

`greeks-pulse` disappears as an app — "pulse" = `transform.velocity()` on every signal, for free.

---

## 9. Lenses (~6 thin frontends, render the store, ZERO compute)

| Lens | Purpose |
|---|---|
| **heat** | THE default screen. Every signal×symbol ranked by `heat = \|velocity_z\| · \|level_z\| · cross_signal_agreement · calibration_weight`. Tags: FLIP / ACCEL / DIVERGE / EXTREME / CLIMBING. |
| **radar** | the decoupling tensor (§6) — extreme/decouple/decorrelate × micro/meso/macro, climbing rail. |
| **chain** | the cross-asset crystal ball (§7) — overlay + corr-delta matrix + residual heatmap + snowball cones. |
| **fuse** | Oracle+Mosaic merged: P(up) ribbon over time, voter-flip timeline, dispersion oscillator, cross-horizon P(up) inversion, family-flip lead. |
| **evolve** | any signal over time: level ↔ velocity ↔ z-score ↔ divergence-vs-price toggle. |
| **surface** | any signal as strike×expiry×time surface with Δ/z toggle. |
| **scan** | multi-symbol board ranked by heat / flips / family-flip-lead. |

Shared `shared/chart.js` = one canvas engine carrying the full R-UI-20 contract (Y-scale
slider+gutter, pan, wheel-zoom, click→dateline x-synced, fullscreen, save-PNG) so no lens
re-implements charts.

---

## 9b. Frontend: live alerting (flash) + the eyeball simulator (`shared/alert.js`, `shared/controls.js`)

The lenses are not passive heatmaps. They actively **flash the moment a break fires** so the
eye catches the *beginning* of a turn, and they expose every detector knob so the user can
tune the radar to match their own eyeball read.

### A. Live alerting — flash on state-change (`shared/alert.js`)
A break **flashes (a transient pulse animation) at the instant it crosses its threshold**, then
settles into a static highlight while the condition persists (with a "since HH:MM" stamp), so a
persistent condition never drowns out a *new* one. Distinct visual language per break-type,
intensity scaled by `heat`:

| Break | Flash treatment |
|---|---|
| **DECORRELATION** | the pair's link pulses; corr-delta cell rings |
| **INVERSE correlation** (corr crosses 0 / flips sign) | stronger alarm — red ring + `INVERSE` tag (the highest-value tell: things that moved together now oppose) |
| **DECOUPLE** (residual spike) | asset cell flashes amber, `⊗` glyph |
| **UNUSUAL TREND / EXTREME** (z-extreme, new regime) | flash + `EXTREME` tag |
| **ACCELERATION** | up-chevron, brightening pulse |
| **DECELERATION** | down-chevron, dimming pulse |
| **CLIMBING** (break ascending micro→meso→macro) | a rising arrow across the scale columns — the earliest warning |

Plus a top **alert ticker / event log** rail: a chronological "what just fired" feed
("10:32 gold:real_yield DECORRELATED → INVERSE · meso · heat 0.88"), click → jumps to it, so
nothing is missed off-screen. Severity = heat; flash intensity scales with it. Palette follows
the house order (R/B/Y/G before O/P); honors a global **calm mode** / reduced-motion toggle and
an optional subtle audio/badge cue (off by default — minimal aesthetic).

### B. The eyeball simulator — live fine-tune controls (`shared/controls.js`)
A persisted control rail on every analytical lens that **re-drives the charts instantly from the
store** (no re-fetch — transforms are parameterized and recompute on read; this is *why* the
record-first architecture makes tuning cheap):

- **Date range** — start/end + quick windows; drives the whole view.
- **Candle / timeframe range** — resolution (1m/5m/15m/1h/4h/1d/1w) + candle width (per the house candle rules: shrink-to-fit then scroll).
- **Algo-curve params (every transform's knobs, as live sliders):**
  - z-score / percentile **lookback window**
  - correlation **window** + the "normal corr" **baseline window**
  - residual-regression **basket membership** (toggle which drivers each asset is regressed against)
  - **smoothing** (EMA span) on overlays
  - **break thresholds** (z, corr-break, accel) — these directly control *what flashes*
  - overlay toggles: moving averages, regression channel, bands, parabolic/curvature
- **Tuning presets** — save/load named parameter sets ("my swing eyeball", "fast intraday").

The goal: the user tunes window/threshold/basket until the flashing matches their own read of
"a trend is changing" — effectively **calibrating the detector to their intuition.** Because the
radar's transforms and the UI's sliders are *the same parameterized functions* over the recorded
series, the eyeball simulator is a first-class property of the architecture, not a bolt-on.

---

## 10. Directory layout

```
apps2/
  ARCHITECTURE.md          ← this file (the map)
  core/
    ingest.php             snapshot once → snapshots/  (cron, RTH-gated)
    store.php              the ONLY I/O module (append/read series, snapshots, radar, fusion)
    transform.php          ★ multi-scale leading-edge library (§5)
    radar.php              ★ extreme/decouple/decorrelate tensor (§6)
    graph.php              load + calibrate causal edges
    propagate.php          forward chain projection
    analog.php             historical analog matcher (the empirical cone)
    fuse.php               ensemble → fusion/ ; calibrate.php → weights.json
    contract.md            the voter + schema spec (single source of truth)
  signals/  hedging/  vol/  flow/  chain/
  lenses/   heat/  radar/  chain/  fuse/  evolve/  surface/  scan/
  shared/   chart.js  alert.js (flash-on-fire, §9b.A)  controls.js (eyeball simulator, §9b.B)  ui.js  swiss.css
  data/     snapshots/  series/  radar/  fusion/  graph.json  weights.json
```

---

## 11. Build order (each phase ships something usable)

1. **Spine** — `core/store.php` + `transform.php` + `contract.md`. Nothing works without it.
2. **Chain ingest** — pull the 7 chain assets (Databento daily GLBX + Yahoo + **FRED real yields**) with volume into `snapshots/`. Start recording immediately — every unrecorded day is lost history.
3. **Radar detector + flash + controls** — `radar.php` + the `radar`/`chain` lens panels 1–3 (normalized overlay, correlation-delta matrix, residual heatmap), wired to `shared/alert.js` (flash-on-fire, incl. INVERSE) and `shared/controls.js` (live date/candle/algo-curve tuning). *This alone is a working, tunable decoupling radar across micro→macro that flashes turns as they begin.*
4. **Crystal ball** — encode `graph.json`, add `analog.php` (empirical cone) + `propagate.php` (projection) + the snowball panel.
5. **Fusion + heat board** — `fuse.php` + `calibrate.php`, wire `chain_projection` + radar as voters; build the `heat` default screen + `fuse` meta-views.
6. **Backfill signal families** — hedging/vol/flow signals into the store; add `surface`/`evolve`/`scan` lenses.

---

## 12. Keep / rebuild / drop from v1

- **Keep & port:** the voter contract, Black-Scholes (`bs.php`), market-hours gating, Massive/Yahoo/Databento/Hyperliquid fetchers, Swiss CSS skin, the calibration concept.
- **Rebuild:** one unified store (not scattered JSONL types), one chart engine, fusion as a recorded time-series, the radar as a first-class detector.
- **Drop on purpose:** per-app re-fetching, per-app re-implemented z/Δ, the "ephemeral" path, greeks-pulse-as-app, the 40-card hub (→ heat board), seasonality-as-app (keep as a static note).

---

## 13. Honesty & limits

- The chain engine detects **broken relationships and matches analogs** — it does not assert causation. Output is always "this pattern historically preceded X, n=K, hit R/K," with cones widening per hop.
- Edge betas are **regime-dependent**; `graph.php` recalibrates on a rolling window and greys out edges whose recent r² is weak. Confident chains only when the recent relationship is stable.
- Rare macro events → small analog samples → wide cones. That haziness is honest and intended.
- Cadence: the chain is **multi-day**, so daily/hourly resolution is the right altitude. Pay for live/tick futures only if intraday chain detection is later proven necessary — not now.

> Companion specs: `data/graph.json` (the causal map), `core/contract.md` (voter + schema spec, written in Phase 1).