Skip to main content

Producer Intelligence Layer

Overview

The 13 base producers are dumb signal emitters. They collect data, normalize it, and publish events. They have no memory of whether they were right or wrong, no awareness of what other producers are saying, and no ability to adapt to different market regimes. The Producer Intelligence Layer (P3 + P4) fixes this. It wraps every producer in an interpreter stack that adds:
  • Per-signal adaptations (P3): regime conditioning, LLM critique, self-memory, adversarial prosecution, novelty filtering
  • System-level intelligence (P4): hierarchical weighting, multi-horizon forecasts, cross-producer awareness, and a meta-producer that learns from the ensemble’s track record
Every layer in the stack defaults to shadow mode (shadow=True): it observes, logs, and learns — but does not mutate the forecast. This is the right default. You observe before you trust.

Architecture: The Interpreter Stack

Each producer’s raw output passes through a layered interpreter chain. Each layer wraps the inner layer, adding one specific capability. The stack is applied automatically by BaseProducer.emit_forecast().
Data flow per cycle: Key invariant: No layer in the stack ever changes the forecast action (long/short/flat). Only confidence is modulated, or the forecast is replaced with an abstention.

P3: Per-Signal Adaptations

P3.1 — LLM Critic (shadow mode)

What it does: An LLM reviews the rule-engine’s candidate forecast and flags mis-calibrated confidence or signals that should be suppressed. How it works:
  1. The rule engine produces a candidate ForecastPayload (action + confidence)
  2. The LLM sees: the candidate, the top 5 raw signals, the regime tag, trailing Brier score, and aggregate conviction
  3. The LLM returns a confidence_delta (±0.3 max) and optionally suppress=True
  4. In shadow mode: the critique is hashed into reasoning_hash and logged to llm_shadow_log but the candidate is returned unchanged
  5. In live mode: confidence is adjusted, regime caps applied, and suppressed forecasts become abstentions
Shadow mode (shadow=True, the default):
  • Critique is computed and stored but does not affect the output
  • Logged to llm_shadow_log table for later analysis
  • Use get_shadow_comparison() to see rule-vs-LLM divergence stats
Meta-guardrail: If the producer’s trailing Brier score exceeds 0.35, the LLM critic automatically reverts to shadow mode for that cycle — even if live mode is configured. This prevents a struggling producer from compounding errors with LLM adjustments. Confidence bounds in live mode:
  • Delta clamped to [-0.3, +0.3]
  • Regime cap applied (BULL: 1.0, BEAR: 0.7, TRANSITION: 0.6, CRISIS: 0.4)
  • Minimum 0.3 confidence required after adjustment (min_live_confidence)
Configuration (env vars): Graceful degradation: If the LLM is unconfigured, returns an error, or times out — the rule-engine’s candidate is returned unchanged. The critic never blocks emission.

P3.2 — Regime Matrix

What it does: Conditions every forecast on the current market regime. Different regimes get different confidence scaling, different minimum thresholds, and optionally full abstention. How it works:
  • Each producer can declare a RegimeMatrix on its interpreter class
  • After interpret() returns a candidate, apply_regime_conditioning() applies the regime-specific RegimeConfig
  • This happens automatically inside safe_interpret() — producers don’t need to add if/else blocks
RegimeConfig fields: Global regime confidence caps (from engine/core/regime.py, 0-10 scale → divided by 10 at use): Example declaration:
When no RegimeMatrix is set: The interpreter’s output passes through unchanged (the default for producers that haven’t declared regime-specific behavior).

P3.3 — Differentiated Inputs

Each domain producer receives signals specific to its domain. This is handled at the collect() + normalize() level — each producer ingests only what it knows how to interpret. The intelligence layer doesn’t change what each producer ingests — it changes how the system evaluates and combines the resulting forecasts.

P3.4 — Producer Self-Memory

What it does: Adjusts a producer’s confidence based on its own historical Brier score. Good calibration → confidence boost. Poor calibration → confidence penalty. A grimoire with teeth: lessons persist because they’re callable. How it works:
  1. SelfMemory.query() reads from forecast_calibration (Brier scores, regime breakdown)
  2. Computes a blended delta from long-term (90-day) and recent (3-day) Brier performance
  3. Optional regime-specific adjustment (20% weight if ≥3 regime-specific samples)
  4. SelfMemoryInterpreter applies the delta to the candidate’s confidence
Brier → delta mapping: Blending formula:
Where streak_weight = 0.35 by default (recent performance gets 35% influence). Guardrails:
  • max_delta = ±0.30 — confidence can never shift more than 30% in either direction
  • min_resolved = 5 — no adjustment until ≥5 resolved forecasts exist
  • Action is never changed (confidence only)
  • Returns no-op when DB is unavailable
SelfMemoryConfig defaults:

P3.5 — Adversarial Prosecutor

What it does: An LLM constructs the strongest possible case AGAINST each forecast. If the bear case overwhelms the bull case, the forecast is suppressed. If the bear case is weak, confidence gets a small boost. How it works:
  1. The prosecutor sees: the candidate forecast, top 5 signals, and regime tag
  2. It returns:
    • bear_strength (0.0-1.0): strength of the counter-case
    • bull_strength (0.0-1.0): strength of the thesis from the data
    • suppress (bool): True if bear_strength > bull_strength by more than 0.15
    • confidence_boost (0.0-0.15): bonus if bear case is weak (bear_strength < 0.25)
    • rationale: one-sentence reason
  3. In shadow mode: result is logged but the candidate passes through unchanged
  4. In live mode: suppress → abstain, confidence_boost → bounded adjustment
Why it matters: Catches correlated inputs — when all signals agree because they’re measuring the same thing from different angles, the prosecutor finds the shared assumption. Configuration (env vars): Graceful degradation: If the prosecutor is unconfigured, returns an error, or times out — the candidate passes through unchanged. The ProsecutorInterpreter also respects prosecutor.config.shadow as an additional shadow check.

P4: System-Level Intelligence

P4.1 — Hierarchical Weighting

What it does: Dynamically adjusts per-domain weights in synthesis based on historical performance, asset fit, regime fit, and cross-domain correlation. The multiplier chain: The HierarchyEngine computes a weight multiplier for each domain per cycle. The multiplier modulates the domain’s prior weight (from config) during brain synthesis.
Factor details: Brier → multiplier conversion: Guardrails:
  • MIN_MULTIPLIER = 0.1 — a domain can lose at most 90% of its prior weight
  • MAX_MULTIPLIER = 2.0 — a domain can at most double its prior weight
  • MIN_BRIER_SAMPLES = 5 — no reliability adjustment until ≥5 resolved forecasts
  • MIN_ASSET_SAMPLES = 3 — no asset_fit adjustment until ≥3 per-asset samples
  • All factors return 1.0 (neutral) when data is insufficient
Graceful degradation: When tables are empty or P2.x data is unavailable, all multipliers return 1.0 and domain priors from config apply unchanged.

P4.2 — Multi-Horizon Forecasts

What it does: Each domain produces forecasts at domain-appropriate horizons with horizon-specific confidence scaling. Design principle: Short horizons are noisier → lower confidence cap. Long horizons are less actionable but carry higher conviction when signals agree. Domain-specific horizon sets: How horizon scaling works:
Note that TradFi gets a slight 24h boost (scale=1.05) because basis/funding signals are more meaningful over 24h. On-chain gets a 3d boost (scale=1.10) because accumulation/distribution patterns play out over days.

P4.3 — Cross-Producer Awareness

What it does: Gives each producer a single aggregate signal about what the rest of the system is thinking, without revealing individual producer identities or domain breakdowns. Aggregate conviction:
  • ConvictionStateReader reads recent FORECAST_V1 events (default: last 2 hours)
  • For each asset: weighted average of (confidence × direction_sign) across all producers
    • long = +1, short = -1, flat/no_forecast = 0
  • Result: a single signed float in [-1, +1] (positive = bullish, negative = bearish)
  • Minimum 2 forecasts required for a non-zero signal
Novelty penalty mechanics:
  • High agreement with strong conviction → suppress confidence (you’re adding noise, not signal)
  • Contrarian signal → slight confidence boost (disagreement = information)
  • Weak conviction → no penalty (brain is uncertain, all signals valuable)
Tuning constants (from engine/core/novelty.py): Decision rules: Shadow mode (shadow=True, the default): The novelty result is logged but the candidate passes through unchanged. Action is never changed — only confidence is modulated.

P4.4 — The Meta-Producer

The meta-producer is the system’s compound learning output. It learns from the ensemble’s historical track record and emits forecasts based on ensemble pattern matching.

Outcome Resolver

What it does: Resolves elapsed FORECAST_V1 events against actual prices, producing FORECAST_OUTCOME_V1 events. This is the data collection mechanism that feeds everything downstream. How it works:
  1. Finds unresolved FORECAST_V1 events whose horizon has elapsed (+ 5-minute buffer)
  2. Fetches prices at forecast time and resolution time:
    • First: local price_history table (seconds and milliseconds epoch)
    • Fallback: Binance public klines API
  3. Computes: return_actual_pct, direction_correct, brier_score
  4. Writes FORECAST_OUTCOME_V1 event (immutable, deduplicated)
  5. Records resolution in forecast_resolution_state table (idempotent)
How to run:
Returns the count of forecasts resolved. Exit code is always 0. Cron setup:
Run every 30 minutes. Idempotent — safe to run as often as you want. What to expect in logs:
  • outcome_resolver_missing_price — price data unavailable for a forecast (normal early on)
  • Normal output: count of resolved forecasts per run

Performance Aggregator

What it does: Computes rolling producer performance statistics from FORECAST_OUTCOME_V1 history. Outputs:
  • producer_performance table: per-producer, per-asset, per-horizon, per-regime stats (win_rate, avg_brier, avg_confidence, confidence_reliability)
  • producer_correlation table: pairwise producer agreement rates, agreement win rates, disagreement win rates
Minimum data: MIN_FORECASTS_FOR_STATS = 5 resolved outcomes per group.

MetaProducer

What it does: Reads performance tables and learns which ensemble patterns historically lead to correct outcomes. Hard constraint: No direct market data reads. Inputs are restricted to:
  • FORECAST_V1 events (current ensemble state)
  • FORECAST_OUTCOME_V1 events (historical outcomes)
  • producer_performance / producer_correlation (derived tables)
Activation gate: MIN_FORECASTS_FOR_ACTIVATION = 500 resolved outcomes must exist before the meta-producer emits any non-abstention forecast. Below this threshold, it always abstains with INSUFFICIENT_DATA. Pattern matching:
  1. Gets current ensemble state: latest action (long/short/flat) per producer for the target asset (last 2 hours)
  2. Searches historical episodes for matching ensemble patterns
  3. Computes win rate and majority direction from matching episodes
  4. Emits a forecast only if:
    • n ≥ MIN_SAMPLE_FOR_PATTERN (10 matching episodes)
    • win_rate ≥ WIN_RATE_THRESHOLD (0.60)
Shadow mode (shadow=True, the default): Even after activation, the meta-producer logs its would-be forecast but emits an abstention with SHADOW_MODE reason. This ensures the pattern library matures before affecting synthesis. Registration: @register("meta", domain="events") — runs on */30 * * * * schedule.

Configuration

Per-layer configuration summary

When to change defaults

Enable LLM Critic live mode when:
  • You’ve run shadow mode for ≥2 weeks
  • get_shadow_comparison() shows the LLM consistently improves calibration
  • You’ve verified the LLM isn’t just agreeing with the rule engine (check suppression rate)
Enable Prosecutor live mode when:
  • Shadow logs show meaningful bear/bull strength separation
  • The prosecutor catches genuine correlated-input problems
  • Suppression rate is reasonable (not suppressing everything)
Enable Novelty live mode when:
  • You have ≥3 producers running and brain conviction is meaningful
  • You want to actively penalize redundant signals
Enable Meta-Producer live mode when:
  • ≥500 resolved outcomes exist
  • Pattern library shows win_rate ≥ 0.60 on ≥10 matching episodes
  • You’ve verified shadow logs look sensible for ≥1 month

Shadow Mode

Shadow mode is the most important design pattern in the intelligence layer. Every LLM-based and adaptive layer defaults to shadow=True. What shadow mode means operationally:
  • The layer runs its full computation (LLM call, novelty penalty, pattern match, etc.)
  • The result is logged to the database or structured log
  • The candidate forecast passes through unchanged
  • No forecast is ever suppressed, boosted, or modified by a shadow layer
Why this is the right default:
  • New layers have no track record. Trusting them immediately is reckless.
  • Shadow data lets you compare “what would have happened” against actual outcomes.
  • You can validate each layer independently before going live.
  • If an LLM starts hallucinating or a pattern goes wrong, shadow mode means zero production impact.
How to enable live mode:
  • LLM Critic: set B1E55ED_LLM_CRITIC_SHADOW=false
  • Prosecutor: set B1E55ED_PROSECUTOR_SHADOW=false
  • Novelty: pass shadow=False to NoveltyInterpreter constructor
  • Meta-Producer: pass shadow=False to MetaProducer constructor
Observation tools:
  • get_shadow_comparison(db, producer, days=30) — LLM critic shadow stats (rule vs adjusted confidence, suppression rate, error rate)
  • llm_shadow_log table — raw per-cycle LLM critic logs
  • Producer logs at INFO level — self-memory, prosecutor, novelty all log their shadow results

Cron Setup

Outcome Resolver

The outcome resolver must run periodically to collect the data that feeds the entire intelligence layer.
Verify it’s running:

Data Activation Timeline

The intelligence layer activates progressively as data accumulates:

Reference: Source Files