Skip to content

0002 — v0.17 scope: bundle four scoring fixes

Status: Accepted — implemented in v0.17.0 Date: 2026-05-23 Deciders: elf (curator), Ben

Context

Two days of analysis (closed-form derivations + four Monte Carlo simulation rounds + architectural brainstorming) explored a wide design space for elfmem's evolution. The exploration produced both empirically validated proposals and speculative architectural extensions.

A decision was needed: ship the validated minimum, or the architecturally ambitious maximum?

Alternatives considered

  1. Original plan (plan_memory_scoring.md): ~340 LOC across v0.16 + v0.17 — sufficient statistics, additive rescore, peer merge, exploration bonus, opt-in power-law
  2. Extended plan (plan_self_architecting_elfmem.md): ~800 LOC across v0.17 → v0.20 — original plan PLUS Architecture M, Model C (ego_strength), Model D (distributed feedback), self-architecting agent
  3. Validated minimum (this ADR): original plan minus power-law decay

Decision

Ship option 3 (~330 LOC):

  1. Sufficient statistics: success_count, failure_count columns. confidence stays as denormalised view.
  2. Additive rescore: α += signal × 0.5; β += (1−signal) × 0.5 instead of overwriting confidence.
  3. Arithmetic peer merge: (local_α + remote_α × trust, local_β + remote_β × trust).
  4. Exploration bonus: kappa × sqrt(beta_variance) with hardcoded kappa = 0.05.

This is the original plan_memory_scoring.md minus power-law decay (rejected per ADR 0001).

Evidence supporting each component

  • Additive rescore: simulation showed 22× reduction in rescore damage on N>5 blocks (0.152 → 0.007 |Δconfidence|); +12pp recent_reach under weekly-rescore workload. Zero cost when no rescore fires.
  • Sufficient statistics: mechanical bookkeeping enabling additive rescore + arithmetic peer merge. Mathematically equivalent to existing (confidence, outcome_evidence) representation.
  • Arithmetic peer merge: the only principled way to combine two peers' Beta-Binomial observations. ~50 LOC.
  • Exploration bonus: simulation showed +5.6pp quality at 730 days under long-horizon workload (72.4% → 78.0%). Small short-term cost (~3pp) at 90 days. Compounds over time. One hardcoded magic number, Critic-endorsed.

What this rejects (and why)

  • Architecture M (constitutional excluded from ATTENTION): structural insight is real, but the simulation showed a −7pp tax under stable identity vs +33pp gain under regime change. Without telemetry on real user-mix distribution, shipping M is a bet that helps some users and harms others. Defer.
  • Model C (ego_strength mechanism): 4 magic numbers + 1 new schema table + new vocabulary. Equivalent shape to FSRS-5 mechanics already rejected in plan_memory_scoring.md. No production evidence of need. Defer.
  • Model D (distributed feedback): marginal over Model C; ships together or not at all. Defer.
  • Self-architecting agent: simulation showed it underperforms fixed strategies in every scenario tested. Directionally correct but not quantitatively better. P2 risk: miscalibrated self-architect worse than no self-architect. Defer indefinitely.

Consequences

  • v0.17 scope is bounded and reviewable. ~330 LOC. Four mechanically independent changes.
  • Each change is empirically validated, reversible, and adds at most a small concept to the public surface.
  • The constitutional / Model C / self-architect work is preserved as research in docs/research/, not committed to the implementation roadmap.
  • Future v0.18+ scope is driven by production telemetry, not pre-designed.

Required before any deferred items earn implementation

  • Dmitry's anonymised DB for real workload calibration
  • N=5+ seeds on validating simulations (current N=2-3)
  • v0.17 in production for ≥3 months
  • Evidence from real instances that additional complexity is warranted

References