Skip to content

elfmem Roadmap

What this is: the public-facing roadmap for elfmem. One source of truth. What's shipped, what's next, what we've considered and explicitly rejected.

What this is not: a calendar. elfmem is solo OSS. Dates are illustrative; sequence is what matters.

Last reviewed: 2026-08-28 (v0.20.0 merge-readiness pass β€” moved the v2 substrate entry from "In Progress" to "Recently Released" now that it's shipping; renumbered the pre-empted "Production signal response" item to v0.21, since v0.20 is now this release). Reviewed quarterly. Open issues at github.com/emson/elfmem/issues.

Project axioms

These don't change between releases. They constrain what we ship.

  • Agent-first: every public API serves the agent's read β†’ call β†’ interpret β†’ next loop
  • Biological grounding: four rhythms (heartbeat / breathing / sleep / deep sleep), four frames (self / attention / task / simulate)
  • No magic numbers β€” hardcoded constants must be defensible from first principles, not from flashcard fits
  • SIMPLE Β· ELEGANT Β· FLEXIBLE Β· ROBUST β€” docs/coding_principles.md
  • Ship minimum, measure, then earn each layer β€” solo OSS cannot sustain unbounded complexity growth
  • SQLite + zero external services β€” backwards-compatible, file-portable, no cloud lock-in

Status legend

Symbol Meaning
βœ… Released
🚧 In Progress
πŸ“‹ Next (committed, not yet started)
πŸ” Exploring (research, no commitment)
❌ Rejected (with reason)

Recently Released

Version Highlights Date
βœ… v0.20.0 v2 substrate β€” file-backed memory, direct edit primitives, review over auto-archival. The largest release since v0.19.3, spanning ADRs 0009–0013. Direct memory management (edit()/forget()/ls()/inbox(), explicit and no-LLM); the markdown file substrate itself (.elfmem/memory/**.md, elfmem migrate export β†’ cutover with a git-history undo path, elfmem index rebuild/parity); corpus & constitutional review replace silent archival (breaking: decay-driven archival retired, ADR 0009); pairwise LLM contradiction detection at consolidate-time retired for near-zero realized value (breaking, ADR 0010); elfmem init --seed now defaults to off (breaking); provider flexibility (api_key_env, .env auto-discovery, doctor --resolve); host-supplied analysis (dream(host_analyses=...)); frame isolation and credit-assignment protection β€” ATTENTION no longer returns self/constitutional blocks and outcome() no longer scores them (docs/frames_and_credit_assignment_report.md); write-path visibility β€” every stage that reduces what the caller asked for now reports it (FrameResult.dropped, elfmem doctor --frames, docs/integration_friction_report.md); the SELF frame's preamble now reads project.agent_name instead of hardcoding "elf" (docs/self_preamble_naming_report.md), with elfmem migrate offering to set it explicitly. See CHANGELOG.md for the full itemized list. 2026-08-28
βœ… v0.19.3 MCP entry default, drift detection, and migration β€” mcp_json_snippet() now resolves the running elfmem executable to an absolute path instead of a bare "elfmem" string (broke on project-local uv venvs); elfmem serve --env-file reliably delivers API keys to the spawned MCP subprocess instead of silently degrading to mock/no-op behaviour; elfmem migrate/doctor --migrate-mcp now scan the real ~/.claude.json (previously omitted) and its nested projects[path].mcpServers shape, with a new drift check catching an MCP entry wired to a different project's config. Unplanned, bug-driven β€” found via elfmem's own dev instance drifting to an unrelated config/db with peer messaging silently broken (#81, ADR 0008) 2026-07-14
βœ… v0.19.2 Bound and checkpoint consolidation for slow LLM adapters β€” consolidation.contradiction_top_k (default 10) caps contradiction-detection LLM calls per inbox block to the K most similar candidates, bounding worst-case cost to O(K) regardless of active-set size; consolidation.max_inbox_per_run (default 5) self-terminates dream()/consolidate() runs, surfaced as --max and ConsolidateResult.inbox_remaining; rescore_blocks() now commits per-block instead of one all-or-nothing transaction. Unplanned, bug-driven (same pattern as v0.19.0) β€” mitigates but does not fully close the elfmem dream kill-and-lose-progress failure mode; per-block commit durability inside consolidate() itself is a follow-up (#78, ADR 0007) 2026-07-02
βœ… v0.19.1 ConsolidationHealthMetrics on ConsolidateResult.health β€” five diagnostic ratios per cycle (edge_creation_rate, contradiction_detection_rate, prefilter_pass_rate, promotion_rate, deduplication_rate). Observability only; same additive shape as v0.18.1. Defers multi-parameter self-tuning (ADR 0006) with explicit reopen triggers. Also: CI now enforces ROADMAP↔docs/roadmap.md sync; AGENTS.md added with the memory-routing rule earned from Mira's peer message (#74, closes #73) 2026-06-06
βœ… v0.19.0 Peer-protocol hardening β€” peers: in config.yaml now load-bearing; canonical-DID routing eliminates outbox/alv/ vs inbox/elf-alv/ slug drift; atomic + idempotent envelope writes (dotfile temp + os.rename); recipient-readiness precondition replaces silent black-hole sends; one-shot legacy folder migration. Wire-compatible with v0.18 peers (#71, ADR 0005) 2026-05-25
βœ… v0.18.1 ContradictionFinding surfaces per-pair detection-time signals (cosine, tag_jaccard, category_match, hours_apart) on ConsolidateResult.contradictions β€” agents can gate suppression rules without recomputing from current block state (#69) 2026-05-24
βœ… v0.18.0 Manual constitutional review β€” review_constitutional() surfaces drifted constitutional blocks; accept_amendment() applies with audit + (Ξ±, Ξ²) preservation; revert_amendment() one-step undo; CLI + MCP surfaces (#67) 2026-05-23
βœ… v0.17.0 Bayesian sufficient statistics (Ξ±, Ξ²); additive rescore (22Γ— damage reduction); arithmetic peer merge (BUNDLE_VERSION 2); exploration bonus (ΞΊ=0.05) (#65) 2026-05-23
βœ… v0.15.3 Cold-start centrality floor for fresh blocks (#61) 2026-05-17
βœ… v0.15.2 Removed confidence cliff at alignment_score=0.70 (#60) 2026-05-16
βœ… v0.15.1 Surface connect() relation conflicts; fix token under-counting (#59) 2026-05-14
βœ… v0.15.0 Embedding-model lock β€” closes silent-corruption risk from changing embeddings.model (#56, #57) 2026-05-17
βœ… v0.14.x Theory of Mind tools in MCP; dream flags exposed 2026-05-10

See CHANGELOG.md for full history.

Next

πŸ“‹ v0.21 β€” Production signal response

Originally slated as v0.19, then v0.20. Pre-empted twice: first by v0.19.0 peer-protocol hardening (an unplanned signal from elf's own peer-messaging usage; see "Recently Released" above and ADR 0005), then by v0.20.0 claiming the number for the v2 substrate release (unplanned in scope, already in flight when this section was last written). v0.19.2/v0.19.3 also slotted in ahead of this without renumbering it β€” the telemetry gate below is unchanged either time β€” date-bound, not version-bound.

v0.17 (sufficient stats + scoring bundle) shipped 2026-05-23. v0.18 (manual constitutional review) shipped 2026-05-23. Telemetry window for both now open (past the ~2026-08-24 gate below as of this release). Concrete v0.21 scope depends on: - Dmitry's follow-up answer (postponed until we have something substantive β€” draft preserved in archived plan) - β‰₯3 months of v0.17 + v0.18 telemetry from real instances (i.e., not before ~2026-08-24) - Any newly-observed systematic failure modes β€” especially around the new amendment loop

Possible v0.21 candidates (each requires its own ADR before committing): - Amendment loop tuning if v0.18 defaults are off (drift_threshold, cooldown_hours, max_proposals) - Stronger rescore tuning if v0.17 defaults need adjustment - Scheduled review triggers β€” dream(review=True) integration so review is part of the deep-sleep rhythm rather than a manual ritual (only if production data shows manual cadence is too sparse)

Exploring

These are research directions, not commitments. Each requires a Decision Record before becoming a roadmap item.

πŸ” Constitutional / identity evolution β€” automatic mechanisms

The manual review cycle (shipped in v0.18) addresses the MANUAL side of constitutional evolution. The automatic mechanisms remain deferred per ADR 0003:

  • Architecture M: exclude constitutional from ATTENTION candidate pool; inject as preamble at frame render. Big help under drift (+33pp), real cost under stability (βˆ’7pp).
  • Model C (ego_strength): Darwinian β€” constitutional earn persistence via positive outcomes. Adds 4 magic numbers + 1 table.
  • Model D: distributed feedback across top-N constitutional (fixes Model C hoarding, adds cost).
  • Self-architecting agent: hill-climb in parameter space; agent picks its own configuration. Simulation showed it underperforms fixed strategies.

Status: deferred until production signal from v0.18 manual review demands an automatic complement. The simulations explored the design space but did not produce a decisively-better-than-baseline result for any user class.

πŸ” MemoryAgentBench / LoCoMo participation

Empirical comparison against MemMachine, A-MEM, Mem0. Would calibrate the simulation harness and validate weight choices against real workloads. Requires effort to integrate; not on the current critical path.

πŸ” Multi-context (work-self vs personal-self)

Per-tag parameter sets or per-frame overrides. Real demand: unconfirmed. Filed for tracking only.

πŸ” Multi-parameter self-tuning (issue #73)

ConsolidationPolicy adapts only effective_threshold; edge_score_threshold and decay-Ξ» remain static constants (contradiction detection and its prefilter were retired entirely β€” ADR 0010 β€” so they're no longer knobs to tune at all). The full design space (5 architectures, 4 scenarios) was explored in docs/plans/issue_self_tune_research.md; every adaptive variant fails on axioms 1 ("no magic numbers") or 3 ("ship minimum, earn each layer"), consistent with ADR 0003's prior deferral of self-architecting parameter search. Decision recorded in ADR 0006.

Observability-only delta shipped: ConsolidationHealthMetrics on ConsolidateResult.health (five diagnostic ratios). Triggers to reopen: β‰₯30 consecutive cycles of any health-metric field outside a sane band on a real deployment, OR concrete underperformance on MemoryAgentBench / LoCoMo traceable to a specific static threshold. The likely fix when triggered is making one constant a config-yaml override β€” not adaptive tuning.

πŸ” Peer-protocol architectural cleanup (phases 5 & 6 of v0.19)

ADR 0005 deferred two phases from v0.19 because no current bug justified the blast radius. They unblock on trigger, not on a date:

  • Phase 5 β€” Envelope schema_version + time-bucketed msg_id. Current msg_id = m_<hash(content)[:8]> collapses two legitimate repeat-content sends from the same sender into one message. Trigger to reopen: production logs show msg_id collisions in real traffic, or wire-format evolution forces a versioned envelope. (Requires observability β€” see "Issues" in the change notes; no collision counter exists today.)
  • Phase 6 β€” Quarantine routing for unknown senders and corrupt envelopes. Unknown senders currently land in their named subdirectory; malformed JSON fails silently in _parse_message. Trigger to reopen: peer roster grows beyond ~10 entries (federation noise becomes material), or federation to a product-elf at scale (per the cloud-architecture sketch in note-to-alv).

πŸ” Goal-directed edge metabolism (tier-2 edges via self frame)

Full plan: docs/plans/plan_edge_metabolism.md. Extends rescore() (Deep Sleep) to propose connections judged against elf's own self/goal blocks, not just cosine similarity β€” ungated, correctable via connect()/forget()/amending self rather than a human-review step.

This reopens a previously-deferred idea: plan_memory_scoring.md explicitly deferred "Zettelkasten auto-linking" pending evidence that manual connect() is undertilised (checked against the real self-hosted DB β€” it is) and a way to validate LLM-judged links aren't phantom edges. ADR 0010 (2026-08-08) retired a different pairwise-LLM mechanism for measured near-zero realized value β€” the most recent and most directly cautionary precedent for this shape of feature.

Status: Stage A (read-only dry-run instrument, no schema change, no edge writes) approved and building. Stage B (live autonomous edge creation) requires reviewing Stage A's real output first β€” recorded as an open decision in the plan doc, to become ADR 0011 once resolved.

Rejected

Things we considered and decided not to do. Documented so they aren't relitigated.

❌ Power-law retrievability decay (FSRS-style)

Considered: replacing exp(-Ξ»t) with (1 + 0.5t/stability)^(-0.5). Why rejected: simulation across 4 scenarios showed βˆ’5 to βˆ’7.6pp quality and βˆ’44 to βˆ’66pp recent_reach (catastrophic). Fat tails keep stale blocks competitive against fresh ones. Power-law works for flashcards; refuted for agent memory. Decision Record: docs/decisions/0001-power-law-decay-rejected.md Trigger to revisit: only if MemoryAgentBench shows power-law wins on agent workloads (none reported as of 2026-05).

❌ FSRS-5 19-parameter stability machinery

Considered: importing the FSRS-5 difficulty/stability mechanics wholesale. Why rejected: 19 magic numbers fitted to flashcard data; no fitting infrastructure for agent traces. Violates "no magic numbers" by an order of magnitude. Trigger to revisit: β‰₯10,000 outcome events in production from real elfmem deployments AND a fitting pipeline.

❌ block_events event log table

Considered: a full per-event log for replay and audit. Why rejected: block_outcomes already captures the only event type with non-trivial signal. Replay is a research affordance, not a user affordance. Trigger to revisit: real user demand for compliance / replay features.

❌ Hierarchical abstract tier (raw / summary / abstract)

Considered: MemoryOS-style three-tier hierarchy. Why rejected: existing summary-block mechanism is sufficient. Adding a third tier is imitation without measured benefit. Trigger to revisit: measurement of context-bloat retrieval defects on real queries.

❌ Zettelkasten auto-linking on consolidate

Considered: LLM-driven automatic connect() calls during dream(). Why rejected: connect() already exists for explicit use; auto-linking introduces LLM failure modes without measured gain. Trigger to revisit: evidence that explicit connect() is undertilised by real agents AND a validation strategy for LLM-judged links.

❌ Renaming confidence β†’ utility

Considered: aligning vocabulary with cognitive-science papers (Park's "importance"). Why rejected: confidence is elfmem's brand-term used across outcome(), consolidate(), MCP, AgentGuide, CLI. Rename has high churn cost for zero behavioural value. Trigger to revisit: never. Terminology decision is settled.

Long-term north star

Not committed, but where this is heading.

  • v0.19: peer-protocol hardening (shipped β€” unplanned, signal-driven)
  • v0.20: v2 substrate β€” file-backed memory (shipped β€” unplanned in scope, see "Recently Released")
  • v0.21+: production signal response (originally v0.19, then v0.20; gated on β‰₯3 months of v0.17/v0.18 telemetry)
  • v0.22+: benchmarking against MemoryAgentBench / LoCoMo if calibration is needed
  • v0.23+: earned architectural features (only the deferred items that empirical evidence supports)
  • v1.0: public API freeze. Stable for years. Backwards-compatible changes only.

Discipline: every subsequent layer must be earned with evidence β€” not designed in advance.

How this roadmap is maintained

  • Quarterly review: at start of each quarter, re-evaluate "In Progress" and "Next" against actual user signal
  • Per-release update: at each minor release, move items from "In Progress" to "Recently Released"; revisit "Exploring"
  • Decision Records: every "Rejected" item or major architectural choice gets an ADR in docs/decisions/
  • Issue links: each roadmap item should link to a GitHub issue once filed; discussions happen there

If something here is wrong, file an issue. Pull requests against this file are welcome.