ArborML

Research programme

Trainable hierarchical memory for language models.

Can a model learn to navigate a very large store of trainable parameters while reading only a small, local working set for each token? Can a small student acquire and combine capabilities from several teacher models through that memory? ArborML tests these questions with preregistered experiments, and reports the negative outcomes as clearly as the positive ones.

The questions

The target architecture is a list of requirements, and each one is a separate question with its own studies. Being clear about what is open matters more here than sounding finished.

Does external memory help prediction?

Measured against the same backbone without memory, against shuffled memory, and against a dense adapter with the same number of parameters.

Can addressing be learned reliably?

Whether the model finds the right slot as the number of stored items and the load per slot grow.

Does a hierarchy earn its keep?

Whether a tree of memory beats a flat memory on quality, work per token and storage locality, or only looks like it should.

Can memory live on an SSD?

Whether learned addressing gives access patterns that make storage-backed inference practical. Not evaluable yet.

Can a student learn from teachers?

Whether one model can acquire, retain and combine capabilities from several open teacher models. Untested.

Does it hold on a real model?

Everything so far uses small backbones trained from scratch. Attaching the memory to a pretrained model is still ahead.

Status: the up-to-date scorecard, with every registered outcome quoted from its record, is in the ArborLM README. This page does not restate results, so it cannot drift from them.

How the work is done

  1. Preregistered

    Each study fixes its design, budget, seeds and pass/fail criteria in writing before it runs. Labels are assigned by the criteria, not after the fact.

  2. Negative results stay published

    Failed hypotheses are recorded with the same care as successes, and later work never rewrites an earlier outcome.

  3. Science and engineering kept apart

    ArborLM holds the research record. The ArborML Lab builds the runtime, telemetry and memory explorer; passing engineering tests is never presented as a scientific result.

  4. Independent review

    Studies and code are audited by a reviewer other than the author, and the review is recorded, including when it was not performed.

Roadmap

  1. Engineering foundation

    A Rust runtime, telemetry, visual tools for the memory, a pretrained backbone and a trainable memory attached to it.

  2. Integrated prototype

    A model in which the memory learns during training, measured against equally sized alternatives.

  3. Multi-teacher investigation

    An experimental test of whether one student can acquire, retain and combine capabilities from open teacher models such as Qwen and GLM.

  4. Model validation

    A separate milestone with independent evaluations. Completing the stages above is not, by itself, evidence that the approach works.

Get in touch

Researchers, engineers and labs working on model memory, retrieval or distillation: collaboration, review and critique are welcome.

Or write directly to

hello@arborml.ai

You will get a reply from a person, usually within a few days.

Your message is emailed to the project owner and used only to reply to you.