Continual learning remains a pre-paradigm field as researchers explore new compaction methods

Continual learning remains a “pre-paradigm” field, with no agreed definition. Some approaches treat it as searching and organizing information in a context window, while others require updating the model with every token; the latter can damage the model in different ways.

Compaction is described as a comparatively mature practice in OpenAI’s Codex and Anthropic’s Claude Code, where summarizer models currently perform the compression. More advanced neural compactors operating in KB-cache space may be possible, but much of the relevant research remains inside closed-source labs.

BaseLabs positions itself as an effort to move this work into the open without the pressure to release the best model in the current quarter. The analysis predicts that research into compaction and continual learning could make the field a more systematic science.