Does an AI research assistant do better when it remembers its earlier thinking? Across eight synthetic investigations, repeated twice per condition, the first model completed all 16 retained-reasoning runs and none with a short rolling history. A second model reproduced the retention advantage on the released suite.
Why it mattered
When an agent forgets its own investigation
An agent can waste its limited steps rediscovering evidence it has already seen. I designed a test that made this failure visible while holding the questions and scoring rules fixed.
What I built
Three memory conditions
I ran the same eight synthetic investigations twice under each memory setting: a short rolling history, retained reasoning, and retained reasoning with compression. Tools performed the calculations; the model chose which evidence to inspect.
What the logs showed
Forgetting consumed the step budget
Every stateless run used all eight allowed steps, repeatedly reconstructing lost evidence. Retained reasoning completed all sixteen runs successfully. This suite was deliberately designed to stress memory loss.
What I would test next
A broader test of the same idea
A larger, independently written hidden set would test whether the result extends beyond these cases. Compression also needs testing at realistic long-context sizes before making a production recommendation.
Second experiment
The retention result repeated
I kept the eight investigations, tools, scoring rules and 48-run design fixed, changing the model from GPT-5.6 Sol to Luna. Luna completed 13/16 retained-reasoning runs, compared with 0/16 for its short-history condition. Its mean retained-mode score was 94.375/100; Sol scored 100/100.
Cost and limits
Cheaper did not mean more accurate
Luna’s complete experiment cost about $0.157 versus $3.754 for Sol under the recorded API-price assumptions. It used 1.5% more output tokens, so the estimated cost advantage came from pricing. The models ran in separate time blocks on an already released suite; this comparison is descriptive, not a new hidden benchmark.
Technical evidence
The harness ran 48 controlled runs across eight hash-locked synthetic investigations in the first experiment. Retained reasoning passed 16/16 runs; stateless history passed 0/16; reasoning plus compaction passed 14/16 but failed its rules. The episode-clustered 95% interval for the primary score difference was [100, 100]. The exact-test p = 0.0078 is the floor for eight independent episodes, not a claim of universal improvement or live-market alpha. The retained condition also cost 41.9% less than the stateless run.
Forced compaction failed the registered quality-preservation and output-token-reduction rules in both experiments. One Luna compaction run encountered an API error and remained recorded as a failure. The suite deliberately stresses memory loss and does not establish an advantage in live market research.