Two settings tripled OpenAI’s ARC-AGI-3 benchmark scores

OpenAI said its GPT5.6 Sol model performed much better on the ARCAGI3 benchmark after two API settings were enabled: retained reasoning and compaction. The company reported that the changes tripled scores on the public task set and reduced output tokens by about six times. The benchmark tests how well AI agents can learn and reason through unfamiliar 2D puzzle games. OpenAI said the low score from its model was partly caused by the evaluation harness, which discarded private reasoning and used rolling truncation that removed older context as tasks grew longer. Using the Responses API, the company kept reasoning across turns and replaced truncation with compaction. OpenAI said this helped the model preserve what it had learned, act more consistently, and solve more of the benchmark tasks. The company added that benchmark results can depend heavily on harness design and API settings, not just the model itself.