Ulam builds research-grade reasoning datasets and evaluation systems for frontier AI labs and AI developers. Our work is grounded in open mathematical research, where progress is measured by the quality of reasoning, not just final answers.
We produce human-in-the-loop reasoning traces, expert-reviewed proof attempts, partial-progress data, and private benchmark programs for teams that need harder-to-game signals on model capability.
We turn frontier mathematical research into data and evaluation assets that model builders can use immediately.
Curated open-problem corpora, structured metadata, and research workflows designed to probe reasoning beyond solved benchmarks.
Iterative traces where models explore, revise, and refine ideas with expert intervention at the points that matter.
Partial progress, failed approaches, and checker-backed artifacts labeled for signal quality rather than surface fluency.
Custom benchmark suites and eval protocols for labs that need private, difficult-to-overfit measures of frontier reasoning.
Run private evaluations that expose where a model’s reasoning breaks, then turn those failure patterns into a clear improvement agenda.
Start with reviewed reasoning datasets or build targeted data packs around your model’s actual errors, tasks, and evaluation results.
Train and evaluate agents in stateful, verifier-backed research environments where progress, revision, and failure become learning signals.
We usually start with a pilot benchmark on ErdősBench and SimoBench to gauge a model’s reasoning capabilities, inspect where and why it fails, and agree on the next research intervention. From there, we can provide off-the-shelf datasets or build RL gyms matched to the failure modes the pilot reveals.
Book a call with us.
We are a team of PhD-level mathematicians and engineers tackling research-level open problems. We use AI to push frontier models to their limits - and beyond - by solving real mathematical problems, not just measuring performance on established benchmarks. We care about the evidence, environments, and training signals that can move LLM reasoning to its next level.
Use verified research trajectories, proof attempts, critiques, repairs, and failure traces off the shelf - or build a targeted data pack around your model’s actual errors.
Best fit: post-training, critic training, and model-specific repair.
Start with ErdősBench and SimoBench to expose reasoning failures, then train and evaluate agents in verifier-backed RL gyms for code, algorithms, science, and mathematics.
Best fit: capability diagnosis, release gates, and verifier-backed RL.
We used GPT-5.4 Pro, Opus 4.6, and GPT-5.2 to attack open problems in mathematics—and one of them is now fully settled, with a machine-checked proof in Lean 4.
Read full article →