The hardest part of building with AI isn't writing the code. It's knowing which assumptions are safe to build on. Most teams inherit their assumptions from whatever the field happens to be repeating that quarter, and they rarely stop to test them, because testing is slow and the shortcuts usually seem to work.
At Aimpoint Labs, testing those assumptions is the work. Our latest research paper takes on one that sits underneath a growing number of production AI systems, and the result gives teams a firmer foundation for a decision that's usually made on instinct.
A shortcut the field rarely checks
If you're building AI agents, you've likely faced a version of this choice. You have several models, each trained to handle a different kind of task, and you want one model that does it all. Retraining everything together from scratch is often impractical. It demands all your data in one place and the full compute cost, paid again. So, most teams reach for the cheaper option and combine the specialists they already have. Take the models, add their learned changes together, and you're done.
Joint retraining is expensive. It requires all your data in one place and the full compute cost, paid again. Merging is cheap: take the specialists, add their learned changes together, and you're done.
Merging has become a popular answer precisely because it works when joint training is impossible, such as when specialists come from different teams whose data can't be pooled. But that convenience rests on an untested premise: when you could train jointly, does merging actually match it? The comparison almost never gets made, because the two situations so seldom coexist. So the field merges, assumes it's close enough, and moves on.
We decided to find out whether "close enough" is actually true.
Building the missing comparison
This is the kind of question that rewards controlled, rigorous experimentation, which is exactly what Aimpoint Labs is built for. Our team trained specialist AI agents on AppWorld, a benchmark of realistic, multi-step digital tasks graded by hidden tests. We built specialists on tasks of different difficulties, merged them using multiple established methods, and separately trained a model jointly on the combined data. Then we put all of them head-to-head on the same held-out tasks.
By holding everything else constant, one benchmark, one recipe, one clean comparison, we could isolate precisely what merging gains or gives up against the baseline it claims to replace. It's the comparison the standard merging setup can't make, because it never has the jointly trained model on hand.
What we found, and why it matters
On the benchmark's primary metric, merging matched joint training. Every merge method we tested came out statistically indistinguishable, and all of them tied the far more expensive jointly trained model.
For teams building with AI, that's a directly actionable result. If you can pool your data, joint training and merging deliver the same performance, so the decision comes down to cost and operational simplicity rather than results, and you can confidently take the cheaper path. If you can't pool your data, whether for privacy, organizational, or logistical reasons, merging doesn't sacrifice what joint training would have given you. Either way, a decision that's usually made on guesswork now has evidence behind it.
Understanding the why, not just the what
Where this work moves from useful to genuinely differentiated is in the explanation. It isn't enough for us to observe that merging works; we wanted to understand the mechanism, because that's what makes a finding portable to new situations instead of a one-off result.
Sophisticated merging methods exist specifically to resolve conflicts between models. If the choice of method made no measurable difference, something about these models had to be unusual. So we measured the geometry of what each specialist had actually learned, looking at the direction of their learned changes rather than just their scores.
The specialists' updates pointed in nearly independent directions, even while modifying overlapping parameters. We calibrated that measurement rigorously, against a random baseline on one end and a same-model ceiling on the other, to prove it reflected real learning rather than an artifact of the setup, and we watched the pattern develop gradually across training.
The picture that emerges is clean: when direction and overlap decouple this way, the clever merge rules built to untangle conflicts have almost nothing left to untangle, so they collapse into simple averaging. That's why the method doesn't matter here. And knowing the why is what lets us predict where it will and won't hold as conditions change, which is the difference between a result and an understanding.
The discipline behind the work
Aimpoint Labs is the AI product development and experimentation arm of Aimpoint Digital. We partner with forward-thinking companies and university systems to research, develop, and deploy applications of traditional and generative AI, pushing on the questions of why these systems behave the way they do and how that understanding translates into better products.
This is the same discipline we bring to client engagements. Getting to the foundations, rather than stopping at what appears to work, is what lets us help organizations make sound architectural decisions, avoid expensive bets on assumptions that don't hold, and move faster with AI to build a real competitive edge. Rigorous, well-founded answers are what separate durable AI strategy from surface-level implementation, and they're what we're here to deliver.
We've also released the full code and statistics so the field can reproduce this result and build on it, because moving the field forward is part of how we stay at its edge.
Read the full paper here.
