Why We Built an Experiment Orchestrator, Not a Prediction Engine

Prediction accuracy is not the story here. In August 2025, our first Discovery Loop build was a prediction model: researchers uploaded results, we fit a surrogate to their data, and it estimated outcomes for untested conditions. The accuracy was reasonable. Two research groups ran it for six weeks. After six weeks, each group was still conducting exactly as many experiments per week as before.

That outcome redirected everything we built next.

What V1 Actually Did

V1 worked technically. Its surrogate learned from a researcher's historical data and predicted candidate-condition outcomes with useful benchmark accuracy. Researchers uploaded a CSV, defined candidate conditions, and received a ranked list of predicted outcomes with confidence intervals.

It did not recommend an action. It answered, "what happens under condition X," but not, "which condition should I run next." Those questions look close but serve different functions. One extends knowledge; the other initiates work.

Both early groups behaved similarly. Researchers uploaded fresh results, inspected predictions, said "interesting" or "that confirms what we thought," then planned the next experiment through their usual intuition and domain knowledge. The model informed them but did not direct them. Good information seldom changes practice without a recommended next action.

The Insight That Forced a Rebuild

About six weeks into the second pilot, one researcher put the issue plainly: "The predictions are accurate, but I still have to decide what to run. That's the hard part. They make that part slightly better informed, not easier."

That comment changed our framing. We had focused on information that could improve a researcher's decisions. We needed to reduce what the researcher had to do, not merely improve what they knew. The bottleneck was the decision step: turning available evidence into a concrete action, such as queuing one experiment instead of another, quickly enough to keep the loop moving.

Prediction improves a decision's inputs. An orchestrator automates the decision step in the human critical path. These are different interventions: one informs the existing workflow, while the other changes it.

What We Rebuilt

The rebuild had three parts. First, we replaced the static prediction screen with a live loop. As results arrive, the model updates and proposes the next experiment batch on its own schedule. Researchers do not start a prediction session. Second, we added an execution layer that accepts proposals and, for labs with our instrument connectors, queues them directly instead of requiring re-entry elsewhere. Third, we built an acquisition function that turns uncertainty and expected improvement into a ranked proposal list. Recommendations therefore follow an explicit decision rule, not predicted outcome alone.

Prediction remained, but as one component of the orchestrator. The V1 surrogate is the same class of model. What changed is the next step: the loop scores candidates with its predictions and builds the next proposal set automatically, rather than displaying predictions for manual follow-through.

What Changed in Practice

The same two V1 pilot groups tried the rebuilt system. Both increased their experiment cycle rate within the first two weeks. The experiments still took the same time; the interval between cycles became much shorter. A decision process that had required 1 to 3 days of informal discussion became a proposal the researchers could review, approve, or adjust in under an hour.

We do not assign the entire improvement to orchestration. These researchers were motivated early adopters, and shorter cycles may reflect enthusiasm as well as lower friction. Still, the direction was clear: prediction without action guidance changed nothing over six weeks, while orchestration with recommendations changed behavior within two.

The Residual Role of Prediction

Prediction is still central to the loop. Every proposal carries an estimated outcome. Its confidence interval helps the acquisition function decide whether to run a candidate for exploration, where high uncertainty has value, or treat it as sufficiently known and not worth running when uncertainty is low and predicted value falls below the current best plus margin.

In V1, prediction sat before the human decision. Now it sits inside a decision architecture. Researchers do not translate confidence intervals into experiment selections by hand. The loop makes that conversion and presents the result: here are the three experiments it recommends next. The researcher can accept, modify, or override the set. That is a narrower cognitive task than turning a prediction table into an experiment plan from scratch.

We built an orchestrator because orchestration changed behavior. A stronger prediction model might have improved V1 at the margin, but it would not have fixed its limiting problem. The constraint was the distance between prediction and action, and the product closes that distance.