Hypothesis Trees: A Model for Experimental Design

A hypothesis tree is not a decision tree. When we introduce Discovery Loop's hypothesis engine, researchers usually ask whether it is simply a decision tree. It is not, and the distinction takes a few minutes to explain.

A decision tree has a predetermined branch structure: when a condition holds, follow its assigned path. The builder defines those branches beforehand, and a new branch requires rebuilding the tree. Scientific exploration needs a different representation because relevant hypotheses are initially unknown, while experiments continually reshape the possible outcome space.

A hypothesis tree differs in one key respect: branches contain probability distributions, not deterministic rules, and incoming data changes the structure itself.

The Data Structure

Each node expresses a hypothesis about the experimental system. Specifically, it stores a distribution over outcomes that testing the hypothesis could produce. Rather than asserting that one parameter combination yields outcome X, it records probabilities: p for X above threshold, q for a middle range, and r for an unexpected result.

Edges link parent hypotheses to their children conditionally. If testing a parent produces result Z, its applicable child hypotheses are the corresponding branches. A tested parent remains in the tree as evidence, updating distributions throughout the downstream structure.

Leaves mark the tree's experimental frontier. They are untested hypotheses whose distributions represent the current estimate of what testing would show. These leaves become experiment candidates.

Bayesian Updates Each Cycle

After an experiment returns a result, we apply a Bayesian update to the tree. Evidence propagates in two directions.

Forward propagation shifts child distributions according to the parent's result. If parameter combination A produces a stronger signal than expected, child nodes representing A's variants gain probability mass around high-signal outcomes. The shift is determined by the covariance model's correlation structure.

Lateral propagation uses the result to partially exclude neighboring hypotheses. If A occupies the same parameter-space region as B, C, and D and yields low output, their prior estimates move downward. The amount depends on the modeled correlation between A and its neighbors. Nearby parameter settings are assumed to have related outcomes, with correlation declining by distance according to a domain-tuned length-scale parameter.

We initialize the covariance model from prior data loaded at loop startup and revise it as experiments accumulate. Without prior data, the covariance matrix uses a length-scale assumption based on parameter type: continuous variables receive a Gaussian kernel, while categorical variables use Hamming-distance weighting.

How the Loop Prunes

Pruning is straightforward. When a hypothesis's posterior probability drops below a threshold, typically 5 percent of the current tree's maximum, we mark its subtree dormant. We do not delete dormant subtrees: later evidence can reactivate them, and a researcher can also reactivate one manually when the model appears to be reading too much into the available data.

The 5 percent cutoff is configurable. During broad exploration, researchers often use 10 to 15 percent to retain more possibilities. In late-stage refinement, they may reduce it to 2 to 3 percent so the loop focuses on a narrow, high-probability region.

How the Loop Extends

Extension is the more informative operation. At each update, the hypothesis engine checks whether an unexplored parameter-space region warrants new leaf nodes. It examines the covariance function's gradient along current tree boundaries. A region next to a high-probability node qualifies when uncertainty remains high and expected value is plausible, prompting new leaves there.

This mechanism changes the tree's shape. It expands toward regions estimated to merit exploration and contracts away from areas ruled out or offering little expected information. After ten cycles, the tree differs from its form after two cycles. Its shape also records where the model considers the search space most productive.

Growth has a limit: each project can hold 2,000 active leaf nodes. This prevents an unmanageable computational structure. A tree nearing that cap usually indicates a sufficiently multivariate project that the researcher should narrow the parameter space using domain knowledge.

Leaf Nodes Are Experiments

The practical result of the probability model is a ranked set of leaves. Each leaf specifies an untested experimental condition, namely a parameter-value set. Ranking uses expected information gain: the anticipated reduction in tree-wide entropy if testing that leaf produces a result.

Expected information gain differs from expected yield and from the expected chance of a favorable result. A leaf in a poorly understood region may offer high information gain despite a low chance of the best outcome, because any result would resolve substantial uncertainty. The ranked list therefore is not merely a list of likely successes.

Where the Model Breaks Down

The hypothesis-tree model is effective when nearby parameter settings tend to produce related outcomes, giving the space a useful correlation structure. Many physical and chemical optimization problems have such a regime.

It can fail on highly multimodal landscapes, where the optimum lies in an isolated region without a gradient path, or near phase transitions, where small parameter changes cause discontinuous jumps. There, the covariance structure can misdirect exploration and keep it from approaching the true optimum.

We do not expect this model to fit every problem. In our experience, it works well for continuous optimization with 5 to 20 parameters. Below 5 parameters, a carefully designed factorial design often performs as well with less overhead. Above 20 parameters, covariance estimation degrades without substantial prior data. Much iterative optimization in materials science, process chemistry, and synthetic biology occupies this middle range.