A vocabulary for multi-agent automated research systems

We introduce a vocabulary for automated research systems built from one or more agents to make their design choices easier to describe and compare. The vocabulary specifies who the agents are, which operations the harness exposes, who may invoke them, how agents communicate, what state is visible within and across runs, how the next action is chosen, how a run begins, and how outputs are scored. A trajectory records one run from the input task to the returned artifact. Because agents, operations, and initialization may be stochastic, repeated runs on the same task induce a distribution over trajectories rather than a single behavior. The vocabulary assigns each structural design question, such as when agents should communicate, gain or lose a capability, or carry information across runs, to a distinct coordinate that can be varied on its own. It also makes the evaluator a component of the system, since reported gains depend on how closely the proxy score matches true quality. That separation also splits the vague complaint that these systems lack taste into two failures with different fixes. Generative taste is the rate at which a system proposes novel trajectories before any score is observed, and evaluative taste is the gap between the proxy score and the quality it should match. We instantiate the vocabulary on recent autoresearch systems to illustrate that it covers designs that differ widely in structure.