The size illusion

A question can be profound, urgent and intellectually fertile while still being a bad unit of work. Research becomes cumulative only when uncertainty is connected to an intervention that can reorganise what comes next.

A topic is not a question, and a question is not a loop

Machine consciousness is a topic. Can machines be conscious? is a question. Neither tells a researcher what to build next Monday. A research loop adds five commitments: the candidate system, live rival explanations, a controlled difference, an observable outcome and a decision that follows from each result.

John Platt's proposal of strong inference emphasised alternative hypotheses, experiments that discriminate among them and repeated refinement (Platt, 1964). Modern Bayesian experimental design formalises a related intuition: choose experiments expected to provide the most useful information under a model, while acknowledging that the calculation can be difficult and sensitive to modelling assumptions (Rainforth et al., 2024). Neither framework says that nature always offers one clean crucial experiment. Auxiliary assumptions, noisy measurements and theory underdetermination remain. Their durable lesson is narrower: progress accelerates when alternatives are made to risk different predictions.

A good problem can fail cleanly

Imagine two proposals about artificial understanding. The first promises to collect increasingly impressive demonstrations until the community is persuaded. The second names a mechanism, constructs a matched system that can reproduce the same baseline behaviour without that mechanism, and states what observation would lower confidence in the preferred account. Both may begin from the same philosophical concern. Only the second has located where disappointment can land.

A clean failure has three properties. It is local, because the result tells us whether the mechanism, measurement bridge or intervention failed rather than merely announcing that the whole topic remains difficult. It is sensitive, because the experiment includes checks showing that a difference of the predicted size could have been detected. It has an aftermath, because the failed claim leaves a test harness, a calibrated measure or a better rival that another study can use. Failure without these properties often becomes a story about insufficient scale, an uncooperative dataset or a future model that might behave differently.

The grand question still has a role. It supplies the horizon that tells us why the local result matters and which neighbouring loop should follow. It should not be asked to serve simultaneously as hypothesis, method and verdict. Ambition belongs in the dependency map; accountability belongs in the next complete loop. This separation lets a researcher pursue a question whose final answer may remain distant while making every year answerable to evidence.

A career frontier does not require a known final answer. It requires a sequence of experiments whose failure modes are informative. This is why a narrow study can belong to a large life, while a grand slogan can consume one. Depth comes from the number and quality of causal loops a question can support, not from the altitude of its nouns.

flowchart LR
  A["The two envelopes"] --> B["The size illusion"]
  B --> C["A problem must separate, start<br/>and matter"]
  C --> D["The frontier after the gates"]
  D -. "re-test" .-> A
  classDef first fill:#0a1424,stroke:#087A98,stroke-width:2px,color:#e6f3ff
  classDef step fill:#0a1424,stroke:#3f7da0,stroke-width:1px,color:#e6f3ff
  class A first
  class B,C,D step

A problem must separate, start and matter

The selection rule has three non-compensatory gates. A spectacular score on one dimension cannot rescue a zero on another, because each missing gate breaks the learning loop in a different place.

Gate one: discrimination

Discrimination is not the ability to measure something precisely. It is the ability to make live explanations risk different outcomes. An additional decimal place on a benchmark may improve measurement while leaving every causal story intact. A coarse intervention can be more valuable if one mechanism predicts a sign reversal and another predicts stability.

Write the rival set before choosing the instrument. For a long-horizon agent, one hypothesis may say failure comes from insufficient model capability. Another may locate failure in stale world state. A third may blame unverified effects after tool calls. Giving the model more tokens does not separate these accounts. Injecting a stale state while holding model, prompt and tools fixed does.

A useful formal compression is:

D = expected separation among rival predictions × intervention fidelity × transport stability.

Each term refers to a physical or computational condition. Expected separation asks how far outcomes diverge. Intervention fidelity asks whether only the intended cause changed. Transport stability asks whether the effect survives reasonable changes of seed, task and implementation. The product is not an objective law. It prevents one impressive plot from hiding a weak intervention or a single-environment artefact.

The rival is part of the apparatus

A weak rival is easy to defeat and therefore teaches little. Calling a system “mere pattern matching” does not create a rival unless that description is implemented well enough to reproduce the target observation. The serious rival generator should match ordinary performance, interface, resource budget and evaluation exposure while differing in the causal property under dispute. It functions like a control condition that can think back.

This requirement prevents a common mistake in frontier research: comparing a carefully engineered preferred system with a deliberately impoverished baseline, then attributing every gain to the favoured mechanism. Strong rivals may share most of the architecture and differ at only one intervention point. They may also expose that all current hypotheses rely on the same shortcut. When that happens, the correct response is not to declare a tie. It is to expand the rival set and redesign the intervention. The strength of a mechanism claim is bounded by the strongest alternative that survived the same test.

Gate two: tractability

Tractability does not mean easy, cheap or guaranteed to publish. It means the first complete loop can finish with present resources. A complete loop includes the rival generator, intervention, measurement, analysis and release of the artefact. “Train a frontier model and inspect what emerges” is not tractable for most researchers. “Plant a known circuit in three small transformers, recover it with competing methods, then test necessity, sufficiency and invariance” may be.

Every difficult programme has dependencies: a testbed, a measurement bridge, access to interventions, a safety protocol or a source of ground truth. Treat each dependency as a mountain pass. A pass is acceptable when crossing it yields an artefact useful even if the final question remains open. A causal chamber, a state-swap harness or a typed candidate dossier can support many later studies.

Gate three: consequence

A result matters when it changes a decision. The decision need not be commercial. It may concern which theory to invest in, which architecture to build, which benchmark to retire, which welfare safeguard to apply or which human capability must remain practised. State the decision branches before seeing the data.

Consider an interpretability method. A positive result may permit its use as one item of release evidence. A negative result may prohibit that use while preserving it as an exploratory tool. A null result may show that the chosen behaviour was not mediated by the proposed circuit. All three can matter. By contrast, a study that merely says one representation “looks more semantic” may change no action under any result.

Design the null branch before seeing it

A null result can mean that the preferred mechanism is absent, that the intervention did not reach it, that the measure was insensitive, that two effects cancelled, or that the rival predictions were never far enough apart. These possibilities cannot be sorted after the fact by adding whichever explanation protects the favoured view. Before collection, specify a manipulation check, the smallest separation worth detecting, the noise model, the stopping rule and the auxiliary assumption most likely to break.

A well-designed null does not automatically settle the rivalry. It does something more disciplined: it lowers confidence in a declared claim by a declared amount, or redirects the programme to a named dependency. For the state-swap example, an unchanged outcome with verified state replacement and adequate power weakens the claim that the state is causally necessary. An unchanged outcome with failed state readback says little about the mechanism but clearly identifies the tooling failure. A null branch is consequential only when it has been given a job before the data arrive.

The frontier after the gates

The following are not declarations that six fields are “the most important”. They are examples of problem statements that can be sharpened into discriminating loops with credible first builds and material consequences.

The frontiers share instruments, not answers

These six agendas are connected because they need many of the same research assets. A configured candidate family makes system boundaries explicit. A typed intervention harness records what changed and what was held fixed. A rival generator tests whether the observation is diagnostic. An evidence ledger separates measured behaviour, mechanism inference, welfare implication and metaphysical interpretation. Building any one of these well can lower the cost of several later programmes.

For example, an aligned checkpoint apparatus built for agent recovery can also test claims about temporal continuity, responsibility after interruption and which state belongs inside the governed system. A causal testbed built for world-model evaluation can provide known interventions for interpretability methods. A retained-skill protocol can reveal whether a human-machine team improved joint output by learning, by temporary scaffolding or by replacing the human contribution. The instrument travels; the conclusion does not. A recovery result is not evidence of consciousness, and a causal representation is not automatically a world model suitable for action.

This gives a practical portfolio rule. Prefer early assets that support several discriminating loops while keeping each claim typed. Shared infrastructure should increase experimental leverage, not manufacture conceptual equivalence. When two fields use the same word, such as state, attention or value, require a separate account of the objects, relations and interventions before transferring a result.

stateDiagram-v2
  [*] --> Claim
  state "A topic is not a question, and<br/>a question is not a loop" as Claim
  state "A good problem can fail cleanly" as Test
  state "The same answer from two machines" as Revision
  Claim --> Test: expose mechanism
  Test --> Decision: evidence survives
  Test --> Revision: rival survives
  Revision --> Claim: narrow or rebuild
  Decision --> [*]

Build a decade from one complete year

A career-sized problem should not require faith that year ten will redeem years one to nine. It should contain a ladder of complete loops whose artefacts reduce the cost and ambiguity of the next loop.

Separate problem quality from career fit

The scorecard evaluates a problem statement and its first loop. A career decision also depends on access, mentorship, health, financial runway, team norms and the freedom to publish negative results. These factors should not be hidden inside the tractability number, because doing so makes disagreement impossible to diagnose. A technically tractable experiment may be inaccessible to one researcher and straightforward for another; that changes timing and route, not the intrinsic importance of the question.

Use two sheets. The first records the three gates, evidence and decision branches. The second records the environment required to execute them: equipment, data rights, collaborators, safety review, compute, protected time and an exit option. A mismatch between the sheets suggests a dependency pass, a different host institution or a smaller first loop. It does not require abandoning the horizon. A durable programme joins a good causal question to an environment that permits honest failure.

Start by designing the first twelve months, not the final theory. The first loop should create a candidate family and a rival generator. The second should test replication and sensitivity. The third should cross a model family, environment or institution. Only later should the programme attempt a field-level decision or governance integration.

This ordering protects against two opposite failures. One is premature scale: spending the first years building infrastructure before knowing which contrast matters. The other is toy imprisonment: producing endless demonstrations that never face a realistic configured system. Each loop must therefore add one dependency and one external challenge while preserving the causal question.

Build an asset ladder, not a publication queue

Papers report results, but assets change the cost of the next result. In year one, the asset might be a state-swap harness with selective intervention checks. In year two, it may become a benchmark containing matched rival generators and preregistered failure injections. In year three, an independent group may reuse the benchmark on another model family and expose a boundary condition. The publication sequence matters because each paper is challengeable; the asset ladder matters because each loop becomes less dependent on private judgement.

At the end of every loop, ask what an unfriendly but competent team could reuse without accepting your interpretation. Code, schemas, calibrated measures, negative controls, synthetic data generators and decision receipts usually survive disagreement better than a new label for the phenomenon. If the only inheritance is a persuasive narrative, the programme may accumulate attention without accumulating experimental power.

How to apply the instrument

Begin with evidence, not enthusiasm. List the strongest source for each factual claim and one rival interpretation. Score the problem statement, not the field. Use confidence to penalise guesses. Then inspect the weakest gate. A low discrimination score calls for sharper rivals or a better intervention. Low tractability calls for a smaller candidate family or a dependency pass. Low consequence calls for explicit decision branches.

A positive result permits commitment only within the stated boundary. A negative result means the current mechanism or design should lose support. A null result may reveal inadequate sensitivity, but that explanation must itself be testable. The tool cannot establish that a problem will be fashionable, funded, personally meaningful or ultimately true.

function lowerBound(score, confidence) {
  const uncertaintyPenalty = 0.35 * (5 - confidence);
  return Math.max(0, score - uncertaintyPenalty);
}

function evaluateProblem(problem) {
  const lower = {
    discrimination: lowerBound(problem.discrimination.score,
                               problem.discrimination.confidence),
    tractability: lowerBound(problem.tractability.score,
                             problem.tractability.confidence),
    consequence: lowerBound(problem.consequence.score,
                            problem.consequence.confidence)
  };

  const weakest = Object.entries(lower)
    .sort((a, b) => a[1] - b[1])[0];
  const threshold = 2.5;
  const recommendation = weakest[1] < threshold
    ? "rewrite"
    : weakest[1] < 3.4 ? "pilot" : "bounded-commitment";

  return { lower, weakestGate: weakest[0], recommendation };
}

sequenceDiagram
  participant Q as Question
  participant M as Mechanism
  participant E as Evidence
  participant D as Decision
  Q->>M: The two envelopes
  M->>E: A topic is not a question, and a question is not a loop
  E-->>M: compare rival explanations
  E->>D: Choose the problem that compounds when you are wrong
  D-->>Q: revise scope or act

Choose the problem that compounds when you are wrong

The familiar career decision is “Which big question do I care about most?” Keep the care, but change the decision procedure. Ask which version of that question can now support a rival set, a selective intervention, a complete first loop and a result-dependent decision.

Do not reject a field because its final question is unresolved. Rewrite the problem until the first loop is decisive at its own scale. Do not accept a problem because its consequences are enormous. High stakes cannot substitute for a measurement bridge. Do not confuse a convenient benchmark with a useful decision. A result that moves no action may still be descriptive science, but it is not yet the selection instrument for a career.

The practical choice is simple. Before committing, produce one page containing: the candidate boundary, two serious rivals, the intervention, the smallest build, the positive branch, the negative branch, the null branch, the reusable asset and the strongest reason the whole design may fail. Score the three gates conservatively. Rewrite the weakest one.

Choose the problem that can keep teaching you when your preferred hypothesis loses. That is what makes an open question durable enough for a research career: not a promise of final victory, but a sequence of honest experiments that leave the world, the field and the next researcher with better choices.