Part oneA cut is a claim about independence

Consider a problem as a set of decisions connected by constraints. A decision might be a route, a date, a calculation, a retrieved fact, a policy interpretation or a tool action. Draw one node for each decision and one edge whenever two decisions must agree, share a resource, change each other’s feasibility, or contribute jointly to the final objective.

A decomposition draws a boundary through that graph. Nodes on one side are sent to one solver; nodes on the other side go elsewhere. The tempting measure of success is whether each side looks simpler. The decisive measure is different: what information must cross the boundary for a locally correct answer to remain globally correct?

Some cuts expose genuine independence. Suppose six invoices can be checked against six unrelated purchase orders, and the final answer is the sum of six independently verified amounts. Each case has its own documents, rules and outcome. The aggregation operator is explicit addition. There is no shared credit limit, duplicate-payment condition or supplier-level exception. Parallel decomposition reduces context and elapsed time without changing the meaning of correctness.

Other cuts merely follow nouns in the request. “Flights”, “hotel” and “rail” sound like modules, but their choices are tied through time and location. “Identity”, “screening” and “payment setup” sound like separate onboarding workstreams, but they must refer to the same legal entity and jurisdiction. Surface categories are weak evidence for independence.

The small separator

The cleanest formal version uses variables and factors. Let the complete assignment be x. Each factor f_j scores or constrains a small subset of variables. The full objective is the sum of those factors:

Now divide the private variables into blocks B₁ to B_k, and place shared variables in a separator S. The split is exact when every factor belongs wholly to one block plus S, or wholly to S. Once S is fixed, the private blocks no longer interact. They may be solved separately and their scores combined.

If each variable has d possible values, brute-force search over all variables considers roughly d^|S|+Σ|Bᵢ| assignments. Separator-conditioned search considers approximately d^|S|Σd^|Bᵢ|. This is an illustrative enumeration bound, not a universal law for language-model latency. Its value is structural: a small separator can turn one exponential search into several smaller ones, while a wide separator restores much of the original difficulty.

Graphical-model research makes this idea precise through AND/OR search and width measures such as treewidth. The size of the useful decomposition is governed less by the number of named parts than by the dependencies that must remain connected.¹ The same intuition should discipline agent design, even when the variables are latent and expressed in prose.

The thought experiment reveals why task count is a poor proxy. The addition of one edge can change a clean parallel decomposition into a coordination problem. Conversely, moving one crucial variable into the interface can restore separability without forcing every detail back into a monolithic context.

flowchart LR
  A["The perfect itinerary that<br/>nobody can take"] --> B["Part one A cut is a claim<br/>about independence"]
  B --> C["Part two The interface carries<br/>the problem"]
  C --> D["Part three What language-model<br/>decomposition…"]
  D -. "re-test" .-> A
  classDef first fill:#0a1424,stroke:#7B4CD8,stroke-width:2px,color:#e6f3ff
  classDef step fill:#0a1424,stroke:#3f7da0,stroke-width:1px,color:#e6f3ff
  class A first
  class B,C,D step

Part twoThe interface carries the problem

An interface is often mistaken for a short hand-off note. That is too weak. In a valid decomposition, the interface is the information that makes the private blocks independent enough to solve. It carries the shared variables, invariants and update rules on which composition depends.

For the itinerary, an adequate interface might include traveller identity, permitted airports, earliest acceptable arrival, event location, total budget, fare flexibility, evidence timestamp and the rule for resolving conflicts. “Find a flight” is a subtask. “Find a flight satisfying interface version 7, and return arrival airport, time, price, fare conditions and evidence” is a composable contract.

A good interface is sufficient, typed and smaller than the problem it connects. Sufficiency means no omitted interaction can change the local optimum or invalidate recomposition. Typed means that units, entities, time semantics, confidence and authority are explicit. Smaller means the interface does not become a disguised copy of the entire global context. When every detail must cross, the proposed decomposition has found no useful independence.

Worked example: seven binary decisions

Take a deliberately small optimisation problem. The left block has three private yes-or-no decisions, l₁ to l₃. The right block has three, r₁ to r₃. Each private choice contributes a local benefit, totalling 18 when all six are selected. One shared binary variable s represents the coordination choice, perhaps “early” or “late”.

The left block earns six extra points when s is early. The right block earns eight when s is late. Solved independently, the left specialist correctly chooses early and the right specialist correctly chooses late. A naive merge rule that lets the left result win produces a score of 24. The globally best answer chooses late and scores 26.

The interface-aware solver does not ask the two specialists to guess s independently. It enumerates the two possible values of s, solves both private blocks conditional on each value, then compares the composed scores. It finds the same optimum as exhaustive search while evaluating 32 local candidates instead of 128 full assignments.

The example is small enough to inspect, but it contains the general failure. Local optimisation chooses different values for a shared variable. An aggregator cannot repair the disagreement by formatting both answers into one document. It must either search over the shared choice, apply an authorised decision rule, or escalate the conflict.

The benchmark also contains a harsher test. Add a factor that directly couples l₃ and r₃, but leave both variables private. The separator solver rejects the cut because a factor spans both private blocks. Rejection is a feature: an invalid decomposition should fail before execution, not after a polished answer has concealed the severed edge.

When subgoals undo each other

The classic Sussman anomaly gives the same lesson in action rather than optimisation. Three blocks must end with A on B and B on C. A planner that treats these as independent subgoals can achieve either one first, only to discover that completing the other requires undoing what it just achieved. The solutions must be interleaved.²

The anomaly matters beyond blocks. A compliance investigation may need to preserve an evidence state while resolving identity. A code agent may make a test pass by changing an interface that another package relies on. A financial plan may satisfy a monthly budget by using funds reserved for an annual obligation. In each case, actions inside one subproblem alter the preconditions of another.

This is why a static hand-off document is sometimes insufficient. The interface may need causal and temporal semantics: who may update a variable, which version another solver used, whether a value is observed or proposed, and what must be recomputed after change. The interface is a live boundary of shared truth, not a message copied between prompts.

Preserve option value when the separator is uncertain

A proposed interface can be wrong in two directions. An underspecified interface hides a dependency and permits incompatible local answers. An oversized interface copies almost the whole problem into every block, eliminating the intended gain. When evidence is incomplete, choosing the separator too early can also freeze assumptions that the task should have kept open.

Suppose the itinerary system has not yet resolved whether the traveller must attend the opening dinner. If it fixes “arrival before 18:00” as a global constraint, every specialist may optimise around an obligation that later proves optional. If it omits arrival from the interface, the rail and hotel choices can conflict. The safer design represents the condition explicitly as unresolved, allows workers to return conditional alternatives, and postpones the irreversible booking decision.

This suggests a staged pattern. First, parallelise reversible evidence gathering: retrieve schedules, fare rules, policies and availability. Second, construct or revise the shared separator from that evidence. Third, solve private blocks conditional on each material shared state. Finally, commit only after the global verifier selects a consistent branch. The pattern retains some parallelism without granting local workers authority to collapse unresolved global choices.

Late binding is often more valuable than maximum concurrency. It keeps option value where the coupling is uncertain and moves commitment to the point where enough evidence exists. In agent systems, this means distinguishing a local finding from a global decision, a candidate action from an authorised effect, and a provisional interface value from verified world state.

The design can still be economical. Conditional alternatives need not enumerate every imaginable world. They should cover the small set of separator values that materially change feasibility or consequence. If that set grows without bound, the task is signalling that the decomposition has not reduced the hard part. Merge the blocks, retrieve more evidence, or narrow the objective before proceeding.

This equation is a practitioner decision model, not a published universal metric. It prevents a common accounting error: celebrating shorter prompts inside each worker while ignoring the tokens, latency and failure surface added by planners, routers, shared state, retries, arbitration and judges.

Part threeWhat language-model decomposition actually buys

Language models can turn a broad instruction into a plausible sequence of smaller instructions with little engineering. That is useful, but linguistic plausibility is not structural validity. A list such as “research, analyse, draft, review” describes stages. It says almost nothing about which evidence each stage may use, which decisions are reversible, which outputs become inputs, or which constraints must remain global.

The strongest evidence for decomposition is therefore task-bounded. The BREAK benchmark formalised question decomposition as an ordered sequence of natural-language operations and released more than 83,000 question-decomposition pairs. Least-to-most prompting showed striking compositional generalisation on selected symbolic, mathematical and SCAN tasks by solving easier subproblems in sequence. Decomposed Prompting routed specialised subtasks to separate prompts, models or symbolic functions.⁴⁵⁶

Later work adds two important qualifications. ADaPT decomposes as needed when an executor cannot complete a subtask, rather than imposing maximum granularity from the start. TaskLAMA found that models could identify plausible steps in complex everyday tasks yet still struggled with pairwise temporal dependencies. Recursive Decomposition with Dependencies explicitly represents ordered subtask relations and error recovery, reporting compute-matched gains as tested problems become more complex.⁷⁸⁹

These results support a mechanism more modest than “models reason better when told to think step by step”. A split can reduce the number of alternatives considered at once, lower the length and heterogeneity of local context, permit a specialised solver, expose intermediate states to tests, and confine recovery to one branch. Kang and colleagues connect some empirical gains from chain and tree structures to lower-complexity intermediate steps, while Dechter and Mateescu’s earlier AND/OR account explains why exposed independence can shrink a search space.¹⁰¹

Long-context work makes the boundary especially clear. Xu and colleagues separate three sources of error: dependence between chunks, confusion that grows with context length, and imperfect aggregation. Chunking helps when reducing model confusion outweighs the information lost at chunk boundaries and the error introduced during merge. It can fail when the question requires strong cross-chunk relations.¹¹

Task-Decoupled Planning applies a related idea to long-horizon agents: a supervisor constructs a dependency graph, while planners and executors receive scoped context for the active subtask. The paper reports gains on its selected benchmarks and token reductions of up to 82 per cent, but it also leaves decomposition granularity as an open design problem and focuses on tasks with verifiable objectives.¹² These boundaries matter. A method that isolates recoverable subgoals in a simulator has not thereby established that free-form strategic synthesis should be divided the same way.

Thought experiment 2: change only the question

Imagine a 400-page case file divided into forty chunks. In version A, the task is to count pages containing a signed declaration. Each chunk can be checked independently, and the counts add. In version B, the task is to determine whether the person named in the first declaration is the same legal entity referenced under aliases, subsidiaries and historic addresses elsewhere.

The document and chunk boundaries are unchanged. Only the query changes. Version A has an almost empty interface: the local count and evidence page. Version B requires cross-chunk identity state, temporal validity, alias resolution and contradiction handling. A map-reduce pattern that excels in A can destroy the decisive relation in B.

The negative control: embarrassingly parallel work

A serious account must include a case where the preferred machinery is unnecessary. Return to six unrelated invoice checks with no shared supplier cap, duplicate detection or portfolio rule. A direct parallel split and an addition step are sufficient. Building a global dependency graph, shared state service and arbitration agent would add failure modes without changing the answer.

This negative control distinguishes two explanations. Perhaps decomposition helps because a sophisticated dependency-aware architecture is intrinsically superior. Or perhaps it helps because the task already contains independent units and any reasonable parallel executor can exploit them. In the control, the second explanation wins. The executable benchmark reproduces this: both naive and separator-aware splits reach the optimum with 16 evaluations rather than 64.

The control yields a disciplined baseline: start with the simplest split that matches known independence, then add interface machinery only when injected interactions make the simple baseline fail. Architectural richness must earn its cost through a detected dependency, a recovery requirement or a consequence that justifies stronger assurance.

There is also a credible competing baseline: a sufficiently capable monolithic model may preserve interactions implicitly and produce a better integrated answer than a brittle pipeline. PlanBench cautions that planning capability itself should be tested rather than inferred from fluent plans.¹³ The correct comparison is therefore empirical: same task set, same evidence, controlled compute, and measures for final correctness, dependency violations, cost and recovery.

stateDiagram-v2
  [*] --> Claim
  state "The small separator" as Claim
  state "Worked example, seven binary<br/>decisions" as Test
  state "When subgoals undo each other" as Revision
  Claim --> Test: expose mechanism
  Test --> Decision: evidence survives
  Test --> Revision: rival survives
  Revision --> Claim: narrow or rebuild
  Decision --> [*]

Part fourRelease the split only after it survives recomposition

A decomposition should be treated like a design change with a release gate. The system is not ready because the subtasks are clear or the agents return valid JSON. It is ready when the proposed boundary preserves the global invariants, conflicts are observable, and injected dependency changes produce the expected recomputation or rejection.

Production-shaped worked example: supplier onboarding

Consider a synthetic onboarding case for Meridian Field Services Ltd. The requested outcome is permission to create a supplier record and enable payment. The work appears naturally divisible: resolve the organisation, screen relevant parties, review the contract, verify the bank account, validate tax details, and prepare the final case summary.

A topic-based design assigns one worker to each lane. The identity worker resolves a registered company. The screening worker finds no relevant match for the name provided. The contract worker confirms that required clauses are present. The bank worker verifies that the account exists. The tax worker validates the supplied registration number. Every lane can produce a locally supported “pass”.

The global case may still be wrong. The screening result could belong to the trading name rather than the resolved legal entity. The contract might name a parent company while the account belongs to a subsidiary. A beneficial owner discovered after screening might never enter the screening lane. The tax registration may have been valid on its source date but inactive at the decision cut-off. These are not five independent quality defects. They are consequences of failing to preserve identity and time across the cut.

The first shared object should therefore be a typed case interface, not a prose summary. It contains the resolved legal-entity identifier, jurisdiction, aliases, controlling parties, contract counterparty, proposed account holder, evidence cut-off, source provenance, unresolved contradictions and current authority state. Each lane reads a version and returns both its finding and the interface fields on which that finding depends.

If the legal entity changes, the system invalidates dependent findings rather than merely appending an update to memory. If a beneficial owner is added, screening becomes stale by rule. If the account holder differs from the contract party, the merge cannot average the two conclusions. It must resolve the relationship or escalate. The decomposition becomes safe because dependency changes have operational consequences.

This design permits useful concurrency without pretending that all work is independent. Document extraction can begin immediately. Screening may start with provisional identities but cannot become final until the identity anchor is accepted. Contract and account checks can proceed in parallel while their conclusions remain conditional. The final release gate checks cross-lane invariants and records which evidence and interface version authorised the outcome.

For a deeper treatment of planning semantics, see Planning Before Action. The applied boundary between evidence, calculation, policy, narrative and accountable decision is developed in Decision Decomposition in Commercial Credit. The narrower claim here is that neither hierarchy nor specialisation is enough unless the cut preserves interaction.

The decomposition release record

Before implementation, complete one record for the proposed split. It is an architecture decision instrument and a test specification. Empty fields are not documentation debt. They are evidence that the cut is not yet understood.

The most revealing test is dependency mutation. Change one crossing variable after local work completes. A correct system identifies every affected output, withdraws stale conclusions and recomputes only the necessary branches. A weak system leaves the old conclusions intact or asks a language model to decide whether they “still look reasonable”.

A second test injects incompatible local optima. The expected outcome is a visible conflict, not an elegant compromise. A third hides a cross-edge that the declared separator does not cover. The decomposition validator should reject the topology. These tests convert the fracture map from documentation into operating evidence.

How to test a cut when the dependencies are latent

The factor model is exact because every variable and interaction is declared. Natural-language work is harder. The system may not know that two mentions refer to one entity, that a “reasonable” local choice consumes a shared resource, or that a later action invalidates an earlier conclusion. A decomposition cannot wait for a perfect graph, but it can test whether its current graph is adequate.

Begin with an edge-perturbation catalogue. For every shared noun, number, identity, deadline, resource, policy condition and irreversible action, ask what would change if its value changed. Do not ask only whether two blocks exchange data. Ask whether a change in block A could alter the correctness, feasibility or authority of block B. The answer identifies a dependency even when no explicit message currently carries it.

Turn each candidate edge into a paired test. Hold the task constant, vary one interaction, and predict the system response before execution. If the arrival time moves beyond the last connection, the rail choice must become stale. If a legal-entity identifier changes, screening and account-holder checks must be withdrawn. If two document chunks switch from independent counting to cross-chunk identity resolution, the topology must change or the system must declare that its current split is insufficient.

Three fault families are particularly discriminating. A deletion test removes an edge from the declared interface while retaining the underlying constraint; the system should reject the cut or fail a global verifier. A mutation test changes a separator value after local work; every dependent conclusion should be identified, invalidated and selectively recomputed. A collision test gives two blocks incompatible local optima; the merge should expose a conflict and invoke a named rule or human authority. A fluent compromise is a failed test.

The composition oracle must be independent of the workers whose outputs it judges. Depending on the domain, it may be an exact constraint checker, a simulator, a database readback, a property test, a policy engine, a second evidence path or accountable human review. Another language model can help generate adversarial cases, but a correlated model judgement is weak evidence that the original model preserved an omitted relation.

Measure more than final accuracy. Record the rate at which every local output passes while the global result fails. That is the signature of a bad cut. Track dependency-edge recall, stale-output invalidation recall, false invalidations, separator width, coordination tokens, elapsed time, number of recomputed blocks and human interventions. Compare these with a coupled baseline under the same evidence and approximate compute budget. A decomposition that raises local scores while lowering verified outcome quality has optimised the wrong level.

Granularity should remain revisable. When a block repeatedly fails because its internal search is too broad, split it around a newly discovered separator. When two blocks exchange a growing interface, invalidate each other frequently or require constant arbitration, merge them. This gives decomposition a feedback loop: topology changes in response to observed coupling rather than remaining fixed because an initial prompt produced an attractive plan.

High-consequence tasks require a stricter stopping rule. If an edge could authorise payment, deny service, alter a legal record or create an irreversible external effect, uncertainty about that edge should narrow authority. Local workers may prepare evidence, but the action remains blocked until the shared state and postconditions are verified. Unknown independence is not independence; it is unresolved model risk at the system boundary.

Executable artefact: a dependency-aware decomposition benchmark

The following Python programme implements the seven-variable example. It compares exhaustive search, separator-conditioned search and independent local solving followed by an unsafe merge. It also verifies the negative control and rejects a hidden cross-block factor. The code uses only the Python standard library and deterministic synthetic scores.

from dataclasses import dataclass
from itertools import product
from typing import Dict, Iterable, Mapping, Sequence, Tuple


@dataclass(frozen=True)
class Factor:
    name: str
    scope: Tuple[str, ...]
    table: Mapping[Tuple[int, ...], int]

    def value(self, assignment: Mapping[str, int]) -> int:
        return self.table[tuple(assignment[v] for v in self.scope)]


@dataclass(frozen=True)
class Result:
    score: int
    assignment: Dict[str, int]
    evaluations: int


class InvalidCut(ValueError):
    pass


def all_assignments(variables: Sequence[str]) -> Iterable[Dict[str, int]]:
    for bits in product((0, 1), repeat=len(variables)):
        yield dict(zip(variables, bits))


def total_score(assignment: Mapping[str, int], factors: Sequence[Factor]) -> int:
    return sum(factor.value(assignment) for factor in factors)


def solve_exact(variables, factors, fixed=None) -> Result:
    fixed = dict(fixed or {})
    free = [variable for variable in variables if variable not in fixed]
    best_score = -10**9
    best_assignment = {}
    evaluations = 0

    for candidate in all_assignments(free):
        assignment = fixed | candidate
        current = total_score(assignment, factors)
        evaluations += 1
        key = tuple(assignment[v] for v in variables)
        old_key = tuple(best_assignment.get(v, -1) for v in variables)
        if current > best_score or (current == best_score and key > old_key):
            best_score, best_assignment = current, assignment

    return Result(best_score, best_assignment, evaluations)


def classify_factors(left, right, separator, factors):
    left_set, right_set, sep_set = set(left), set(right), set(separator)
    if left_set & right_set or (left_set | right_set) & sep_set:
        raise InvalidCut("Blocks and separator must be disjoint.")

    left_factors, right_factors, sep_factors, crossing = [], [], [], []
    for factor in factors:
        scope = set(factor.scope)
        if scope <= sep_set:
            sep_factors.append(factor)
        elif scope <= left_set | sep_set:
            left_factors.append(factor)
        elif scope <= right_set | sep_set:
            right_factors.append(factor)
        else:
            crossing.append(factor)

    if crossing:
        names = ", ".join(f"{f.name}{f.scope}" for f in crossing)
        raise InvalidCut(
            "Cut severs factors not represented by the separator: " + names
        )
    return left_factors, right_factors, sep_factors


def solve_with_separator(left, right, separator, factors) -> Result:
    left_factors, right_factors, sep_factors = classify_factors(
        left, right, separator, factors
    )
    best_score, best_assignment, evaluations = -10**9, {}, 0

    for shared in all_assignments(separator):
        left_result = solve_exact([*left, *separator], left_factors, shared)
        right_result = solve_exact([*right, *separator], right_factors, shared)
        evaluations += left_result.evaluations + right_result.evaluations
        assignment = (
            shared
            | {v: left_result.assignment[v] for v in left}
            | {v: right_result.assignment[v] for v in right}
        )
        current = (
            left_result.score
            + right_result.score
            + total_score(shared, sep_factors)
        )
        if current > best_score:
            best_score, best_assignment = current, assignment

    return Result(best_score, best_assignment, evaluations)


def solve_naively(left, right, separator, factors):
    left_factors, right_factors, _ = classify_factors(
        left, right, separator, factors
    )
    left_result = solve_exact([*left, *separator], left_factors)
    right_result = solve_exact([*right, *separator], right_factors)
    disagreement = {
        variable: (left_result.assignment[variable], right_result.assignment[variable])
        for variable in separator
        if left_result.assignment[variable] != right_result.assignment[variable]
    }
    # Unsafe deterministic merge: the left solver wins shared-variable conflicts.
    merged = (
        {v: left_result.assignment[v] for v in [*left, *separator]}
        | {v: right_result.assignment[v] for v in right}
    )
    return total_score(merged, factors), disagreement, (
        left_result.evaluations + right_result.evaluations
    )


def unary(name, variable, reward):
    return Factor(name, (variable,), {(0,): 0, (1,): reward})


def contextual_preference(name, private, shared, preferred, reward):
    return Factor(
        name,
        (private, shared),
        {(p, s): reward if s == preferred else 0
         for p, s in product((0, 1), repeat=2)},
    )


def equal_bonus(name, first, second, reward):
    return Factor(
        name,
        (first, second),
        {(a, b): reward if a == b else 0
         for a, b in product((0, 1), repeat=2)},
    )


LEFT = ("l1", "l2", "l3")
RIGHT = ("r1", "r2", "r3")
BASE = [
    unary("l1-benefit", "l1", 3), unary("l2-benefit", "l2", 2),
    unary("l3-benefit", "l3", 4), unary("r1-benefit", "r1", 2),
    unary("r2-benefit", "r2", 3), unary("r3-benefit", "r3", 4),
]
SHARED = BASE + [
    contextual_preference("left-prefers-early", "l1", "s", 0, 6),
    contextual_preference("right-prefers-late", "r1", "s", 1, 8),
]

print("CASE 1: separable")
print("monolithic", solve_exact([*LEFT, *RIGHT], BASE))
print("separator ", solve_with_separator(LEFT, RIGHT, (), BASE))
print("naive     ", solve_naively(LEFT, RIGHT, (), BASE))

print("\nCASE 2: shared interface")
print("monolithic", solve_exact([*LEFT, "s", *RIGHT], SHARED))
print("separator ", solve_with_separator(LEFT, RIGHT, ("s",), SHARED))
print("naive     ", solve_naively(LEFT, RIGHT, ("s",), SHARED))

print("\nCASE 3: hidden cross-edge")
HIDDEN = SHARED + [equal_bonus("late-coupling", "l3", "r3", 20)]
try:
    solve_with_separator(LEFT, RIGHT, ("s",), HIDDEN)
except InvalidCut as error:
    print("REJECTED:", error)

CASE 1: separable
monolithic Result(score=18, ..., evaluations=64)
separator  Result(score=18, ..., evaluations=16)
naive      (18, {}, 16)

CASE 2: shared interface
monolithic Result(score=26, ..., evaluations=128)
separator  Result(score=26, ..., evaluations=32)
naive      (24, {'s': (0, 1)}, 32)

CASE 3: hidden cross-edge
REJECTED: Cut severs factors not represented by the separator:
late-coupling('l3', 'r3')

The programme’s most important line is not the optimiser. It is the validation branch that identifies a factor whose scope crosses both private blocks. Real tasks rarely provide complete factor tables, so the equivalent engineering work is to discover edges through domain modelling, traces, counterexamples and failure injection. The benchmark makes the obligation visible: unknown coupling must remain a risk, not be assumed away.