Bellman’s target is a bounded or unbounded sequence of operations performed to achieve a result. Each operation may produce a stochastic event whose outcome informs what should happen next. Two objectives frame the field: maximise yield in a given time, or minimise the time or cost required to accomplish a task.

A naïve planner lists every possible complete sequence. If there are four choices at each of ten stages, that means more than a million fixed sequences before uncertainty branches the tree further. Bellman asks a different question: what is the best operation first, assuming the continuation will also be chosen optimally once its new state is known?

The value of a first operation has two parts. One is immediate return or cost. The other is the value of the remaining problem after the state changes. Compare those combined values across first operations. The winning first step, joined to an optimal continuation, constitutes an optimal sequence under the model.

This is not greed. A greedy rule ranks only immediate gains. Bellman’s comparison includes downstream value, so an operation with a smaller present reward can win because it leaves a much better state. The local decision is globally informed through the value function.

`p` is the present state. `k` indexes the operation selected now. `gₖ(p)` is its immediate contribution. `Sₖp` is the point transformation that produces the next state. `f(Sₖp)` is the best value attainable from that remainder. `hₖ(p)` scales continuation.

The minimum and maximum versions share the same structure. Costs invite a minimum; yields invite a maximum. Switching the word without switching signs and interpretation causes silent errors. A negative reward can convert one form to the other, but the accounting must remain consistent.

The operator notation is more general than a table of stages. It permits continuous states, stochastic belief states and unbounded horizons. It also makes assumptions visible. If `Sₖp` leaves the valid region, the recursion is not closed. If immediate terms are unbounded or continuation is not controlled, a bounded solution need not exist.

The apparent self-reference is productive. `f` appears on both sides because the same decision problem returns after the first operation at a new state. The equation is not a verbal circle; it is a fixed-point condition. Bellman’s theorems specify cases in which exactly one bounded function satisfies it.

Bellman writes the continuation as `f(Sₖp)`, not as a function of the entire path. This is a substantive modelling commitment. Two histories that produce the same state `p` are treated as equivalent for all remaining decisions. If their future probabilities or returns differ, the state is too small.

In a fully observed physical problem, position, velocity and remaining resource may suffice. Under uncertainty, the state may be a probability vector. A failed search does not merely advance time; it changes beliefs about where the hidden ball lies. The posterior vector becomes the next state, making the stochastic past usable without retaining every observation separately.

State compression is therefore conditional independence engineered for decisions. Given the state, earlier history should add no decision-relevant information about the modelled future. This does not require the world itself to forget. It requires the state representation to preserve whatever the optimisation needs.

When recursion fails, enlarging the state is often the repair. Add the last action if transition costs depend on switching. Add accumulated damage if failure risk depends on usage. Add belief parameters if observations are partial. Dynamic programming does not erase memory; it identifies the memory that must travel forward.

Consider `f(p)=maxₖ[gₖ(p)+hₖ(p)f(Sₖp)]` on a region `R`. The transformed state must remain in `R`. Immediate terms are bounded. Continuation multipliers satisfy `|hₖ(p)|≤c₂<1`, and the theorem assumes non-negative `gₖ` and `hₖ`. Under these conditions, Bellman states that there is a unique bounded solution.

Why does the strict bound matter? Take two candidate value functions `u` and `v`. Immediate terms cancel when the same action is compared. For each action, continuation differences are at most `c₂‖u−v‖∞`. Taking a maximum cannot enlarge that worst-case bound. The Bellman operator therefore brings functions closer by at least a uniform factor below one.

This sup-norm explanation uses modern fixed-point language; the paper states the theorem and says its proof uses successive approximations. The reconstruction is faithful to the assumptions but should not be mistaken for wording on the page. It makes the role of `c₂` memorable: future disagreement is damped every time the recursion moves one step.

Remove strict contraction and the conclusion can fail. With continuation multiplier one and zero immediate return, every constant function solves `f=f`; uniqueness vanishes. With positive immediate return, `f=1+f` has no finite solution. A recursive form alone is not a licence to iterate indefinitely.

Choose an initial bounded function, often zero. Define `f₁=T(f₀)`, then `f₂=T(f₁)` and continue. With zero initial continuation, the first update values one operation. The second values an operation plus the best value represented by one further update. In a finite-horizon interpretation, the horizon grows one step at a time.

Under contraction, the functions converge to the unique bounded fixed point regardless of the bounded initial guess. This yields both a constructive procedure and an existence argument. It also gives an error certificate: the residual or difference between successive functions can bound remaining error when the contraction factor is known.

Iteration is not always computationally cheap. A continuous state region requires representation or approximation. A large discrete state space can be impossible to enumerate. The 1952 theorem establishes mathematical convergence under assumptions; it does not abolish the cost of applying the operator.

Nor is every horizon extension semantically safe. If a terminal value hides an omitted consequence, early iterates optimise the wrong truncated objective. If the continuation factor encodes discount or survival, changing it changes the problem rather than merely accelerating a solver. Approximation and modelling choices must be kept separate.

Let `pₖ` be the probability that box `k` contains the ball. Let `qₖ` be the probability that an examination of that box fails to reveal its contents. Let `tₖ` be the time required for one attempt. Choosing a box incurs time, may obtain information and changes what should be tried next.

If box `k` is examined and the ball is not obtained, two possibilities remain: the ball was elsewhere, or it was in box `k` but the attempt failed. The probability of this failure event is `1−pₖ(1−qₖ)`. Bayes’s rule shrinks the mass on box `k` by factor `qₖ` and renormalises every component by that event probability.

The posterior vector is the transformed state `Sₖp`. It retains exactly the information needed to compare the next examination under the model. Reusing original priors after failure would discard evidence and break the recursion’s meaning.

A failed test can raise every alternative probability while leaving residual mass on the tested box. When `qₖ=0`, failure rules that box out. When `qₖ` is near one, failure says little. The action’s informational value and chance of obtaining the ball are coupled through the same observation.

The numerator `pₖ(1−qₖ)` is the probability that one attempt at box `k` obtains the ball: the ball must be there and the examination must succeed. Dividing by `tₖ` prices that chance by time. The index compares immediate acquisition rates, but its optimality is embedded in the recursive posterior update, not asserted as a free-standing heuristic.

The ranking can reverse after failure. The tested box loses posterior mass when `qₖ<1`, while the untested boxes gain relative mass. Recompute the index from the new belief vector. A policy is a state-dependent rule, not a fixed permutation chosen once.

Bellman distinguishes obtaining the ball from merely locating its box. If the task ends only when the ball is physically obtained, repeated attempts at an unreliable box may be required. If the task ends when location is known, elimination can finish the problem without opening the last box. Bellman says the latter solution is more complicated: the box with the largest ratio is either examined first or never examined.

This objective dependence is a warning for modern rankings. An index can be exact for one stopping condition and wrong for a neighbouring question. Before borrowing a policy, preserve what counts as success, what an observation reveals and when the process terminates.

Start with `x>0` and choose `y` for one use, leaving `x−y` for another. Immediate returns are `g(y)` and `h(x−y)`. The remaining resource is `ay+b(x−y)`, where `0<a,b<1`. The value equation is:

Every allocation changes both present return and future capacity. The coefficients `a` and `b` encode retention. A use with a smaller current payoff may preserve more of the resource. Once again, the first allocation is chosen by including its transformed remainder.

Bellman’s Theorem 2 treats a more general allocation among `N` non-negative parts summing to `x`. Continuity, non-negativity, contraction of the remaining resource and a summability condition yield a unique continuous solution with zero value at zero. The details matter: compact allocation sets and shrinking resources support the recursion.

For a particular two-use case, Theorem 5 shows an extreme-point simplification under monotonicity and curvature conditions: compare allocating everything to the first use with allocating everything to the second. Bellman explicitly warns that the opposite curvature produces a much more complicated situation and the simple reduction does not hold in general.

Theorems 1–3 provide existence and uniqueness for three families. The first uses a bounded continuation multiplier. The second treats allocation on a simplex with continuous, non-negative functions, contraction of remaining resource and summable attainable returns. The third treats a stochastic probability-vector recurrence under a strict expected contraction condition.

Theorems 4–7 exhibit particular solutions. A two-variable maximum recursion acquires a threshold structure and can be piecewise linear under a commensurability condition. The allocation problem can reduce to two extreme choices under stated shape restrictions. The box problem receives its acquisition index. A one-dimensional minimum recursion receives a threshold solution determined by an integer minimisation.

Equation OCR is unreliable in the searchable copy: subscripts disappear, inequalities reverse and vector notation collapses. This companion therefore reproduces only expressions checked against the rendered pages. It focuses on the load-bearing structures rather than pretending that an extraction artefact is a proof.

The notes matter. Bellman credits S. Karlin and H. N. Shapiro for unpublished existence results, M. Shiffman for one solution and D. Blackwell for another. References place the paper beside Wald’s sequential decisions and work by Arrow, Blackwell, Girshick, Harris and Marschak. Dynamic programming emerges within a live network of sequential analysis, inventory and decision theory.

The objective comes from outside the recursion. Maximise yield and minimise acquisition time can produce different policies over the same world. The model also determines which actions exist and how they transform state. Bellman optimises those supplied possibilities; he does not claim that the procedure invents its own ends.

A mathematical value function is not automatically tractable. Continuous regions may require approximation; state dimension can explode; exact expectations may be unavailable. The 1952 paper proves solutions for structured cases. Later fields would confront the computational burdens now associated with dynamic programming, but those later diagnoses should not be read backwards as claims made in this note.

Uncertainty does not by itself imply learning. In the box example, observations update a specified probability state. The rule for updating and the model of examination reliability are already given. A learning system would also revise unknown structure from evidence. Belief update is inference inside a decision process, not a universal synonym for learning.

Finally, optimal sequential behaviour is not evidence of consciousness. It can make a machine appear farsighted because continuation values encode later consequences. That is a functional achievement. Awareness, understanding, moral responsibility and self-authored goals require separate operational evidence. The same discipline that protects theorem assumptions protects mental attribution.