The inverse question before the celebrated theorem

Suppose a workshop produces small clockwork switches. Each switch is tested once. Some click into place and some fail. Eleven switches have been tried: eight succeeded and three failed. The forward probability problem begins by specifying a success chance, perhaps three quarters, and asks how likely an eight-three record would be. The inverse problem begins with the record and asks where the unknown success chance probably lies. The direction of explanation has changed. An assumed cause once generated possible observations; now actual observations must constrain an unobserved cause.

That converse problem governs Thomas Bayes’s posthumous essay. Richard Price states it near the opening and Bayes repeats it as the formal problem: the numbers of happenings and failures are given; the required quantity is the chance that the probability of a happening on a single trial lies between any two named degrees. The phrase is cumbersome because eighteenth-century probability had no settled notation for a density over an unknown parameter. Its intent is precise. Bayes wants a numerical probability for a proposition about the single-trial chance, not merely a point estimate or a long-run guarantee.

Price contrasts this task with work by Bernoulli and Abraham de Moivre. Their direct question starts with a known chance and bounds how far a sufficiently large observed frequency might depart from it. Bayes asks for the converse: after a finite count has arrived, how strongly does it locate the chance that produced it? The two directions share binomial arithmetic, yet one cannot simply read the first theorem backwards. Conditioning changes the denominator, and a rule is needed for how possible unknown chances are represented before the count is known.

The familiar identity P(H|D) = P(D|H)P(H)/P(D) is therefore only one hinge in a longer machine. Bayes proves conditional results early in the essay, including Proposition 5, which permits a probability to be reversed once the probability of the evidence and the joint event are known. His distinctive labour is to build those quantities for a continuum of possible chances. He does so with geometry. A first random point determines the unknown chance; later random points produce observations. Areas on a diagram then carry the probabilities needed for the reversal.

This order guards against a common historical shortcut. It is correct that Bayes’s early propositions contain the logic now associated with his name. It is misleading to make the essay sound like a two-line theorem followed by examples. The central achievement is an exact solution to an inverse binomial problem, together with practical methods for calculating the required areas. The paper asks how experience supports induction when the chance itself is not known in advance.

Probability measured by expectation

Bayes begins with definitions and general propositions because, as Price explains, he did not know where to send a reader for clear demonstrations of the needed laws. His fifth definition measures the probability of an event by a ratio. Take an expectation that pays a specified value if the event happens and nothing if it fails. The event’s probability is the ratio between the fair present value of that expectation and the value delivered on success. Bayes is not defining a private feeling. He is supplying an operational numerical measure that can apply to past facts as well as future events.

The first proposition adds probabilities for inconsistent events. If only one of several alternatives can happen, an expectation paying the same prize on any of them has a value equal to the sum of the separate expectation values. The third proposition is the product rule: the probability that two successive events both happen compounds the probability of the first with the conditional probability of the second given the first. Independence later becomes the special case in which the second probability does not change when the first happens.

Propositions 4 and 5 prepare the reversal. Imagine two successive events repeated under a stopping arrangement. Bayes shows how the value of an expectation is preserved when information arrives. Proposition 5 states, in his own expectation language, that if the probability of a second event and the probability of both events are known, then after learning that the second happened one may calculate the probability that the first happened. In modern symbols, P(A|B) = P(A∩B)/P(B). This is conditional probability, not yet the full inverse-binomial solution.

Proposition 7 supplies the forward engine. If each trial has chance a of happening and chance b of failing, then the probability of exactly p happenings and q failures in p+q trials is the binomial coefficient E multiplied by aᵖbᑫ. Each particular order has the same product because the trials are independent and similar; E counts the possible orders. This result will become the height of Bayes’s curve after the unknown chance is represented by position.

The modern line is a translation. Bayes uses ratios, letters a and b, and the coefficient of a term in a binomial expansion. What matters is the mechanism: when a fixed chance governs exchangeable independent trials, every ordering with the same success and failure counts has the same probability. Counting the orderings produces the coefficient. Later, when the expression is normalised across possible values of θ, that coefficient will cancel because it does not vary with θ.

The square table experiment

Section II introduces the construction that carries the paper’s originality. Bayes supposes a square, level table ABCD. Either of two balls, W and O, is equally likely to rest on any equal part of the plane. Ball W is thrown first. Through its resting point a line is drawn parallel to two sides. Ball O is then thrown repeatedly. Event M occurs when O comes to rest on one specified side of the line; otherwise M fails.

If W lands one fifth of the way across the table, the region counted as M occupies one fifth of the area, so each later O throw has chance one fifth of producing M. If W lands four fifths of the way across, the chance is four fifths. W’s horizontal position therefore determines the single-trial chance. Because W was itself uniformly placed, equal horizontal intervals represent equal antecedent probability for the chance. The physical construction turns an unknown number between zero and one into a random geometrical coordinate.

The object is a square table and the text speaks simply of balls. Calling it a billiard table adds period colour that the primary does not supply. More importantly, the table is not a casual illustration appended to a modern prior. It is Bayes’s route around a disputed postulate. Price reports that Bayes first considered directly supposing an equal chance for the unknown probability to lie in equal intervals. Concerned that readers might reject this, Bayes instead stated a proposition in another form and attached a scholium explaining why it could stand for complete antecedent ignorance.

The construction has two random stages that must remain distinct. The first throw fixes θ. Repeated O throws are random conditional on that fixed θ. Before W’s position is known, θ varies across imagined repetitions of the whole experiment. After W has landed, θ does not drift between O throws. Mixing these levels would replace Bayes’s model with a different process.

Lemma 1 proves that W’s horizontal coordinate falls between any two points with probability equal to their distance divided by the whole line. Later propositions combine that coordinate with the binomial count. At coordinate x, the chance of p happenings and q failures is proportional to xᵖ(1-x)ᑫ. Plot that quantity as a height above x. An interval of possible x values sweeps out an area under the curve. Geometry now records both a range of unknown chances and the evidence expected under each chance.

From a count curve to an area ratio

Proposition 8 joins the two stages. Before W is thrown, consider both the event that its coordinate lies in a named interval and the event that M happens p times and fails q times. For each possible coordinate x, the conditional count probability is E xᵖ(1-x)ᑫ, where E is the binomial coefficient. Bayes represents this value as a height. The probability of the joint event is the area under that curve over the named interval, measured relative to a fixed surrounding rectangle.

The corollary integrates across the entire line. Because W must land somewhere, the prior probability of the observed count is the whole area under the curve. Proposition 9 then applies Proposition 5. It is first discovered that the count occurred. One guesses that W lies in the interval, which is equivalent to saying that the single-trial probability of M lies between the interval’s endpoints. The chance that the guess is right equals the interval’s area divided by the whole area.

This is the crucial normalisation. A high curve value says that a particular θ makes the observed count comparatively plausible. It is not yet a probability that θ is correct. Dividing by the total area makes all possible θ values jointly exhaust probability one. The numerator keeps the possibilities named in the question; the denominator sums the evidential support across the complete parameter range. Bayes’s geometry performs what modern notation calls posterior normalisation.

The coefficient E has disappeared because it multiplies both integrals. No information about the order of successes and failures remains. The curve is highest near s/(s+f) when both counts are positive, which is the maximum-likelihood point. Yet Proposition 9 asks for an area, not merely the height at the peak. A narrow high peak may carry less probability than a broader, slightly lower region. The whole shape matters.

Bayes next develops exact series and approximation rules for calculating these areas. The mathematics is laborious because tables, numerical integration software and standard beta functions do not yet exist. Price says the first rule gives a direct and perfect solution, while the later rules approximate when direct labour becomes too great. Computation is therefore not incidental. The inferential rule is useful only if the area can actually be evaluated.

Why the denominator is evidence, not a decorative constant

It helps to name the two events in Proposition 9. Let A say that W lies between the chosen bounds, and let B say that M produced the observed numbers p and q. Proposition 8 calculates P(A and B) as the bounded area relative to Bayes’s reference rectangle. Its corollary calculates P(B) as the whole curve area relative to the same rectangle. Proposition 5 then divides the first by the second. The rectangle cancels just as the binomial coefficient does. What remains is P(A|B), the selected area divided by the total.

This anatomy shows why the denominator depends on the observation. For two successes and one failure, it integrates θ²(1−θ); for ten successes and no failures, it integrates θ¹⁰. Each value is the prior predictive probability of that evidence under the whole table experiment, apart from any count-order convention that cancels consistently. A datum that few possible chances predict well has a small total probability. Conditionalisation rescales the surviving possibilities more strongly. The denominator is the probability of seeing what was seen under the full antecedent model.

The same proof also reveals what would change under a non-uniform start. Suppose W were placed by a device that favoured the centre of the table. Its horizontal density, call it g(θ), would multiply the count curve. Both numerator and denominator would then integrate g(θ)θˢ(1−θ)ᶠ. Proposition 5 would still reverse the conditioning. Bayes’s conditional law is not tied to uniformity; the particular inverse solution is tied to how his auxiliary experiment distributes W.

Bayes’s scholium moves from the constructed event M to an event about whose probability nothing is known before trials. His argument is that there is no reason, before observation, to prefer one possible number of happenings in a fixed number of trials over another. The table event has exactly that predictive symmetry: integrating the binomial probability across its uniformly distributed chance gives the same antecedent probability for every count. This is subtler than merely asserting that every numerical chance looks equally plausible. It connects ignorance about the chance to uniformity over possible counts.

Modern readers may accept or reject that bridge. Its value is that Bayes exposes it. The inverse conclusion does not emerge from observed frequency alone. It emerges from frequency plus a symmetry judgement represented by an auxiliary chance experiment. Later Bayesian work generalises the starting distribution; later critics ask whether such probability assignments to unknown constants are justified. Both developments begin where the scholium leaves a modelling choice visible.

Beta distributions after Bayes

Modern probability condenses Bayes’s geometry. Write θ for the unknown success chance. A uniform density before observations assigns equal probability to equal intervals, exactly as W’s horizontal coordinate does. The likelihood of s successes and f failures is proportional to θˢ(1−θ)ᶠ. Multiplying prior and likelihood leaves the same kernel. Normalising it produces a beta distribution with parameters s+1 and f+1.

The words prior, likelihood, posterior density and beta distribution were not Bayes’s vocabulary. They are useful because they expose the reusable structure, but the article keeps their date visible. Bayes talks of chances, expectations, guesses, happenings, failures and areas. His result is not made more faithful by silently rewriting the whole essay as a contemporary statistical textbook.

The beta form gives several summaries. Its mean is (s+1)/(s+f+2). If s and f are both positive, its mode is s/(s+f), the observed success fraction. The difference between mean and mode records the contribution of the uniform construction, especially with little data. An interval probability is the integral of the density over that interval. A prediction for the next success averages θ over the posterior and therefore equals the posterior mean.

Learning changes concentration as well as location

Two data sets can have the same success proportion and very different implications. One success and one failure produce Beta(2,2); one hundred successes and one hundred failures produce Beta(101,101). Both are centred at one half, but the second curve is much narrower. The location records the balance of evidence. The concentration records how much repeated evidence supports that balance under the fixed-chance model. A point estimate alone discards this distinction.

The addition of one to each count is sometimes described as two imaginary observations, one success and one failure. That mnemonic can help with arithmetic, but it should not replace the derivation. The ones are the parameters of the uniform Beta(1,1) density. Different starting beta parameters add different amounts and patterns of concentration. More generally, a prior need not correspond to literal past trials at all. Treating every prior as a bag of pseudo-data can conceal substantive information or distort its meaning.

Prediction introduces another integral. The probability of success on the next trial is ∫θ p(θ|data)dθ. For the beta density this integral equals its mean. The result combines uncertainty about θ with randomness of the next outcome. Plugging the mode into a Bernoulli model would instead pretend that θ had become known. With little evidence, the two answers can differ markedly; with much balanced evidence, they approach one another.

One may describe that last equality as a rule of succession in the uniform beta-binomial model, but historical attribution needs care. Price’s Appendix draws predictive conclusions from Bayes’s rules, and Laplace later systematised the rule and made it famous. The sunrise example belongs to Laplace’s later exposition, not to Bayes’s 1763 text. A companion can show the mathematical successor without awarding every later illustration to the earlier author.

The uniform starting density is also not neutral in every description. If equal intervals of θ receive equal mass, equal intervals of the odds θ/(1−θ) do not. A transformation of coordinates changes which regions look equal. Bayes’s table answers the representation question by naming θ as a spatial proportion. Modern analysts must still defend why that parameter and that prior suit their problem.

Executable inverse-probability laboratory

This teaching artefact computes the modern beta reconstruction for integer success and failure counts. Change the count, interval and starting beta parameters. The chart shows the normalised density, while the text reports the mean, interval probability and next-trial prediction. Setting α = β = 1 reproduces Bayes’s uniform table construction. Other values are modern sensitivity probes, not variants stated in the 1763 essay.

The numerical integration uses Simpson’s rule over a fine grid. That choice makes the mechanism inspectable rather than hiding it in an external package. Counts are capped to keep the calculation stable in an ordinary browser. For α or β below one, the mathematical density is infinite at an endpoint; the display samples just inside the endpoints and the integral remains finite.

Bayes, Price, Laplace and Fisher

The publication is a joint historical object with unequal and partially uncertain authorship. Bayes died in 1761. Price found the essay among his papers, prepared it for the Royal Society and addressed it to John Canton in a letter dated 10 November 1763. The paper was read on 23 December. Its journal heading places it in volume 53 for 1763, pages 370–418; printing occurred in 1764. That difference explains why bibliographies alternate between 1763 and 1764. This collection follows the canon’s 1763 date while stating the publication history.

Price identifies his own interventions. He says that he wrote short notes in some places and added an Appendix applying the rules to particular cases. He also says he, not Bayes, should answer for any computational errors in that material. The letter’s discussion of induction, stable laws in nature and an intelligent cause is Price’s framing. It cannot be quoted as Bayes’s private declaration. Bayes’s surviving contribution includes the definitions, propositions, table construction, scholium and computational rules as Price transmitted them, but the original papers are mostly lost, so a perfectly clean editorial boundary is unavailable.

Stephen Stigler later found that 1764 offprints carried a more ambitious title: “A Method of Calculating the Exact Probability of All Conclusions founded on Induction.” He uses that evidence to reconstruct Price’s purpose and suggests a possible connection to replies to David Hume. The offprint title is documentary evidence. The claim about Bayes’s own motivation is a historical inference, which Stigler marks as the most speculative step. This companion preserves that modal status.

Laplace is the great successor. His work from the 1770s onward developed inverse probability independently and systematically, extended approximation methods and popularised predictive applications. The beta-binomial calculation that gives (s+1)/(s+f+2) for a future success became associated with his rule of succession. Bayes and Price contain the mathematical seed and early applications; Laplace turned inverse probability into a broad programme.

Fisher supplies a sharp rival rather than a routine objection. In his 1922 foundations paper, a population parameter is fixed but unknown. The data have a distribution conditional on it; the likelihood compares parameter values without becoming a probability distribution over the parameter. Fisher objected that inverse probability often smuggled in an unsupported distribution over a fixed constant. He developed likelihood, sufficiency, efficiency and sampling distributions as a different foundation for estimation.

The rivalry clarifies what Bayes’s construction buys. The square table does not infer a parameter distribution from likelihood alone. It supplies a probability model for the first point and therefore for θ before the later count. Once that starting model is accepted, conditional reversal is legitimate within the joint experiment. Whether an application outside the table warrants an analogous starting distribution remains a modelling question. Fisher’s criticism bites precisely where the physical or substantive basis for that distribution is missing.

What the solution establishes, and what it cannot

The first boundary is model stability. One fixed chance must govern independent similar trials. Selection effects, learning, fatigue, contagion or time trends can make the binomial likelihood wrong. More observations then concentrate the wrong model’s answer. Posterior sharpness measures conditional uncertainty inside the model; it is not a certificate that the model describes the world.

The second boundary is the starting representation. Bayes offers a principled spatial construction for equal intervals of θ. Applying that construction to disease prevalence, machine reliability or witness accuracy requires an argument that θ and its antecedent distribution are appropriate. Alternative priors can encode real background knowledge or different invariance commitments. Sensitivity analysis is not an embarrassment added after inference; it reveals which conclusions belong to the data and which still depend on the starting model.

The third boundary is interpretation. A posterior interval is a probability statement about θ conditional on the model, prior and observed count. A frequentist confidence interval has a different guarantee under repeated sampling and need not assign probability to a fixed θ after data arrive. The numerical endpoints can sometimes resemble each other, but the questions and error accounts differ. Calling either one simply “the uncertainty interval” hides the philosophical work.

The fourth boundary is decision. Proposition 9 says how probable an interval is. It does not by itself state what action to take, what loss follows from error or how observations should be acquired. A high posterior probability may still be insufficient for an irreversible decision with asymmetric harm. Later Bayesian decision theory joins probabilities to utilities and actions; that is a successor layer, not a result silently present in the 1763 essay.

Memory studio

Remember the paper as an apparatus, not a slogan. The first throw sets a hidden divider. Repeated throws generate a count. The count raises a curve over every possible divider. A selected interval becomes an area, and the whole area supplies the denominator. These physical and geometric images preserve the direction of the argument when the modern formula has faded.

Questions that expose a shallow reading

Place in the canon

The preceding papers established limits on computation, measures of information, universal prediction, learnability and information-priced choice. Paper 0013 reaches further back to a prerequisite beneath all of them: how evidence can support a probability distribution over an unknown regularity. Its mechanism is modest enough to draw on a table and powerful enough to become a template for statistical learning.

For intelligence and agency, the lesson is not that one equation solves induction. It is that inference needs a complete generative story. A hidden quantity, an observation rule and an antecedent distribution together determine how evidence changes uncertainty. The result can guide prediction and later decision only relative to those parts. Bayes’s square makes them visible.

The enduring question is therefore not merely “What is the posterior?” It is “What joint experiment makes this posterior a valid conditional probability?” That question separates a derivation from a ritual. It also joins historical care to modern practice: preserve the first ball, the later trials, the count curve and the whole area, and the inverse problem becomes intelligible rather than magical.