A notebook that nobody can check

Dr Meenakshi Natarajan works alone in a rented room above a bookshop in Mysuru. She has a university affiliation that pays for nothing, a laptop, a cheap EEG headset and a question she expects to spend ten years on. Her question is whether a simple breath-counting practice, done for twenty minutes a day, makes people better at knowing when they are right. Psychologists call this metacognitive accuracy. In the language of this series it is a claim about reflexivity, the third layer of the Recognition Theory of Mind.

She keeps a notebook, and on its first page, in January 2026, she has written forty claims. Some are bold: "Eight weeks of practice will raise metacognitive efficiency by at least a tenth in novices." Some are cautious: "The effect, if any, will be smaller in people who already meditate." A few are close to articles of faith: "Self-recognition is the same in a meditator and a non-meditator; only the noise differs."

The notebook has a problem she has not yet noticed. In 2036, when a funder asks what her decade of work established, the notebook will be the only record. It will have been edited, and some pages will have been rewritten after results came in. Some predictions will have been forgotten, and nobody, including Meenakshi, will be able to say which. The words in it will be vague enough to fit almost any outcome. "Raise by at least a tenth" is clear. "Likely smaller" is not.

The funder's auditor in 2036 is a methods reviewer called Sam Okafor. He will have a single afternoon, and he will want to know three things. Did she say in advance what she expected? How often was she right, given how sure she said she was? And did she correct herself in public when she was wrong? The notebook cannot answer any of them.

This paper is about what Meenakshi should keep instead. I will call it a claim ledger, and the rules for keeping it the Claim Ledger Protocol. The series has used a claim ledger at the end of every paper so far, and this paper sets out what that ledger is for, how it should be built, and how a stranger could check it. The answer has five parts, and each part answers one of the ways the notebook fails.

Why claims drift when nobody keeps score

Science is often described as self-correcting. In practice, the correction depends on someone noticing the error, caring enough to say so, and having a record clear enough to show what was claimed. A solo researcher has no colleagues to notice, no lab meeting to object and no institution that keeps her old drafts. All the correction has to be built into the record itself.

Claims drift in three ways. The words drift, so that a prediction that failed can be reread as one that succeeded. The memory drifts, so that unfavourable results fade first. And the public record drifts, because losing confidence in your own finding is something most researchers do privately. Each drift has a true story attached to it.

Words that mean different odds to different readers

The same two words, agreed by the people who wrote them, covered everything from one chance in five to four chances in five. Nobody had lied. The phrase had simply never been tied to a number, so each reader filled in their own. Kent's response was to propose a table linking words to odds. "Probable" would mean about 75 per cent, give or take 12. "Almost certain" would mean about 93, give or take 6. "Chances about even" would mean 50, give or take 10.

Meenakshi's notebook is full of phrases like "serious possibility". "Likely smaller." "Probably real." "Unlikely to replicate." Each one will be read in 2036 by someone who was not there when it was written, and who will fill in their own odds. If her prediction fails, a generous reader will decide she meant 40 per cent. A hostile reader will decide she meant 90. A verbal forecast cannot be scored, because its meaning is set by the person scoring it.

The forecaster who is always right

The sunrise forecaster is a joke with a real problem inside it, and I return to it in the instrument. The protocol needs some way to tell a record full of easy wins from a record that took risks. The standard answer is to compare a forecaster with a dull baseline, such as always predicting the long-run rate. If she cannot beat the baseline, her accuracy is borrowed from the world.

The memory that forgets the wrong things first

Darwin did not trust his memory to be fair to his theory, so he wrote the unfavourable things down at once. The habit is simple and it has a clear logic. Memory keeps what fits and lets go of what does not. Any record that relies on memory to decide what goes in will drift towards the owner's views.

A notebook that can be rewritten has the same flaw as memory. If Meenakshi revises a page after a result comes in, she is doing what memory does, only in ink. The protocol therefore needs a rule that entries are added and never changed. Darwin wrote memoranda against himself, and a claim ledger turns that personal discipline into a structural rule.

The quiet loss of confidence

The finding I take from the project is about structure. Most researchers who lost confidence in a result did not hide it. They simply had no place to put it, since a journal article cannot be updated by its author. A correction requires the editor's agreement and usually a reason more serious than "I now think this is probably wrong." A doubt that has no home stays private.

A claim ledger gives the doubt a home. If Meenakshi's confidence in a claim falls from 70 per cent to 40, she adds an entry saying so, with the reason. The original claim stays where it was, so anyone reading the ledger sees both entries and the date between them. The ledger is built so that changing your mind leaves a trace instead of a gap.

An intuition that fails

The quiet deletion is the failure the ledger is built to prevent. The fix cannot be a promise, because a promise is an entry that can itself be deleted. The fix has to make deletion visible to someone who was not watching. That is what the hash chain in the next part does.

What a decade of claims would need

Put the four drifts together and the requirements are plain. A claim must be stated with a number, so it can be scored. It must carry a date by which it will be judged, so it cannot wait for ever. It must be entered in a record that cannot be edited, so memory and convenience cannot rewrite it. And revisions and retractions must be new entries, so changing your mind is visible. A fifth requirement comes from the sunrise forecaster. The record must allow a reader to see whether the claims were difficult.

None of these is new on its own. Weather forecasters have scored numeric forecasts since 1950, and intelligence analysts have argued about Kent's table since the 1960s. Computer scientists have chained records with hashes since 1991, and medical evidence has been graded systematically since the 2000s. What this paper adds is a way to put them together so that one person, working alone for a decade, can keep them all without an institution.

The Claim Ledger Protocol

The protocol has five rules, each of them short. I state them first and then explain each one with Meenakshi's ledger.

1. State it so it can fail. Every claim gets an identifier, a plain statement, a resolution criterion and a date by which it will be judged.
2. Put a number on it. Confidence is recorded as a probability, with a verbal band as a label, never the other way round.
3. Grade the evidence separately. How strong the evidence is gets its own grade, independent of how confident the claimant feels.
4. Chain it and publish the head. Entries are linked by cryptographic hashes, and the latest hash is published somewhere outside the researcher's control at least once a year.
5. Correct by adding. Revisions, retractions and resolutions are new entries that point to the old one. Nothing is ever edited or removed.

Rule one: state it so it can fail

Meenakshi's first claim in the notebook read: "Breath counting improves metacognition." It cannot fail, because it does not say by how much, in whom, by when or measured how. Rewritten for the ledger, it becomes claim M-01:

M-01. In adult novices (no prior meditation), eight weeks of daily twenty-minute breath counting raises metacognitive efficiency on a perceptual task by at least 0.10, compared with an active listening control. Resolves by 31 December 2028, on the first preregistered study with at least forty people per group.

The resolution criterion does most of the work. It names the measure, the size of effect, the comparison and the kind of study that will count. It also names a date. Without the date, a claim can wait for ever for the right study, and the sunrise forecaster's trick returns in a slower form.

Some claims cannot be written this way, and the protocol does not pretend otherwise. Meenakshi's third claim, that self-recognition is the same in a meditator and a non-meditator, has no measure that anyone agrees on. It goes into the ledger anyway, marked unscorable with the reason. An unscorable claim is still a claim on the record. It tells a later reader where the programme's beliefs outran its methods.

Rule two: put a number on it

Kent's story shows why words alone fail. The fix is to record a number and attach the word as a label. Meenakshi gives M-01 a confidence of 0.70, which carries the band label "likely". The number is what will eventually be scored. The label is there for readers who want a quick sense of the claim.

The series uses three bands, and this paper fixes what they mean in numbers for the first time. Established means 0.90 or above. Likely means 0.60 to 0.89. Speculative means 0.30 to 0.59. A claim the author thinks less than 0.30 likely is entered as its negation, so every claim in the ledger is one the author leans towards. The bands are coarse on purpose. A solo researcher who reports confidence to two decimal places is claiming a precision she probably lacks.

The bands sit close to the likelihood scale used by the Intergovernmental Panel on Climate Change. In its 2010 guidance note for authors, "likely" means 66 to 100 per cent and "very likely" means 90 to 100 per cent. The note says the categories have fuzzy boundaries and that a full range is preferable when there is enough information. The IPCC's ranges overlap, which suits an assessment written by many authors. A personal ledger is better served by bands that do not overlap, so each claim falls into exactly one.

Rule three: grade the evidence separately

Confidence and evidence are different things, and the ledger records them in different columns. A researcher can be very confident on weak evidence, and that is exactly the case a reader most needs to see. Medicine learned this through the GRADE system, which in a 2008 BMJ paper by Gordon Guyatt and colleagues separated the quality of evidence, rated high, moderate, low or very low, from the strength of a recommendation.

The ledger uses four evidence grades, adapted for a programme that mixes experiments, simulations, argument and first-person report.

M-01 starts at grade C. Its only support in January 2026 is an argument from earlier work on attention training and a pilot with six people. It carries a confidence of 0.70 on grade C evidence, a combination that is allowed and informative. It tells Sam, ten years later, that Meenakshi was fairly sure of something she had not yet shown. If she turns out right, that is evidence of good judgement. If she turns out wrong, it is evidence of the opposite.

The matrix above shows why both columns are needed. The top-left cell, high confidence on weak evidence, is where honest researchers and overconfident ones both sit, and only the later resolution record can tell them apart. The bottom-right cell, low confidence on strong evidence, is rare and interesting. It usually means the researcher suspects a flaw that the evidence grade does not yet reflect.

Rule four: chain it and publish the head

The trick is simple to describe: a hash function turns any text into a short fingerprint. Change one comma in the text and the fingerprint changes completely. To chain entries, each new entry's fingerprint is computed from its own text plus the fingerprint of the entry before. The last fingerprint, called the head, therefore depends on every entry in the ledger, in order.

A chain on its own does not stop a determined researcher. If Meenakshi deletes an entry, she can recompute every later fingerprint and produce a new chain that looks perfectly consistent. The chain becomes evidence only when its head has been published somewhere she cannot reach back and change. Haber and Stornetta's company used the newspaper, but a solo researcher has cheaper options. She can post the head to a public code repository, send it to two colleagues by email, deposit it with an archive that stamps dates, or print it in the back of each year's annual report.

Meenakshi publishes her head on the last day of each year. In 2036, Sam does not need to trust her. He recomputes the chain from the entries she gives him and compares each year's head with the one she published. If they all match, nothing was changed after publication. If one does not, he knows the first year in which something changed.

You can try this below: edit any entry and watch which fingerprints change.

An analogy from the sea

A ship's logbook is the oldest version of this idea I know. Entries are made in ink, in order, with the time, on pages that are numbered and bound. Mistakes are struck through with a single line so that the original stays legible, and the correction is written beside it. A log that has a page torn out is suspect in any inquiry, even if the remaining pages are accurate.

Rule five: correct by adding

The last rule is the one that makes the others bearable. A researcher who could never correct anything would refuse to make claims at all. So the ledger allows every kind of change, provided each change is a new entry that names the old one.

There are four kinds of follow-up entry. A revision changes the confidence or evidence grade of an open claim, with a reason. A resolution records that a claim's date has arrived and states the outcome against the criterion. A retraction withdraws a claim whose basis was faulty, such as a coding error, so that it should never have been entered at the confidence it had. A supersession replaces a claim with a better-worded one, while keeping the old one scored on its own terms.

In 2027, Meenakshi finds that her pilot data for M-01 had two participants' files swapped. With the files corrected, the pilot shows nothing. She enters a revision: M-01 confidence falls from 0.70 to 0.50, evidence grade stays at C, and the reason names the error. The original entry stays where it was. Anyone reading the ledger later sees what she believed in January 2026, what she learned in 2027, and how she responded.

I think this is the rule that a reader should weigh most. A ledger full of revisions shows that the researcher kept looking. A ledger with no revisions over ten years is either the work of an extraordinary forecaster or of someone who stopped looking, and the score in the next part will usually tell you which.

An analogy from the counting house

Double-entry bookkeeping is the other old practice the ledger borrows from, and the borrowing is close. A bookkeeper never erases a wrong figure; instead she writes a correcting entry that reverses it, and the books still balance. An auditor can follow every change from the first entry to the last, because each one left its own line. A clean set of books with no corrections is something auditors are trained to find suspicious.

Scoring a decade

A ledger that follows the five rules can be scored. This part explains how, and what the score can and cannot show. The scoring methods come from weather forecasting, where people have been asking how to judge a probability for seventy-five years.

A true story about a forecaster who played the system

Brier's score is easy to compute. For each claim, take the stated probability and subtract the outcome, counted as 1 if the claim came true and 0 if it did not. Square the difference, and then average across all the claims. A perfect forecaster scores 0, while a forecaster who says 50 per cent on everything scores 0.25. A forecaster who says 100 per cent and is always wrong scores 1.

For M-01, suppose the eight-week study arrives in 2028 and shows no effect. At Meenakshi's original 0.70, the claim would have scored 0.49. At her revised 0.50, it scores 0.25. The revision earns credit only if it was entered before the result was known, and the chain is what proves that it was.

The property that matters most is one Brier pointed to and later statisticians proved in general. Tilmann Gneiting and Adrian Raftery, in a 2007 review of scoring rules, call a scoring rule strictly proper when a forecaster gets the best expected score only by reporting her true belief. The Brier score is one of these. If Meenakshi believes 0.70, she cannot improve her expected score by writing 0.90 to look bold or 0.50 to look careful. Honesty is her best strategy, given the rule.

Calibration, resolution and the dull baseline

A single score hides two different skills. In 1973, Allan Murphy showed in a paper on partitioning the probability score that the Brier score can be split into three parts. The first part, reliability, measures calibration: whether claims at 0.80 come true about 80 per cent of the time. The second part, resolution, measures discrimination: whether the claims that came true were given higher numbers than the ones that did not. The third part, uncertainty, depends only on how often claims came true overall, and says how hard the set was.

The Brier score equals reliability minus resolution plus uncertainty. Low reliability is good, because it means small calibration errors. High resolution is good, because it means the forecaster told true from false. The uncertainty term is the score the dull baseline would get by always predicting the overall rate.

The quiz shows why Sam will read the parts as well as the total. A forecaster who is overconfident but often right can beat a perfectly calibrated forecaster on the total score. The calibration part shows the overconfidence, while the resolution part shows the skill. Both are facts about the record, and a reader who sees only the total sees neither.

Here is the reliability diagram for Meenakshi's invented ledger in 2036, as computed by the auditor later in this paper. Each marker represents a confidence level, and its height shows how often claims at that level came true. A perfectly calibrated ledger would put every marker on the diagonal.

The diagram is noisy, as any diagram built from fifty-four claims will be. Her claims at 0.30 came true 36 per cent of the time, which is close. Her claims at 0.70 came true only half the time, which is the clearest miss. Her claims at 0.90 all came true, but there were only six of them. Ten years is short for one person. Sam should read the diagram as suggestive and the confidence bands on it as wide.

What a large record of expert forecasts showed

Tetlock's later work was more hopeful. In 2014, Barbara Mellers and colleagues reported in Psychological Science on a geopolitical forecasting tournament in which five university-based groups competed. Three interventions improved both calibration and resolution: training in probability, working in teams, and placing the best forecasters together in elite teams. Forecasting well turned out to be a skill that could be measured, and measurement was what let it be improved.

For a solo researcher, the lesson is that the ledger is a feedback device as well as a public record. Meenakshi can read her own reliability diagram each year. If her 0.70 claims keep coming true only half the time, she can move them down. A ledger kept for ten years is a ten-year training course in her own judgement, with the marks published.

Two notebooks with the same numbers

The two notebooks matter for this series in a particular way. Meenakshi studies metacognitive accuracy, which is the ability to know when you are right. A calibration score is a public, scored measure of exactly that ability, applied to her own research claims. It is tempting to read a good score as proof of good reflexivity. That reading would be the error this series keeps warning against. L2 evidence never stands in for L3. A calibrated record shows that the outputs of her judgement track the world. It does not show how her judgement does it.

The gap that no score can see

Every score so far has been computed on claims that were resolved. That opens a door that Brier did not have to worry about, because the weather always arrives. A researcher who resolves her successes promptly and leaves her failures open will have a better score than she deserves. Nothing in the chain is broken and nothing has been deleted. The failures are simply never judged.

I call this selective resolution, and it is the hardest drift to see. The protocol's answer is the resolution date in rule one. When a claim's date passes, the claim becomes due, and the auditor counts how many due claims have been resolved. If the share left open is high, the score is suspect, however good it looks.

Public prediction sites have faced the same problem. Long Bets, run with the Long Now Foundation, describes itself as "a public arena for enjoyably competitive predictions" and says that the foundation "furnishes the continuity to see even the longest bets through to public resolution." Resolution is the part that needs an institution, because it is the part a predictor has least reason to do.

A bathroom scale shows the problem in domestic form. Someone who weighs himself only after weeks of careful eating will build a record of steady progress, and every individual reading on it will be accurate. The scale has not been tampered with and no figure has been altered. The record misleads because the decision about when to step on it was made by the person with an interest in the answer, after he already had a fair idea what it would say.

The comparison with public betting is instructive. A solo researcher has to make resolution automatic, by deciding in advance who will judge each claim and how.

For Meenakshi, that means each claim names a resolver; for most claims it is herself, using the stated criterion. For claims where her own judgement is most likely to be biased, such as claims about her own theory, the resolver is a named colleague who has agreed to look at the evidence when the date arrives. If the colleague does not reply, the claim is resolved as open, and the open entry counts against her overdue share.

Retraction as a new entry

Revision handles a change of confidence. Retraction handles something harder: the discovery that a claim should never have been made at the confidence it had, because its basis was faulty. A coding error, a misread table, a data file in the wrong column. In a journal, retraction is a public event, often a painful one, whereas in a ledger it should be routine.

What journals have learned about retraction

The Committee on Publication Ethics sets out how journals should handle retraction. Its 2019 guidelines begin with the purpose: "The main purpose of retraction is to correct the literature and ensure its integrity rather than to punish the authors." The guidelines say a notice should state the reasons, say who is retracting, be linked to the original and be freely available. They also say the retracted article should normally stay online with a clear notice, because "retaining the original work ensures transparency of the published record."

Every one of these points carries over to a ledger. A retraction entry names the claim, states the reason, says who found the problem and links back. The original entry stays, marked by the later one. The essential difference is that a ledger needs no editor. Meenakshi is both author and journal, so the guidelines become rules she applies to herself.

The wider record is also changing. In 2023, Retraction Watch announced that Crossref had acquired its database of retractions, launched in 2018, and would make it completely open. Retractions in journals are becoming searchable facts. A personal ledger lets the same be true of the smaller, more frequent retractions that never reach a journal: the pilot that was coded wrongly, the claim that rested on a misread paper.

The objection that free retraction means nothing

I want to be clear about the scoring rule, because it is easy to get wrong. If retracting a claim removed it from the score, the ledger would reward exactly the quiet deletion that the chain was built to prevent, only done in public. So a retraction leaves the original claim scored. The ledger reports two numbers: the score with retracted claims included, and the count and timing of retractions. A reader can see whether retractions came before or after the outcomes were known. Retractions entered before resolution suggest a researcher checking her own work. Retractions entered just after failures look like something else.

Publishing both kinds of result

In 1974, Richard Feynman gave a commencement address at Caltech that has been read by many students since. In the published text, he described "a specific, extra type of integrity" that meant "bending over backwards to show how you're maybe wrong." His first principle was that "you must not fool yourself," and he added that "you are the easiest person to fool." On publication, he said that if you set out to test a theory, "you should always decide to publish it whichever way it comes out."

The ledger is a way to make that decision once, at the start, so that it never has to be made again when a result arrives. When Meenakshi enters M-01 with its resolution criterion, she has already decided to publish the outcome, whichever way it goes. The resolution entry is the publication. It may be only one line long, but it still counts.

This connects to a broader change in how science records predictions. In 2013, Christopher Chambers proposed in Cortex that journals review study plans before results exist, so that publication does not depend on the outcome. In 2018, Brian Nosek and colleagues described in PNAS a wider move to preregistration, which separates predictions made before data from explanations made after. The claim ledger applies the same separation to a whole programme, including the claims too small or too odd for any journal.

Reading a programme's direction from its ledger

Imre Lakatos argued, in his essay on falsification and research programmes, that a programme should be judged by its direction over time. A progressive programme predicts new facts, some of which are then found. A degenerating one mainly explains facts after they have been found, adjusting its theory to fit.

Lakatos gave no practical way to measure direction from outside, and a ledger offers one. Count the claims that predicted something new and were entered before the evidence existed. Count the revisions that came after evidence. If the second count keeps growing while the first stays flat, the programme is spending its effort fitting what it already knows. A ledger makes a programme's direction visible to someone who has never met its author.

Meenakshi's ledger in 2036 shows a mixed picture, which I think is the honest one for most programmes. In her first three years, most entries were new predictions. In the middle years, revisions outnumbered predictions, as the first studies came in and her early claims met data. In the last three years, new predictions rose again, built on what the middle years had taught her. Sam will read that shape as a programme that learned, and then used what it learned.

Ledgering claims across the five layers

The Recognition Theory of Mind, or RTM, describes five layers: responsiveness, access, reflexivity, self-recognition and the luminous ground. It is a hypothesis under test, and the ledger is part of how it is tested. Each layer raises a different question about what a claim can be scored against. This part works through them, using Meenakshi's ledger and the ledger at the end of each paper in this series.

Responsiveness and access: the easy rows

Claims at the first two layers are the easiest to score. A claim about responsiveness says that a system changes in a stated way when something acts on it. A claim about access says that some content is available for report, reasoning or control. Both can be written with measures, effect sizes and dates. Meenakshi's claim that breath counting changes a lab measure of attention is an L2 claim, and it resolves like any other experimental prediction.

Most of the claims in this series so far sit here. Of the thirty-six claims entered in the first four papers, the large majority concern what methods can show, what published studies report or what simulations demonstrate, which are all claims about access-level evidence or about method itself. Only a handful concern reflexivity directly, and none is a scorable claim about self-recognition or the ground. Paper 3's prediction that a marker of awareness will be tested in infants born before 30 weeks, with a check against arousal, is a claim about what the literature will contain by a date. It is checkable by a literature search, and so is Paper 2's prediction about a marketed device. These are the rows where the ledger works best.

Reflexivity: a claim about knowing when you are right

Meenakshi's central claims are at the third layer. Metacognitive efficiency is a measure of how well a person's confidence tracks their own accuracy. It is an L3 measure in the sense that it scores the system's representation of its own states. But the measure is still built from reports and choices, which are L2 outputs. A claim that breath counting raises metacognitive efficiency is scored against those outputs.

The ledger should record this honestly. When Meenakshi enters an L3 claim, she tags it L3 and states what kind of evidence will resolve it. If the evidence is a behavioural score, she notes that the resolution rests on L2 outputs taken as a sign of L3. A reader then knows that a successful resolution supports the L3 claim only as far as the measure is valid for reflexivity, which is a question Paper 3 addressed and did not close.

Self-recognition and the ground: the hard rows

Claims at the fourth and fifth layers are where the ledger's limits show. A claim about self-recognition, in RTM's sense, is a claim that a system recognises its own awareness as its own. A claim about the luminous ground is a claim about awareness itself, as distinct from anything it shows. The series takes no position on whether any machine has the fifth layer, and it has no validated measure of either layer in anyone.

The protocol does not exclude these claims; it requires them to be entered with an honest status. Take Meenakshi's third notebook claim, that self-recognition is the same in a meditator and a non-meditator. It enters the ledger as unscorable, evidence grade D, with the reason: "No agreed measure of self-recognition exists; the claim is recorded so its role in the programme is visible." If someone later builds a measure, the claim can be superseded by a scorable version. Until then it sits in the ledger, marking a belief that no method yet reaches.

An old argument about how a cognition earns trust

Classical Indian epistemology had a long debate that bears directly on the ledger. The question was whether a cognition's validity, prāmāṇya, is intrinsic, svataḥ, or extrinsic, parataḥ. According to the Stanford Encyclopedia entry on Kumārila, the Mīmāṃsā philosopher Kumārila held that "all pramāṇas' being pramāṇas obtains intrinsically." On the reading the entry emphasises, a cognition is justified "until and unless some specifically overriding cognition happens to be forthcoming." Such an overriding cognition was called a bādhaka, a defeater. Kumārila's reason was a regress: if every cognition needed a second to certify it, the second would need a third.

Nyāya took the other side. According to the encyclopedia's entry on classical Indian epistemology, Nyāya held an extrinsic view, in which knowing that you know requires a further, apperceptive certification. It avoided the regress by treating veridicality as the default and checking only when there was a reason to. The entry summarises the Nyāya strategy as taking "veridicality as cognitive default" and certifying cognitions "by inference from the success or failure of the activity that they provoke and guide."

I find both views useful for the ledger. Kumārila is right about the default. A researcher who demanded independent certification of every claim before entering it would never enter anything. The ledger accepts claims as entered and waits for defeaters. Nyāya is right about the check. Validity, for a claim about the world, is eventually shown by what happens when you act on it, which is what a resolution records. The ledger is a parataḥ device with a svataḥ default. It trusts the entry until a defeater arrives, and it makes sure the defeater, when it arrives, is written down.

A thought experiment from practice

The meditator's case has a practical lesson for Meenakshi. Some of her claims are about her own practice, because she sits every day and uses her experience to shape hypotheses. Those claims should be tagged first-person and kept apart from claims about other people. Their scores should be read with the meditator's problem in mind. A first-person claim that comes true may have come true partly because it was written.

[AUTHOR NOTE: [AUTHOR NOTE: if you keep any personal record of predictions about your own practice, describe its form and what you have noticed about writing forecasts changing what you observe. If you keep none, delete this note.]]

An uncomfortable thought experiment

I think every long programme about mind, including this series, should expect to face the 2036 auditor. RTM's claims at the top two layers are, at present, unscorable in the same way. The ledger at the end of each paper tags them, and the honest thing to do is to count them each year. If the count of unscorable claims does not fall over a decade, the ledger will have shown that the theory's most distinctive claims stayed out of reach. That would itself be a finding about RTM, and I would have to report it.

[AUTHOR NOTE: [AUTHOR NOTE: state, in your own words, how you would respond if the series ledger showed after ten years that RTM's L4 and L5 claims had stayed unscorable. Do not let me write this for you.]]

The strongest case against the protocol

The protocol has serious critics, and I give the strongest versions I can, and then say what each gets right.

"Numbers give false precision"

The first objection is that a number like 0.70 pretends to a precision no researcher has. Words like "likely" are honest about vagueness, whereas numbers hide it. A ledger full of numbers will look scientific and mean little, and readers will take the numbers more seriously than they deserve.

I take this objection seriously, and the protocol's coarse bands are my answer to it. Kent's story shows the cost of words alone: one phrase, read as anything from 20 to 80 per cent. A number with a coarse band is less precise than it looks, but it is far more precise than a word, and its imprecision can be measured. If Meenakshi's 0.70 claims come true half the time, that is a measured fact about her numbers. Words never get measured, which is why they never get better.

"Researchers will game the score"

The second objection is Goodhart's law in a new form. Once researchers know they will be scored, they will make easy claims, avoid bold ones and pad their records. The ledger will reward caution and punish exactly the risky conjectures that science needs.

This is the sunrise forecaster again, and it is a real risk. The protocol's defence is only partial. The uncertainty term shows how difficult the set was, and comparison with a dull baseline shows whether the forecaster beat it. And the ledger reports claims by layer and grade, so a record of easy L1 claims is visible as such. But a determined researcher could still log many safe claims and few bold ones. Readers should look at the bold ones directly. A good protocol makes gaming visible, though it cannot make gaming impossible.

"The real claims are never entered"

The third objection is the strongest. The chain stops anyone editing entries after the head is published, but it does nothing about claims that were never entered. A researcher could keep a second, private notebook, enter only the claims she feels sure of, and present a flawless public record. The drawer, as statisticians say of unpublished studies, is where the failures live.

I concede most of this. The ledger cannot see what was never entered. Its defences are indirect. Papers published under the programme should cite ledger entries for every prediction they make, so a prediction in a paper with no ledger entry is visible as a gap. Collaborators and reviewers can be invited to suggest claims for the ledger, which makes the selection less private. And a ledger whose claims never fail is itself suspicious, in the way a clean set of books is. None of these closes the drawer, but together they make keeping one more expensive.

"Demanding checks for every claim leads to regress"

The fourth objection comes from Kumārila. If a claim's standing depends on a later check, then the check's standing depends on a further check, and so on. A ledger that requires every claim to be resolved before it counts would never count anything.

The protocol accepts Kumārila's point, as the tradition section argued. Claims enter the ledger with their stated confidence and are used at that confidence until something defeats or resolves them. The ledger does not require resolution before use. It requires a resolution date, so that defeaters have a scheduled place to arrive. That is a check without a regress.

"Public ledgers expose researchers to punishment"

The fifth objection concerns the people who keep ledgers rather than the ledgers themselves. A public record of failed predictions is a gift to anyone who wants to discredit a researcher. Early-career scientists, independent researchers and people working on unfashionable questions are the most exposed, because they have the least institutional protection. A protocol that asks them to publish their misses, while established figures keep theirs private, could make the field less fair.

I think this objection is correct about the risk and mistaken about the remedy it implies. The asymmetry it describes already exists: established researchers' misses are forgotten because nobody keeps score, while newcomers' misses are remembered because they have no record of hits to set against them. A ledger gives the newcomer something the reputation economy does not, which is a complete, dated record in which her failures appear alongside her successes and her corrections. Even so, I would not ask anyone to keep a public ledger without being free to delay publication of the entries, provided the yearly heads are published on time.

What each critic gets right

The precision critic is right that numbers can mislead, and the bands and diagrams should always be shown with their sample sizes. The gaming critic is right that a score invites strategy, and readers should never judge a ledger by its total alone. The drawer critic is right that the most important claims may never be entered, and no technical device fixes that. Kumārila is right that trust must start somewhere. What none of them shows is that keeping no ledger is better. Each objection, followed through, leads to a better ledger.

The ledger auditor

The paper's main claims can be tested directly. I wrote a short program that builds an invented ledger for Meenakshi, chains it with SHA-256 hashes, publishes a head for each year, and then audits it. It reports the Brier score and its three parts, the share of due claims left unresolved, and whether the chain still matches the published heads. It then runs the same audit on three altered ledgers.

The invented ledger has fifty-four claims, six a year from 2026 to 2034, each due two years after entry, so that every claim is due by 2036. Confidences are drawn from the levels 0.30 to 0.90. Outcomes are drawn so that Meenakshi is slightly overconfident: each claim comes true with a probability five points below her stated number. The altered ledgers are as follows. In B, one failed claim, M-17, is deleted and the chain is rebuilt so that it looks consistent. In C, about six in ten failed resolutions are never entered, so the failures stay open, and the chain is built honestly on what remains. In D, the negative control, every claim is entered at 0.50.

import hashlib, json, random
def h(prev, event):
    return hashlib.sha256((prev + json.dumps(event, sort_keys=True)).encode()).hexdigest()
def chain(events):
    out, prev = [], "GENESIS"
    for e in events:
        prev = h(prev, e); out.append(prev)
    return out
def anchors(events, hashes):
    last = {}
    for e, x in zip(events, hashes):
        last[e["year"]] = x
    return last
def make_ledger(seed=11, n=54, bias=0.05, flat=False):
    rng = random.Random(seed); ev = []
    for i in range(n):
        yr = 2026 + i // 6
        p = 0.5 if flat else rng.choice([0.3, 0.5, 0.6, 0.7, 0.8, 0.9])
        truth = rng.random() < max(0.0, (0.5 if flat else p) - bias)
        ev.append({"kind": "claim", "id": f"M-{i+1:02d}", "year": yr, "p": p, "due": yr + 2})
        ev.append({"kind": "resolve", "id": f"M-{i+1:02d}", "year": yr + 2, "outcome": truth})
    ev.sort(key=lambda e: (e["year"], e["kind"] != "claim", e["id"]))
    return ev
def score(events, now=2036):
    claims = {e["id"]: e for e in events if e["kind"] == "claim"}
    res = {e["id"]: e["outcome"] for e in events if e["kind"] == "resolve"}
    due = [c for c in claims.values() if c["due"] <= now]
    pairs = [(claims[k]["p"], 1 if v else 0) for k, v in res.items()]
    brier = sum((p - o) ** 2 for p, o in pairs) / len(pairs)
    base = sum(o for _, o in pairs) / len(pairs)
    bins = {}
    for p, o in pairs:
        bins.setdefault(p, []).append(o)
    rel = sum(len(v) * (p - sum(v) / len(v)) ** 2 for p, v in bins.items()) / len(pairs)
    reso = sum(len(v) * (sum(v) / len(v) - base) ** 2 for v in bins.values()) / len(pairs)
    overdue = sum(1 for c in due if c["id"] not in res) / len(due)
    return brier, rel, reso, base * (1 - base), overdue
def audit(name, events, published):
    hs = chain(events); now = anchors(events, hs)
    broken = [y for y in sorted(published) if now.get(y) != published[y]]
    b, rel, reso, unc, od = score(events)
    flags = []
    if broken: flags.append(f"CHAIN BROKEN from {broken[0]}")
    if od > 0.10: flags.append(f"SELECTIVE RESOLUTION ({od:.0%} of due claims left open)")
    if reso < 0.005: flags.append("NO RESOLUTION SKILL")
    print(f"{name}\n  Brier {b:.3f} | reliability {rel:.3f} | resolution {reso:.3f} | uncertainty {unc:.3f} | overdue {od:.0%} | {'; '.join(flags) or 'passes'}")
honest = make_ledger()
pub = anchors(honest, chain(honest))
audit("A. Honest ledger, resolved on time", honest, pub)
deleted = [e for e in honest if e["id"] != "M-17"]
audit("B. Same ledger, failed claim M-17 quietly deleted and chain rebuilt", deleted, pub)
rng = random.Random(3)
selective = [e for e in honest if not (e["kind"] == "resolve" and not e["outcome"] and rng.random() < 0.6)]
pub_s = anchors(selective, chain(selective))
audit("C. Failures left unresolved, chain intact", selective, pub_s)
flat = make_ledger(flat=True, bias=0.0)
audit("D. Negative control: every claim at 50%", flat, anchors(flat, chain(flat)))
print("  M-17 in honest ledger:", [e for e in honest if e["id"] == "M-17"])

Running it gives the following output. It needs only the Python standard library and runs in under a second.

A. Honest ledger, resolved on time
  Brier 0.205 | reliability 0.016 | resolution 0.045 | uncertainty 0.233 | overdue 0% | passes
B. Same ledger, failed claim M-17 quietly deleted and chain rebuilt
  Brier 0.196 | reliability 0.015 | resolution 0.049 | uncertainty 0.230 | overdue 0% | CHAIN BROKEN from 2028
C. Failures left unresolved, chain intact
  Brier 0.185 | reliability 0.054 | resolution 0.035 | uncertainty 0.165 | overdue 20% | SELECTIVE RESOLUTION (20% of due claims left open)
D. Negative control: every claim at 50%
  Brier 0.250 | reliability 0.000 | resolution 0.000 | uncertainty 0.250 | overdue 0% | NO RESOLUTION SKILL
  M-17 in honest ledger: [{'kind': 'claim', 'id': 'M-17', 'year': 2028, 'p': 0.8, 'due': 2030}, {'kind': 'resolve', 'id': 'M-17', 'year': 2030, 'outcome': False}]

The positive case is ledger A. The chain matches every published head, no due claim is left open, and the audit passes. Its Brier score of 0.205 beats the baseline of 0.233, the score she would have got by always predicting the overall rate. That is an improvement of about 12 per cent, which is modest and real. The calibration error, 0.016, is small, and the resolution, 0.045, shows she told true claims from false ones better than chance.

The negative cases are B and C, and they show why the protocol has two separate defences. In B, deleting the failed claim M-17 improves the Brier score from 0.205 to 0.196. Cheating pays on the score. But the rebuilt chain no longer matches the head published at the end of 2028, so the audit flags a broken chain from that year. In C, leaving failures open improves the score further, to 0.185, and the chain is intact because nothing was changed after publication. Only the overdue share catches it: 20 per cent of due claims were never judged.

The negative control is D. A ledger of claims all at 0.50 scores exactly 0.250, with zero calibration error and zero resolution. The audit labels it as showing no resolution skill. If the auditor had found skill in a ledger built to have none, that would have meant a bug.

A minimal worked case: three claims

Take the smallest ledger that shows every step, in which Meenakshi enters three claims in 2026. Claim one, at 0.90, says the headset will record usable data in at least nine sessions out of ten; it comes true. Claim two, at 0.60, says a pilot will show a positive effect; it does not. Claim three, at 0.30, says a rival lab will report the same effect first; it does not.

The Brier score is the average of the three squared errors: 0.01, 0.36 and 0.09, giving 0.153. The chain has three links. The head for 2026 is the fingerprint of the third entry, which folds in the second and first. She publishes it on 31 December. If she later changed claim two's number from 0.60 to 0.40, the fingerprint of claim two would change, so would claim three's, and the recomputed head would no longer match the published one. Three entries are enough to show the whole idea.

A realistic worked case: Sam's audit in 2036

In 2036 Sam receives a file of 108 events, the claims and their resolutions, and a list of ten published heads. He runs the auditor, which reports ledger A's numbers. He then does three things the auditor cannot do for him.

First, he reads the reliability diagram and notes that the 0.70 band is the weakest, with claims there coming true only half the time. Second, he sorts claims by layer and grade, and finds what the 2036 auditor thought experiment predicted: the scorable claims are at L1 and L2, and the L4 and L5 claims are unscorable. Third, he checks the papers Meenakshi published during the decade and finds two predictions that were not in the ledger. He lists them in his report as gaps.

His report says four things. The ledger is intact and honestly resolved. Her forecasting is modestly better than baseline and slightly overconfident at 0.70. Two published predictions were never logged. And the programme's central claims remain untested by any method in the ledger. Every sentence in it can be checked by anyone with the same file.

The negative control and the counterexample

The negative control is the flat ledger, D, and it behaved as it should. The counterexample is the drawer. Suppose Meenakshi had kept a private notebook alongside the ledger and entered only claims she was sure of. The auditor would have reported a clean chain, no overdue claims and an excellent score. Nothing in the program could catch it. Sam's third check, comparing published papers against the ledger, is the only line of defence, and it works only for claims that reached print.

Where the method stops working

The protocol has three boundaries, and the first is sample size. With fifty-four claims, calibration at any single level rests on five to fifteen claims, and the diagram is noisy. Below about thirty resolved claims, I would not read calibration at all, only the overdue share and the chain.

The second is claims with no possible resolution. The ledger can hold them indefinitely, but it cannot score them. A programme whose central claims are all unscorable will have a ledger that is honest and silent at the same time.

The third is the self-fulfilling claim, which the meditator's ledger showed. When writing a forecast changes what is forecast, the score measures a mixture of judgement and influence. The ledger can tag such claims, but it cannot separate the two.

What would change my mind

Four results would make me revise the protocol or give it up.

The first would be evidence that numeric confidence makes researchers worse. If a careful study found that researchers asked to give numbers became more overconfident, or made fewer bold claims, than researchers using words, then rule two would be doing harm. I would then want to know whether coarser bands, or words with numbers attached only at resolution, did better.

The second would be evidence that published heads do not deter tampering in practice. If ledgers with published heads were altered as often as ledgers without them, and the alterations went unnoticed because nobody ever checked, the chain would be theatre. The cryptography would still work, but the social practice around it would not.

The third would be a long-run comparison showing that ledgers do not improve calibration. Mellers and colleagues found that tracking and training improved forecasters in a tournament. If researchers who kept ledgers for a decade were no better calibrated at the end than at the start, the feedback argument in this paper would fail, and the ledger would be a record and nothing more.

The fourth would be a programme in which the ledger made things worse for the field. If public ledgers led to harassment over failed predictions, or to researchers avoiding the hard questions that the ledger cannot score, the protocol would be pushing science away from where it is most needed.

What I still don't know

I don't know how many claims a solo researcher can keep and resolve over ten years without the ledger becoming a burden. Six a year is easy, whereas sixty might not be. The right number probably depends on the field, and I have no data on it.

I don't know how to score claims whose resolution depends on judgement, such as "the field will come to regard this measure as valid." A named resolver helps, but two resolvers might disagree, and I have no rule for that case. Paper 5 takes up a closely related problem.

I don't know how to handle the self-fulfilling claims that the meditator's ledger raised. Tagging them is a start, but whether their scores should be discounted, and by how much, is open.

I don't know whether readers will look at the overdue share and the layer breakdown, or only at the headline score. If they look only at the score, the protocol's most important defences will be invisible to the people who most need them.

Claim ledger

These are the paper's main claims, entered under the rules the paper sets out. Confidence uses the three bands defined in the protocol section: established (0.90 or above), likely (0.60 to 0.89) and speculative (0.30 to 0.59).

There are two dated predictions. First, by 31 December 2030, at least one peer-reviewed journal in psychology, neuroscience or consciousness science will publish a paper whose author or authors report a calibration score computed from their own earlier, dated predictions. I will judge it by a literature search and record the first example, or record the prediction as failed. Second, by 31 December 2031, this series' own ledger will contain at least twenty resolved claims with a published Brier score and overdue share. That one is a prediction about my own conduct, and it is scored the same way as the others.

Glossary

Bādhaka बाधक

A defeater: a later cognition that overrides an earlier one. In this paper, the role played by a resolution or a discovered error.

Brier score

The average squared difference between stated probabilities and outcomes counted as 1 or 0. Lower is better.

Calibration

The match between stated confidence and how often claims at that confidence come true. Measured by the reliability term.

Chain head

The latest hash in a hash chain, which depends on every earlier entry. Publishing it fixes the ledger's state at that moment.

Evidence grade

A separate rating, A to D, of how strong the support for a claim is, independent of the claimant's confidence.

Hash chain

A sequence of entries in which each fingerprint is computed from the entry and the previous fingerprint, so that any change breaks every later link.

Overdue share

The proportion of claims past their resolution date that have not been resolved. A check for selective resolution.

Parataḥ prāmāṇya परतः प्रामाण्य

Extrinsic validity: the view, held by Nyāya, that a cognition's validity is certified by something beyond it.

Prāmāṇya प्रामाण्य

The validity or epistemic reliability of a cognition.

Resolution (in scoring)

The part of the Brier score that measures how well a forecaster separated true claims from false ones.

Retraction

A new ledger entry withdrawing a claim whose basis was faulty. The original stays and is still scored.

Selective resolution

Resolving favourable claims promptly and leaving unfavourable ones open, which improves a score without changing any entry.

Svataḥ prāmāṇya स्वतः प्रामाण्य

Intrinsic validity: Kumārila's view that a cognition is justified by default until a defeater arrives.

Vimarśa विमर्श

Reflective awareness. RTM layer 3, reflexivity, borrows the name.

The ledger that went to the funder

It is March 2036, and Sam Okafor sends his two-page report to the funder. The first page gives the numbers: a chain intact against ten published heads, no overdue claims, a Brier score of 0.205 against a baseline of 0.233, slight overconfidence in the 0.70 band. The second page gives the rest: two published predictions never logged, and a programme whose central claims about self-recognition sat unscorable for the whole decade.

Meenakshi reads it in the room above the bookshop and does not like the second page. But she can check every line of it, and so can anyone else. She adds three entries to her ledger that afternoon. The first logs the two missing predictions, late, with the date of entry and a note that they were found by an auditor. The second revises her confidence in the self-recognition claim downwards, from 0.60 to 0.40, citing ten years without a measure. The third is a new claim, scorable this time, about a task that might one day reach the fourth layer.

The notebook from 2026 is still on her shelf. It has forty claims on its first page, some crossed out, some rewritten, none dated after the first week. Nobody, including her, can say what it shows. The ledger beside it has a decade of entries that anyone can check. It shows a researcher who was often wrong, said so, and got a little better.

For the funder, the decision is simple. The ledger is silent on whether her theory is true, and what it shows instead is how far her claims can be trusted, which is the question a funder can act on. They renew the grant for three years, on the condition that the self-recognition claim gets a resolution date.

Sam keeps a copy of the published heads in his own files, as auditors do. If he is asked to look again in 2039, he will not need to trust anyone's memory, including his own, about what the ledger said in 2036. The comparison will take him a few minutes, and the answer will be one that Meenakshi, the funder and any rival can reproduce exactly. That reproducibility is the whole point of the protocol, and it is the one thing the notebook on her shelf could never offer.

[AUTHOR NOTE: [AUTHOR NOTE: if you have kept any form of dated prediction log for your own research, describe it here in one paragraph, including what it taught you. If you have not, delete this note.]]

Sources

Cite this work

@misc{mahapatra_U0016,
  author       = {Mahapatra, Rajesh Ranjan},
  title        = {The Claim Ledger Protocol: confidence, evidence grades
                  and retraction for a ten-year programme},
  howpublished = {Mind Frontier, paper U0016},
  url          = {https://rajeshrmahapatra.com/papers/mind-frontier/U0016-claim-ledger-protocol-confidence-evidence-grades-and-retraction.html},
  note         = {DOI: to be assigned}
}