A StellarMinds.ai research demo · private preview

Seven agents vote. How much should each vote count?

A mesh of agents decides by voting. Three of them are confidently wrong, and they are three of seven, so on a close call they carry it. Counting heads can't tell them from the four that know something.

Here, every vote comes with a stated confidence, and every decision is later confirmed right or wrong. An issuer signs each agent's record. Whoever relies on the vote checks that record, and weighs the vote by how often the agent has actually been right when it spoke that way.

Checking for a live service…
Counting heads

Every vote counts the same

  • Four agentsKnow something, and say how sure they are.
  • Three agentsBuilt a good record, then started voting wrong at 0.99.
  • The tallyThree wrong votes need only one honest mistake to win.
The group is right about one time in four.
Weighing by record

A vote counts for what its record supports

  • The issuerConfirms each outcome and signs each agent's record: how often it was right at each level of confidence.
  • The relying partyChecks the signature, recomputes the numbers, and reads each new vote through that record.
  • A confident strangerSays 0.99. Its record has nothing at 0.99, so the claim counts for nothing.
The group is right about three times in four.
01 / How it works

Five steps. The issuer saw the outcomes, so the issuer signs the record.

The agent can't write its own score. The only way to change the record is to take part in a decision whose outcome is later confirmed.

1 · Vote with a confidence

Each agent names its answer and says how sure it is. The confidence is a prediction that will be checked, not a weight to be taken on trust.

answer + 0.82
2 · Confirm the outcome

After the decision, the issuer confirms what happened. Each vote becomes one line in the agent's record: what it said, and whether it was right.

{c: 0.82, y: 1}
3 · Sign the record

The issuer signs a summary: the calibration error, and a table of how often the agent was right at each level of confidence. A Merkle root commits to the lines behind it.

CalAt: ece, dist, der.root
4 · Check and recompute

The relying party checks the signature against the issuer's key, then recomputes the error from the table. It doesn't take the issuer's arithmetic on trust.

verify → recompute
5 · Weigh the vote

A new vote at 0.9 counts for how often this agent has been right at 0.9. An agent that is calibrated but no better than a coin earns the floor.

0.9 → record → weight

Nothing is added to the signed object to do step 5. The weight is computed from what the attestation already carries.

02 / The stories

Seven populations, eight ways to count. Run one yourself.

Each run is 200 rounds of warm-up, in which records build up, and 200 scored rounds. The bars show how often the group got the right answer in that one run, under each way of counting. The line under them is the average over 200 runs, which is the number to trust.

03 / What the issuer signs

Check the latest attestation yourself, in this browser.

This page checks the issuer's signature itself, with your browser's built-in cryptography, and recomputes the numbers from the signed table. It doesn't ask a server whether the attestation is valid.

No run yet.

A valid signature proves that the issuer's key signed exactly this record. It doesn't prove the outcomes in it happened, or that the issuer confirmed them honestly.

For the technically minded: the attestation is RFC 8785 canonical JSON with the sig member removed, signed with Ed25519. The issuer is a did:key, so its public key is in its name, and sig.kid must sit under iss. The demo's keys are derived from public labels and protect nothing.

04 / What it is

What this is, and what it isn't.

What this is

  • A working demo of the Calibration Attestation (CalAt): a signed, portable record of how an agent's stated confidence has matched confirmed outcomes.
  • The runs execute the reference library, sm-calat, and its benchmark. The attestation in section 03 is a real one: signed, and checkable.
  • The honest result of the benchmark: weighting by calibration error alone, the first design, was no better than counting heads in five of seven populations. Two rules that replaced it are better in all seven.

What it isn't

  • Not a deployment. The agents are simulated. Their mistakes are independent, every outcome is confirmed, and each adversary follows one fixed plan.
  • Not better than everything. Plain "how often was it right before" is a strong rule that nothing here beats everywhere. What an attestation adds is that the record is signed, travels between organizations, and can be recomputed.
  • Not a defense against copies. Four copies of one agent are one opinion with four votes. Every rule here over-counts them.
  • Not proof the issuer is honest. The signature shows who issued a record, not that the record is true.
  • Not published. The draft and the library are private and unsubmitted. This page is a preview.
05 / Resources

The draft, the code, and the evidence.

These open in the private repository, for people who have access to it.