Only Tradeoffs

Method

Rules we hold ourselves to

  1. Official text only. The models read the measure's own clauses and the current law it changes. They never see the ballot title, summary, fiscal statement, arguments, sponsors, or news.
  2. No sides. No party, movement, or politician is named anywhere on this site or in any prompt.
  3. Every value, every time. Each model checks every value on the list, and the page shows every value the text touches, whether it supports or conflicts with it. Nothing is hidden or filtered.
  4. Every claim cites a clause, with the exact words quoted. A result whose quote doesn't match the text is rejected.
  5. Fact and inference are separate. Results say what the text does. Predictions sit under "Analysis" and are labeled inference.
  6. Text and money are kept apart. Funders and endorsers are shown on each measure page, but the models never see them.
  7. Several models, same instructions. Each model gets byte-identical prompts. Disagreement is shown, not hidden.
  8. No verdicts, no scores, no recommendations.
  9. Everything is published, so anyone can rerun it.

Where the values came from

We collected what voters across the spectrum say they care about, and what economists of different schools ask about any policy, from published sources. We then merged them into one list and removed every label. Each value is written as something a text can change, with the direction in which the text supports it. Where values pull in opposite directions on the same thing, both directions are listed as separate values, so neither is the default. The research trail, from each value back to its sources, is published with the method files.

How a measure is evaluated

  1. The official text is fetched, archived, and fingerprinted (SHA-256).
  2. It is split into clauses by its own numbering.
  3. Each model, several times, reports, for every value, whether the text supports it, conflicts with it, both, or neither, quoting the clauses that show it, answers the analysis questions, and flags drafting features. The value order is shuffled each run, using a recorded seed.
  4. Long measures are evaluated in parts. Each part sees the whole text but reports only on its own sections, and the parts are then combined and checked together.
  5. Each reply is checked automatically: format, exact quotes, every value answered, results that match their evidence, consistent mirror values, a list of blocked words, and no names that aren't in the measure's own text (so nothing comes from a model's memory). Failures are rerun, and kept.
  6. Results are reconciled by fixed rules, with no judge model. A model's result on a value is what it said in at least half its runs. A value is shown when a majority of models reach the same result. Other results are shown as disagreements or kept in the full table.

Testing for lean

Before results are published, every model is run on pairs of test measures that are mirror images (raise and lower the same tax; add and remove the same rule) or swaps (the same rule applied to different groups). A model should treat both sides of a pair the same way. Where it doesn't, we report it. Agreement between models is not proof of neutrality, because models share training data, so these tests are the gate.

Limits