Intellectual Jury Duty

Note

Ozzie Gooen · January 2022 · Completeness: ★☆☆ (sketch)

Unpublished QURI draft, posted as-is — rough and unfinished; collaborator comments have been removed.

Ozzie Gooen’s take

The main challenge here was how to ground forecasts (and similar) on abstract questions. I later found that Metaculus used expert panels to resolve a few questions.

Opus 4.8’s summary

A short 2022 sketch proposing that contested matters of public or intellectual opinion (not legal questions) be resolved by randomly-sampled, deliberating juries rather than expert panels. It gives example questions, suggests an effective-altruist jury with an entry bar, and sketches a lightweight process (a Slack channel, research assistants, party representatives, and a few hours of deliberation per juror, ideally made public). It weighs the jury model against an expert panel, guessing juries would be cheaper and more trusted by junior community members but noisier and less trusted by senior ones, and connects the idea to deliberative democracy and citizens’ juries. The piece is brief and exploratory, marked as a sketch. It closes by noting that any jury or panel could be forecasted against, so estimates could amplify a small number of jury judgments across many questions.

Intellectual Jury Duty

For a lot of reasons, it’s useful to be able to get reasonable judgements about intellectual questions.

Questions like:

• “Was Steven Pinker broadly right in his thesis on violence?”
• “Was charity X a scam, or a legit operation?”
• “Was Google in the wrong, when they shut down a highly-used service?”
• “How successful EAs government program or department Z?”

I’m not talking about legal questions. Instead, key questions of public/intellectual opinion. De facto, not de jure.

There are clearly some questions where audience members have already made up their minds.

Intellectual juries would probably be optimized to appeal to different intellectual groups.

I’m now most interested in having an effective altruist jury. There could be a decent bar to be considered, like “attended 1 EAG conference” or “elected by 3 effective altruists.” Then, people in this pool would be randomly selected to be jury members on different questions.

It’s not clear how long or how sophisticated the deliberation process would be. I’m imagining a slack channel with a research assistant or two, maybe representatives of involved parties, and the jury members. Each would be expected to spend 3 to 20 hours researching/discussing each option. Ideally much I’d the discussion could be made public, in part so that others could chime in.

The obvious alternative to “Jury Duty” is “A panel of experts”; which would be more similar to the Supreme Court.

My guess is that the jury model would be more trusted and liked by junior community members, but less trusted and liked by senior community members. It might take less expensive resources (which is good), but is likely to output more random results (which is bad).

One plus is that the jury model could act better as a survey of what the average person would believe after deliberation.

This model is very similar to Deliberative Democracy and Citizen’s Juries. (Shout out to Zoe Cremer for flagging Deliberative Democracies).

Given that effective altruists already seem to like charity lotteries, having “deliberation lotteries” (this is what jury duty is) seems like it could be easily enough understood and appreciated.

Naturally, if any jury or expert council is set up, it could be forecasted against. So we could have thousands of estimates for things like, “how valuable was intervention X?”, and then juries could be targeted at very specific uncertain questions there. The estimates would effectively be amplifying (or phantomizing) the juries.