Enlightened Willingness-To-Pay Forecasting Exploration
Author: Ozzie Gooen
I’m interested in running an exploratory experiment to use forecasting to help with minor decision-relevant interventions. The goal is to produce a list of the costs and benefits of various interventions in a way that would be interesting and useful.
This experiment may get written up on LessWrong/the EA Forum, but will not be published. It’s not meant to get statistically meaningful results, but rather to get a feel for how this kind of thing would work and how it may break. If you participate or provide help, we will give proper credit in the blog post if we write one.
I’m currently looking for:
- People to give feedback & intervention ideas
- 2-5 people to act as evaluators
- People to act as forecasting
The proposal:
- We produce a list of 30-100 possible interventions or things that we may have willingness-to-pay for.
- A few individuals sign up and agree to be polled (in step 5). These are called the “evaluators”
- The evaluators join a custom Slack channel made for this purpose.
- A team of forecasters spend a few weeks talking to the evaluators (via slack) and making forecasts of (5)
- We randomly sample 5-20 of the items. On each item, the evaluators do the following:
- We can discuss these options in detail and spend 10-60 minutes each (or more, if you want) investigating them.
- We each give our own probability distribution of our willingness-to-pay for that variable.
- The aggregate of (5 b) is used to resolve this question. Everyone gets equal weight. Example: I make estimate my willingness to pay as =normal(5,2). Someone else says normal(8,3). The resolution would be =mm(normal(5,2), normal(8,3), [.5,.5]).
Enlightened Willingness-To-Pay
Enlightened willingness-to-pay should be considered equivalent to what one believes they would think if they had a very large amount of information about an activity, but not the actual result for a certain case. Think of it as the expected value in terms of money for yourself.
Things you should imagine having when being “enlightened”:
- New scientific studies are released.
- You get to read all of the literature now, and over the next 10-30 years on this topic.
- You get to talk to all the possible experts on this topic.
- You can think about this topic for 1-20 years solid.
Things you should not imagine having when being “enlightened”:
- Outcomes of events. For instance, if you were to consider getting laser eye surgery, you won’t know in advance if an error would actually happen; you would just have a very good idea on what the true probability would be.
FAQ:
Question: What if an intervention could be handled in different ways, with different levels of effectiveness? For instance, maybe there’s good and bad running technique.
Answer:
Assume that the evaluators read comments on the questions, but do not assume that they do the optimal thing. For instance, imagine that they read comments suggesting good running technique, but only some realistic fraction actually takes that advice. The reasoning for this is that we care about making the information maximally value for those who read it, but only for those who read it, and can assume that people reading forecasts also read the comments.
Question:
What if the intervention has a few distinct varieties?
Answer:
Assume that the participants do the expected combination of varieties. For instance, if you estimate the “benefits of ketchup”, and there are 3 different types of ketchup, but you know expect that participants will follow some distribution regarding ketchup consumption, then estimate their weighted-average willingness to pay for that distribution.
Question:
What exactly do you mean by “marginal” interventions?
Answer:
Say person X mediates for 20 minutes a day, and you are estimating the willingness to pay for a marginal “10 minutes per day”. Interpret that to mean the difference between person X spending 20 minutes vs. them spending 30 minutes meditating on a random day.
Question:
Why is this focussed so much on personal things, as opposed to more impactful or large-scale things?
Answer:
First, personal things can be valuable to better understand. Second, more impactful things are likely to be more controversial. Starting with personal “life hacks” is expected to be more playful and acceptable.
Question:
Should we assume these interventions take into account the opportunity cost? For instance, when estimating the impact of “10 minutes meditating”, should we include the cost of losing 10 minutes of time of other things?
Answer:
No. Imagine the time comes for free. That way it will be easy to compare costs & benefits separately later on. For instance, an evaluator can weigh the “$38 of value from meditating” from the cost of “10 minutes of their time” by themselves.
Example:
Intervention: The health effects of a marginal 10 grams of white sugar.
Assume that 10 grams of sugar will be added to your body one day without being able to taste it. How much would you be willing to pay to have or stop this from happening? This is likely an unhealthy addition, so perhaps you would be willing to pay $0.10 to stop this. However, you’re unsure about what you would think if you were “enlightened”; had you spent a lot of time investigating the issue and getting science from the future. So you estimate “=normal(-1.2 to -.01)”.
Average American…
Current List (Feel free to add comments with suggestions of other things)
Health
10g white sugar (just health effects)
10g brown sugar (just health effects)
1 pack stevia sugar-replacement (just health effects)
1 diet coke
1 cup coffee, black
5 ounces red wine, 12% alcohol
12 ounces regular beer, 5% alcohol
1 Orthomega Fish Oil Capsule
6 Enhanced Zinc Lozenges taken correctly, for first day feeling sick
6 Enhanced Zinc Lozenges taken correctly, for third day feeling sick
10-minutes “mediocre headache”
10-minutes “mild cold”
2-minutes vomiting
10-minutes meditation
10-minute walk, casual pace
10-minute bike ride, casual pace
1 marginal glass of water
10-minutes extra of primary sleep period
10-minutes walking in nature
10-minutes commuting, driving
10-minutes commuting, walking
10-minutes commuting, riding bicycle
10-minutes of stretching
30-seconds washing hands
1 day of having a cold
1 day of wearing a face mask
Media / Education
10-minutes browsing Reddit
10-minutes browsing Facebook
10-minutes watching Netflix
10-minutes reading the EA Forum
10-minutes reading LessWrong
10-minutes of reading political news on the New York Times
10-minutes reading articles on The Economist
10-minutes reading articles on Breitbart
10-minutes reading a math textbook
10-minutes listening to podcasts
10-minutes watching an in-person academic presentation
Social
10-minutes social dancing
10-minutes talking with close friends
10-minutes talking with casual friends
10-minutes video games with friends
Other
1 micromort
1 microtopia (A 1 of a million chance of a positive AGI Future vs. total sentient extinction)
1 extra year of healthy life, conditional on dying in the next 90 years (to exclude cryonics/AGI).
10 minutes spent commenting on the Slack or Prediction pages for this experiment
1 hour lost today
$1, given 1 year from now
$1, given to self 1 year ago
Spending 10 hours reading this table of forecasts, vs. not being able to have seen them at all.
- This is only to get at the benefit that evaluators will get from making life choices differently. It is not meant at all to cover differences they may make in the actual evaluations had they not seen this information. Spending 10 hours reading this table of forecasts, vs. not being able to have seen them at all, conditional to 10 times as much effort being spent on forecasting.
Meta Predictions