Decision Value Research Agenda
Decision Value Technologies Research Agenda
Section titled “Decision Value Technologies Research Agenda”Corresponding Video: https://www.youtube.com/watch?v=X_NvXIFoZG0
Status: In-progress. Seeking feedback on the large details in particular (naming, groups, missing areas, overall utility). Also would be interested in which specific aspects excite people, or if there are references that would be particularly useful for some of this that I probably don’t know of. I may ‘reframe’ the agenda significantly, though I imagine a lot of the overall approach will remain.
Notes: Currently this is an individual project by Ozzie Gooen. It is not apparent how much it will actually be carried out.
Introduction
We can call the total value of information to an agent for its future decision making “Decision Value.” Technologies that assist in this aim could thus be called “Decision Value Technologies.” We think that it’s possible and tractable to build software technologies to help optimize decision value. This will require a lot of engineering effort, but also a lot of original theoretical work to properly ground and guide that engineering.
The decision value technologies research agenda seeks to optimize the ability of organizations to optimize decision value, with the larger aim of optimally benefiting future sentient life. The research will focus on principled approaches of combining human intuitions and formal calculations in order to understand how to best use concepts like decision value and expected value.
Agenda Rigidity
Work on decision value should aim to be maximally useful. Constraining research with a rigid scope would likely limit this intention; therefore, any laid out intentions should be considered as a useful framework for research guidance, rather than as an exact plan. The following is considered a best current guess at a plan to optimize decision value on research on decision value, but it is expected to change dramatically to maximize effectiveness.
Key Research Approaches
Section titled “Key Research Approaches”There already exists a lot of research that has to do with decision making, but we feel like there are some important gaps. Here are a few ideas we intend to focus on that we feel are particularly tractable.
Principled / Probabilistic
We seek to ground our work in Bayesian theory and epistemology. The research will focus on “principled” approaches, in contrast to “pragmatic tools”. This is similar to choosing “neat” research in the neats and scruffies distinction.
Engineering-Oriented
One focus is on the development of software tools to augment humans in using advanced estimation and decision making techniques. Eventually, this may include machine learning, with the goal of advancing the use of narrow but safe intelligence for human intelligence improvement.
Rapid Experimentation
Rather than focus on formal experiments, we seek to engage in lots of individual and technical experimentation. This is in the vein of some Silicon Valley groups and engineering research.
Self-Use
If research goes well, then hopefully it can be recursively applied to improve prioritization for future decision value research. The research effort itself could be a good testbed for ideas that it produces. If research doesn’t go well, it should be canceled.
Research Domains & Methods
Section titled “Research Domains & Methods”On a high level, one can look at research through three important dimensions: domains, methods, and projects.
Research Domains
Section titled “Research Domains”- Epistemology
- Questions
- If one agent sees a prior of another, how much should that agent update?
- Are there structured approaches for understanding magnitudes of uncertainty and how they propagate in systems?
- This could look like a combination of category theory and Bayesian statistics.
- If we have uncertainty about what calculations to perform, can we use simple models but add uncertainty to the results in a structured manner?
- How can one best handle overconstrained* beliefs?
- Questions
- Categorization
- This domain seeks to understand how categorization systems can be used to best optimize decision value. This includes issues around ontologies, knowledge management, and practical tools.
- Questions:
- How can one best estimate the decision value of a given ontology to a given agent?
- How can an organization work together to optimize their primary ontologies?
- Can we find patterns or high-level ontologies that agents can use to maximize decision value?
- What web collaboration tools would be most effective to allow groups to build useful ontologies?
- Can ML systems help teach us to make better ontologies or make them for us?
- Estimation
- Estimation really lies at the heart of what is expected to make up decision value research. We’re mostly interested in estimation that includes the use of human intuitions combined with calculation. There already exist many mathematical techniques for estimation in the case of certain input, but there is significantly less literature in cases where many of the inputs come from human intuitions.
- Questions
- Overarching: How can a large group of people best estimate a single variable?
- How can an agent best adjust for model uncertainty?
- How can a group of people best share a knowledge base of common priors?
- What are the best probability distributions and techniques for humans to describe their intuitions about important variables?
- How do we probabilistically combine the results of multiple models, especially in cases where they share multiple parameters?
- What are the best ways for probability distributions to be entered and stored in software systems?
- How can forecasting be best used for maximizing decision value?
- Optimization
- Optimization here describes the use of estimation in order to optimize certain variables, the main ones being variants of expected value.
- Questions
- What are good ways of discussing things around expected value and decision value?
- Instead of treating estimation as a constraint optimization problem, perhaps it should be an expected value optimization problem. How should this work best in practice?
- What common patterns can we use to estimate things in ways that will optimize decision value?
- Descriptive Expected Value
- Most attempts at optimizing human utility so far have typically focussed on things that are relatively simple to understand, like income and commuting convenience. However, many people take actions that make it seem like they care a lot about things that are much more difficult to measure. These should be better understood if one would want to better use numbers to help humans. “Descriptive expected value” is meant to attempt to best model agent’s actions as optimizations of utility functions, rather than “normative expected value”, which more attempts to make decisions using possible utility functions.
- Questions
- Can we better understand human actions in terms similar to expected values, and use this knowledge to better optimize their utility?
- How can we numerically estimate:
- The expected value of important but strange things to humans, like signaling and identity preservation?
- The expected value agents gain from making decisions to be strategically incorrect about things?
- What models of expected value maximization best explain why people and organizations have things that seem like biases and irrationalities?
- Are there ways to model subsystems of the brain, to act as alternatives to treating entire people as “rational agents”?
- Feasibility
- Questions
- How can individuals, organizations, and large networks best adapt practices to better maximize expected value? Where will the incentives correctly align to make this possible?
- Historically, why have tools such as Bayesian analysis, prediction markets, and probability distributions, seen so little adoption in comparison to what one might expect?
- Existentialism: Attempting to optimize for expected value can seem “cold-hearted.” How far can humans reasonably go in this dimension, and how can they feel comfortable with it?
- Questions
Research Methods
Section titled “Research Methods”- Historic Analysis
- Summaries & Meta-analyses
- Finding, collecting, and documenting existing research most useful for decision value.
- Conceptual Research
- Mathematics and philosophical study would be considered “conceptual research.”
- Statistical Analyses
- This encompasses most of the work that could also be called “data science.” While statistical analysis could be done in scientific experiments, it could also be very useful to do on existing data.
- Scientific Experiments
- This includes formal scientific experiments.
- Technology Development
- The main aspect here is web application and software application work.
- Direct Applications
- This includes attempting to directly apply decision value research to fields where it would be high in expected value. This can be called “applied” decision value research, as opposed to “theoretical” decision value research. This research agenda is focussed on optimizing theoretical decision value research, so all direct applications would be considered based on how much they would help the theoretical research. There are two main ways this could occur: one, to better test the theoretical ideas in order to improve them, and two, to be used to optimize prioritization and strategy on the theoretical research.
- Advocacy & Training
- The main benefit of decision value research would be from other people, optimizing for global wellbeing. The main groups that would seem to use the research to optimize this are expected to be specific organizations involved in charity and high-leverage global issues.
Research Domain / Method Matrix
Section titled “Research Domain / Method Matrix”Each combination is rated from 0-5 for how much emphasis we expect it to have under this agenda.
| Epistemology | Categorization | Estimation | Optimization | Descriptive Expected Value | Feasibility | |
|---|---|---|---|---|---|---|
| History | 1 | 1 | 1 | 2 | 1 | 2 |
| Summaries & Meta-analyses | 1 | 1 | 2 | 1 | 1 | 1 |
| Conceptual Research | 2 | 3 | 4 | 3 | 2 | 1 |
| Statistical Analysis | 0 | 1 | 1 | 1 | 1 | 1 |
| Formal Scientific Experiments | 0 | 1 | 2 | 1 | 1 | 0 |
| Technology Development | 0 | 2 | 5 | 3 | 2 | 1 |
| Direct Applications | 0 | 1 | 3 | 2 | 2 | 2 |
| Advocacy & Training | 1 | 1 | 2 | 2 | 1 | 1 |
Benefits / Costs / Risks
Section titled “Benefits / Costs / Risks”Benefits:
- Improved decision making
- Research would allow people close to it to make better decisions.
- Positive signaling
- Researchers and research users can signal that they have thought about things thoroughly. This could be useful for convincing others if one is confident enough to rigorously test their beliefs.
- Consequentialist encouragement
- Research may encourage users to adopt consequentialist principles
Costs:
- Opportunity costs
- (funding, talent, attention)
- Information hazards
- We could make bad groups more effective
- Risk of negative signaling
- We could make ourselves look bad if we do poor work
- If we do provocative work (rate external groups), it could upset other groups.
- Risks of worsening decision making
- If we do a poor job, we could, on the whole, make groups make worse decisions
- Risk of Negative Founder effects
- We could displace better people who would come along after
Risks to effectiveness (distinct from risks that could make the project net-costly):
- Challenges getting funding
- Intractability
- Perhaps all of this territory is very difficult to make progress on and isn’t worthwhile because of that.
- Ozzie-specific
- Current “bus-factor” of 1.
- Not enough experience for either good work or signaling
- Personal accident or different job opportunity
- Short Timelines
- Other global problems may happen too quickly for research to be useful.
Key Research Influences
Section titled “Key Research Influences”Key Academic Influences:
- Management
- Peter Drucker
- Douglas Hubbard
- LEAN
- Bayesian Epistemology
- Forecasting
- Consequentialism / Utilitarianism
- Strategy Research
- Effectiveness Research
- GiveWell / Open Phil’s writings on expected values.
- FHI’s work on Macrostrategy
- Automated/factored reasoning research
- OpenAI & Ought
- Logical Induction
- Cybernetics
- Crowdsourcing
Key Technology Influences:
- Ontology Tools
- Probability Tools
- Forecasting Tools
Explicit Non-Coverage
Section titled “Explicit Non-Coverage”“Decision Making” Research
- Psychology
- There are large departments aimed at understanding decision making. These are typically focussed on “descriptive” decision making, rather than “normative” decision making. In their descriptive studies, they typically don’t study the specific issue of how human preferences could be best numerically estimated in terms of expected values or similar.
- Business
- There are many popular business-focused books and managerial courses. Most of these study “best practices” for businesses, based on historical information. In contrast, our work is aimed to be more normative, theoretical, and statistically principled.
Data Engineering & Data Science
While software engineering, data engineering, and data science are all useful in decision making, they are also already very well studied, and also don’t optimize decision value in a systematic or explicit way.
Advanced Decision Theory
Logical Fallacies and Cases of Poor Decision Making
Rather than focus on the possible and frequent errors of existing decision making, we mainly seek to understand optimal normative decision making. In some ways, this is a much smaller space.
Online Debate Platforms
Several debate platforms have launched in the last few years. These could be useful, but typically do not focus on probabilities and are not principled in the ways we will focus on.
Concept and Mind-Mapping Software
Mind-mapping software can be quite difficult to make, and also typically does not focus on probabilities and mathematical integrations.
Comments from Nuño Sempere
Section titled “Comments from Nuño Sempere”Restored with permission (Nuño’s comments, with Ozzie’s replies).
On “How can one best handle overconstrained beliefs?”:*
Nuño Sempere: This seems interesting, and is also something I might have thought about before
On “What are the best probability distributions and techniques for humans to describe their intuitions about important variables?”:
Nuño Sempere: Interesting
On “and how can they feel comfortable with it?”:
Nuño Sempere: This seems interesting, but also maybe \not that hard to solve\ by creating better ideologies