Inconvenient Truths
Value of Information calculations typically assume that information is always beneficial. But I think most observant people who have dealt with people and organizations realize that this is clearly not always true in real settings.
I think it should be quite clear that there are many cases where individuals or organizations get more value when they either have wrong beliefs or where they lack specific knowledge. Historically, there were many cases where having correct beliefs were highly risky, and the much more advantageous thing to do was to have specific convenient beliefs.
I would like to better understand these phenomena by adapting models or rational agents to incorporate some of these effects.
The purpose of this work is not to argue that we should have more incorrect beliefs. It’s to try to understand what is actually happening, so we can get a better picture of what to expect and how to improve things going forward. I think Step 1 for overcoming inaccurate beliefs isn’t to directly identify these beliefs, but to accept that one has inaccurate beliefs and figure out what purposes they are serving. Then one could be strategic about optimizing.
Some examples of inconvenient truths:
- A parent might not want a list of all the ways they did a poor job at raising their children.
- If someone thought that society would hate them for their sexual orientation, they might feel safer not discovering this themselves. Doing so would pose a risk.
- If a financial institution was to discover that employees were committing large amounts of fraud to external parties, it might well prefer not to know. It’s possible that fixing the problem would be more costly than it would be worth, and having any documented evidence that they were aware and didn’t fix the problem could open them up to substantial legal risk. Better not to know.
An ideal Bayesian agent may be fine at handling these sorts of knowledge, but humans are not ideal Bayesian agents. Humans are often quite poor at keeping deep secrets, and humans are prone to significant feelings of guilt and shame upon knowing certain information. Organizations are made up of humans so have similar biases, plus have additional and operational legal issues.
There are clearly cases of truths that are expected value negative to realistic individual agents. Let’s call these inconvenient truths. Likewise, truths that are EV-positive to an agent is a convenient truth.
Inconvenient truth:
A truth that is EV-negative for a given agent, when either revealed to that agent or to another agent. The agent could be defined as having a high discount function. For example, one could say that a specific truth is “inconvenient in the short term”, but “convenient in the long term.”
EV(truthful claim, agent A, agent B) < 0
Convenient truth:
A truth that is EV-positive for a given agent, when either revealed to that agent or to another agent.
EV(truthful claim, agent A, agent B) > 0
Note: If my group finds out that my previous work turned out poorly, then that would be considered an inconvenient truth for myself. If they find that the passcode to my personal safe was 38492, and use that to unlock the safe, that would also technically be an inconvenient truth. I’m not a huge fan of this latter part, as it less matches my intuitions of how the word “inconvenient” should be used. I’m curious if there is any clear line in the sand here.
This choice of definition is clearly different from its use in the movie, “An Inconvenient Truth”. I’ve thought a fair bit about this and couldn’t come up with better terminology. If you have ideas I’d be very open to changing this. My main priority here is to have a solid structure. I like the term “convenient” because I think the connotation is generally correct. When someone asks, “Wouldn’t it be convenient for you if your argument were true” hints at a definition similar to what I am using.
Now that we have this clear definition of convenience, we can expand it for the following related terms:
Convenient Ignorance:
The not knowing of truths, or the ability to keep on not knowing truths, which results in positive expected value.
Convenient Belief:
A belief that is convenient to hold as a belief, even if it might not be true. More specifically, a belief is convenient for agent A, with respect to being known by agent B, if it would be EV-positive for agent A, for agent B to believe this information. Agent B could be agent A.
Convenient oversight:
Ways of hiding information
Secrets from others
Self-secrets
“Known” by unaccepted (Repressed) “Many people know it, but they refuse to say anything”
Active Repression, No action, Lack of Action
Active Repression, Inaction, Active Truth Seeking
Inaction
Passive vs. Active Repression
Instantial vs. Domain Repression
Instantial: Don’t say “Bloody Mary” 3 times.
Domain: Don’t go into the basement.
1st level, 2nd level, n-level.
Inconvenient Repressed Space (The space of all of the inconvenient truths that are being awaited to be discovered)
Correlated vs. uncorrelated inconveniences …
And correlated vs. uncorrelated unknowns
Convenient oversight
Could also be called desirable vs. undesirable truths.
Secrets, Inconvenient Public Truth, Inconvenient Private Truths