Calculating the Shapley Value of Impact

Note

Ozzie Gooen · December 2019 · Completeness: ★★☆ (developing)

Unpublished QURI draft, posted as-is — rough and unfinished. Nuño Sempere’s comments are restored at the end (with permission); other collaborators’ comments have been removed.

Ozzie Gooen’s take

A very short and humorous post, with the goal of highlighting the importance of ontology in Shapley values. A lot of people seem to think Shapley values are the ideal solution for credit assignment; I still don’t agree.

Opus 4.8’s summary

A December 2019 draft structured as a problem set on credit assignment using Shapley values, built around a recurring scenario in which a person named Emma rescues drowning puppies under varying circumstances (a helper named Phil, an inattentive bystander, a rock, paperclip maximizers, splinter activist groups, etc.). The opening case is worked through twice: first arguing the “correct” attribution would recurse into ancestors and counterfactual replacements (yielding only infinitesimal individual shares), then offering a bounded version that gives intuitive numbers. The remaining scenarios are posed as exercises with the answer fields largely left blank. The draft is short and incomplete as written; Nuño Sempere’s restored comments at the end supply intuitive answers to several of the questions and debate whether the “ancestors” point is a genuine objection, with Ozzie clarifying it is meant to surface counterintuitive features rather than refute Shapley values.

Question:

Your name is Emma. You see 50 puppies drowning in a pond. You think you only have enough time to save 30 puppies yourself, but you look over and see a person in the distance. You yell out, they come over (their name is Phil), and together you save all the puppies from drowning.

Calculate the Shapley values for:

• You (Emma)
• Phil

---

The correct answer, of course, for both, should have been “an infinitesimal fraction” of the puppies. In your case, your parents were necessary for you to exist, so they should get some impact. Their parents too. Also, there were many people responsible for actions that led to your being there through some chaotic happenstance. Also, in many worlds where you would have not been there, someone else possibly would have; they deserve some Shapley value as well.

In moral credit assignment, it seems sensible that all humans should be included. That includes all those who came before, many of whom were significant in forming the exact world we have today.

However, maybe we want a more intuitive answer for a very specific version of the Shapley value; we’ll only include value from the moment when we started the story above.

Now the answer is Emma: 40 puppies, Phil: 10 puppies. In total, you share 50 saved puppies. You can tell by trying it out in this calculator.

Now that we’ve solved all concerns with Shapley values, let’s move on to some simpler examples.

---

Question:

You (Emma again) are enjoying a nice lonely stroll in the park when you hear a person talking loudly on their cell phone. Their name is Mark. You stare to identify the voice, and you spot some adorable puppies drowning right next to him. You yell at Mark to help you save the puppies, but he shrugs and walks away, continuing his phone conversation. You save 30 puppies. However, you realize that if it weren’t for Mark, you wouldn’t have noticed them at all.

Calculate the Shapley values for:

• You (Emma)
• Mark

---

Question:

You (Emma again) are enjoying a nice lonely stroll in the park when you hear a rock splash in a pond. You look and notice some 30 adorable puppies drowning right to it. You save all of the puppies. You realize that if it weren’t for the rock, you wouldn’t have noticed them at all.

Calculate the Shapley values for:

• You (Emma)
• The Rock

---

Question:

You (Emma again), are enjoying a nice stroll in the park. Alarmedly, 29 paperclip maximizers inform you that a paperclip is going to be lost forever, and 30 adorable puppies will drown unless you do something about it. You, together with the paperclip maximizers, spend three grueling hours saving the 30 puppies and the paperclip.

Calculate the Shapley values for:

• You (Emma)
• Each paperclip maximizer

---

Question:

You (Emma again), decide that this drowning puppies business must stop, and create the Puppies Liberation Front. You cooperate with the Front for the Liberation of Puppies, such that the PLF gets the puppies out of the water, and the FLP dries them, and both activities are necessary to rescue a puppy. Together, you rescue 30 puppies.

Calculate the Shapley values for:

The Puppies Liberation Front:

The Front for the Liberation of Puppies:

---

Question:

The Front for the Liberation of Puppies splits off a subgroup in charge of getting the towels: The Front for Puppie Liberation. Now;

• The Puppies Liberation Front gets the puppies out of the water

• The Front for the Liberation of Puppies dries them

• The Front for Puppie Liberation makes sure there are enough clean & warm towels for every puppie.

All steps are necessary. Together, you save 30 puppies. Calculate the Shapley value of:

The Puppies Liberation Front:

The Front for the Liberation of Puppies:

The Front for Puppie Liberation:

---

Your name is Emma. Phil sees 30 puppies drowning in a pond, and he yells at you to come and save them. To your frustration, Phil just watches while you do the hard work. But you realize that without Phil’s initial shouting, you would never have saved the 30 puppies.

Calculate the Shapley values for:

• You (Emma)
• Phil

---

You are Emma, again. You finally find the person who has been trying to drown so many puppies, Lucy. You ask how many puppies she threw into the water: 100. Relieved, you realize you (and you alone) have managed to save all of them.

Calculate the Shapley values for:

• You (Emma)
• Lucy

Comments from Nuño Sempere

Restored with permission (Nuño’s comments, with Ozzie’s replies).

On “The correct answer”:

Nuño Sempere: Thought: Suppose that you have a decision theory. This decision theory tells you which agents will act differently if you add differently / would have acted diferently if you choose to act diferently. Use that subset of agents as the participants in your SV calculations.

Nuño Sempere: (This fails because it works badly with actions which people have taken in the past. But I think it points at something).

On “parents”:

Nuño Sempere: I don’t think this is a take-down argument. Suppose that our ancestors are responsible for 99.99% of all Shapley value; maximizing for SV and maximizing for SV*0.01 has the same results.

Ozzie Gooen: It’s not meant as a take-down argument. I like shapley values (better than counterfactuals, where applicable), but am just trying to point out some counterinuitive things. If I release this as a blog post I’ll write a paragraph or so to make that more clear.

On “Calculate”:

Nuño Sempere: Intuitive: 30 & 0

Nuño Sempere: Another interesting thing is that if Mark is not optimizing for Shapley values, sure, in this case he’ll get some SV (and some counterfactual impact), but over time, altruists will still get more SV.

Ozzie Gooen: If that’s the case though, there’s a big gradient between Mark and a rock that we could get into. Would an urn of someone’s dead ashes count? Someone in a coma? Dirt that contains a fraction of the remains of someone who died?

Nuño Sempere: Intuitively: 30 and 0

On “Calculate the Shapley values for:”:

Nuño Sempere: Intuitively, 30 puppies and 1/30th of a paperclip

Nuño Sempere: So the answer to this is +50 & - 50, respectively, which I still find unintuitive.