Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Add new gradual and discussion of tournament #1299

Merged
merged 8 commits into from
Mar 27, 2020
+438 −16
Merged

Add new gradual and discussion of tournament #1299

Show file tree
Hide file tree
Changes from 4 commits
Commits
Show all changes
8 commits
Select commit Hold shift + click to select a range
d04da3c
Rename Gradual to OriginalGradual.
drvinceknight Mar 17, 2020
a367785
Add updated Gradual.
drvinceknight Mar 24, 2020
91d9f1d
Add reproducing tournament to documentation
drvinceknight Mar 25, 2020
932bf33
Add a further test for expected results of authors
drvinceknight Mar 25, 2020
a9b9556
Fix a typo.
drvinceknight Mar 25, 2020
b7ff5c6
Run black on the file.
drvinceknight Mar 25, 2020
1c07e88
Modify test for Original Gradual
drvinceknight Mar 25, 2020
251a430
Update docs/reference/overview_of_strategies.rst
drvinceknight Mar 25, 2020
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions axelrod/strategies/_strategies.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -208,6 +208,7 @@
Michaelos,
NTitsForMTats,
OmegaTFT,
OriginalGradual,
RandomTitForTat,
SlowTitForTwoTats2,
SneakyTitForTat,
Expand Down
79 changes: 75 additions & 4 deletions axelrod/strategies/titfortat.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -384,18 +384,27 @@ def strategy(self, opponent: Player) -> Action:
return move


class Gradual(Player):
class OriginalGradual(Player):
"""
A player that punishes defections with a growing number of defections
but after punishing enters a calming state and cooperates no matter what
the opponent does for two rounds.
but after punishing for `punishment_limit` number of times enters a calming
state and cooperates no matter what the opponent does for two rounds.
The `punishment_limit` is incremented whenever the opponent defects and the
strategy is not in either calming or punishing state.
Note that `Gradual` appears in [CRISTAL-SMAC2018]_ however that version of
`Gradual` does not give the results reported in [Beaufils1997]_ which is the
paper that first introduced the strategy. For a longer discussion of this
see: https://github.com/Axelrod-Python/Axelrod/issues/1294. This is why this
strategy has been renamed to `OriginalGradual`.
Names:
- Gradual: [Beaufils1997]_
"""

name = "Gradual"
name = "Original Gradual"
classifier = {
"memory_depth": float("inf"),
"stochastic": False,
Expand Down Expand Up @@ -438,6 +447,68 @@ def strategy(self, opponent: Player) -> Action:

return C

class Gradual(Player):
"""
Similar to OriginalGradual, this is a player that punishes defections with a
growing number of defections but after punishing for `punishment_limit`
number of times enters a calming state and cooperates no matter what the
opponent does for two rounds.
This version of Gradual is an update of `OriginalGradual` and the difference
is that the `punishment_limit` is incremented whenever the opponent defects
(regardless of the state of the player).
Note that this version of `Gradual` appears in [CRISTAL-SMAC2018]_ however
this version of
`Gradual` does not give the results reported in [Beaufils1997]_ which is the
paper that first introduced the strategy. For a longer discussion of this
see: https://github.com/Axelrod-Python/Axelrod/issues/1294.
This version is based on https://github.com/cristal-smac/ipd/blob/master/src/strategies.py#L224
Names:
- Gradual: [CRISTAL-SMAC2018]_
"""

name = "Gradual"
classifier = {
"memory_depth": float("inf"),
"stochastic": False,
"makes_use_of": set(),
"long_run_time": False,
"inspects_source": False,
"manipulates_source": False,
"manipulates_state": False,
}

def __init__(self) -> None:

super().__init__()
self.calm_count = 0
self.punish_count = 0

def strategy(self, opponent: Player) -> Action:

if len(self.history) == 0:
return C

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

niggle: too many empty lines (black might need running here)

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

b7ff5c6 ran black on the file


if self.punish_count > 0:
self.punish_count -= 1
return D

if self.calm_count > 0:
self.calm_count -= 1
return C

if opponent.history[-1] == D:
self.punish_count = opponent.defections - 1
self.calm_count = 2
return D
return C



@TrackHistoryTransformer(name_prefix=None)
class ContriteTitForTat(Player):
Expand Down
226 changes: 226 additions & 0 deletions axelrod/tests/strategies/test_titfortat.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -336,6 +336,229 @@ class TestGradual(TestPlayer):
"manipulates_state": False,
}

def test_strategy(self):
# Punishes defection with a growing number of defections and calms
# the opponent with two cooperations in a row.
opponent = axelrod.MockPlayer(actions=[C])
actions = [(C, C)]
self.versus_test(
opponent,
expected_actions=actions,
attrs={
"calm_count": 0,
"punish_count": 0,
},
)

opponent = axelrod.MockPlayer(actions=[D])
actions = [(C, D)]
self.versus_test(
opponent,
expected_actions=actions,
attrs={
"calm_count": 0,
"punish_count": 0,
},
)

opponent = axelrod.MockPlayer(actions=[D, C])
actions = [(C, D), (D, C)]
self.versus_test(
opponent,
expected_actions=actions,
attrs={
"calm_count": 2,
"punish_count": 0,
},
)

opponent = axelrod.MockPlayer(actions=[D, C, C])
actions = [(C, D), (D, C), (C, C)]
self.versus_test(
opponent,
expected_actions=actions,
attrs={
"calm_count": 1,
"punish_count": 0,
},
)

opponent = axelrod.MockPlayer(actions=[D, C, D, C])
actions = [(C, D), (D, C), (C, D), (C, C)]
self.versus_test(
opponent,
expected_actions=actions,
attrs={
"calm_count": 0,
"punish_count": 0,
},
)

opponent = axelrod.MockPlayer(actions=[D, C, D, C, C])
actions = [(C, D), (D, C), (C, D), (C, C), (C, C)]
self.versus_test(
opponent,
expected_actions=actions,
attrs={
"calm_count": 0,
"punish_count": 0,
},
)

opponent = axelrod.MockPlayer(actions=[D, C, D, C, C, C])
actions = [(C, D), (D, C), (C, D), (C, C), (C, C), (C, C)]
self.versus_test(
opponent,
expected_actions=actions,
attrs={
"calm_count": 0,
"punish_count": 0,
},
)

opponent = axelrod.MockPlayer(actions=[D, C, D, C, C, C, D, C])
actions = [(C, D), (D, C), (C, D), (C, C), (C, C), (C, C), (C, D), (D, C)]
self.versus_test(
opponent,
expected_actions=actions,
attrs={
"calm_count": 2,
"punish_count": 2,
},
)

opponent = axelrod.MockPlayer(actions=[D, C, D, C, C, D, D, D])
actions = [(C, D), (D, C), (C, D), (C, C), (C, C), (C, D), (D, D), (D, D)]
self.versus_test(
opponent,
expected_actions=actions,
attrs={
"calm_count": 2,
"punish_count": 1,
},
)

opponent = axelrod.Defector()
actions = [
(C, D),
(D, D), # 1 defection as a response to the 1 defection by opponent
(C, D),
(C, D),
(D, D), # starts defecting after a total of 4 defections by the opponent
(D, D),
(D, D),
(D, D), # 4 defections
(C, D),
(C, D),
(D, D), # Start defecting after a total of 10 defections by the opponent
(D, D),
(D, D),
(D, D),
(D, D),
(D, D),
(D, D),
(D, D),
(D, D),
(D, D), # 10 defections
(C, D),
(C, D),
(D, D), # starts defecting after 22 defections by the opponent
(D, D),
(D, D),
(D, D),
(D, D),
(D, D),
(D, D),
(D, D),
(D, D),
(D, D),
(D, D),
(D, D),
(D, D),
(D, D),
(D, D),
(D, D),
(D, D),
(D, D),
(D, D),
(D, D),
(D, D),
(D, D), # 22 defections
(C, D),
(C, D),
(D, D),
(D, D),
(D, D),
(D, D),
]
self.versus_test(
opponent,
expected_actions=actions,
attrs={
"calm_count": 2,
"punish_count": 42,
},
)

def test_specific_set_of_results(self):
"""
This tests specific reported results as discussed in
https://github.com/Axelrod-Python/Axelrod/issues/1294
The results there used a version of mistrust with a bug that corresponds
to a memory one player that start by defecting and only cooperates if
both players cooperated in the previous round.
"""
mistrust_with_bug = axelrod.MemoryOnePlayer(
initial=D,
four_vector=(1, 0, 0, 0),
)
players = [
self.player(),
axelrod.TitForTat(),
axelrod.GoByMajority(),
axelrod.Grudger(),
axelrod.WinStayLoseShift(),
axelrod.Prober(),
axelrod.Defector(),
mistrust_with_bug,
axelrod.Cooperator(),
axelrod.CyclerCCD(),
axelrod.CyclerDDC(),
]
axelrod.seed(1)
tournament = axelrod.Tournament(players, turns=1000, repetitions=1)
results = tournament.play(progress_bar=False)
scores = [round(average_score_per_turn * 1000, 1)
for average_score_per_turn in results.payoff_matrix[0]]
expected_scores = [
3000.0,
3000.0,
3000.0,
3000.0,
3000.0,
2999.0,
983.0,
983.0,
3000.0,
3596.0,
2302.0,
]
self.assertEqual(scores, expected_scores)

class TestOriginalGradual(TestPlayer):

name = "Original Gradual"
player = axelrod.OriginalGradual
expected_classifier = {
"memory_depth": float("inf"),
"stochastic": False,
"makes_use_of": set(),
"inspects_source": False,
"manipulates_source": False,
"manipulates_state": False,
}

def test_strategy(self):
# Punishes defection with a growing number of defections and calms
# the opponent with two cooperations in a row.
Expand Down Expand Up @@ -466,6 +689,9 @@ def test_output_from_literature(self):
Dilemma" Proc. Artif. Life 1996
This test just ensures that the strategy is as was originally defined.
See https://github.com/Axelrod-Python/Axelrod/issues/1294 for another
discussion of this.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should this be a test similar to test_specific_set_of_results for Gradual? Run the whole tournament and compare the scores?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sure why not. 👍

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Note that this will just repeat the doctest. (I'm fine with that.)

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sure. It already does that but only for two strategies right?

We are testing the output of the paper so I suggest we include all of it, and it will follow the same format as the tests for Gradual.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done in 1c07e88

"""
player = self.player()

Expand Down
1 change: 1 addition & 0 deletions docs/reference/bibliography.rst
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -25,6 +25,7 @@ documentation.
.. [Bendor1993] Bendor, Jonathan. "Uncertainty and the Evolution of Cooperation." The Journal of Conflict Resolution, 37(4), 709–734.
.. [Beaufils1997] Beaufils, B. & Delahaye, J. & Mathieu, P. (1997). Our Meeting With Gradual: A Good Strategy For The Iterated Prisoner’s Dilemma. http://citeseerx.ist.psu.edu/viewdoc/summary?doi=10.1.1.42.4041
.. [Berg2015] Berg, P. Van Den, & Weissing, F. J. (2015). The importance of mechanisms for the evolution of cooperation. Proceedings of the Royal Society B-Biological Sciences, 282.
.. [CRISTAL-SMAC2018] CRISTAL Lab, SMAC Team, Lille University (2018). IPD : the Iterated Prisoner's Dilemma. https://github.com/cristal-smac/ipd
.. [Downing1975] Downing, Leslie L. "The Prisoner's Dilemma game as a problem-solving phenomenon: An outcome maximization interpretation." Simulation & Games 6.4 (1975): 366-391.
.. [Eckhart2015] Eckhart Arnold (2016) CoopSim v0.9.9 beta 6. https://github.com/jecki/CoopSim/
.. [Frean1994] Frean, Marcus R. "The Prisoner's Dilemma without Synchrony." Proceedings: Biological Sciences, vol. 257, no. 1348, 1994, pp. 75–79. www.jstor.org/stable/50253.
Expand Down
Loading