getting_started/03_Action.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# This Notebook focuses on the Action class\n",
    "\n",
    "Try me out interactively with: [![Binder](./img/badge_logo.svg)](https://mybinder.org/v2/gh/rte-france/Grid2Op/master)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "It is recommended to have a look at the [00_Introduction](00_Introduction.ipynb), [00_SmallExample](00_SmallExample.ipynb) and [02_Observation](02_Observation.ipynb) notebooks before this one."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**Objectives**\n",
    "\n",
    "This notebook covers the basics of how to efficienty modify the powergrid by using the *Action* class. Indeed, there are multiple concepts behind this class that may not be very clear at first glance.\n",
    "\n",
    "This notebook focuses on the manipulation of Actions from an expert system point of view in order to demonstrate how a desired action is fundamentally taken in the Grid2Op environment. We will give a more detailed example later which will focus on a more automatic way to handle actions (for example using machine learning, in the notebook [04_TrainingAnAgent](04_TrainingAnAgent.ipynb))."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<img src=\"https://colab.research.google.com/assets/colab-badge.svg\" width=\"200\">\n",
    "Execute the cell below by removing the # character if you use google colab !\n",
    "\n",
    "Cell will look like:\n",
    "```python\n",
    "!pip install grid2op[optional]  # for use with google colab (grid2Op is not installed by default)\n",
    "```\n",
    "<img src=\"https://colab.research.google.com/assets/colab-badge.svg\" width=\"200\">"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# !pip install grid2op[optional]  # for use with google colab (grid2Op is not installed by default)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "import numpy as np\n",
    "import grid2op\n",
    "from grid2op.Action import PlayableAction"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "res = None\n",
    "try:\n",
    "    from jyquickhelper import add_notebook_menu\n",
    "    res = add_notebook_menu()\n",
    "except ModuleNotFoundError:\n",
    "    print(\"Impossible to automatically add a menu / table of content to this notebook.\\nYou can download \\\"jyquickhelper\\\" package with: \\n\\\"pip install jyquickhelper\\\"\")\n",
    "res"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## I) A few comments on actions"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "It is recommended to have built locally the documentation, and to refer to the [action](https://grid2op.readthedocs.io/en/latest/action.html) pages of the documentation or to the [Action.py](grid2op/Action/Action.py) file for a more detailed view on these two classes below."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### I.A) \"change\" vs \"set\""
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "To modify a powergrid, we introduce two distinct (yet close) concepts that will affect the objects differently:\n",
    "\n",
    "- **change** will:\n",
    "  - connect a powerline if it was disconnected, or disconnect if it was connected\n",
    "  - assign the object to the second bus if it was connected to bus 1 or assign it to bus 1 if it was connected to bus 2\n",
    "- **set** will:\n",
    "  - connect a powerline (regardless of its previous status) or disconnect it (regardless of its previous status)\n",
    "  - assign the object to a specific bus (regardless of its previous bus and status -- for powerline)\n",
    "  \n",
    "This is different to the previous pypownet implementation, where only the `change` concept was implemented. The option of having these two types of modifying the powergrid supports our understanding of the system and the intention of the Agent, especially in the debugging phase.\n",
    "\n",
    "Of course, it is perfectly possible to use only the `change` capability and thus being closer to the original implementation.\n",
    "\n",
    "The following is specific example to highlight the differences between these two type of methods to modify the powergrid. Suppose we have a substation with 5 elements:\n",
    "- the origin of powerline $l_1$\n",
    "- the extremity of powerline $l_2$\n",
    "- the extremity of powerline $l_3$\n",
    "- a load $c_1$\n",
    "- a generator $g_1$\n",
    "\n",
    "Let's also assume the original configuration (before the action is applied, *ie* the configuration of the observation at time *t*) is:\n",
    "\n",
    "| Object Name      | Original Bus | Original Status|\n",
    "|------------------|--------------|----------------|\n",
    "| $l_1$ (origin)   | 1            |    connected   |\n",
    "| $l_2$ (extremity)| 2            |    connected   |\n",
    "| $l_3$ (extremity)| NA           | disconnected   |\n",
    "| $c_1$            | 1            | NA             |\n",
    "| $g_1 $           | 2            | NA             |"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Let's say:\n",
    "* action $a_1$ is \"**change** the status of the origin of powerline $l_1$\". After applying this action, the status of the origin of powerline $l_1$ is: \"disconnected\", because it was \"connected\".\n",
    "* action $a_2$ is \"**change** the status of the extremity of powerline $l_3$\". After applying this action, the status of the extremity of powerline $l_3$ is: \"connected\", because it was \"disconnected\". ***\\****.\n",
    "* action $a_3$ is \"**set** the bus of $c_1$ to bus 1\". It is equivalent to doing nothing since $c_1$ is already connected on bus 1.\n",
    "* action $a_4$ is \"**set** the bus of $g_1$ to bus 1\". It will change the bus of $g_1$ and assign it to bus 1.\n",
    "\n",
    "\\* **NB** Another breaking change compared to the pypownet implementation is the introduction of \"ambiguous\" actions. When an action can be understood in different ways or have different meanings, then it will be replaced by a \"do nothing\" action by the environment."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "In this situation, the previous actions are **equivalent** to:\n",
    "* $a_1 \\to$  **set** status of $l_1$ to \"disconnected\"\n",
    "* $a_2 \\to$  **set** status of $l_3$ to \"connected\"\n",
    "* $a_3 \\to$ do nothing\n",
    "* $a_4 \\to$ **change** bus of $g_1$ "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### I.B) \"Ambiguous\" action"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "When some actions are \"ambiguous\" it means that they cannot be properly and / or univocally interpreted. **These actions will be ignored if attempted to be used on the powergrid. This will be equivalent to doing nothing.**\n",
    "\n",
    "For a detailed list of ambiguous actions, the documentation is the **only** official source. Only some examples are presented here. The documentation is available at [\\_check\\_for\\_ambiguity](https://grid2op.readthedocs.io/en/latest/action.html#grid2op.Action.Action._check_for_ambiguity).\n",
    "\n",
    "An action can be ambiguous in the following cases:\n",
    "\n",
    "  - It affects the \"*injections*\" in an incorrect way:\n",
    "\n",
    "    - it tries to modify a load (setting active or reactive values) that doesn't exist in the powergrid\n",
    "    - it sets the values of a generator that doesn't exist (setting its voltage setpoint or active production)\n",
    "\n",
    "  - It affects the \"*powerlines*\" in an incorrect manner:\n",
    "\n",
    "    - it tries to change the status or to assign to a specific bus a powerline that doesn't exist\n",
    "    - ~~somes lines are reconnected but the action doesn't specify on which bus.~~ It used to be like this, but now an \"automaton\" directly coded in the environment will assign the previous bus if that is the case. **You can reconnect a powerline without specifying on which bus** and in that case the last known buses when the powerline was connected will be used.\n",
    "    - for some powerline, the status is both **changed** and **set**\n",
    "\n",
    "  - It has an ambiguous behaviour concerning the topology of some substations\n",
    "\n",
    "    - the state of some bus for some element is both **changed** and **set** \n",
    "    - the bus is trying to be modified (**set** or **changed**) on a object that is not present in the powergrid"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## II) The different types of actions"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**IMPORTANT NOTICE** Each *Agent* has its own `action space` attribute that can be accessed with `self.action_space`. This is the only recommended way to create a valid *Action*. It is strongly recommended NOT to use its constructor, as it requires a deep knowledge of all the elements in the powergrid, as well as their names, their type, the order in which they are used in the backend, etc. For performance reasons, no sanity check are performed to make sure the action (that would be created) is compatible with the backend.\n",
    "\n",
    "In the next cell, we retrieve the action space used by the Agent."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Two main classes are useful when dealing with *Actions* in Grid2Op. The *Action* class is the most basic one. The *ActionHelper* is a tool that helps to create and to manipulate some actions in most of our notebooks, we start by creating an Environment. We will use the `case14_fromfile` provided as an example.\n",
    "\n",
    "We will then extract the complete action space (`action_space`, that is, the actions that can be performed on the power grid) as a dictionary. When a specific action such as *change* or *set* is needed to be performed then we can apply this change to the *action_space* dictionary by accessing it with the relevant key as discussed below."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# create an environment, and use its action space\n",
    "env = grid2op.make(\"educ_case14_storage\", test=True, action_class=PlayableAction)\n",
    "action_space = env.action_space"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "As opposed to the previous plateform, [pypownet](https://github.com/MarvinLer/pypownet), there are no restrictions on actions in Grid2Op. Generally speaking, an Action can modify production, loads, topology, etc. By default though, an Action that an Agent can perform is a [TopologyAction](https://grid2op.readthedocs.io/en/latest/action.html#grid2op.Action.TopologyAction), which is a specific type of action. A TopologyAction can  :\n",
    "\n",
    "- change the status of a powerline (reconnect / disconnect it)\n",
    "- change the way the objects (end of a powerline, generator or load) are interconnected at substations.\n",
    "\n",
    "We will focus on this class in this notebook."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Then best way to get an action is to give a `dictionnary` to the \"action space\" of the player. For example, to get the \"do nothing\" action, you can just pass the empty dictionnary."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "do_nothing = action_space({})"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### II.A) Main principles\n",
    "\n",
    "As explained actions can be done in multiple ways. And signifcant more detail can be found in the documentation [https://grid2op.readthedocs.io/en/latest/action.html#usage-examples](https://grid2op.readthedocs.io/en/latest/action.html#usage-examples) .\n",
    "\n",
    "Since grid2op version 1.5.0 we recommend to use the \"property\" way that tries to unify the way to perform every type of actions.\n",
    "\n",
    "Basically, this always behaves like:\n",
    "\n",
    "```python\n",
    "action = env.action_space()\n",
    "action.property_name = property_description\n",
    "```\n",
    "\n",
    "For example, you can perform a \"set_bus\" action corresponding to \"moving the load with id 1 and set it on bus 2\":\n",
    "\n",
    "```python\n",
    "load_id = 1\n",
    "new_bus = 2\n",
    "action0 = env.action_space()\n",
    "action0.load_set_bus = [(load_id, new_bus)]\n",
    "```\n",
    "\n",
    "Another example, you can perform the action \"change status\" which \"changes the status of powerlines id 5 and 7\":\n",
    "```python\n",
    "line_ids = [5, 7]\n",
    "action1 = env.action_space()\n",
    "action1.line_change_status = [5, 7]\n",
    "```\n",
    "\n",
    "In order to check that the action implemented is the one you want to implement, you can always \"print\" the action:\n",
    "```python\n",
    "print(action0)\n",
    "print(action1)\n",
    "```"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### II.B) Line status modification"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "If you want to change (or set) the status of most of the powerlines, you can create a vector having the same size as the number of powerlines in the grid, and pass it to the dictionary with the relevant keys (\"set_line_status\", \"change_line_status\") and the proper values (a vector of booleans to set line status, or a vector of integers to change line status). An example is given below. Note that this example only modifies the status of a few powerlines, but this way of defining actions is more adapted when you need to modify the status of many powerlines.\n",
    "\n",
    "The following code will:\n",
    "- change the status of powerlines with id 0,1,2\n",
    "- set the status \"connected\" for powerline with id 3,4\n",
    "- set the status \"disconnected\" for powerlines with id 5 and 6"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "line_id_to_change = [0,1,2]\n",
    "change_status = action_space()\n",
    "change_status.line_change_status = line_id_to_change\n",
    "\n",
    "print(\"The action `change_status` is \")\n",
    "print(change_status)\n",
    "print() \n",
    "\n",
    "line_id_to_set = [3, 4, 5, 6]\n",
    "new_status = [1, 1, -1, -1]\n",
    "set_status = action_space()\n",
    "set_status.line_set_status = [[l_id, status] for l_id, status in zip(line_id_to_set, new_status)]\n",
    "print(\"The action `set_status` is \")\n",
    "print(set_status)\n",
    "print()\n",
    "\n",
    "both_action = change_status + set_status\n",
    "print(\"When i combine them its\")\n",
    "print(both_action)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "You have the option to do that \"all at once\" if you use the description of an action as a dictionary:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "both_action = action_space({\"set_line_status\": [[l_id, status] for l_id, status in \\\n",
    "                                                zip(line_id_to_set, new_status)],\n",
    "                            \"change_line_status\": line_id_to_change\n",
    "                           })\n",
    "print(\"both_action is:\")\n",
    "print(both_action)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### II.C) Topology modification\n",
    "\n",
    "One of the interesting aspects of `Grid2Op` is the ability to modify the topology of the powergrid. In other words, it allows to reconfigure the way the objects (generators, loads, side of powerlines) are interconnected at their substations. Comparable to the status change, topological change can be interpreted in two disctinct manners, as described above. Topologycal changes include some of the most interesting interactions with the environment. In this section we study how to modify the topology of the powergrid. \n",
    "\n",
    "In principle, there are two equivalent ways to modify the topology, however, we do not recommend to use both, as it will introduce redundancies (the same thing can be done in two manner, which might badly intearct with each other):\n",
    "\n",
    "- **set** way: you specify the target, and grid2op affect the right objects to match your target\n",
    "- **change** way: you specify what object you want to change\n",
    "\n",
    "Both are more or less equivalent. Basically, if you imagine in grid2op you have to \"press switches\", then:\n",
    "\n",
    "- **set**: you say if the switch should be \"on\" or \"off\"\n",
    "- **change**: you say if the switch should be toggled or not\n",
    "\n",
    "#### a) Basic manipulation\n",
    "\n",
    "The basic ideas to modify the topology is:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "change_buses = action_space({\"change_bus\": [3, 4]})\n",
    "# equivalently: \n",
    "# change_buses = action_space()\n",
    "# change_buses.change_bus =  [3, 4]\n",
    "print(change_buses)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "set_buses = action_space({\"set_bus\": [(3, 2), (4, 1)]})\n",
    "# equivalently: \n",
    "# set_buses = action_space()\n",
    "# set_buses.set_bus =  [(3, 2), (4, 2)]\n",
    "print(set_buses)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**NB**: in the above code, it's not explicit that the \"element with id 3\" is \"line 0, extremity side\" and the \"element with id 4 is \"line 2, origin side\".\n",
    "\n",
    "This information can be retrieved either using the `action_space.load_pos_topo_vect`, `action_space.gen_pos_topo_vect`, `action_space.line_or_pos_topo_vect`, `action_space.line_ex_pos_topo_vect` or `action_space.storage_pos_topo_vect` vectors.\n",
    "\n",
    "For example you can see that `action_space.line_ex_pos_topo_vect[0] = 3` so this means that... the \"*line extremity 0 (extremity side of powerline 0) is encoded as the element 3*\" and `action_space.line_or_pos_topo_vect[2] = 4` meaning that \"*the line origin 2 (origin side of powerline 2) is encoded as the element 4*\".\n",
    "\n",
    "\n",
    "Alternatively, this information can also be retrieved using the `action_space.grid_objects_types` matrix and looking at the rows `3` and `4` (because we are interested in element 3 and 4 in this example) and this gives:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "action_space.grid_objects_types[[3,4],:]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "This matrix can be understood as the following:\n",
    "- first column represents the substation to which this object is connected (between `0` and `env.n_sub-1`)\n",
    "- second column represents loads: its either `-1` if the object is not a load, otherwise the load id if this is a load\n",
    "- third column represents generators\n",
    "- fourth column is origin side of powerline. Here the first row has \"0\" at this place, so this means this object is the origin of powerline 0.\n",
    "- fifth column is extremity side of powerline. Here the second row has \"2\" here, this means this object represents powerline extremity 2\n",
    "- sixth colum represents storage units."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### b) Manipulating by object types\n",
    "\n",
    "If you want to modify the \"load 5\" you can of course do:\n",
    "\n",
    "```python\n",
    "set_buses = action_space({\"set_bus\": [(action_space.load_pos_topo_vect[5], 2)]})\n",
    "```\n",
    "\n",
    "But it's not really convenient (and very verbose...). To do exactly the same, you can:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "set_load_5_bus = action_space({\"set_bus\": {\"loads_id\": [(5, 2)]}})\n",
    "\n",
    "# equivalently: \n",
    "# set_load_5_bus = action_space()\n",
    "# set_load_5_bus.load_set_bus =  [(5, 2)]\n",
    "print(set_load_5_bus)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**NB** likewise, you can do that for the \"change\" way to modify the grid with:\n",
    "```python\n",
    "change_buses = action_space({\"change_bus\": [action_space.load_to_pos_topo_vect[5]]})\n",
    "```\n",
    "\n",
    "#### c) Manipulating by substation\n",
    "\n",
    "Sometimes it can also be interesting to look explicitly at the substation level, and to explicitly say we want to modify this substation and not anything else. This is also an option with grid2op.\n",
    "\n",
    "To that end, you need to:\n",
    "\n",
    "- know the substation you want to modify [In this example, we will modify the substation 1]\n",
    "- know which object(s) you want to modify [The object, connected at this substation, that you want to modify, in this example \"line 2 (origin side)\" and \"line 0 (extremity side)\"]\n",
    "- know how to say it with grid2op **what we will focus on here**"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "sub_id = 1\n",
    "topo_sub1 = np.zeros(action_space.sub_info[sub_id], dtype=int)\n",
    "topo_sub1[action_space.line_or_to_sub_pos[2]] = 2  # \"line 2 (origin side)\"\n",
    "topo_sub1[action_space.line_ex_to_sub_pos[0]] = 2  # \"line 0 (extremity side)\"\n",
    "modify_sub_1 = action_space({\"set_bus\": {\"substations_id\": [(sub_id, topo_sub1)]}})\n",
    "# equivalently: \n",
    "# modify_sub_1 = action_space()\n",
    "# modify_sub_1.sub_set_bus =  [(sub_id, topo_sub1)]\n",
    "\n",
    "print(modify_sub_1)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**NB** likewise, you can do that for the \"change\" way to modify the grid with:\n",
    "```python\n",
    "change_buses_sub_1 = action_space({\"set_bus\": {\"substations_id\": \n",
    "                                                   [action_space.line_or_to_sub_pos[2], \n",
    "                                                    action_space.line_ex_to_sub_pos[0]]\n",
    "                                               }\n",
    "                                  }\n",
    "                                 )\n",
    "```"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### II.D) Redispatching\n",
    "\n",
    "Redispatching is explained in depth in its own notebook, see [06_Redispatching_Curtailment](06_Redispatching_Curtailment.ipynb). We provide here a brief introduction to perform these types of continuous action.\n",
    "\n",
    "In summary, redispatching aims at asking generators to change their production setpoint.\n",
    "\n",
    "\n",
    "For further details, the documentation explains in detail how the generators behave here [https://grid2op.readthedocs.io/en/latest/modeled_elements.html#generators](https://grid2op.readthedocs.io/en/latest/modeled_elements.html#generators))\n",
    "\n",
    "It can be done with:\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "gen_id = 0  # on which generator to apply this action\n",
    "amount = 3.14159262359  # how do you want to modify the generator\n",
    "redisp_act = action_space({\"redispatch\": [(gen_id, amount)]})\n",
    "\n",
    "print(redisp_act)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Alternatively, you can use the \"property\" method to do it, which would be, in this case:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "redisp_act2 = action_space()\n",
    "redisp_act2.redispatch = [(gen_id, amount)]\n",
    "print(redisp_act)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### II.E) Curtailment\n",
    "\n",
    "As the \"redispatching, the detailed behaviour of the curtailment is explained in depth in its own notebook, see [06_Redispatching_Curtailment](06_Redispatching_Curtailment.ipynb). We only provide here a brief introduction on how to perform these types of continuous action.\n",
    "\n",
    "In summary, curtailment aims at limiting the production of renewable generator. For example, if there is too much wind in a certain area, you can ask windmill to decrease their production in order to keep the grid safe.\n",
    "\n",
    "\n",
    "For further details, the documentation explains in detail how the generators behave here [https://grid2op.readthedocs.io/en/latest/modeled_elements.html#generators](https://grid2op.readthedocs.io/en/latest/modeled_elements.html#generators))\n",
    "\n",
    "**NB** By default, curtailment are given in ratio of pmax. You can also do it in MW if you prefer by adding the \\_mw (see example below)\n",
    "\n",
    "\n",
    "It can be done with:\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "gen_id = 1  # on which generator to apply this action\n",
    "ratio_curtailment = 0.15  # this is a ratio, between 0.0 and 1.0 \n",
    "# if not in range [0, 1.0] an \"ambiguous action\" will be raised when calling \"env.step\"\n",
    "curtail_act = action_space({\"curtail\": [(gen_id, ratio_curtailment)]})\n",
    "\n",
    "print(curtail_act)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Alternatively, you can use the \"property\" method to do it, which would be, in this case:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "curtail_act2 = action_space()\n",
    "curtail_act2.curtail = [(gen_id, ratio_curtailment)]\n",
    "print(curtail_act2)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Or you can also do it in MW:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "amount_mw = 15  # this should be between 0. and gen_pmax[gen_id], otherwise \n",
    "curtail_act2_mw = action_space()\n",
    "curtail_act2_mw.curtail_mw = [(gen_id, amount_mw)]\n",
    "print(curtail_act2_mw)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### II.F) Manipulating storage units\n",
    "\n",
    "Storage units also consist of continuous actions.\n",
    "\n",
    "Their main property is that they can produce and absorb power (behaving as a load or as a generator). They are also limited in the amount of energy they can store:\n",
    "\n",
    "- if they are at their maximum capacity, they cannot absorb any more power from the grid, but they can of course still behave like generators\n",
    "- if they are at their lowest capacity, they cannot produce any more power from the grid, but they can of course still absord it.\n",
    "\n",
    "We chose, in grid2op to have storage units in the \"load convention\", this means that:\n",
    "\n",
    "- if the value is positive, they will absorb power from the grid, they will charge\n",
    "- if the value is negative, they will produce power from the grid, they will discharge.\n",
    "\n",
    "See the notebook [10_StorageUnits](10_StorageUnits.ipynb) about this element.\n",
    "\n",
    "For further details, the documentation explains in detail how the storage units behave here [https://grid2op.readthedocs.io/en/latest/modeled_elements.html#storage-units-optional](https://grid2op.readthedocs.io/en/latest/modeled_elements.html#storage-units-optional)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "storage_id = [0, 1]\n",
    "values = [-1.7, 2.3]\n",
    "action_descr = [(stor_id, val) for stor_id, val in zip(storage_id, values)]\n",
    "# alternatively: action_descr = np.array([-1.7, 2.3], dtype=float)\n",
    "\n",
    "# with the dictionnary\n",
    "action_storage0 = action_space({\"set_storage\": action_descr})\n",
    "# or with the property\n",
    "action_storage1 = action_space()\n",
    "action_storage1.set_storage = action_descr\n",
    "\n",
    "print(\"The storage action applied is\")\n",
    "print(action_storage1)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### II.G) Important note on the powerline status\n",
    "\n",
    "#### Direct impact on the status\n",
    "As of grid2op version 1.2.0 the behavior of the platform with respect to line status modification has been clarified and rationalized (we hope).\n",
    "\n",
    "The powerline status (connected / disconnected) can now be affected in two different ways:\n",
    "- by `setting` / `changing` its status directly (using the \"set_line_status\" or \"change_line_status\" keyword).\n",
    "- [NEW] by modifying the bus on any of the end (origin or extremity) of a powerline\n",
    "\n",
    "In that case, the behavior is:\n",
    "- if the bus of a powerline end (origin or extremity) is \"set\" to -1 and not modified at the other and if the powerline was connected, it will disconnect this powerline\n",
    "- if the bus of a powerline end (origin or extremity) is \"set\" to 1 or 2 at one end and not modified at the other and if the powerline was connected, it will reconnect the powerline\n",
    "- if the bus of a powerline end (origin or extremity) is \"set\" to -1 at one end and set to 1 or 2 at its other end the action is **ambiguous**.\n",
    "\n",
    "The way to compute the impact of the action has also been adjusted to reflect these changes. \n",
    "\n",
    "In the table below we try to summarize all the possible actions and their impact on the powerline. This table is made considering that \"`LINE_ID`\" is an id of a powerline and \"`SUB_OR`\" is the id of the origin of the substation. If a status is 0 it means the powerlines is disconnected, if the status is 1 it means it is connected.\n",
    "\n",
    "| action      | original status | final status | substations affected | line status affected |\n",
    "|-------------|-----------------|--------------|----------------------|----------------------|\n",
    "| {\"set_line_status\": [(LINE_ID, -1)]}             | 1 | 0 | None   | LINE_ID |\n",
    "| {\"set_line_status\": [(LINE_ID, +1)]}             | 1 | 1 | None   | LINE_ID |\n",
    "| {\"set_line_status\": [(LINE_ID, -1)]}             | 0 | 0 | None   | LINE_ID |\n",
    "| {\"set_line_status\": [(LINE_ID, +1)]}             | 0 | 1 | None   | LINE_ID |\n",
    "| {\"change_line_status\": [LINE_ID]}                | 1 | 0 | None   | LINE_ID |\n",
    "| {\"change_line_status\": [LINE_ID]}                | 0 | 1 | None   | LINE_ID |\n",
    "| {\"set_bus\": {\"lines_or_id\": [(LINE_ID, -1)]}}   | 1 | 0 | None   | LINE_ID |\n",
    "| {\"set_bus\": {\"lines_or_id\": [(LINE_ID, -1)]}}   | 0 | 0 | SUB_OR | None    |\n",
    "| {\"set_bus\": {\"lines_or_id\": [(LINE_ID, 2)]}}    | 1 | 1 | SUB_OR | None    |\n",
    "| {\"set_bus\": {\"lines_or_id\": [(LINE_ID, 2)]}}    | 0 | 1 | None   | LINE_ID |\n",
    "| {\"change_bus\": {\"lines_or_id\": [LINE_ID]}}      | 1 | 1 | SUB_OR | None    |\n",
    "| {\"change_bus\": {\"lines_or_id\": [LINE_ID]}}      | 0 | 0 | SUB_OR | None    |\n",
    "\n",
    "of course we could have set `{\"set_bus\": {\"lines_ex_id\": [(LINED_ID, 2)]}}` (ie the extermity bus of the powerline) and it would have the same impact on its status. Assign the powerline extremity to bus 1 (instead of bus 2) by sending the dictionnaries `{\"set_bus\": {\"lines_or_id\": [(LINED_ID, 1)]}}` or `{\"set_bus\": {\"lines_ex_id\": [(LINED_ID, 1)]}}` would also lead to the same results.\n",
    "\n",
    "#### Consequences on the bus at both powerline ends\n",
    "In grid2op there is a convention that if an object is disconnected, then it is assigned to bus \"-1\". For a powerline this entails that a status changed affects the bus of \n",
    "\n",
    "As we explained in the previous paragraph, some action on one end of a powerline can reconnect a powerline or disconnect it. This means they modify the bus of **both** the extremity of the powerline.\n",
    "\n",
    "Here is a table summarizing how the buses are impacted. We denoted by \"`PREVIOUS_OR`\" the last bus at which the origin end of the powerline was connected and \"`PREVIOUS_EX`\" the last bus at which the extremity end of the powerline was connected. Note that for clarity when something is not modified by the action we decided to write on the table \"not modified\" (this entails that after this action, if the powerline is connected then \"new origin bus\" is \"`PREVIOUS_OR`\" and \"new extremity bus\" is \"`PREVIOUS_EX`\"). We remind the reader that \"-1\" encode for a disconnected object.\n",
    "\n",
    "| action      | original status | final status | new origin bus | new extremity bus |\n",
    "|-------------|-----------------|--------------|----------------|-------------------|\n",
    "| {\"set_line_status\": [(LINE_ID, -1)]}             | 1 | 0 | -1           | -1           |\n",
    "| {\"set_line_status\": [(LINE_ID, +1)]}             | 1 | 1 | Not modified | Not modified |\n",
    "| {\"set_line_status\": [(LINE_ID, -1)]}             | 0 | 0 | Not modified | Not modified |\n",
    "| {\"set_line_status\": [(LINE_ID, +1)]}             | 0 | 1 | PREVIOUS_OR  | PREVIOUS_EX  |\n",
    "| {\"change_line_status\": [LINE_ID]}                | 1 | 0 | -1           | -1           |\n",
    "| {\"change_line_status\": [LINE_ID]}                | 0 | 1 | PREVIOUS_OR  | PREVIOUS_EX  |\n",
    "| {\"set_bus\": {\"lines_or_id\": [(LINE_ID, -1)]}}   | 1 | 0 | -1           | -1           |\n",
    "| {\"set_bus\": {\"lines_or_id\": [(LINE_ID, -1)]}}   | 0 | 0 | Not modified | Not modified |\n",
    "| {\"set_bus\": {\"lines_or_id\": [(LINE_ID, 2)]}}    | 1 | 1 | 2            | Not modified |\n",
    "| {\"set_bus\": {\"lines_or_id\": [(LINE_ID, 2)]}}    | 0 | 1 | 2            | PREVIOUS_EX  |\n",
    "| {\"change_bus\": {\"lines_or_id\": [(LINE_ID, 2)]}} | 1 | 1 | \\*           | Not modified |\n",
    "| {\"change_bus\": {\"lines_or_id\": [(LINE_ID, 2)]}} | 0 | 0 | Not modified | Not modified |\n",
    "\n",
    "\\* means that this bus is affected: if it was on bus 1 it moves on bus 2 and vice versa.\n",
    "\n",
    "\n",
    "#### Extra note on the actions\n",
    "As we can see here, but this is true in general in grid2op, each action you do is labeled to have at least an impact on some object. Some actions are considered to have impact on powerline status, some have impact on substations. \n",
    "\n",
    "This is always the case regardless of the actual impact of this action on the powergrid. For example, if we look at the second and third row of the tables above we notice that:\n",
    "- these actions does not impact the grid (nothing is modified)\n",
    "- these actions are counted towards action affecting \"powerline\".\n",
    "\n",
    "This is particularly important for what we call \"cooldown\" (see the notebook [0_Introduction](./0_Introduction.ipynb) section `Introduction of \"operational constraints\" in grid2op` for more information). You action can trigger a cooldown (preventing future action on the same element) while not impacting the grid at all."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### II H) Sampling random actions\n",
    "\n",
    "Sampling random actions uniformly is not an easy task. Especially because \"unform random actions\" (taken on all the action space) will most likely be either ambiguous or illegal.\n",
    "\n",
    "\n",
    "#### a) action_space.sample() and its limitations\n",
    "Grid2op comes with a tool to sample random actions. A possibility for such puporse is simply:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "print(action_space.sample())"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "print(action_space.sample())"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**NB** This method will sample action at random that meets the following properties:\n",
    "- the action will either perform `redispatching` OR `set_bus` OR `change_bus` OR `set_status` OR `change_status` OR `storage` (these are \"exclusive or\"). It will not act on multiple of these at the same time. The choice of the \"type\" of the action is uniform at random among the supported types.\n",
    "- if the action is a `set_status` or a `change_status` it will only affect 1 powerline at a time\n",
    "- if the action is a `set_bus` or a `change_bus` it will only affect 1 substation\n",
    "- if the action is a `redispatching`: will sample a single generator at random, and apply unformly at random redispatching on this single generator\n",
    "- if the action is a `redispatching`: will sample a single storage unit at random, and only act on this particular one.\n",
    "\n",
    "You can combine these \"unary actions\" with the `+=` operator if you want, for example if you want to combine three such action:\n",
    "\n",
    "```python\n",
    "action = action_space()\n",
    "action += action_space.sample()\n",
    "action += action_space.sample()\n",
    "action += action_space.sample()\n",
    "```"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**NB** This method is not uniform on all the actions. It's uniform among the type of actions.\n",
    "\n",
    "#### More control on the type of actions\n",
    "\n",
    "If you want more control on these action types you can use the following code snippet, that sample 2 actions only of types \"set_status\""
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "random_set_line_action = action_space()\n",
    "random_set_line_action.update(action_space._sample_set_line_status())\n",
    "random_set_line_action.update(action_space._sample_set_line_status())\n",
    "print(random_set_line_action)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The other methods are:\n",
    "- `action_space._sample_set_line_status()`: to sample action of type \"set_line_status\" (still acting only on one powerline)\n",
    "- `action_space._sample_change_line_status()`: to sample action of type \"change_status\" (still acting only on one powerline)\n",
    "- `action_space._sample_set_bus()`: to sample action of type \"set_bus\" (still acting on one substation)\n",
    "- `action_space._sample_change_bus()`: to sample action of type \"set_bus\" (still acting on one substation)\n",
    "- `action_space._sample_redispatch()`: to sample action of type \"redispatching\" (still acting on one single generator)\n",
    "- `action_space._sample_storage_power()`: to sample action of type \"storage unit\" (still acting on one single storage unit)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### II J) Redispatching, curtailment and storage units\n",
    "\n",
    "One of the main \"properties\" of a powergrid is that, at every time the total demand (including the losses) should match exactly the total generation.\n",
    "\n",
    "This entails that the \"redispatching\", \"curtailment\" and \"storage units\" type of actions have \"side effects\".\n",
    "\n",
    "For example, if you curtail `xxx` MW of power from renewable energy sources, they will be compensated by a corresponding increase of `xxx` MW from the controlable generators. Hence, if you perform a \"curtail\" action you will also affect the \"redispatching. The same goes for the actions on storage units.\n",
    "\n",
    "Generators are in turn limited in their capacity. For example (for each controlable generator), its production $p_t$ is limited between $p_min$ and $p_max$ as well as the variation of this production $-ramp_{min} <= p_t - p_{t-1} <= ramp_{max}$. One the main consequences if that you are limited by the total amount of MW you can curtail and or absorb / produce by the storage units.\n",
    "\n",
    "In particular, at each time, the equations:\n",
    "\n",
    "- for each controlable generators $p_{min} <= p_t <= p_{max}$\n",
    "- for each controlable generators $-ramp_{min} <= p_t - p_{t-1} <= ramp_{max}$\n",
    "- at each step the sum of MW curtailed and the total contribution of storage units is absorbed by the controlable generators so that the total amount of power injected at this step does not change: $\\sum_{\\text{all generators } g} p^{(g, scenario)}_t = \\sum_{\\text{controlable generators } c}  p^{(c)}_t + \\sum_{\\text{storage unit } s} p^{s}_t + \\sum_{\\text{renewable generator} r} p^{(r)}_t$ where $p^{(g)}_t$ denotes the productions of generator $g$ in the input data \"scenario\" (*ie* \"in the current episode\", \"before any modification\", \"decided by the market / central authority\").\n",
    "\n",
    "If there is no solution to these equations there is a \"game over\" (done=True) and the `info[\"exception\"]` will say something like:\n",
    "\n",
    "```\n",
    "InvalidRedispatching('\\nThis is an attempt to explain why the dispatch did not succeed and caused a game over.\\nTo compensate the decrease of loads and / or increase of renewable energy (due to naturl causes but also through curtailment) and / or variation in the storage units, the generators should decrease their total production of -634.39MW (in total).\\nBut, if you take into account the generator constraints (pmin and max_ramp_down) you can have at most -151.20MW.\\nIndeed at time t, generators are in state:\\n\\t[369.6   25.5  228.71 228.71 369.6  231.2   28.9  135.88 128.08 369.6\\n  40.41 231.2    0.   231.2  231.2   37.47 231.2  150.    81.88  25.16]\\ntheir ramp max is:\\n\\t[ 3.5  8.5 11.2 11.2  3.5 11.2  8.5  5.3  5.3  3.5  9.9 11.2  5.7 11.2\\n 11.2  8.5 11.2  5.3  5.3  5.7]\\n and pmax is:\\n\\t[0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0.]\\nWrapping up, each generator can decrease at minimum of:\\n\\t[ 3.5  8.5 11.2 11.2  3.5 11.2  8.5  5.3  5.3  3.5  9.9 11.2  5.7 11.2\\n 11.2  8.5 11.2 -0.   5.3  5.7]\\nNB: if you did not do any dispatch during this episode, it would have been possible to meet these constraints. This situation is caused by not having enough degree of freedom to \"compensate\" the variation of the load due to (most likely) an \"over usage\" of redispatching feature (some generators stuck at pmin as a consequence of your redispatching. They can\\'t increase their productions to meet the decrease in demand or increase of renewables)'), Grid2OpException AmbiguousAction InvalidRedispatching InvalidRedispatching('Game over due to infeasible redispatching state. The routine used to compute the \"next state\" has diverged. This means that there is no way to compute a physically valid generator state (one that meets all pmin / pmax - ramp min / ramp max with the information provided. As one of the physical constraints would be violated, this means that a generator would be damaged in real life. This is a game over.')\n",
    "```\n",
    "\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## III) Combining actions and other manipulations"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### III A) Combine different type of actions\n",
    "\n",
    "It is absolutely **NOT** recommended to use *Actions* outside of the action space, for example building an action directly with the class constructor.\n",
    "\n",
    "However, it is possible, in grid2op to combine different action, or to create an action \"step by step\".\n",
    "\n",
    "Let's take an example. Let's say we want an action to:\n",
    "- perform redispatching on generator 0 of 3.1 MW\n",
    "- change the bus on which load 0 and line (origin side) 1 are connected\n",
    "- have the storage unit 0 produce 1.5 MW on the grid.\n",
    "\n",
    "It is possible to do the following \"at once\", with:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "combined_action = action_space({\"redispatch\": [(0, 3.1)],\n",
    "                                \"change_bus\": {\"loads_id\": [0], \"lines_or_id\": [1]},\n",
    "                                \"set_storage\": [(0, -1.5)]\n",
    "                               }\n",
    "                              )\n",
    "print(combined_action)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "But it is also possible to build it \"little by little\" by using the `+=` (or `+`) python operator. This can be done like this:"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "This can be useful particularly if different \"entities\" are responsible to perform different kind of actions. \n",
    "\n",
    "In the example given above, we could imagine having a \"stuff\" that takes `redispatching` action, another that takes `change bus` actions and finally another affecting `storages`, and then combine the action easily, giving a code looking like:\n",
    "\n",
    "```python\n",
    "action_redispatch = entitiy_handling_redispatching(...)\n",
    "action_change_bus = entitiy_handling_change_bus(...)\n",
    "action_storage = entitiy_handling_storage(...)\n",
    "combined_action = action_redispatch + action_change_bus + action_storage\n",
    "```"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### III B) Combine same type of actions on different objects\n",
    "\n",
    "Using the same principle as above, it is also possible to combine the type of action (say disconnecting a powerline) using the `+` operator.\n",
    "\n",
    "For example, to disconnect powerline `i` and powerline `j` you can do:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "i = 0\n",
    "j = 1\n",
    "disco_line_i = action_space({\"set_line_status\": [(i, -1)]})\n",
    "disco_line_j = action_space({\"set_line_status\": [(j, -1)]})\n",
    "\n",
    "disco_both = disco_line_i + disco_line_j"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "This might be especially usefull if you have different agent acting on different part of the grid for example. One agent could be responsible to handle the \"top part\" (whatever it means) of the grid and another the \"bottom part\" (whatever it means) of it and you can combine the actions of both agents this way.\n",
    "\n",
    "### III C) Caveat on actions combination\n",
    "\n",
    "Combining actions this way might come with different caveat:\n",
    "1) the combination of actions can be **illegal** depending on the game rules in place. For example if at any given step you can only do an action on one single powerline, but you combine an action disconnecting two (or more) powerlines. In this case the resulting action will be illegal.\n",
    "2) the combination of actions might make the resulting action \"**ambiguous**\" (meaning that \"grid2op cannot make sense of what you are trying to do\"). This can happen for example if you mix action on topology and on powerline status. For example, if you decide to affect the \"origin\" side of powerline `i` to bus 1 in an action but you also say to disconnect powerline `i` in a second action. Grid2op currently will not \"chose for you\" what you want to do\\*.\n",
    "\n",
    "\\* Depending on the outcome of issue https://github.com/rte-france/Grid2Op/issues/201 we might chose in this case to have the last action added \"take the priority\" over the previous one. This might \"solve\" this specific problem but will also introduce a really important role in the order on which actions are summed. "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## IV Legacy codes\n",
    "\n",
    "As of grid2op version 1.5.0 we simplified and rationalized the way to interact with the powergrid. We do not recommend to use the older API described here.\n",
    "\n",
    "However, for backward compatibility (and for people using grid2op version 1.4 or below) we still kept these features in the notebook.\n",
    "\n",
    "### A) Lines status modification\n",
    "\n",
    "#### a) Modify multiple powerlines  (legacy)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Each powerline / load / generator has an **ID** and a **name**.\n",
    "\n",
    "**Keep in mind that the IDs of the powerlines are 0, 1, 2, ..., `env.n_line` - 1** (where `env.n_line` is the number of lines in the environment). The same goes for the loads and generators.\n",
    "\n",
    "On the other hand, the names are more human-friendly identifiers for the different objects in the grid.\n",
    "\n",
    "Therefore, for any `vector` containing information about the powerlines, **the variable relative to the powerline of id `i` can be accessed or modified with `vector[i]`** since it is the `i+1`th powerline in the grid.\n",
    "\n",
    "This is how we will proceed in the next cell.\n",
    "However, the IDs are easy to use but less meaningful for a human (since it is the names of the powerlines that we see) and sometimes, when inspecting an observation for example, we may want to look at a specific powerline by specifying its name. This will be covered later in the notebook [7_PlottingCapabilities](7_PlottingCapabilities.ipynb)."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Another way to achieve the same things is:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "change_status = action_space.get_change_line_status_vect()\n",
    "change_status[0] = True\n",
    "change_status[1] = True\n",
    "change_status[2] = True\n",
    "\n",
    "set_status = action_space.get_set_line_status_vect()\n",
    "set_status[3] = 1\n",
    "set_status[4] = 1\n",
    "set_status[5] = -1\n",
    "set_status[6] = -1\n",
    "\n",
    "this_first_act = action_space({\"set_line_status\": set_status, \"change_line_status\": change_status})"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**NB** even if it can handle different types, for performance reasons it's better to follow the type of data mentionned below : The dictionnary values should be:\n",
    "* for **change** actions, booleans :\n",
    "  * `True` means \"*change*\"\n",
    "  * `False` means \"*don't change*\"\n",
    "* for **set** actions, integers :\n",
    "   * `0` means \"*do nothing*\"\n",
    "   * `1` means \"*connect it*\"\n",
    "   * `-1` means \"*disconnect it*\"\n",
    "   \n",
    "For convenience, an Action object can be inspected easily by using the `print` method. It will summarize on which object it has an impact:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "print(this_first_act)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "this_first_act.is_ambiguous()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**NB** This action is ambiguous so it cannot be implemented on the powergrid. Indeed, powerlines 3 and 4 are reconnected, but we don't specify on which bus! Implementing this action on a grid will be equivalent to doing nothing."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### b) Modify a single powerline, or a few powerlines"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "It's not always convenient to manipulate all the status of all the powerlines, or change it. For mor convenience, it's possible to modify only a few of them. The syntax is the following."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "the_same_act = action_space({\"set_line_status\": [(3,1), (4,1), (5,-1), (6,-1)],\n",
    "                             \"change_line_status\": [0,1,2]\n",
    "                            })\n",
    "print(the_same_act)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We can check that the two actions are indeed equal:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "the_same_act == this_first_act"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### B) Substations reconfiguration / topology changes"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "One of the interesting aspects of Grid2Op is to be able to modify the topology of the powergrid. In other words it allows to reconfigure the way the objects (generators, loads, end of powerlines) are interconnected at their substations.\n",
    "\n",
    "Comparable to the status change, topological change can be interpreted in two disctinct manners, as described above. Topologycal changes include some of the most interesting interactions with the environment.\n",
    "\n",
    "In this section we study how to modify the topology of the powergrid.\n",
    "\n",
    "The underlying way to represent the topology is through a integer vector, having the same dimension as the number of objects of the grid. For each object in the grid, this vector tells on which bus it's connected. Manipulating this vector can be done, but is absolutely not handy. We present here the way to change the topology through the helped, which can be done more easily.\n",
    "\n",
    "To **set** the bus to which a load is connected, it is recommended to do:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "set_bus_load_0 = action_space({\"set_bus\": {\"loads_id\": [(0,2)]}})\n",
    "print(set_bus_load_0)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "To **change** the bus, a similar interface can be used:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "change_bus_load_0 = action_space({\"change_bus\": {\"loads_id\": [0]}})\n",
    "print(change_bus_load_0)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The API is really similar for generator:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "change_bus_gen_0_and_1 = action_space({\"change_bus\": {\"generators_id\": [0,1]}})\n",
    "set_bus_gen_3_and_4 = action_space({\"set_bus\": {\"generators_id\": [(3,2), (4,2)]}})\n",
    "print(set_bus_gen_3_and_4)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The same goes for each ends of the powerlines:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "change_bus_lines_or_0 = action_space({\"change_bus\": {\"lines_or_id\": [0]}})\n",
    "set_bus_lines_or_4 = action_space({\"set_bus\": {\"lines_or_id\": [(3,2)]}})\n",
    "change_bus_lines_ex_15 = action_space({\"change_bus\": {\"lines_ex_id\": [15]}})\n",
    "set_bus_lines_ex_18 = action_space({\"set_bus\": {\"lines_ex_id\": [(18,2)]}})\n",
    "print(set_bus_lines_ex_18)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "When reconnecting a powerline, if the bus to which this powerline is reconnected is not specified, the action is ambiguous and thus will not be implemented. It is, in that case, recommended to use the `reconnect_powerline` method as followed:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "reconnecting_line_1 = action_space.reconnect_powerline(line_id=1,bus_or=1,bus_ex=1)\n",
    "print(reconnecting_line_1)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Finally, if you know how many objects are in a substation, you can also modify all their buses in one call:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "reconfigure_substation_id_1 = action_space({\"set_bus\": {\"substations_id\": [(1, (1,2,2,1,1,2))]}})"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "In the above code we knew that the substations of id 1 had 6 elements, and we assign to the elements of id 1,2 and 5 [second, third and sixth] the bus 2 and the others to bus 1.\n",
    "\n",
    "Using this way does not make explicit which objects are modified. You need to know beforehand that the element of id 0 of this substation is the extremity of powerline 0, the element of id 1 of this substation is the origin of powerline 1 etc. We explain how to do so in the next paragraph.\n",
    "\n",
    "#### Know the order of the objects on a substation\n",
    "You can also have a look at the `grid_objects_types` object that is a representation, as a numpy array, of which element is connected to each substation. It counts as many rows as there are elements on the powergrid (here 56: 5 generators, 11 loads and 20 lines each having two ends - making `5 + 11 + 2*20 = 56` objects) and exactly 5 columns. The first column of this matrix of this matrix represents the id of the substation at which the object represented by the row is located.\n",
    "\n",
    "Then the next 4 columns each account for an object type: either it's a load (column 1) or a generator (column 2) or the extremity of a powerline (column 3) or the extremity of a powerline (column 4). Let's take some example instead of getting lost in the details:\n",
    "\n",
    "If at row 0 of `grid_objects_types` i see `[ 0, -1, -1,  0, -1]` this means that: the object of id 0 (remember we were looking for the row 0) is connected to substation 0 (first element of this vector). This is not a load nor a generator (there are `-1` in columns 1 and 2 encoding for load and generators respectively. This is not an extremity of a powerline (there is a `-1` in the 4th column encoding for the powerline extremity). We see a \"0\" on column 3, encoding for \"powerline origin\" this means that this is the origin of the powerline of id 0.\n",
    "\n",
    "On the row 1 of `grid_objects_types` we see `[ 0, -1, -1,  1, -1]`. With the same reasoning we know that it corresponds to the origin of powerline 1 that is connected to substation 0.\n",
    "\n",
    "On the row 2 we see `[ 0, -1,  4, -1, -1]`. This means that the third object of the substation 0 is the generator of id 4.\n",
    "\n",
    "And a last example is the following. The row 12 of `grid_objects_types` is `[ 2,  1, -1, -1, -1]` (see below) this means that the 12th element of the grid is: connected to substation 2, is a load, this load has id 1."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "action_space.grid_objects_types[12]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "This representation in terms of `grid_objects_types` is rather explicit for a human but is also particularly suited to look for information informatically thanks to numpy indexing. For example if you want to know where the loads are located you can do:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "is_load = action_space.grid_objects_types[:,action_space.LOA_COL] != -1\n",
    "print(\"The substation with at least one load are: {}\".format(\n",
    "    action_space.grid_objects_types[is_load,action_space.SUB_COL]))\n",
    "\n",
    "is_sub1 = action_space.grid_objects_types[:, action_space.SUB_COL] == 1\n",
    "is_gen = action_space.grid_objects_types[:, action_space.GEN_COL] != -1\n",
    "print(\"The generator ids connected to substations 1 are: {}\".format(\n",
    "    action_space.grid_objects_types[is_sub1 & is_gen, action_space.GEN_COL]))\n",
    "\n",
    "print(\"The object connected to substation 1 are: {}\".format(\n",
    "     action_space.grid_objects_types[is_sub1,:]))\n",
    "# etc."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "However, the method \"`get_obj_connect_to`\" is clearer to read for human. This gives the following information:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "action_space.get_obj_connect_to(substation_id=1)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "In this case it means on the substation 1 are connected:\n",
    "- the load of id 0\n",
    "- the generator of id 0\n",
    "- the origin of powerlines 2, 3 and 4\n",
    "- the extremity of powerline 0\n",
    "- no storage units\n",
    "And this substation counts 6 elements.\n",
    "\n",
    "Note that this does not allow easily to know which object is assign to which bus with the action `reconfigure_substation_id_1`"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### C) Indexed actions"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "For convenience, it might be better sometimes to change the bus of an object from its name instead of its ID in case the ID is not known. Grid2Op allows to do that, but only for changing or setting a bus. These methods take longer than the methods shown above. If they are used at all, it's recommended NOT to use them for training an Agent. Their main goal aims at debugging and / or understanding the behaviour of an Agent.\n",
    "\n",
    "These methods are:\n",
    "* [`action_space.change_bus`](https://grid2op.readthedocs.io/en/latest/action.html#grid2op.Action.HelperAction.change_bus) ($\\leftarrow$ this is a link)\n",
    "* [`action_space.set_bus`](https://grid2op.readthedocs.io/en/latest/action.html#grid2op.Action.HelperAction.set_bus) ($\\leftarrow$ this is a link)\n",
    "\n",
    "Please refer to the official documentation for a complete description of their behaviour. To sum up, we can use them this way:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "my_act = action_space.set_bus(\"gen_1_0\", # mandatory name of the element\n",
    "                               new_bus=2, # mandatory the new bus to connect it too\n",
    "                               type_element=\"gen\", # optional the type of the element, one of \"line\", \"gen\" or \"load\"\n",
    "                               previous_action=None  # optional: if you want to combine multiple action, you can do it with this\n",
    "                              )\n",
    "print(my_act)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "action_space.set_bus(\"1_3_3\", # mandatory name of the element\n",
    "                     extremity=\"or\", # mandatory, which extrmity to change\n",
    "                     new_bus=2, # mandatory the new bus to connect it too\n",
    "                     type_element=\"line\", # optional the type of the element, one of \"line\", \"gen\" or \"load\"\n",
    "                     previous_action=my_act  # optional: if you want to combine multiple action, you can do it with this\n",
    "                    )\n",
    "print(my_act)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "action_redispatch = action_space()\n",
    "action_redispatch.redispatch = [3.1, 0, 0, 0, 0, 0]  # or action_redispatch.redispatch = [(0, 3.1)]\n",
    "action_change_bus = action_space()\n",
    "action_change_bus.load_change_bus = [0]\n",
    "action_change_bus.line_or_change_bus = [1]\n",
    "action_storage = action_space()\n",
    "action_storage.storage_power = [-1.5, 0.] # or action_storage.storage_power = [(0, -1.5)]\n",
    "combined_action2 = action_redispatch + action_change_bus + action_storage\n",
    "print(combined_action2)"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.8.10"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}