This repo contains the full implementation of the E-MAML algorithm from the paper Some Considerations on Learning to Explore via Meta-Reinforcement Learning
The main implementation is contained in the e_maml_tf
directory. Inside the e_maml_experiments
directory we provide a light weight half-cheetah baseline for verification. The original
KrazyWorld codebase is not opensourced. So we implemented a new KrazyWorld environment. To run E-MAML
on this new KrazyWorld, you need to add a thin adaptor following the convention in custom_vendor
and sampler.py
.
-
Setup conda environment with python 3.6.4 or above. (this is required for all of the
f-string
literals.) -
if on mac, run
brew install mpich
. this is the MPI version thatbaseline
andmpi4py
relies on. -
run
pip install -e .
. If thempi4py
installation fails, trypip install mpi4py
in a new terminal session. -
if
mujoco-py
complains (which fails the installation), make sure you have installed mujoco and have a working license key. -
If not, you should download mujoco for your environment and place the license key
mjkey.txt
under~/.mujoco/
. -
Distributed Setup: Add a file
.yours
insidee_maml_experiments
that contains the following content:username: <your-id> project: e_maml logging_server: http://<your-ml-logger-logging-server>:8081
If you are not using a distributed logging setup, you can leave the logging_server to
none
or leave it empty. In that case it would be logged to you~/ml-logger-outputs
directory.
To cite E-MAML please use
@article{stadie2018e-maml,
title={Some considerations on learning to explore via meta-reinforcement learning},
author={Stadie, Bradly C and Yang, Ge and Houthooft, Rein and Chen, Xi and Duan, Yan and Wu, Yuhuai and Abbeel, Pieter and Sutskever, Ilya},
journal={arXiv preprint arXiv:1803.01118},
year={2018}
}