Introducing Deep RL Zoo

A collection of Deep Reinforcement Learning algorithms implemented with PyTorch to solve Atari games and classic control tasks like CartPole, LunarLander, and MountainCar

By Michael Hu
May 3, 2022 9:00 pm
2 min read

Reinforcement learning is a very exciting field in machine learning, with the success of the DQN agent playing Atari games at a human level [1], and AlphaGo playing the game of Go at a superhuman level [2].

We are excited to share our latest work Deep-RL-Zoo, an open-source implementation of common deep reinforcement learning algorithms, implemented with PyTorch. Our work is based on DQN Zoo [3], where we adapted the code from JAX to PyTorch. In addition, we have also implemented new state-of-the-art (SOTA) algorithms like PPO, RND, and Agent57.

This project is intended for educational and research purposes only, following the following principle:

  • Research-Friendly: we focus a detailed exploration of individual algorithms rather than creating a standard library.

  • Simple Code: We keep the code simple and mostly in a single file module, so you can easily adapt it to support other algorithms.

  • Low Dependencies: We only use basic tools like Numpy, PyTorch, and Gym for the project, with no reliance on third-party reinforcement learning libraries.

References

  • [1]

    Volodymyr Mnih, Koray Kavukcuoglu, David Silver, Alex Graves, Ioannis Antonoglou, Daan Wierstra, Martin Riedmiller. Playing Atari with Deep Reinforcement Learning. arXiv:1312.5602, 2013.

  • [2]

    Silver, David, Huang, Aja, Maddison, Chris J., Guez, Arthur, Sifre, Laurent, van den Driessche, George, Schrittwieser, Julian, Antonoglou, Ioannis, Panneershelvam, Veda, Lanctot, Marc, Dieleman, Sander, Grewe, Dominik, Nham, John, Kalchbrenner, Nal, Sutskever, Ilya, Lillicrap, Timothy, Leach, Madeleine, Kavukcuoglu, Koray, Graepel, Thore, Hassabis, Demis. Mastering the game of Go with deep neural networks and tree search. Nature, 529 (484--489), 10.1038/nature16961, 2016.

  • [3]

    John Quan, Georg Ostrovski. DQN Zoo: Reference implementations of DQN-based agents. 2020. http://github.com/deepmind/dqn_zoo