Introduction to RLPark

RLPark is a Java reinforcement learning library to experiment with online learning algorithms on robots and benchmarks. RLPark uses Zephyr for visualization and real-time display.

Demos of RLPark without visualization can be run directly just using rlpark.jar. Demos of RLPark with visualization are available in Zephyr. To install them, download Zephyr standalone application, then install all the RLPark features in Zephyr. After the installation, a new Demos menu will appear from which you can start the demos. See the documentation for more information.

RLPark features and algorithms:

• On-policy control: Sarsa(λ), Expected Sarsa(λ), Actor-Critic with normal distribution (continuous actions) and Boltzmann distribution (discrete action), average reward actor-critic
• Off-policy control: Off-PAC, Q-Learning, Q(λ), Greedy-GQ, Softmax-GQ
• On-policy prediction: TD, TD(λ)
• Off-policy prediction: GTD(λ), GQ(λ), TDC
• Acting: softmax, greedy, ε-greedy
• Online supervised learning: Adaline, IDBD
• Representations and function approximation: tile coding (with no hashing, hashing with UNH CMAC or mumur2), Linear Threshold Unit, Radial Basis Function, observation history, feature normalization
• Robots: the Critterbot, iRobot Create
• Compatible with Zephyr
• Problems: mountain car, swing-up pendulum, random walk, continuous world
• Eligibility traces: accumulating traces, accumulating traces with a max, replacing traces
• The Horde architecture (see OpenCL-Horde for an implementation on GPU by Clement Gehring)
• A framework for running parallel code on clusters
• A framework for on-policy control experiments and off-policy control experiments

RLPark related projects:

• OpenCL-Horde: implementation of Horde on GPU by Clement Gehring
• RLLib: port of RLPark to C++ by Saminda Abeyruwan

To learn more about Reinforcement Learning, the following two books are available online:

• Reinforcement Learning: An Introduction (Sutton & Barto, 1998)
• Algorithms for Reinforcement Learning (Szepesvári, 2010)

Contributors

Main author: Thomas Degris
Contributors: Jérome Béchu, Adam White, Joseph Modayil, Patrick M. Pilarski, Christian Denk