TF-Agents simulates multiple environments in parallel, and groups them to perform the neural network computation on a batch rather than individual observations. This allows the TensorFlow execution engine to parallelize computation, without the need for manual synchronization. Environments are stepped in separate Python processes to progress them in parallel without interference of the global interpreter lock.
An "agent" is a core element of reinforcement learning which encompasses two main responsibilities:
- defining a Policy to interact with the Environment; and
- determining how to learn/train that Policy from collected experience.
Currently the following algorithms are available under TF-Agents:
- DQN: Human level control through deep reinforcement learning.
- DDQN: Deep Reinforcement Learning with Double Q-learning.
- DDPG: Continuous control with deep reinforcement learning.
- TD3: Addressing Function Approximation Error in Actor-Critic Methods.
- REINFORCE: Simple Statistical Gradient-Following Algorithms for Connectionist Reinforcement Learning.
- PPO: Proximal Policy Optimization Algorithms.
- SAC: Soft Actor Critic.
In their paper, TensorFlow Agents: Efficient Batched Reinforcement Learning in TensorFlow, authors Hafter, Davidson, and Vanhoucke also introduced BatchPPO, which is an efficient implementation of the proximal policy optimization algorithm.
Setting up a Python Environment with Unity ML-Agents and TensorFlow for macOS
TensorFlow Agents: Efficient Batched Reinforcement Learning in TensorFlow
Danijar Hafner, James Davidson, Vincent Vanhoucke
Documentaries, videos and podcasts
TensorFlow at DeepMind (TensorFlow Dev Summit 2017)
February 15, 2017
- Reinforcement LearningAn area of machine learning focusing on how machines and software agents react in a specific context to maximize performance and achieve reward known as reinforcement signal.
- TensorFlowA machine learning software library for numerical calculation
- Artificial neural networkA computational model biologically inspired network of artificial neurons applied in computers to execute specific tasks
- DeepMindDeepMind is an artificial intelligence company creating programs that use deep neural networks to teach themselves how to play a variety of games like Go and chess.
- Q-learningA model-free reinforcement technique in machine learning and data mining that compares available actions to states of the expected actions for a given machine state.
- Show More