At Unity, we aim to maximize the transformative impact of Machine Learning for researchers and developers alike. Our Machine Learning tools, combined with the Unity platform, promote innovation. To further strengthen the Machine Learning community, we provide a forum where researchers and developers can exchange information, share projects, and support one another to advance the field.
Learn what Unity is up to in the area of Machine Learning.Download ML-Agents
Obstacle Tower Environment
The Obstacle Tower Environment is a procedurally generated Unity environment designed for machine learning researchers to push the limits of artificial intelligence (AI). Combining platform-style gameplay with puzzles and planning problems, the environment has an endless number of floors that are progressively more difficult as the agent advances. It natively supports the Unity ML-Agents Toolkit.Download the project Enter the Challenge
Understand the true potential of AI
Unity ML-Agents offers a flexible way to develop and test new AI algorithms quickly and efficiently across a new generation of robotics, games, and beyond.
Dr. Danny Lange, VP of AI and Machine Learning at Unity Technologies, former head of Machine Learning at Uber and Amazon.
Unity Machine Learning Agents beta
Unity Machine Learning Agents, the first of Unity’s machine learning product offerings, trains intelligent agents with reinforcement learning and evolutionary methods via a simple Python API, which enables:
- Academic researchers to study complex behaviors from visual content and realistic physics
- Industrial and enterprise researchers to implement large-scale parallel training regimes for robotics, autonomous vehicles, and other industrial applications
- Game developers to tackle challenges, such as using agents to dynamically adjust the game-difficulty level
ML-Agents Community Challenge Winners
Metal Warfare is a Real-time strategy (RTS) game designed for both Entertainment and Education purpose. The education perspective will be on Artificial Intelligence (AI) and Machine Learning (ML), that allow player to design or script AI and may do ML on it.
Inspired by the stealth genre, the project was designed to train an ML Agent to successfully run and hide from a traditional AI which patrols from room to room. A second variation was created to train against a faster AI by utilizing curriculum learning.
It is a Unity AI version, trained with ml-agents, of a little game which is recently popular on WeChat App in China, aiming to make the player jump to next box and never fall down to the ground. It only took two hours to train the AI to play very well.
Autocar with only camera learns to drive using pure RL.
- Process the camera image by detecting lane edges
- Build a model using PPO taking the edge image as input
- Train on a short track for 200 eps
- Train on a long track for another 200 eps
- Upon testing, act deterministically
SpaceY project demonstrates how Unity Machine Learning Agents can be used to teach a rocket to take off from one planet, then fly and land on another one.
This is a vehicle environment with static objects. The vehicle should evade tire piles (- reward) and get stars (+ reward), and it can only move in the lateral direction (3 actions -> left, right, keep).
The sea waves should change so as to bring the drifting ship to the target, overcoming obstacles. However, control over sea waves proved to be very difficult even for people. It seems that wave changes are unpredictable, but AI has learned how to control them!
The submission aims to automate breakfast, with one Machine Learning Agent flipping a pancake from a pan to a plate, and the “Pass the Butter Robot” from Rick and Morty dodging obstacles to deliver butter.
This project uses ML-Agents to stabilize satellite rotating in one axis. AI controls two satellite engines which can be turned on or off. Session starts with rotating satellite. It took two hours of ML-Agents learning to achieve a goal to stabilize satellite.
The rewards function
With Unity ML-Agents, a variety of training scenarios are possible, depending on how agents, brains, and rewards are connected. These include: Single Agent, Simultaneous Single Agent, Cooperative and Competitive Multi-Agent, and Ecosystem.
This tennis example shows an Adversarial Self-Play rewards function. Two interacting agents with inverse reward functions linked to a single brain. In two-player games, adversarial self-play can allow an agent to become increasingly more skilled, while always having the perfectly matched opponent: itself.
From the blog
Is your dream job at Unity?
Unity democratizes game development with tools that help developers create games and immersive experiences, work more productively, and connect with millions of users across multiple platforms.The Unity engine reaches nearly 3 billion devices worldwide.
We are currently looking for exceptional engineers to build the next-generation Machine Learning platform for Unity developers. Working closely with a stellar team of engineers and scientists, you’ll put Machine Learning and AI to work for Unity–and for the rest of the world.