On December 4th 2016 OpenAI released Universe described as:

a software platform for measuring and training an AI’s general intelligence across the world’s supply of games, websites and other applications.

It is a pretty exciting release so I thought I’d write up a simple guide for getting started with Universe & Ubuntu 16.04.  Let’s get started with the required Ubuntu packages

For Ubuntu 16.04:

$ sudo apt-get update
$ sudo apt-get install golang libjpeg-turbo8-dev make tmux htop chromium-browser git cmake zlib1g-dev libjpeg-dev xvfb libav-tools xorg-dev python-opengl libboost-all-dev libsdl2-dev swig

For Ubuntu 14.04

$ sudo add-apt-repository ppa:ubuntu-lxc/lxd-stable  # for newer golang
$ sudo apt-get update
$ sudo apt-get install golang libjpeg-turbo8-dev make tmux htop chromium-browser git cmake zlib1g-dev libjpeg-dev xvfb libav-tools xorg-dev python-opengl libboost-all-dev libsdl2-dev swig

Anaconda Python 3.5 is by far the easiest python environment to use in conjunction with Universe .  So lets gets started by installing that.

$ wget https://repo.continuum.io/archive/Anaconda3-4.2.0-Linux-x86_64.sh

And install by:

$ bash Anaconda3-4.2.0-Linux-x86_64.sh

When Anaconda asks if you would wish prepend the Anaconda install location to your bash type in ‘yes’, but if you accidentally defaulted no by pressing enter you can.

$ gedit ~/.bashrc

and copy and paste this in the bottom

export PATH="/home/$USER/anaconda3/bin:$PATH"

You will have to open up a new terminal to use Anaconda.  Now the best thing to do is to create a new isolated environment to manage package versions and so that you don’t have to reinstall Anaconda if you flub your python packages

$ conda create --name universe

and activate the environment

$ source activate universe

We will need to build additional pylons, I mean packages.

(universe) $ conda install pip six libgcc

Next let’s install the Computer Vision package opencv:

 (universe) $ conda install -c conda-forge opencv

and finally let’s install Tensorflow 11.0 (the starter agent works best with this version), if using tensorflow with GPU please follow all the instructions here and stop at Install Bazel and then follow by using pip installation.  Choose one of the following:

# Ubuntu/Linux 64-bit, CPU only, Python 2.7
export TF_BINARY_URL=https://storage.googleapis.com/tensorflow/linux/cpu/tensorflow-0.11.0-cp27-none-linux_x86_64.whl

# Ubuntu/Linux 64-bit, GPU enabled, Python 2.7
export TF_BINARY_URL=https://storage.googleapis.com/tensorflow/linux/gpu/tensorflow-0.11.0-cp27-none-linux_x86_64.whl

# Ubuntu/Linux 64-bit, CPU only, Python 3.5
export TF_BINARY_URL=https://storage.googleapis.com/tensorflow/linux/cpu/tensorflow-0.11.0-cp35-cp35m-linux_x86_64.whl

# Ubuntu/Linux 64-bit, GPU enabled, Python 3.5
export TF_BINARY_URL=https://storage.googleapis.com/tensorflow/linux/gpu/tensorflow-0.11.0-cp35-cp35m-linux_x86_64.whl

Followed by:

(universe) $ pip install --ignore-installed --upgrade $TF_BINARY_URL

Next we can get started by installing Docker:

$ sudo apt-get install apt-transport-https ca-certificates
$ sudo apt-key adv \
--keyserver hkp://ha.pool.sks-keyservers.net:80 \
--recv-keys 58118E89F3A912897C070ADBF76221572C52609D

For Ubuntu 16.04:

$ echo "deb https://apt.dockerproject.org/repo ubuntu-xenial main" | sudo tee /etc/apt/sources.list.d/docker.list

For Ubuntu 14.04:

$ echo "deb https://apt.dockerproject.org/repo ubuntu-trusty main" | sudo tee /etc/apt/sources.list.d/docker.list

Followed by:

$ sudo apt-get update
$ sudo apt-get install linux-image-extra-$(uname -r) linux-image-extra-virtual
$ sudo apt-get install docker-engine

And test installation by:

$ sudo service docker start
$ sudo docker run hello-world

You should see a message Hello from Docker! informing you that your installation appears correct.  I found to get docker working correctly on my Ubuntu Mate laptop I also had to do this and reboot

$ sudo usermod -aG docker $(whoami)
$ sudo reboot
$ sudo groupadd docker
$ sudo gpasswd -a ${USER} docker
$ sudo service docker restart   
$ sudo reboot

Next make sure your still in your Anaconda environment and lets clone and install Gym:

$ source activate universe
(universe) $ cd ~
(universe) $ git clone https://github.com/openai/gym.git
(universe) $ cd gym
(universe) $ pip install -e '.[all]'

Followed by Universe

(universe) $ cd ~
(universe) $ git clone https://github.com/openai/universe.git
(universe) $ cd universe
(universe) $ pip install -e .

Let’s do a test by downloading test.py running it in our conda environment you will also want to make sure that chromium browser is your default web browser.

(universe) $ python ~/Downloads/test.py

You should see Pong load and the green paddle do nothing but random actions, a pretty poor policy which will eventually yield results.  Press ctrl-c to exit.

So let’s download Open AI’s starter agent which will train an agent using the A3C Algorithim.  For pong you have two options for which environment to run your algorithm with.  This fastest is the ALE (Arcade Learning Environment) or you can also run on a VNC remote desktop which allows you to view and interact with the simulation.  The VNC option though will take longer to solve but is a lot more interesting so we will use that.

(universe) $ cd ~
(universe) $ git clone https://github.com/openai/universe-starter-agent.git
(universe) $ cd ~/universe-starter-agent
(universe) $ python train.py --num-workers 2 --env-id gym-core.PongDeterministic-v3 --log-dir /tmp/vncpong

If everything was installed properly this should start training an agent, to observe workers enter in a new terminal.

$ tmux a

Then press ctrl – b followed by 0-4 to observe individual windows in tmux.

To peak into the VNC connect to:


You can also checkout your TensorFlow’s graph, diagnostics and performance visually using Tensorboard by connecting to http://localhost:12345/ in your web browser.  I modified my train.py file to use TensorBoard’s default port ‘6006’.

Can’t wait to see where Universe leads to next…