On December 4th 2016 OpenAI released Universe described as:

a software platform for measuring and training an AI’s general intelligence across the world’s supply of games, websites and other applications.

It is a pretty exciting release so I thought I’d write up a simple guide for getting started with Universe & Ubuntu 16.04.  Let’s get started with the required Ubuntu packages

For Ubuntu 16.04:

$ sudo apt-get update
$ sudo apt-get install golang python3-dev python-dev libcupti-dev libjpeg-turbo8-dev make tmux htop chromium-browser git cmake zlib1g-dev libjpeg-dev xvfb libav-tools xorg-dev python-opengl libboost-all-dev libsdl2-dev swig

For Ubuntu 14.04

$ sudo add-apt-repository ppa:ubuntu-lxc/lxd-stable  # for newer golang
$ sudo apt-get update
$ sudo apt-get install golang python3-dev python-dev libcupti-dev libjpeg-turbo8-dev make tmux htop chromium-browser git cmake zlib1g-dev libjpeg-dev xvfb libav-tools xorg-dev python-opengl libboost-all-dev libsdl2-dev swig

Anaconda Python 3.5 is by far the easiest python environment to use in conjunction with Universe .  So lets gets started by installing that.

$ wget https://repo.continuum.io/archive/Anaconda3-4.2.0-Linux-x86_64.sh

And install by:

$ bash Anaconda3-4.2.0-Linux-x86_64.sh

When Anaconda asks if you would wish prepend the Anaconda install location to your bash type in ‘yes’, but if you accidentally defaulted no by pressing enter you can.

$ gedit ~/.bashrc

and copy and paste this in the bottom

export PATH="/home/$USER/anaconda3/bin:$PATH"

You will have to open up a new terminal to use Anaconda.  Now the best thing to do is to create a new isolated environment to manage package versions and so that you don’t have to reinstall Anaconda if you flub your python packages

$ conda create --name universe python=3.5 anaconda

and activate the environment

$ source activate universe

We will need to build additional pylons, I mean packages.

(universe) $ conda install pip six libgcc swig

Next let’s install the Computer Vision package opencv:

 (universe) $ conda install opencv

and finally let’s install Tensorflow.  If using tensorflow with GPU please follow all the instructions here and stop at Install Bazel and then follow by using pip installation.  Choose one of the following:

(universe) $ pip install --upgrade tensorflow     
(universe) $ pip install --upgrade tensorflow-gpu

Next we can get started by installing Docker:

$ sudo apt-get install \
    apt-transport-https \
    ca-certificates \
    curl \
    software-properties-common

For Ubuntu 14.04:

$ sudo apt-get install \
    linux-image-extra-$(uname -r) \
    linux-image-extra-virtual

Followed by:

$ curl -fsSL https://download.docker.com/linux/ubuntu/gpg | sudo apt-key add -

Followed by:

$ sudo add-apt-repository \
   "deb [arch=amd64] https://download.docker.com/linux/ubuntu \
   $(lsb_release -cs) \
   stable"

And finish with:

$ sudo apt-get update
$ sudo apt-get install docker-ce

And test installation by:

$ sudo service docker start
$ sudo docker run hello-world

You should see a message Hello from Docker! informing you that your installation appears correct.  To make it so you don’t have to use sudo to use docker you can:

$ sudo groupadd docker
$ sudo usermod -aG docker $USER
$ sudo reboot
# IF LATER YOU GET DOCKER CONNECTION ISSUES TRY:
$ sudo groupadd docker
$ sudo gpasswd -a ${USER} docker
$ sudo service docker restart   
$ sudo reboot

Next make sure your still in your Anaconda environment and lets clone and install Gym:

$ source activate universe
(universe) $ cd ~
(universe) $ git clone https://github.com/openai/gym.git
(universe) $ cd gym
(universe) $ pip install -e '.[all]'

Followed by Universe

(universe) $ cd ~
(universe) $ git clone https://github.com/openai/universe.git
(universe) $ cd universe
(universe) $ pip install -e .

Let’s do a test by downloading test.py running it in our conda environment you will also want to make sure that chromium browser is your default web browser.

(universe) $ python ~/Downloads/test.py

You should see Pong load and the green paddle do nothing but random actions, a pretty poor policy which will eventually yield results.  Press ctrl-c to exit.

So let’s download Open AI’s starter agent which will train an agent using the A3C Algorithim.  For pong you have two options for which environment to run your algorithm with.  This fastest is the ALE (Arcade Learning Environment) or you can also run on a VNC remote desktop.  The VNC option though will take longer to solve but is a lot more interesting so we will use that.

(universe) $ cd ~
(universe) $ git clone https://github.com/openai/universe-starter-agent.git
(universe) $ cd ~/universe-starter-agent
(universe) $ python train.py --num-workers 2 --env-id gym-core.PongDeterministic-v3 --log-dir /tmp/vncpong --visualise

If everything was installed properly this should start training an agent, to observe workers enter in a new terminal.

$ tmux a

Then press ctrl – b followed by 0-4 to observe individual windows in tmux.

You can also checkout your TensorFlow’s graph, diagnostics and performance visually using Tensorboard by connecting to http://localhost:12345/ in your web browser.

Can’t wait to see where Universe leads to next…

Advertisements