I have decided to move my blog to my github page, this post will no longer be updated here. You can find the newest revision here.
On December 4th 2016 OpenAI released Universe described as:
It is a pretty exciting release so I thought I’d write up a simple guide for getting started with Universe & Ubuntu 16.04. Let’s get started with the required Ubuntu packages
For Ubuntu 16.04:
$ sudo apt-get update $ sudo apt-get install golang python3-dev python-dev libcupti-dev libjpeg-turbo8-dev make tmux htop chromium-browser git cmake zlib1g-dev libjpeg-dev xvfb libav-tools xorg-dev python-opengl libboost-all-dev libsdl2-dev swig
For Ubuntu 14.04
$ sudo add-apt-repository ppa:ubuntu-lxc/lxd-stable # for newer golang $ sudo apt-get update $ sudo apt-get install golang python3-dev python-dev libcupti-dev libjpeg-turbo8-dev make tmux htop chromium-browser git cmake zlib1g-dev libjpeg-dev xvfb libav-tools xorg-dev python-opengl libboost-all-dev libsdl2-dev swig
Anaconda Python 3.5 is by far the easiest python environment to use in conjunction with Universe . So lets gets started by installing that.
$ wget https://repo.continuum.io/archive/Anaconda3-4.2.0-Linux-x86_64.sh
And install by:
$ bash Anaconda3-4.2.0-Linux-x86_64.sh
When Anaconda asks if you would wish prepend the Anaconda install location to your bash type in ‘yes’, but if you accidentally defaulted no by pressing enter you can.
$ gedit ~/.bashrc
and copy and paste this in the bottom
export PATH="/home/$USER/anaconda3/bin:$PATH"
You will have to open up a new terminal to use Anaconda. Now the best thing to do is to create a new isolated environment to manage package versions and so that you don’t have to reinstall Anaconda if you flub your python packages
$ conda create --name universe python=3.5 anaconda
and activate the environment
$ source activate universe
We will need to build additional pylons, I mean packages.
(universe) $ conda install pip six libgcc swig
Next let’s install the Computer Vision package opencv:
(universe) $ conda install opencv
and finally let’s install Tensorflow. If using tensorflow with GPU please follow all the instructions here and stop at Install Bazel and then follow by using pip installation. Choose one of the following:
(universe) $ pip install --upgrade tensorflow (universe) $ pip install --upgrade tensorflow-gpu
Next we can get started by installing Docker:
$ sudo apt-get install \ apt-transport-https \ ca-certificates \ curl \ software-properties-common
For Ubuntu 14.04:
$ sudo apt-get install \ linux-image-extra-$(uname -r) \ linux-image-extra-virtual
Followed by:
$ curl -fsSL https://download.docker.com/linux/ubuntu/gpg | sudo apt-key add -
Followed by:
$ sudo add-apt-repository \ "deb [arch=amd64] https://download.docker.com/linux/ubuntu \ $(lsb_release -cs) \ stable"
And finish with:
$ sudo apt-get update $ sudo apt-get install docker-ce
And test installation by:
$ sudo service docker start $ sudo docker run hello-world
You should see a message Hello from Docker! informing you that your installation appears correct. To make it so you don’t have to use sudo to use docker you can:
$ sudo groupadd docker $ sudo usermod -aG docker $USER $ sudo reboot # IF LATER YOU GET DOCKER CONNECTION ISSUES TRY: $ sudo groupadd docker $ sudo gpasswd -a ${USER} docker $ sudo service docker restart $ sudo reboot
Next make sure your still in your Anaconda environment and lets clone and install Gym:
$ source activate universe (universe) $ cd ~ (universe) $ git clone https://github.com/openai/gym.git (universe) $ cd gym (universe) $ pip install -e '.[all]'
Followed by Universe
(universe) $ cd ~ (universe) $ git clone https://github.com/openai/universe.git (universe) $ cd universe (universe) $ pip install -e .
Let’s do a test by downloading test.py running it in our conda environment you will also want to make sure that chromium browser is your default web browser.
(universe) $ python ~/Downloads/test.py
You should see Pong load and the green paddle do nothing but random actions, a pretty poor policy which will eventually yield results. Press ctrl-c to exit.
So let’s download Open AI’s starter agent which will train an agent using the A3C Algorithim. For pong you have two options for which environment to run your algorithm with. This fastest is the ALE (Arcade Learning Environment) or you can also run on a VNC remote desktop. The VNC option though will take longer to solve but is a lot more interesting so we will use that.
(universe) $ cd ~ (universe) $ git clone https://github.com/openai/universe-starter-agent.git (universe) $ cd ~/universe-starter-agent (universe) $ python train.py --num-workers 2 --env-id gym-core.PongDeterministic-v3 --log-dir /tmp/vncpong --visualise
If everything was installed properly this should start training an agent, to observe workers enter in a new terminal.
$ tmux a
Then press ctrl – b followed by 0-4 to observe individual windows in tmux.
You can also checkout your TensorFlow’s graph, diagnostics and performance visually using Tensorboard by connecting to http://localhost:12345/ in your web browser.
Can’t wait to see where Universe leads to next…
http://localhost:15900/viewer/?password=openai is giving blank screen did you got this problem ?
LikeLike
Sometime it takes a second which browser you using?
LikeLike
Thanks for writing this up, it helped out a lot. I was able to get the test.py running pretty quickly, but had trouble getting the starter agent to work. Take a look at the issues and some of the pulls, you need to carefully install the prerequistes and might have to change the port for tensorboard.
LikeLike
Hi, thanks for the writeup! Got everything working, except I can’t vnc into the pong instances running under universe-starter-agent. Even when I run them as part of your test.py there’s no vnc server running on my machine, though it seems to be running fine otherwise. I am however able to VNC in if I start up an env like e.g. “flashgames.DuskDrive-v0” in both situations.
I’m assuming that this has to do with the fact that by default it’s running directly through ALE and not as flash in a sandbox, but I can’t find any anything anywhere that notes this — even the universe-starter-agent readme assumes you can just vnc in immediately. Do I need to start the remotes manually to enable vnc for pong?
LikeLike
Great Tutorial ,Thanks ! However , stuck up at installing opencv as it is throwing up conflict with python 3.6* :
universe) hrishikesh@DeepState:~$ conda install -c conda-forge opencv
Fetching package metadata ………
Solving package specifications: ….
UnsatisfiableError: The following specifications were found to be in conflict:
– opencv
– python 3.6*
Use “conda info ” to see the dependencies for each package.
LikeLike
Thank you very much for the tutorial
It works perfect in Ubuntu 16.04, both, locally and in an instance EC2 the Amazon AWS
In Ubuntu 14.04 fail in:
conda install -c conda-forge opencv
and in
pip install –ignore-installed –upgrade $TF_BINARY_URL
But in 16.04 works perfect
LikeLike
Hi
A big thanks for your tutorial!
I’m unable to run your test.py file.
I get this error:
File “/home/USERNAME/universe/universe/envs/vnc_env.py”, line 495, in _handle_connect
for remote in self.remote_manager.pop(n=n):
AttributeError: ‘VNCEnv’ object has no attribute ‘remote_manager’
However the example file given here https://github.com/openai/universe/blob/master/README.rst#system-overview
seems to work though the game doesnt start
LikeLike
Thanks for this! FYI I had to add a couple of steps to get test.py going
– Needed to add extra line to test.py file as per here http://stackoverflow.com/questions/41852625/vncenv-object-has-no-attribute-remote-manager-when-running-openai-universe-e
– Needed to add $USER to docker group to enable universe/gym working https://docs.docker.com/engine/installation/linux/linux-postinstall/
LikeLike