README.md 9.41 KB
Newer Older
Benjamin Beyret's avatar
Benjamin Beyret committed
1
2
3
4
5
# Animal-AI Olympics

## Overview

Welcome to the repository for the Animal-AI Olympics competition where you will find all the code needed to compete in 
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
this new challenge. Note that for the moment this repo contains **only the training environment** (v0.5) that will be 
used for the competition and **does not contain any competition tests or information for entering**. If everything goes 
well the competition will be live on June 30th. Until then we will be continually updating with bug fixes and small 
changes to the environment. However, the general structure will stay the same so it's not too early to start working with 
the environment. For more information on the competition itself and to stay updated with any developments, head to the 
[Competition Website](http://www.animalaiolympics.com/) and follow [@MacroPhilosophy](https://twitter.com/MacroPhilosophy) 
and [@BenBeyret](https://twitter.com/BenBeyret) on twitter.

The environment contains an agent enclosed in a fixed sized arena. Objects can spawn in this arena, including positive 
and negative rewards (green, yellow and red spheres). All of the hidden tests that will appear in the competition are 
made using the objects in the training environment. We have provided some sample environment configurations that should 
be useful for training, but part of the challenge will be experimenting and designing new configurations.

The goal of this first release is to **seek feedback from the community** as well as to provide the environment for 
research prior to the launch of the competition itself. The competition version of the environment will be similar to 
this one, however we are open to suggestion (for minor changes) and especially bug reports! Head over to the 
[issues page](https://github.com/beyretb/AnimalAI-Olympics/issues) and open a ticket using the `suggestion` or `bug` 
labels respectively.
Benjamin Beyret's avatar
Benjamin Beyret committed
24
25
26

To get started install the requirements below, and then follow the [Quick Start Guide](documentation/quickstart.md). 
A more in depth documentation <!--, including a primer on animal cognition,--> can be found on the 
27
[Documentation Page](documentation/README.md).
Benjamin Beyret's avatar
Benjamin Beyret committed
28

mdcrosby's avatar
mdcrosby committed
29
30
## Development Blog

31
32
You can read the development blog [here](https://mdcrosby.com/blog). It covers further details about the competition as 
well as part of the development process.
mdcrosby's avatar
mdcrosby committed
33
34
35
36
37

1. [Why Animal-AI?](https://mdcrosby.com/blog/animalai1.html)

2. [The Syllabus (Part 1)](https://mdcrosby.com/blog/animalai2.html)

Benjamin Beyret's avatar
Benjamin Beyret committed
38
39
## Requirements

40
41
The Animal-AI package works on most platforms. 
<!--, for cloud engines check out [this cloud documentation](documentation/cloud.md).-->
Benjamin Beyret's avatar
Benjamin Beyret committed
42

43
44
First of all your will need `python3.6` installed, we recommend using virtual environments. We provide two packages for 
this competition:
Benjamin Beyret's avatar
Benjamin Beyret committed
45

46
47
48
49
- The main one is an API for interfacing with the Unity environment. It contains both a 
[gym environment](https://github.com/openai/gym) as well as an extension of Unity's 
[ml-agents environments](https://github.com/Unity-Technologies/ml-agents/tree/master/ml-agents-envs). You can install it
 via pip:
Benjamin Beyret's avatar
Benjamin Beyret committed
50
```
51
pip install animalai
Benjamin Beyret's avatar
Benjamin Beyret committed
52
```
53
54
55
56
57
58
59
60
61
Or you can install it from the source, head to `animalai/` folder and run `pip install -e .`.

- We also provide a package that can be used as a starting point for training, and which is required to run most of the 
example scripts found in the `examples/` folder. It contains an extension of 
[ml-agents' training environment](https://github.com/Unity-Technologies/ml-agents/tree/master/ml-agents) that relies on 
[OpenAI's PPO](https://openai.com/blog/openai-baselines-ppo/), as well as 
[Google's dopamine](https://github.com/google/dopamine) which implements 
[Rainbow](https://www.aaai.org/ocs/index.php/AAAI/AAAI18/paper/download/17204/16680) (among others). You can also install 
this package using pip:
Benjamin Beyret's avatar
Benjamin Beyret committed
62
```
63
pip install animalai-train
Benjamin Beyret's avatar
Benjamin Beyret committed
64
```
65
Or you can install it from source, head to `examples/animalai_train` and run `pip install -e .`.
Benjamin Beyret's avatar
Benjamin Beyret committed
66

Benjamin Beyret's avatar
Benjamin Beyret committed
67
Finally download the environment for your system:
Benjamin Beyret's avatar
Benjamin Beyret committed
68
69
70

| OS | Environment link |
| --- | --- |
Benjamin Beyret's avatar
Benjamin Beyret committed
71
72
73
| Linux |  [download v0.4](https://www.doc.ic.ac.uk/~bb1010/animalAI/env_linux_v0.4.zip) |
| MacOS |  [download v0.4](https://www.doc.ic.ac.uk/~bb1010/animalAI/env_mac_v0.4.zip) |
| Windows | [download v0.4](https://www.doc.ic.ac.uk/~bb1010/animalAI/env_windows_v0.4.zip)  |
Benjamin Beyret's avatar
Benjamin Beyret committed
74

75
76
77
You can now unzip the content of the archive to the `env` folder and you're ready to go! Make sure the executable 
`AnimalAI.*` is in `env/`. On linux you may have to make the file executable by running `chmod +x env/AnimalAI.x86_64`. 
Head over to [Quick Start Guide](documentation/quickstart.md) for a quick overview of how the environment works.
Benjamin Beyret's avatar
Benjamin Beyret committed
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92

## Manual Control

If you launch the environment directly from the executable or through the VisualizeArena script it will launch in player 
mode. Here you can control the agent with the following:

| Keyboard Key  | Action    |
| --- | --- |
| W   | move agent forwards |
| S   | move agent backwards|
| A   | turn agent left     |
| D   | turn agent right    |
| C   | switch camera       |
| R   | reset environment   |

93
94
95
**Note**: on some platforms, playing manually in full screen makes the environment slow, keep the environment in window 
mode for better performance.

96
97
## Competition Tests

98
99
100
101
102
103
104
We will be releasing further details about the tests in the competition over the coming weeks. The tests will be split 
into multiple categories from the very simple (e.g. **food retrieval**, **preferences**, and **basic obstacles**) to 
the more complex (e.g. **working memory**, **spatial memory**, **object permanence**, and **object manipulation**). For 
now we have included multiple example config files that each relate to a different category. As we release further 
details we will also specify the rules for the type of tests that can appear in each category. Note that the example 
config files are just simple examples to be used as a guide. An agent that solves even all of these perfectly may still 
not be able to solve all the tests in the categories but it would be off to a very good start.
105

mdcrosby's avatar
mdcrosby committed
106
107
108
109
## Citing

For now please cite the [Nature: Machine Intelligence piece](https://rdcu.be/bBCQt): 

110
111
Crosby, M., Beyret, B., Halina M. [The Animal-AI Olympics](https://www.nature.com/articles/s42256-019-0050-3) Nature 
Machine Intelligence 1 (5) p257 2019.
mdcrosby's avatar
mdcrosby committed
112

Benjamin Beyret's avatar
Benjamin Beyret committed
113
114
115
116
117
## Unity ML-Agents

The Animal-AI Olympics was built using [Unity's ML-Agents Toolkit.](https://github.com/Unity-Technologies/ml-agents)

The Python library located in [animalai](animalai) is almost identical to 
118
119
120
[ml-agents v0.7](https://github.com/Unity-Technologies/ml-agents/tree/master/ml-agents-envs). We only added the 
possibility to change the configuration of arenas between episodes. The documentation for ML-Agents can be found 
[here](https://github.com/Unity-Technologies/ml-agents/blob/master/docs/Python-API.md).
Benjamin Beyret's avatar
Benjamin Beyret committed
121

122
123
Juliani, A., Berges, V., Vckay, E., Gao, Y., Henry, H., Mattar, M., Lange, D. (2018). [Unity: A General Platform for 
Intelligent Agents.](https://arxiv.org/abs/1809.02627) *arXiv preprint arXiv:1809.02627*
Benjamin Beyret's avatar
Benjamin Beyret committed
124

125
126
127
## Known Bugs

Occasionally will spawn an empty arena in play mode. Temporary fix: just press R to respawn.
mdcrosby's avatar
mdcrosby committed
128

129
130
131
132
Occasional slow frame rates in play mode. Temporary fix: reduce screen size. 

## TODO

133
- [ ] Add protobuf for arena spawning feedback
134
- [x] Offer a gym wrapper for training
Benjamin Beyret's avatar
Benjamin Beyret committed
135
- [x] Improve the way the agent spawns
Benjamin Beyret's avatar
Benjamin Beyret committed
136
- [x] Add lights out configurations.
mdcrosby's avatar
mdcrosby committed
137
138
- [x] Improve environment framerates
- [x] Add moving food
mdcrosby's avatar
mdcrosby committed
139

Benjamin Beyret's avatar
Benjamin Beyret committed
140
## Version History
Benjamin Beyret's avatar
Benjamin Beyret committed
141

142
143
144
145
146
147
148
149
150
- v0.5 
    - Separate environment API and training API in Python
    - Release both as `animalai` and `animalai-train` PyPI packages (for `pip` installs)
    - Agent speed in play-mode constant across various platforms
    - Provide Gym environment
    - Add `trainBaselines,py` to train using `dopamine` and the Gym wrapper
    - Create the `agent.py` interface for agents submission
    - Add the `HotZone` object (equivalent to the red zone but without death)

Benjamin Beyret's avatar
Benjamin Beyret committed
151
152
153
- v0.4 - Lights off moved to Unity, colors configurations, proportional goals, bugs fixes
    - The light is now directly switched on/off within Unity, configuration files stay the same
    - Blackouts now work with infinite episodes (`t=0`)
154
155
156
157
    - The `rand_colors` configurations have been removed and the user can now pass `RGB` values, see 
    [here](documentation/configFile.md#objects)
    - Rewards for goals are now proportional to their size (except for the `DeathZone`), see 
    [here](documentation/definitionsOfObjects.md#rewards)
Benjamin Beyret's avatar
Benjamin Beyret committed
158
    - The agent is now a ball rather than a cube
Benjamin Beyret's avatar
Benjamin Beyret committed
159
    - Increased safety for spawning the agent to avoid infinite loops
Benjamin Beyret's avatar
Benjamin Beyret committed
160
161
    - Bugs fixes
    
Benjamin Beyret's avatar
Benjamin Beyret committed
162
- v0.3 - Lights off, remove Beams and add cylinder
163
164
    - We added the possibility to switch the lights off at given intervals, see 
    [here](documentation/configFile.md#blackouts)
Benjamin Beyret's avatar
Benjamin Beyret committed
165
    - visualizeLightsOff.py displays an example of lights off, from the agent's point of view
Benjamin Beyret's avatar
Benjamin Beyret committed
166
167
    - Beams objects have been removed
    - A `Cylinder` object has been added (similar behaviour to the `Woodlog`)
Benjamin Beyret's avatar
Benjamin Beyret committed
168
    - The immovable `Cylinder` tunnel has been renamed `CylinderTunnel`
Benjamin Beyret's avatar
Benjamin Beyret committed
169
    - `UnityEnvironment.reset()` parameter `config` renamed to `arenas_configurations_input`
Benjamin Beyret's avatar
Benjamin Beyret committed
170
    
mdcrosby's avatar
mdcrosby committed
171
172
- v0.2 - New moving food rewards, improved Unity performance and bug fixes 
    - Moving rewards have been added, two for each type of reward, see 
Benjamin Beyret's avatar
Benjamin Beyret committed
173
    [the details here](documentation/definitionsOfObjects.md#rewards).
mdcrosby's avatar
mdcrosby committed
174
175
176
    - Added details for the maze generator.
    - Environment performance improved.
    - [Issue #7](../../issues/7) (`-inf` rewards for `t: 0` configuration) is fixed.
Benjamin Beyret's avatar
Benjamin Beyret committed
177
178

- v0.1 - Initial Release
Benjamin Beyret's avatar
Benjamin Beyret committed
179