Projectiles
Pew Pew! Sometimes in games we want to create objects that move across the environment under their own power. In this tutorial we learn how to do this using GDY.
We build an environment where the jelly
agent can shoot projectiles to break box
es that are sitting on an island in the middle of an ocean of slime.
The agent receives a reward of 1 for every time a projectile hits a box, and a reward of 10 when all boxes are destroyed.
There’s several game mechanics here that we will explain in detail:
Spawning Objects
Initial Actions
Input Mappings
Delayed Actions
Internal Actions
Action Spaces
Collisions
Spawning the “flame” object
To spawn an object in a particular direction we use the following code:
- Name: flame_shoot
InputMapping:
Inputs:
1:
OrientationVector: [ 0, -1 ]
VectorToDest: [ 0, -1 ]
MetaData:
image_idx: 0
2:
OrientationVector: [ 1, 0 ]
VectorToDest: [ 1, 0 ]
MetaData:
image_idx: 1
3:
OrientationVector: [ 0, 1 ]
VectorToDest: [ 0, 1 ]
MetaData:
image_idx: 2
4:
OrientationVector: [ -1, 0 ]
VectorToDest: [ -1, 0 ]
MetaData:
image_idx: 3
Behaviours:
- Src:
Object: jelly
Dst:
Object: [ grass, _empty ]
Commands:
- spawn: flame
Firstly want to be able to spawn
our pink flame
object in a particular direction from our jelly
agent.
We can do this by defining an action which we will call flame_shoot
. In this action, we have 4 Input
objects, with ids 1-4 (0 is reserved for NOP actions).
Each Input is associated with a particular vector, which defines the direction and magnitude of the action. In this particular case the 4 actions correspond to up
, right
, down
and left
respectively.
Behaviours
We define which objects can perform actions and which objects can be the destination of actions by using Behaviours
.
In our snippet above we only have a single Behaviour
. This definition says that: If the object jelly
has grass
or _empty
(a special object name for “an empty space”)
in the destination location of the action, then spawn
a flame
object there.
The destination location is calculated as the location of the source object (jelly
) plus the vector given in VectorToDest
.
The action flame_shoot
will automatically be exposed as an action_type
with 4 action_ids
in the environment’s action space.
See also
You can find much more information about action spaces here
Setting the flame
tile image and initial direction
There are 4 images that we are going to use for the flame
object:
|
0 |
1 |
2 |
3 |
---|---|---|---|---|
Image |
When the flame
spawns, we want to make sure we set the correct tile based on the direction. For this we can use action MetaData
variables and InitialActions
:
Action MetaData
MetaData:
image_idx: 0
In the previous section, we defined the the flame_shoot
action. In each defined action_id
of the InputMapping
of this action, we include the VectorToDest
and also the MetaData
of this action.
For each action_id
you can define as many MetaData
variables as you like. Think of them as constants that are available in the behaviour of the action.
For each of the action_ids
we set a image_idx
variable which we can then use to set the current tile on the flame
object.
In the GDY we define 4 tiles which can be used to render the flame
object:
Objects:
- Name: flame
...
Observers:
Isometric:
- Image: oryx/oryx_iso_dungeon/fire-pink-up.png
- Image: oryx/oryx_iso_dungeon/fire-pink-right.png
- Image: oryx/oryx_iso_dungeon/fire-pink-down.png
- Image: oryx/oryx_iso_dungeon/fire-pink-left.png
Now we have defined our 4 images for UP, DOWN, LEFT and RIGHT and our image_idx
for each direction, we can make sure the right image is selected using InitialActions
Initial Actions
For this game in particular, we are going to create two initial actions. The first will only set the correct tile for the corresponding direction and the second will set the flame
object in motion.
- Name: flame
...
InitialActions:
- Action: set_flame_direction
- Action: flame_projectile_movement
Delay: 2
set_flame_direction
- Name: set_flame_direction
InputMapping:
Internal: true
Behaviours:
- Src:
Object: flame
Commands:
- set_tile: meta.image_idx
Dst:
Object: [ grass, _empty, flame, box ]
When an object is spawned, it automatically inherits the MetaData
and VectorToDest
of the spawning action (in this case flame_shoot
).
This means that the destination location for the Behaviours
will be calculated relative to the source object using the previous VectorToDest
.
For example:
* The jelly
at \([5,5]\) spawns a flame
object using action_id
2. The destination location of the action is \([6,5]\)
* The flame
object is spawned at location \([6,5]\)
* The flame
object then executes set_flame_direction
. This also uses action_id
2 from the previous action, meaning the destination location will be \([7,5]\)
We don’t really care what is in location \([7,5]\), so we can set the possible destination objects as any of the possible objects in the environment.
Finally we perform a set_tile
command using the action MetaData
. We can reference this variable using the meta.
prefix:
Commands:
- set_tile: meta.image_idx
flame_projectile_movement
We add a delay to the flame_projectile_movement
action so that it’s only called after 3 game ticks.
Like the set_flame_direction
this action will inherit the action MetaData
and VectorToDest
.
We don’t need the MetaData
in the flame_projectile_movement
action as we have already set the tile, but the VectorToDest
can be used to set the direction of travel of the projectile.
We will cover this in the next section!
Projectile movement
- Name: flame_projectile_movement
InputMapping:
Internal: true
Behaviours:
- Src:
Object: flame
Commands:
- mov: _dest
- eq:
Arguments: [ range, 0 ]
Commands:
- remove: true
- gt:
Arguments: [ range, 0 ]
Commands:
- decr: range
- exec:
Action: flame_projectile_movement
Delay: 3
Dst:
Object: [ _empty, grass ]
When flame_projectile_movement
is called, we check the destination location (using the inherited VectorToDest
) of the object to see if there is _empty
or grass
object. If there is, we run some commands.
Lets break these down line by line:
Firstly move the
flame
object to the_dest
variable, which contains the calculated destination location.- mov: _dest
Next we check a
range
variable. This is initialized in the flame object. If therange
variable is 0. We remove theflame
object.- eq: Arguments: [ range, 0 ] Commands: - remove: true
Then we check the
range
variable again, but this time we are looking if its larger than 0. If it is, then we decrement the value by 1.- gt: Arguments: [ range, 0 ] Commands: - decr: range
Finally we call the
flame_projectile_movement
function from within itself. But with a delay of 3 game ticks. So the process repeats again!- exec: Action: flame_projectile_movement Delay: 3
Putting all of these commands together, the flame
object moves one square in the initial direction every 3 game ticks. If the flame
object moves more than it’s range
. Then it will be removed.
However, what happens if the flame
encounters something thats not _empty
or grass
?
What we want to happen is that we want the flame
to destroy boxes, we also want to make sure that flames that bump into each other, or go off the edge of the map disappear.
This can be achieved by adding two more Behaviours
that handle these collisions.
Projectile Collisions
Behaviours:
...
- Src:
Object: flame
Commands:
- remove: true
- reward: 1
Dst:
Object: box
Commands:
- remove: true
- Src:
Object: flame
Commands:
- remove: true
Dst:
Object: [flame, _boundary]
In the snippet above, we have two Behaviours
the first one executes if the flame
object has the destination location of a box
object. In this case, the we remove both the flame
and the box
and give a reward of 1.
The second Behaviour
will remove the flame if it has the destination location of another flame
or the _boundary
object (which is a special pseudo object referring to the boundary of the environment.)
Gym Interface
Load the GDY and create a gym environment
Loading the environment is super simple, you can just point the GymWrapper
class at the projectiles.yaml
:
env = GymWrapper('projectiles.yaml', player_observer_type=gd.ObserverType.ISOMETRIC)
env.reset()
You now have an env
that you can use in Reinforcement Learning or any other experiments.
Action Space
So how can we now use this environment? How are the actions that we have defined exposed in the gym interface?
We have defined 4 actions in our GDY:
move
Move the
jelly
(UP,DOWN,LEFT,RIGHT)We didn’t actually mention this one in the tutorial above because its super simple, just a single behaviour that uses the
mov: _dest
command and the defaultInputMapping
(UP,DOWN,LEFT,RIGHT).
flame_projectile_movement
Defines the movement of projectiles
flame_shoot
Shoot a projectile in a particular direction (UP,DOWN,LEFT,RIGHT)
set_flame_direction
Defines the movement of projectiles
But we only want to be able to expose the move
and flame_shoot
actions. All actions defined in GDY are exposed by default, so to stop an action being exposed we use the following:
InputMapping:
Internal: true
This tells the Griddly engine that these actions are only used internally in the game, and cannot be called by an agent.
The actions that are exposed can then be used in the env.step
function:
env.step([0, 1]) # move UP
env.step([0, 2]) # move RIGHT
env.step([0, 3]) # move DOWN
env.step([0, 4]) # move LEFT
env.step([1, 1]) # flame_shoot UP
env.step([1, 2]) # flame_shoot RIGHT
env.step([1, 3]) # flame_shoot DOWN
env.step([1, 4]) # flame_shoot LEFT
See also
For more information on how Griddly deals with any action space you should look here
And thats about it for this tutorial!