Every bad AI moment in a game — the enemy who ignores you standing right in front of them, the turret that fires through a wall — usually traces back to the same root cause: a broken perception system. Let’s build one badly first, so you understand exactly why the better approach matters.
We’re going to build a simple automated turret that can detect potential threats and shoot at them. We will build this from first principles, using the quickest, simplest methods possible.
Why build the wrong system first? Unreal Engine provides a full featured perception system right out the box. Why bother with building a bad one?
You can treat UE’s AI systems like a black box and feed it inputs and read the outputs and everything will work just fine. You can stop reading now. But what if you’re trying to do something a little different? What if you’re building a game with multiple different factions and hundreds of living agents? Does the UE system scale to these numbers? What if you’re trying to build a complex stealth game where agents need to perceive changes in the environment like footsteps or moved objects? What if you don’t like the results you’re getting from UE’s system and want to change how it works?
Aside: If your first instinct is to setup a virtual camera on your agent, and render the world from their point of view to a texture and then run a deep convolution neural network to process that image to detect objects, I’ll stop you right here. We’re getting to the point where this approach can be done by real world robots, but in a video game, all of these steps are complex and take a lot of computing power, which we don’t have. We need to something much much simpler. Something that can simulate vision, without a ton of computation required.
A simple system is very quick to build. It can help you understand the problem space better. And you may discover some issues with the approach that you hadn’t anticipated before.
The best place to start is to think about the desired behavior for our turret – start from the desired results and then work back to determine what we need to get those results.
So what do we want our turret to do?
For the game I’m building, I want the turret to scan back and forth slowly until it can see a potential target and then it should track the target and fire at it. If it loses sight of the target, after a few seconds, it should go back to scanning back and forth.
For our turret, we have three requirements to check if we can see a target:
- It needs to be in front of the direction it’s facing
- It needs to be within some maximum distance and
- There needs to no obstacles between the turret and the target.
But before that, we need to decide what counts as a potential target? There is going to be asteroids, ship wreckage, resource pickups, projectiles, missiles and torpedoes, and space-ships or various sizes and factions. For now, we only care about enemy space-ships. Maybe in the future, we will want to handle decoys and maybe missiles, but for now we only want to check if we can see ships that we want to shoot.
The algorithm for our vision sensor is going to look something like this:
Find All objects of Type (SpaceShip) in MaxViewDistance of Turret
For each potential target:
check if target is in the field of view of Turret Facing.
if in FOV:
check if raycast from Turret to Target hits anything that isn’t the TargetThat’s pretty much it. Get everything we care about in range, check if it’s in the FOV, check if we have a clear line of sight to it.
Here is a quick and dirty Blueprint setup:
We might add a Sphere Collision component with a size matching the max view range. A collision type that only overlaps Pawns will work as we only care about player or AI controlled characters.
In a Tick, or Timer Event, get the pawns inside this overlap volume, and check if each one is inside the view-cone and has a clear line-of-sight trace.
Now, this works. It’s nothing fancy, but it works. And we can confirm it works with some debug visualizations.
But there are problems with this approach. It’s not very efficient and it doesn’t scale. It might be ok if you only ever has one or two turrets in your map and a single player to detect, but as the number of turrets increase and the number of potential targets increase, we’re going to be doing more and more work. Each turret checks against each target, so in the worst case ,the amount of work scales by the number of turrets multiplied by the number of targets – or O(N * M) in comp-sci speak. If you’re building a tower defense or an RTS game, using the approach above would cause your system to crawl!
Another problem is that we’ve built this code right into our turret class. If we derive more turrets, we can share that code, but if we want to make enemy ships that fly around, we’ll need to either copy this code into the enemy ship class, or move it down to a common NPC Actor class. And not only that, but having lots of large overlap trigger volumes that are moving around can cause slow downs in the physics and character movement systems too!
So what is the right way to build a perception system?
Next post, we’ll look at how flipping the model — moving perception off the agent and onto a system in the world — solves the scaling problem entirely. We’ll also look at how Unreal’s built-in system handles this, and where it falls short.
How not to build a perception system, pt2.
The views, thoughts, and opinions expressed on this website are my own and do not reflect those of my employer or any affiliated organizations.