How Games Outsmart Players: The Evolution of Game AI Ai4 · Aug 2021 · YouTube: Ai4 With John R. Diaz · 27 min Page: https://outofplayarea.com/videos/how-games-outsmart-players-the-evolution-of-game/ · Video: https://www.youtube.com/watch?v=JbASnZcYHII Transcript (YouTube's automatic captions, lightly cleaned; names and terms may be wrong): [0:08] all right welcome back I hope you're having an amazing time so far our next speaker is john r diaz he is a senior technical game designer at electronic arts please join me in welcoming john to our virtual stage hello thank you everyone for coming to this if you're looking to get a crash course into all the ways AI is used in then to develop the video games you hopefully play watch stream or know someone who does then you've come to the right place AI from my perspective is an autonomous robot that can either help humans simplify problems that are difficult for us or a piece of cake for a machine over the recent years it's been incredible to watch the progress of abilities we have been able to build into robots and the machines of today just look at you know a tesla factory or an amazon fulfillment center or my favorite the roomba cars are on the cusp of being able to see and model the world around them and intelligently drive themselves safely around in the dynamic and unpredictable [1:26] world that we live in we pretty much have jarvis from iron man aka vision for you comic book fans out there at the tip of our fingers in all of these natural language processing assistants in siri cortana alexa and whatever google calls theirs it's an astounding feat something I realized only just now when being invited to come speak for ai4 about how video games use AI and my work as an AI systems designer and a technical designer is the fact that inside of a video game it's not about extending an AI's capabilities but really limiting them what I mean by that is in a simulation we as developers create the game the AI has access to the entire state of the world at any given moment it's all cold right so the practice of AI design and development for video games is meant to entertain train educate so it's not about creating perfect AI but it's making fair AI and so while there are fair share of interactive simulations out there that don't have a direct antagonist or an [2:44] adversary or a threat the majority of games that I know and love and play with my peers and friends all build artificial intelligence for very specific reasons and that's to provide a challenge for the player and encourage that loop of mastery and learning to triumph over the challenge at hand and repeat that over and over right gradually increasing that that carrot and that pursuit and helping them become more proficient in the game's mechanics so we as developers look to proportionately increase that challenge to keep the sensation of growth that drives the player growth in their learning practicing adapting and overcoming to feel that wonderful feeling of winning so with that welcome to this talk how AI outsmarts players in the evolution of AI and video games my goal is to catch you up to speed on where we are today in AI application where we've come from and where the future may lead so if you haven't looked at the billion dollar video game industry for talent creators engineers [3:53] I want to leave you in believing that our worlds are more closely aligned than you may have so we'll start off by going through AI and games on the second part of this we'll talk how AI is used to help us make these games and then finally we'll get a bit speculative and predict on where we may go in the next 10 to 15 years my background has been as a game designer working on games since way back in 2007 I studied game design development at Full Sail I've been wrestling with controlling AI and NPC's non-playable characters to bend to the will of the designer in creating the perfect experience for the player and this although it changes over the there's still some core foundational bits that haven't changed even through I'll mention also that these are tricks that we've used in games dating far back from the xbox 360 days even up till now on pc and the current generation of technology and even now today where I work on a core tech in Frostbite the engine that powers some of [5:28] ea's most notable games like madden battlefield and things like this I also have a little podcast there out of play area where we speak to developers more on these topics when you think about AI and games I'm curious what comes to mind is it perhaps alphabet's own deep mind teaching itself go and becoming the first computer program to defeat a gold world champion lisa dole in south korea back in march of 2016 or is it perhaps ibm's watson showing the world the seeds of natural language processing winning first place prize of a million dollars on jeopardy in 2011 beating brad rutter and ken jennings for me I want to go back a little bit before all of that to some of the key algorithms that came and are applied today in games so dating back to the 1950s we have some key algorithms discovered dykstra where he conceived an algorithm to find the shortest path between nodes in a graph as he tells it it took him 20 minutes allegedly around in that well in that decade we had george mealey presented a concept [6:59] for a method for synthesizing sequential circuits what was now known as the melee in addition to edward forrest moore's work on the moore fsm or finite state those two works combined would later lead to the state machine which is heavily used even to today in how we organize and instruct NPCs and AI in the world fast forward to the 60s this is where in stanford peter hart nils nelson bertram raphael would build upon dexter's work and found a star as a path planning algorithm or a graph traversal and path search algorithm so it's a staple of how we in games plot where an NPC or AI can traverse to or not introducing weights and and and costs to you know going over rocky terrain or going down over water or something like that and it's a staple in all of my AI game classes in fulso university now with those little building blocks we can go over to the 70s so in 1972 atari's palm came out into the field developed by alan alcorn this black and white image is beautiful [8:27] to me two paddles dotted line in the center a ball score for each side all bound to one screen so you know once you hit the top of the bottom that was the limit of your imagine developing this game and needing to test or someone to play against and not always will you have someone readily available to jump on the sticks so in this nice confined space telling a paddle to move up or down which I imagine if you kind of plot uh x and y axis and zero is the center of the you pretty much only have to move in one dimension from probably one being the topmost point zero the center negative one to the bottom you can plot a point for moving the paddle with some speed and some direction and so pretty much as soon as the ball flies over the net and makes contact with a paddle we can already map out a trajectory and the computer will know when it will make contact with the right hand side of the it knows exactly where to be in the perfect spot to return that ball every time but remember what I was [9:46] discussing about earlier is that it's not about perfect AI in games but it's about fair AI and the way we go about achieving that is by trying to make it more human-like and all we have to do are model two reaction time where we'll simply delay for x tenths of a second before the paddle starts making choices on which way it's going to go and how fast it's going to go and additionally accuracy right so we'll introduce some probability of percent that it's gonna make the wrong decision and those are key elements that you'll see reused over and over and over in a game designer's AI tool bag right so moving on to the 80s we have the beloved namco classic pac-man or puck man as it was originally titled in japan developed by toru so in this game you know how it goes you play pac-man eating pellets clearing the map and while avoiding being touched by the ghosts I want to call out this state machine that I have in the center there where you see chase stagger and frightened again [11:03] building off of the state machine work that we identified built in the 50s you can see how this is to map out the three states that the ghosts can be in right so when the game starts you have the red ghost and they're going chase they're going to fluctuate between chase and stagger and there's a 2d grid and the way that we calculate your path to the player's position is through using those path finding algorithms aka a star at the start of the game ghosts are initialized to scatter and you could see that each ghost has a designated quadrant of the map over some time they'll jump to chase where each ghost knows the position of the player and then they each have a unique behavior associated to them and so this is the introduction of as game designers we are taught to train the player to distinguish NPCs with unique appearances or just you know and that they have distinct behaviors or decisions that they make given the same choice so back in those days a cheap option was a [12:19] simple color swap each ghost has a distinct behavior as a result of their unique color and together they combine for using those roles in how we do today right so the red ghost targeted to target exactly the player's the pink ghost is set to target four tiles in front of the player the blue ghost is set to target tiles in front of the player but two tiles in front of the red ghost right so helping to kind of create this pincer attack movement and then the armist ghost that we call the zoner he has an interesting pattern that is more get into a zone around the player but never directly chase him and so together all four create this very interesting dynamic where it feels like they're working together to catch pac-man and then of course they have the frightened state where when the player eats the power pellet they kind of scatter and so it tends to create a very smart looking AI I think so okay now we start moving over to the 90s and this is some might say the finest [13:41] era of gaming will essentially making the jump to 3d and adding that third dimension uh many a game developers top five games come from this era 98 was particularly impactful with hideo kojima's metal gear solid published by konami that I credited as having pioneered the concept of stealth video games available on the first sony playstation this game was mind-blowing to me in my teenage years you see the game enemy took that state machine pattern this was denoted back to the player to look the the soldiers are patrolling and this is your opportunity to and what was added here that's of note you can see that the dots the red dots presented above are your threats and then they have a vision code because again in the 3d world there's no reason that the AI doesn't know everything about the state of the world and again we want to limit their knowledge to give players a fighting instead of letting them see as far as the they could in a space you limit and restrict their sense of [14:58] their sense of hearing and even their sense of smell and so you feed that back to the player to give them a chance to sneak and move around right unique very unique behaviors that this resulted in was again the cycle of a cat and mouse gameplay where you could be detected if you're within the vision cone or you create a sound that is a short distance away they would come search they would kick in you would kick into alert to be careful there's something suspicious going on they're aware of you and if they identified you then they would go into combat right and so start shooting at you and this is when you're told to and you have to get away for x amount of seconds before they lose sight of you and then they go back to patrol right so you've reset that loop another huge game that decade was half-life half-life is credited for some very robust AI in the 90s in first person shooters there's something like 80 some odd states right so really pushing the limits of what a finite [16:16] state machine could do in those times the half-life sdk is widely available and open and I encourage any developer looking to poke behind the scenes on that to give that a download and check it out something of note that they did that I thought was very intelligent was assigning squads and so that created the illusion of mpcs working to um team up against the threat the player right so the way it worked is you would have a squad leader that they would identify something happened in the world right very similar to what you saw metal gear and anybody who was in that leader's squad would immediately become informed of what that soldier is seeing right so again creating this this uh this illusion of tactical coordination amongst the AI right it was it was very groundbreaking back then and it continues to be a common tool that we use to this day we get into the 2000s and with that one particular game that resonates with is halo 2 and they're credited for pioneering what [17:42] we call behavior trees and that pattern is still very much used today and ships integrated into a lot of dev engines and middlewares one particular behavior that kind of made me jump out of my seat the first time I saw it again they're utilizing squads right the concept introduced in half-life to to me anyway um where you have your different roles and archetypes like we saw in the pac-man ghosts you have your grunts the smaller NPCs and you have your elites the taller more powerful prominent ones and there was this effect that happened when you killed the elite or their general or their leader it would cause the grunts to flee and this was so empowering as a player you're like how why are they doing that it's something you didn't expect and it completely surprises you and it's uh an effect of having these behaviors mapped out in a tree right where they are in states that say fight the player when you go down a branch of the tree that says hey do you have a captain or [18:55] not if you have the captain fight if you don't have the captain flee right so it's an evolution of states as we know them but behavior trees became a way to the huge complexity that having so many states can lead to any developer trying to debug this later on in 2005 monolith introduced gop which is goal-oriented action planning they simplify the state machine down to three states where you move right you navigate in a space on a nav mesh that is running pathfinding algorithms you animate right you you interact with something you you sneeze you pick up a weapon and you use a smart object right so they hit all the complexity behind these three small states in what is called an action planner right this is based off of strips a 1971 stanford research institute problem solver uh so shout out to jeff orkin for a lot of his great work on this and the way they did it right is it give you a simple goal and the AI is left to form a plan and then run a star to figure out okay [20:13] to get to my goal which could be open a door find the player what is the fastest way there so it just becomes a sequence of actions in 2008 we would see aius very creatively in left 4 dead they use what was called an AI director so for pretty much my entire game design career it's common for us to manually specify where an NPC is going to spawn what spawns and and so in left 4 dead they kind of left this up to a robot and they called it their director so in a zombie survival game split between four players trying to cooperately cooperatively get out alive it would lead to a unique experience every time you played so you see depending on the anxiety in the map right like is it too calm that's when the director kicks in to build them right so they start spawning zombies and and and detect and sending them after you in floods and it leads to a peak where they spawn the special archetypes right like the pac-man ghosts uh of a brute tank or the damage dealing witch if you somehow managed to survive those [21:28] the director would let you then relax and repeat that cycle over and over moving on to 2013 forza motorsport intro or has always had the notion of drivatars and it was always learning how you drove and it would save those behaviors a big moment came in 2013 for forza motorsport 5 when these information was uploaded to the cloud and then other players could download it and play against them right so you're not playing against AI you're playing against a version of another player who has been trained and then they called it a chi number right so you could see that it's very small data set from zero meaning you're inside of the track one meaning you're outside of the track to the right five be 0.5 in the center and that information is what would your behavior at any point right whether you're braking whether you're accelerating how hard you're doing that this was all done using deep learning and then the last big thing that I want to talk to you about is the nemesis system in middle of our [22:52] shadow of mordor and war right this is revolutionary for so many reasons um imagine you're in a third person action adventure game you're fighting an orc you cut him down and then later on he comes back and tells you are you surprised to see me you got lucky you got me last time or you take him down again with an arrow to the head and in another playstation on a completely different day he comes back and says taste my blade uh and you will not get me again with arrows right he's wearing a helmet and completely adapts to you this system was so groundbreaking that warner brothers actually applied for a patent ford in 2015 and it finally went through today patenting several key aspects of that very robust I want to get to how AI works in helping developers make our lives simpler we have various middleware and engine tools available that package up a lot of these pathfinding algorithms and basic behaviors state machines behavior trees you'll see them in Unreal nav power is a [23:57] prominent one I've used kythera AI is another one that I've used machine learning has been applied at ubisoft laforge in the form of a commit right this helps coders track down possible bugs to 79 precision I imagine it's come a lot farther now procedural animation right it takes a lot of time to put someone in a mocap film them with very high fidelity equipment nowadays machines are being able to package and go through all that data and procedurally animate all the little leans and tilts and stutters and stops in a way that humans would take so much more time of course there's also automation testing where we can train the game to play itself and identify some low-hanging fruit and bugs there's a ton more here there's sentiment analysis right being able to read uh forums to see players how they're feeling about a particular feature or run it through a stream chat and check out and even filter out inappropriate wording text to speech is another very helpful one right we spend a lot of time writing [25:08] dialogue and casting voice actors to come in record and it's very expensive so in writing text that is then given some type of personality and even accent or language it helps us get much farther before we ever have to commit to getting talent in the booth content generation is another one where games can help us build the world in a very precise rule set you know where can we go in the future there's a lot here that I'd love to see eventually come to fruition I think in the next 10 years we'll start seeing a blend of machine learning being applied to make these games move and feel more lifelike you'll be able to train them to generate environments and keep building worlds to keep refreshing experiences satisfy our players insatiable appetites for more and just as it has always done it will continue to offload some of our tedious tasks while challenging us to make new roles or master new skills and expertises to fully adopt and realize the new capabilities that AI affords us [26:16] in bringing the world the most engaging fun and thought-provoking simulations and manage imaginable in this decade and the next thank you so much I enjoy talking about this all the time please find me at link tree slash jay kingpin and let's connect thank you so much wow thank you john that was amazing if you can stop sharing your slides this virtual audience wants to give you a huge virtual round of applause I know that made me want to break out my old xbox 360. all right for the audience it's time for you to make your way to your next session along the way make sure you accept your connection request and take some time to check out our amazing exhibits thanks so much and we'll see you around