What sort of game problems can neural networks trained/evolved with evolutionary algorithms solve, and how are they typically implemented?

Question

I'm interested mostly in the application of AI in gaming; in case this adjusts the way you answer, but general answers are more than welcome as well.

I was reading up on Neural Networks and combining them with Genetic Algorithms; my high-level understanding is that the Neural Networks are used to produce a result from the inputs, and the Genetic Algorithm is employed to constantly adjust the weights in the Neural Network until a good answer is found.

The concept of a Genetic Algorithm randomly mutating the weights on the inputs to a Neural Network makes sense to me, but I don't understand where this would be applied with respect to gaming.

For example, if I had some simple enemy AI that I want to have adapted to the player's play-style, is this a good opportunity to implement the AI as a Genetic-Algorithm combined with a Neural Network?

With these different suitable applications, how does one go about deciding how to encode the problem in such a way that it can be mutated by the Genetic Algorithm and serve as suitable on/off inputs to a Neural Network (actually, are Neural Networks always designed as on-off signals?)?

Answer 1

Without going in too much detail on how exactly Neural Networks and Generic Algorithms work, I can tell you that both the algorithms are not good candidates for computer games. They work well in scientific environments where the system is "trained" on a huge data set to adjust the "weights" (variables) for a given problem. This "training" process requires a lot of processing power, time and a large data set.

Computer games, however either needs to run in real-time (no time for training) or turn-based (not enough data for training).

Another problem is that computer games need to free up as much as possible system resources for physics, graphics, sounds and the user interface to improve the player's experience so game developers usually use other lighter techniques (like a rule-based system) to create the illusion of an AI player.

Answer 2

For your question there's a brilliant playground emerging!

Go to https://gym.openai.com/ and explore!
You'll get interfaces to games if you want to try applying your machine learning skills and compare the performances of your trained AIs with others. And you can let yourself be inspired by the ideas discussed in the community.
If you're especially into Genetic Algorithms you'll find dicussions there too but I'd suggest digging deeper into Reinforcement Learning.

If you look at what Google Deep Mind accomplished playing

• Breakout
• Montezumas Revenge
• various other Atari Games ..

and obviously !

• the sensational victory at Go

you can say that Reinforcement Learning with (Deep) Neural Networks can be a very promising approach when it comes to training an AI to master games!

Answer 3

"if I had some simple enemy AI that I want to have adapt to the players play-style, is this a good opportunity to implement the AI as a Genetic-Algorithm combined with a Neural Network"

Sure. Just provide a quality measure for the GA that's related in some manner to the effect of the player's actions on the game state/opponent(s).

For example, if defining an opponent's intelligence, one of the conceptually simplest things would be to give a GA population member a fitness that's inversely proportional to the increase in the player's score over some period of time.

are Neural Networks always designed as on off signals?)?

No. In general, they can be considered to perform nonlinear regression, i.e. a mapping from a vector of real numbers of length n to another of length m. Classification (i.e. 0/1 outputs can be seen as a restricted case of this).

As per my answer to this AI SE question, there is a large body of literature (and mature software libraries) for using evolutionary computation to encode neural nets.

More generally, some early work in 'online adaptivity using GA-encoded NNs' appeared in the Creatures http://mrl.snu.ac.kr/courses/CourseSyntheticCharacter/grand96creatures.pdf series of games by Steve Grand (details).