So, today I’m going to muse on the Alpha Star AI playing Starcraft II that I kinda talked about here. These are just musings on the topic and so might be a bit inaccurate, and I’m not doing any extra research and so am just relying on what Shamus and the commenters said about it (links to the posts are in the post I linked above).
As best I can understand it, the system was essentially a neural net that was trained using what were in my day called Genetic Algorithms and potentially have evolved a much more “cool” name, where there were a number of agents that played the game against each other and the best ones were kept to play against each other again and so on and so forth. What I did way back in university — as part of my Honours Project I did a GA simulating a “Diplomacy Problem”, with various countries getting advantages or disadvantages based on whether or not they agreed with the others — was create a set number of agents — 50? — and rank them by score, and then drop the bottom 10, double the top 10 for the next run, and leave the rest. I hope they did something similar, but at any rate the overall idea is the same: run the agents, see which ones get the best score, keep those, keep or introduce some new agents so that they can learn new behaviour, rinse, repeat.
What this meant is that they needed to have agents that could play each other in order to generate the massive data sets that you need to train a neural net, which led them to limit the agents to being able to play as Protoss against players that are playing as Protoss. Right away, this is a little unimpressive, as humans can learn to play as pretty much any combination of the races a lot faster than the agents learned to play as Protoss against Protoss. This also led me to comment on the posts that there’s a risk with trying to get it to learn to play as other races or against other races because of the nature of neural nets. The big advantage of neural nets is that you don’t need to program any rules or semantics into it to get it to solve the problems. There aren’t really any rules or semantics in a neural net. Sure, there may be some in there somewhere, and it often acts like it has rules or semantics, but internally to the system there aren’t any. The system learns by semi-randomly adding and removing nodes and connections and adjusting the weights of connections, but the system doing that, at least in a pure neural net (supposedly Deep Learning systems combine the semantics of inference engines and the flexibility of neural nets, but I haven’t looked at them yet) doesn’t have any idea what actual rules or decisions those things are involved in. Thus, a common problem with early neural nets was that when you decided to train it to do something different or learn anything there was always a risk that you’d break existing behaviour unless you also trained it on the old functionality at the same time, which is not how things seem to work in humans. You can limit that by restricting how much it can change the original net, but then it has a harder time learning anything new. Make it static and the machine can’t learn, but make it random and it will forget lots of things it used to know.
What this means for our agents is that teaching them to play as another race or against another race might cause them to forget important details about how to play as a Protoss against a Protoss. I opined that what they’d probably do instead is build separate agents for each case and then have a front-end — which could be an inference engine since this is all deterministic — pick which agent to use. After all, while there are nine different combinations — the AI playing each race potentially against all other races — that’s set at the beginning of the game and so it’s a pretty straightforward decision of which agent to use, and there’s no real reason to try to teach the AI to try to find the ideal match-up given who they’re playing against. So this seems to me to be the easier way to go than trying to build a generic agent that can play all combinations, and it’s actually even less artificial than some of the other things that the agents were already committed to.
So, after the AI beat all the players the first round, how did the one player get a rematch and beat it rather handily? What he did was adopt a strategy that the AI was vulnerable to, which was harassment. The player waited until the AI had built a big army and sent it off towards his base, and then sent a few units in to attack the base. The AI turned its army around to meet the threat, and he moved the units away. After they were chased off and/or destroyed, the AI started out again … and the player repeated the harassing attack. Rinse, repeat, and eventually win the game.
One issue with neural net type AIs is that since they learn through repetition over massive data sets, they don’t really have the ability to learn or adapt on the fly. They don’t really learn much from one event or run. Inference engines actually can learn on the fly because their actions are driven by the premises and logic of their systems, and so if one event doesn’t turn out right they can immediately try to reassess their inferences. In this case, for example, the AI was probably bringing the army back because it anticipated that it was a mass invasion that it needed to repel. A neural net won’t store this explicitly, but an inference engine will. So there’s a chance that after a few repetitions it concludes that this doesn’t indicate a mass invasion and will learn to ignore it. Which, then, would leave it vulnerable to a “Cry Wolf” strategy: harass it until it learns to ignore the harassment, and then launch a full-scale attack to catch it napping. Which it could then learn to defend against as well, and so on and so forth.
People in the comments asked if you could just teach it to ignore the harassment, but the problem with neural nets is that you can’t really teach them anything, at least by explaining it or adding it as an explicit rule. Inference engines can be tweaked that way because they encode explicit rules, but neural nets don’t. To add a rule to the system you have to train it on data sets aimed at establishing that rule until they learn them. There are approaches that allow for feedback and training of that sort from what I’ve seen (mostly through short presentations at work) but either those will establish explicit rules which the system has to follow — even if wrong — or else they can be overridden by the training and so would need to be trained and retrained. In short, you can explain things to an inference engine, but not really to a neural net. You can only either let the net learn it itself or flat-out tell it the answer.
Neural nets, I think, excite people for two reasons. First, because they don’t generally have explicit rules they can come up with unique correct answers that we, ourselves, can’t figure out, or that at least are extremely difficult for us to figure out. This makes them look more intelligent than we are for coming up with answers that we couldn’t see. Inference engines and expert systems can come up with novel solutions as well, but all of those systems can explain how they came to that conclusion and so seem less “mysterious”, in much the same way as when we see Sherlock Holmes explain his reasoning it seems less mysterious and, often, more of a “Why didn’t we see that?”. We aren’t that impressed by computers having access to all the data and never forgetting or forgetting to consider any of it since that’s kinda what they do, but we are impressed by what seem like leaps of intuition that we can’t match. The other reason is that they loosely resemble the structure of the human brain — although anyone doing AI will tell you that they aren’t really that close, but as they are designed to model that in at least some ways the point still stands — and so people impressed by neuroscience will think that it’s closer to what we really do. Personally, I’m more interested in the reasoning aspects of intelligence and am more interested in finding the algorithm we use rather than emulating the hardware, so I’m less impressed by them. Still, they do manage to do the pattern-matching aspects of intelligence well and far better than more reasoning based systems, which has led me to opine that the ideal AI has an inference engine front-end and a neural net back-end. The inference engine answers what it can and passes off anything else to the neural net, assess the answer, adopts it if it seems to work and retrains the net if it doesn’t. Again, some people commented that this seems like what Deep Learning does.
All of this starts to tie back into the heart of questions about AI leading all the way back to Searle: does the Alpha Star agent actually understand how to play Starcraft II? There’s no semantics to a neural net. You could take those agents and hook them up to something that is, say, trying to run a factory and if the weights were correct the system could do that just as well (and people have indeed taken neural nets trained for one specific task and had them perform entirely different tasks and noted that they can more or less work). So what does the agent actually understand about Starcraft II itself? Does it know what the units are and what they mean? It doesn’t have to, as it doesn’t really encode that information in the neural net itself? If you don’t have the semantics, do you really understand anything at all? With Searle’s Chinese Room, most will agree, at least, that the person inside the room is not doing anything intelligent by simply taking in a symbol, looking up the answer, and passing it back out. That person doesn’t understand Chinese. What people say the error is with the thought experiment is assuming that the room itself can’t understand, or couldn’t if it had the right context and information. But all of that is semantic information about meaning. Does a neural net in and of itself ever have meanings? Does the Alpha Star agent store any semantic information at all, even to the extent that an inference engine does? Having the right output doesn’t guarantee meaning, especially if it can be used for things that mean things that are completely different. So does it have meaning? And if it doesn’t, does it really understand?
These may not be questions that the creators are worried about. They may simply want to build an AI to beat humans at a video game. But these questions will obviously be raised by these things, and the answers — and attempts to answer them — are of great interest to fields like Philosophy of Mind, Cognitive Science and Psychology.
Tags: AI
The Verbose Stoic 
Leave a Reply