In Towards Integrated Imitation of Strategic Planning and Motion Modeling in Interactive Computer Games, Dublin City University researchers Bernard Gorman and Mark Humphrys detail their experiments in using imitation learning techniques to teach an artificial agent to play a first person shooter—in this case, Quake—in a way that would convince onlookers and other players that it is human. They describe a superior agent, which imitates "the observed goal-oriented behaviors of a human player," (p. 2) in order to play with competence, exhibit strategic thinking, and employ "believably human-like motion" (p. 1). In other words, they want to create a bot that can pass a kind of Turing deathmatch.
Gorman and Humphrys describe the behavior model which serves as the basis for their work. The model specifies several levels of control, where each level corresponds with roughly how much time the agent has to form a plan. These range from immediate ("scrambled control") to long-term ("strategic control"). Furthermore, their model relates these levels of control to three kinds of behavior—strategic, tactical, and reactive—which together are supposed to form the building blocks for an agent's motion modeling capabilities (p. 3).
In their experiments, Gorman and Humphrys emphasize the "human-like motion" mentioned above. Specifically, they limit their agent's problems to those of environmental navigation and item acquisition. Their methods involve topology learning (p. 3), Markov decision processes or MPDs (p. 4), and motion modeling based on "action primitives." The action primitives, the recorded gameplay, and relevant state space information comprise the training data (p. 6-7). They applied their agent in two types of tests: "pickup sequence" tests, in which the agent must pick up a series of items, and "continuous gameplay" tests, which were of a more free-roam nature (p. 8-9). The researchers noted that the agent performed "showed impressive performance and adaptability" during the pickup sequence tests (p. 8). Otherwise, they noted one issue in the continuous gameplay tests: the agent would sometimes struggle to choose among competing utilities (p. 10-11).
Gorman and Humphrys note in their conclusion that their work relates closely to that of Christian Thureau of Bielefeld University. They mention several works from this author throughout the paper. Please consider joining me next time as I explore this researcher's work.
Gorman and Humphrys describe the behavior model which serves as the basis for their work. The model specifies several levels of control, where each level corresponds with roughly how much time the agent has to form a plan. These range from immediate ("scrambled control") to long-term ("strategic control"). Furthermore, their model relates these levels of control to three kinds of behavior—strategic, tactical, and reactive—which together are supposed to form the building blocks for an agent's motion modeling capabilities (p. 3).
In their experiments, Gorman and Humphrys emphasize the "human-like motion" mentioned above. Specifically, they limit their agent's problems to those of environmental navigation and item acquisition. Their methods involve topology learning (p. 3), Markov decision processes or MPDs (p. 4), and motion modeling based on "action primitives." The action primitives, the recorded gameplay, and relevant state space information comprise the training data (p. 6-7). They applied their agent in two types of tests: "pickup sequence" tests, in which the agent must pick up a series of items, and "continuous gameplay" tests, which were of a more free-roam nature (p. 8-9). The researchers noted that the agent performed "showed impressive performance and adaptability" during the pickup sequence tests (p. 8). Otherwise, they noted one issue in the continuous gameplay tests: the agent would sometimes struggle to choose among competing utilities (p. 10-11).
Gorman and Humphrys note in their conclusion that their work relates closely to that of Christian Thureau of Bielefeld University. They mention several works from this author throughout the paper. Please consider joining me next time as I explore this researcher's work.
Comments
Post a Comment