"Every action that you take in the game doesn't have a predetermined outcome, because the game or the opponent can randomly react to what you do. So, you need a technique that can handle very complex scenarios that react in potentially random ways."
Barzilay and Branavan's system begins with virtually no knowledge of the task it's intended to perform, or the language in which the instructions are written.
It has a list of actions it can take, like right-clicks or left-clicks, or moving the cursor.
It also has access to information displayed on the screen and ways of gauging its success. But it doesn't know what actions correspond to what words in the instruction set.
But as it takes various actions, different words appear on the screen and it can look for instances of those words in the instruction set.
It can also search the surrounding text for associated words and develop hypotheses about what actions those words correspond to.
this
Users
Comment
article