MarI/O - Machine Learning for Video Games

Jason Weston (Facebook) etc's work on Memory Nets[1] Anything to do with transfer learning (the 2013 Zero-Shot Learning Through Cross-Modal Transfer [2] paper is a good place to start) The increasing amount of demos around using NNs to generate "things" that look kinda-almost "intelligent". I can't point at a paper, but Andrej Karpathy demo of generating Shakespere-like writing, "Wikipedia" pages and "C" code in The Unreasonable Effectiveness of Recurrent Neural Networks [3] is the kind of thing I'm talking about. The beginnings of work around goal-seeking. The (now Google) DeepMind Atari demo[4] and Marl/O[5] Finally, the work being done on making this stuff usable by programmers (Torch etc). [1] link [2] link [3] link [4] link [5] this video

If someone did this in the US, can they get in trouble? When exactly does "hacking" a game become illegal? What about people doing it for research like the machine learning mario guy: this video . Or tool-assisted speedruns? Or the starcraft broodwar ai? Is it only when you breach the TOS or copyrights? Just curious since it never even occurred to me that such things are punishable.

It's true that game AI is surprisingly under-developped compared to what you see in academia on other topics... but it is not always that basic. What you describe in your football example is often called "Behavior Tree", and there are ways to learn those using machine learning (usually by reinforcement learning, using neural networks and/or genetic programming). There was a popular video showing this applied on Mario [1] ... and a paper published 7 years before that video doing the same thing [2] (more context about this in [3]). I remember seeing something on Gamasutra saying that similar methods are used in some AAA games. [1] this video [2] link [3] link

This is from the same guy who made the MarI/O video ( this video - using Neuroevolution). In the video he also compares MariFlow vs MarI/O.

connectionism is an approach to the philosophy of mind, the most successful approach as evidenced by recent deep learning hype the only book in the OP I thought was even mildly relevant was the one by Dennet, and it's generally acknowledged that it's pretty bad. A more interesting but flawed approach, that resonates as someone who loves Dawkins and Hitchens (atheists and advocates of evolution), is Gerald Edelman's Neural Darwinism (1978). There is even modern research (that is real, working code) at the University of Texas that could be considered a successor to Edelman: NeuroEvolution of Augmenting Topologies (NEAT) this video

Also the video discusses previous work starting at 4:40, so I'm confused what OP is complaining about unless the MarI/O guy re-uploaded with this attribution. this video

Thanks for the commentary togelius [1, 2] and deepnet [above]. Anyone interested in adding a soupçon of scholarship to Seth's project? Words do mean different things to different people, in different contexts.. There is a recent body of literature that explores the modern "maker" [3] movement. However, "maker" as a term may not have been a good fit for the OP's argument [1], which contrasted (academic) researchers with so-called "tinkerers". An alternative term for "tinkerer" might be "bricoleur", a loanword from French. (Roughly, it still means one who tinkers: link but has other meanings depending on the academic lens.) Given that we are discussing AIs that play, in the context of education, we can also go back to Seymour Papert's work on link . Originally known for work on _Perceptrons_ with Marvin Minsky, AI researcher Papert later adapted theories from education towards the vision of "learning-by-making" and the (young) bricoleur [4]. This can approach can be seen in the evolution from 1960s graphical [turtle] link to Lego link to modern day efforts to encourage coding-for-kids [5,6,7]. One of Papert's later collaborators, Sherry Turkel, discusses bricolage as it applies to programming -- link . When it comes to early education, Turkel argues for epistemological pluralism [8] and cites anthropologist Levi-Strauss in comparing analytic science with a "science of the concrete". We can appreciate both Seth Bling's concreteness [9] and Togelius's original papers fo

If you watch videos of AIs learning to run [1] or playing Mario [2], you see that there's definitely "try random shit until your reward function gives you positive feedback" as a method for training AIs. The difficult part is developing the reward function. There's a lot of intelligence in our hormone-based incentive system - that we feel pain, that we feel sad, etc. Many of those emotions are pre-programmed. If we find a way to design reinforcement systems with the right goals, we can do it like mother nature did it for several billion years. But it's still a ton of computational power required. If we look at nature as a massively parallelized computational system that defined "just live" as the reward (not because it "wants" it, but because it just emerged), it shows us how much power we would need if we would try to build a completely random process that gains intelligence. I think your proposed method works ("just assemble random solutions and test them") because we've already seen it in nature. But it takes a lot of time and energy, regardless of our computational power. I think we're faster when we find the right boundaries and reward functions instead of randomly trying stuff. [1]: link [2]: this video

This is very similar to Picbreeder [1], which allows you to evolve art in a very similar manner, with evolutionary algorithms. It runs on the same evolutionary algorithm behind MarI/O [2], called NEAT [3]. [1] link [2] this video [3] link

If your goal is to get the network as small as possible, you could try using NEAT[1]. A cool example of a small network for a hard task using NEAT is MarI/O[2]. [1]: link [2]: this video