0

I saw so many examples showing people making it using Deep Reinforced Learning but that is something I am yet to know, I have learned supervised training methods and then I started learning Q learning thinking that maybe it will turn out more efficient, I made a simple snake game that works nicely, all rules are applied, added rewards states and actions, whatever that's needed, and made a Q loop with an updating Q tables, everything is normal, and working perfectly (btw no libraries used other then Tkinter, math and random) except the one thing where the machine does not really manage to learn much no matter how much time passes, the table updates, I am sure of that, but what I am thinking from past experience is that maybe Q learning doesn't fit this kind of game? I also have never seen anyone making the game "Snake" using Q learning so that came up in my mind...

Appreciating every comment! :)

(Btw if you think there's an error with my code you can ask for it/a part of it and I'll sand, it is really no big deal :P )

1Mangomaster1
  • 346
  • 1
  • 6
  • 16
  • 1
    Do you have a simple table with an entry for each possible state and action? I don't see how that could realistically be implemented, since the number of states in snake is exponential in the board size, so even a table for a small board wouldn't fit in memory. – interjay Jan 13 '20 at 16:03
  • Yeah... I do have only a simple table, it contains every state for every action, after thinking about it, it makes sense the machine didn't learn, fitting into it all of the actions(4) for all of the states(BoardLength^2, I started with a small 10x10 board so 400 options..) is basically stupid, and so inefficient. And with that being said, I'd like to ask, what is Q-Learning good for exactly? I have made several things with it, but I am not quite sure – 1Mangomaster1 Jan 13 '20 at 16:07
  • The number of states is not boardlength^2. It's much more than that, because the snake can be long so you need to keep track of whether every possible cell is part of the snake, leading to 2^(boardlength^2) states. If you really had only 100 states then q-learning with a table could probably work. – interjay Jan 13 '20 at 16:11
  • I was pretty sure that if I let the snake see to the next move and that every time it does it checks if there is a body part or a border where it moves it will be enough... – 1Mangomaster1 Jan 13 '20 at 16:13
  • Then the only information you're feeding the q-learning algorithm is the snake head location? It wouldn't be able to learn much if you don't give it all the relevant information. – interjay Jan 13 '20 at 16:17
  • I feed it with, Head Position, body positions, "direction" as the next action, food location – 1Mangomaster1 Jan 13 '20 at 18:27
  • If that was true then you'd have a lot more than 100 states. You'd need more states than there are atoms in the known universe due to all the possibilities for the body positions. – interjay Jan 13 '20 at 18:42
  • Thanks so much for your answers, I did not consider the body position on the screen live which makes it too much of a problem, I guess I'll just have to learn Deep Q-Learning... Thanks so much for everyone :) – 1Mangomaster1 Jan 13 '20 at 18:44

0 Answers0