Model free or model based deep reinforcement learning for car racing?

Question

I'm new in the field of reinforcement learning. So I'm quite confused with "model based" or "model free" terms.

For example, in a video game, if I want to train an agent (a car) to drive on a racetrack.

If my input is a 256x256x3 first person image of the game, should I use a model free RL algorithm ?

And if I want to do the same, but with a 3rd person view above the racetrack, knowing coordinates, speed of the car and all obstacles, etc..., should I use model based RL ?

Thank you for your time.

Answer 1

In model-based you learn a model of the dynamics of your system and use it for planning or for generating "fake" samples. If you can learn the dynamics well, it can be extremely helpful, but if your model is wrong then it can me disastrous.

That said, there is no general rule for when to use model-free or model-based. Usually it depends on how much prior knowledge you have that can help you learning a good dynamics model.