What is the best way to save Q table to file?

Question

I'm planning to save my q table to a text file (as a string) for future use, but I wondered what the pitfalls of this might be? Also, any advice on what might be a better way to store the q table would be appreciated – would it better to store it as JSON, for example?

Answer 1

In machine learning, a common solution to save models and data is to use the HDF5 format. You could try http://www.h5py.org/, which is a Python library that allows you to save, read and manipulate HDF5 files in a quite easy way.

I don't know if it's the best solution for your purpose or if there's a more specific solution (because this also depends e.g. on the size of your Q tables), but this format/library allows you to save data in a hierarchical way, among other advantages.

For example, if you have multiple trained agents, you could save them as a dictionary e.g. d = {"agent1": q_table1, "agent2": q_table2 }. Also, not only can you save them in this hierarchical fashion, you can also read them and then work with their content as if they were dictionaries. Of course, this is just an example to give you an idea of what you can do with this library.

So, HDF5 is probably a good solution if you plan to have multiple agents and your Q tables can be large, but, at the same time, you want to be able to inspect and change your Q-tables easily and flexibly.

For more details about the advantages of this format compared to other common alternatives (such as databases, simple text files, etc.), see e.g. the questions Is there an analysis speed or memory usage advantage to using HDF5 for large array storage (instead of flat binary files)? and What are the advantages of HDF compared to alternative formats?.