What is the purpose of optimizer's state_dict in PyToch Big Graph's embedding dataset?

Question

The documentation for PyTorch Big Graph (PBG) states that "An additional dataset may exist, optimizer/state_dict, which contains the binary blob (obtained through torch.save()) of the state dict of the model’s optimizer." When inspecting this dataset, it seems to be stored as an array of bytes. Could someone conceptually explain the point of state_dict and why it's stored as an array rather than a dictionary?

Answer 1

Could someone conceptually explain the point of state_dict

If you know about Adam or SGD's momentum you probably know that there're some parameters in the optimizer that change in every step. When resume training on top of loading the model weights it'll make convergence faster if you load these parameters too.

You can get away without it, just that sometime it'll almost as if you start training from scratch.

why it's stored as an array rather than a dictionary?

If it's really obtained through torch.save() then it is in fact stored in dictionary or at least a list of dictionaries. Just that your process of "inspecting" it is wrong. Try

print(torch.load('path_to_the_file'))