As you've found the hierarchical object store (HOS) is stored in the configuration database, not the content database.
If you're running in the context of a site other than the central admin then the user account the application pool uses may not have permissions to Write to the configuration database.
This is a bit of a doozy as you probably won't see this on a developer installation (as you probably won't have it setup with domain level accounts for app pools, only NETWORK SERVICE) and you will only trip up when you get to production.
Note - Using SPSecurity.RunWithElevatedPrivileges will not change this as that just eleveates from the restricted user account to the application pool account and not the central admin app pool account.
The article Managing Application State gives a breakdown of the options and it says about the HOS
Security model - Users need
administrator privileges to access the
related SharePoint object (such as
SPFarm or SPWebApplication).
For these reasons I think the HOS is very limited in real world usage and you're better off using the Property Bag Storage model which can store settings at the Farm/WebApp/Site and List Levels :-
Advantages - It is probably the most
light-weight and easy to use of all
the configuration storage options.
There are also reported performance and stability benefits.
In fact the MSDN SharePoint Guidance Library also uses the Property Bag storage for their implementation of a hierarchical Configuration Manager.