I think I now know the answer to this. Please correct me if I'm wrong. Bindings are not removed. Instead referential links are made (e.g. INotifyDataError events are wired up) as a result of a particular instantiation of a Binding. In the simplest case that's when the DataContext changes.
Let's say you set the DataContext of a Page to a new, different INotifyDataError object as you load it. If there are Bindings on your Page (to the DataContext) the Page is not a candidate for garbage collection until the DataContext object is destroyed. That's because the DataContext holds a reference to the Page through its ErrorsChanged event. If you want the DataContext object to be collected you'll have to set the DataContext to null in the Page's Unloaded event.
As far as I understand it, it seems the proper pattern for implementing a DataContext (that is different from the Page) is setting the DataContext in the Loaded event on the page, and then setting the DataContext to null as the page fires its Unloaded event. This concept applies any Framework element.
As an aside, I haven't delved into element Bindings. For instance, when one element on a Page is bound to a property on another element. I'm not sure when the referential links between these objects are removed. I'm presuming it's when the page is unloaded. Anyone know the answer to this?