4

I have a requirement to read the memory occupied by an object based on some events. Is there's a simple method that I can use from the standard library ? I need to access the memory usage of the following object which contains nested structs.

HashMap<String, HashMap<AppIdentifier, HashMap<String, u64>>>
Lahiru Udayanga
  • 309
  • 4
  • 13
  • I think you would have to calculate it manually. But lets see if someone surprises us here :) – Netwave Jul 05 '21 at 11:05
  • @Netwave So if I do manually how would I do it ?? – Lahiru Udayanga Jul 05 '21 at 11:11
  • Well, iterate over the contents and use the size of the `String`, of the `u64`, etc multiplied by the amount of them. then add up the size of the hashmaps structs themselves too. – Netwave Jul 05 '21 at 11:14
  • I think you need to define better what you mean by "memory occupied". There's its size on the stack (pretty small, mostly a pointer to the heap and some bookkeeping). There's the heap allocated to the outer map, the middle map and the inner map. There's the space actually occupied by each of those (there'll be allocated space that's unused). And even then, there's transitively further allocations to (potentially) include: for example the values of the `String` keys in two of the maps. Depending on what you're trying to do here (and why), you could tackle this problem in different ways. – eggyal Jul 05 '21 at 11:14
  • @eggyal I'm actually looking for the allocated heap size. I want to clear the whole hashmap when allocated heap size is greater than some predefined threshold. – Lahiru Udayanga Jul 05 '21 at 11:21
  • 1
    But the allocated heap size *of what*? There are many allocations here, including the `String` keys. Do you want to count everything that is transitively owned by your outer `HashMap` (so that, for example, moving an already allocated `String` into the map increases the calculated size by not *just* the pointer to the underlying string but *also* by the size of the string's allocation too)? – eggyal Jul 05 '21 at 11:24
  • @eggyal Yeah exactly. I want to count all the heap allocations of outer hashmap and its inner hashmaps including allocation for String keys. Like treat the outer hashmap like a black box and get the heap allocations for all the keys and values inside that. – Lahiru Udayanga Jul 05 '21 at 11:27

1 Answers1

6

I recommend you create a trait with a method that returns the size of self's transitively-owned allocations:

trait AllocatedSize {
    fn allocated_size(&self) -> usize;
}

And then implement that for everything involved:

impl AllocatedSize for u64 {
    fn allocated_size(&self) -> usize {
        0
    }
}

impl AllocatedSize for String {
    fn allocated_size(&self) -> usize {
        self.capacity()
    }
}

impl AllocatedSize for AppIdentifier {
    fn allocated_size(&self) -> usize {
        todo!()
    }
}

impl<K: AllocatedSize, V: AllocatedSize> AllocatedSize for HashMap<K, V> {
    fn allocated_size(&self) -> usize {
        // every element in the map directly owns its key and its value
        const ELEMENT_SIZE: usize = std::mem::size_of::<K>() + std::mem::size_of::<V>();

        // directly owned allocation
        // NB: self.capacity() may be an underestimate, see its docs
        // NB: also ignores control bytes, see hashbrown implementation
        let directly_owned = self.capacity() * ELEMENT_SIZE;

        // transitively owned allocations
        let transitively_owned = self
            .iter()
            .map(|(key, val)| key.allocated_size() + val.allocated_size())
            .sum();

        directly_owned + transitively_owned
    }
}
eggyal
  • 122,705
  • 18
  • 212
  • 237