slots does not reduce memory

Question

I have the following class:

class DataGenerator(keras.utils.Sequence):
    
    __slots__ = 'path','batch_size','dim','mode','split_proportion','indices','processes'
    
    def __init__(self, path: str,
                       batch_size: int,
                       dim: Tuple[int] = (12,86,98),
                       mode: str = 'train',
                       split_proportion: float = None,
                       indices: List[List[int]] = None,
                       processes: Optional[int] = None):

        self._path = path
        self._mode = mode
        self._split_proportion = split_proportion
        self._k_indices = indices
        self._dim = dim
        self._batch_size = batch_size
        self._processes = processes

        # Change mode to retrieve from folders
        if mode == 'validation':
            mode = 'train'
        self._im_path = os.path.join(self._path, mode,'image')
        self._msk_path = os.path.join(self._path, mode,'mask')

If I instantiate it, it should not have the attribute dict since it contains slots. However:

training_data = DataGenerator(path = '/path', batch_size = 1, 
                              mode = 'train', split_proportion = 0.1)
training_data.__dict__

{'_path': '/path',
 '_mode': 'train',
 '_split_proportion': 0.1,
 '_k_indices': None,
 '_dim': (12, 86, 98),
 '_batch_size': 1,
 '_processes': None,
 '_im_path': '/path/train/image',
 '_msk_path': '/path/train/mask'}

Additionally, if I check memory requirements, they seem to be higher than without the slots.

# with __slots__
sys.getsizeof(training_data)
112
sys.getsizeof(training_data.__dict__)
152

# without __slots__
sys.getsizeof(training_data)
56
sys.getsizeof(training_data.__dict__)
152

Does `keras.utils.Sequence` define `__slots__`? If not, your class will still have a `__dict__` no matter what you do (`__slots__` must be defined on every layer in the class hierarchy to avoid `__dict__`). The fact that the `__dict__` has your attributes in it is odd, but could be the result of either a weird implementation of `__setattr__`, or a bad override of `__dict__` as a `@property`, both of which would be seen on the parent. — ShadowRanger, Jun 22 '20 at 18:01
I think the larger result from `sys.getsizeof()` is expected. The point of `__slots__` is that your attributes are being stored compactly in the instance itself (where they're visible to `getsizeof()`), rather than in a separate dict object that has a lot of overhead (but doesn't count towards the size of the instance itself). — jasonharper, Jun 22 '20 at 18:14

__slots__ does not reduce memory

0 Answers0

slots does not reduce memory