1

This relates to another question I asked a while back at Size of Qt containers: is QMap much larger than Qlist?.

Basically I would really like to re-implement a VERY LIGHT container which allows to add items (does not matter if at the end or at the beginning - and no, they don't have to be continous in memory... very much like lists). No other functions are required, just appending, and a "count()" or "size()" function to return the number of elements in the container, so I can loop through each.

This is because I create a large number of these containers, and they take up a lot of memory space (see my other question).

Has anybody come up with such challenge? I looked around the web, bust most re-implemented a STL container, which is still too large for me. The idea I am following now is to re-write the simplest (forward_list) STL container somehow, but I am not that Kung Fu, but I'll try.

Thank you for your time.
Francesco

Community
  • 1
  • 1
Francesco
  • 47
  • 1
  • 7
  • 1
    Erm, `std::vector`? Isn't it good enough? – jrok May 16 '13 at 17:49
  • 1
    The amount of space that a data structure takes up is mostly going to depend on the size of the "containers" themselves. Adding additional functions to it will not appreciably change the amount of space it takes up, unless you change the containers. – Robert Harvey May 16 '13 at 17:50
  • 1
    I don't know about relative size but look into `QSet`. However in general I have left Qt's containers for STL ones and not looked back, especially since C++11. – Matt Phillips May 16 '13 at 17:58
  • 1
    @jrok is right: if you want to minimize space (especially if each item you're storing is fairly small, like an `int`) `vector` will usually be preferable to a linked list. A singly linked list has a pointer in each node, so if you're storing `int`s, it's likely to use as much space to store the pointers as to store the data you care about (and a doubly-linked list may use twice as much space for pointers as the real data). – Jerry Coffin May 16 '13 at 18:00
  • @JerryCoffin I am storing a small class holding six shorts, and in the future I might add more stuff, so it is bigger than a single int. I posted an answer with some discussion – Francesco May 17 '13 at 05:49

3 Answers3

3

I think std::vector is about as lightweight as containers get. It only has constant overhead, doesn't add any data per-element. You can see here that gcc's implementation of std::vector takes up 12 bytes (on a 32-bit system). You won't get much less than this.

Angew is no longer proud of SO
  • 167,307
  • 17
  • 350
  • 455
  • 1
    I'd certainly start by trying `std::vector`, but it's not without it's problems: As items get added the reallocation (element copy construction and destruction) can be very painful on pre C++11 systems, and can cause bad memory fragmentation as well. – Roddy May 16 '13 at 18:21
  • That is, unfortunately a significant point, as a lot of stuff gets added. @Angew I really liked the "12 bytes" part, I was looking for references on that. Sounds pretty small. I will look up how much std::list takes up, which sounds quite good. – Francesco May 16 '13 at 18:37
  • 1
    @Francesco Do note that `std::list` adds an overhead of 2 pointers *per element.* `std::forward_list` adds 1 pointer per element. If you're concerned with memory footprints, these can add up quickly (especially if your elements are small). – Angew is no longer proud of SO May 16 '13 at 18:38
  • @Francesco Also note that if you know in advance you'll be doing lots of insertions, and you can roughly estimate how many, you can call `reserve()` on the vector to remove the need for re-alloations. – Angew is no longer proud of SO May 16 '13 at 18:39
  • @Angew - I could estimate roughly the amount of points in each container – Francesco May 16 '13 at 21:17
  • @Francesco Then you can call `reserve(yourUpperSizeEstimate)` after you create the vector. That will pre-allocate memory (without creating elements in it), so that insertions don't force re-allocation and element moving. Insertions will then be guaranteed constant time until you exceed the `reserve`d capacity. – Angew is no longer proud of SO May 17 '13 at 08:01
0

Thanks to all, good inputs here. I finally trimmed it down to the following:

1 - Below is the class holding my info: any suggestions for padding here? Looks ok as float=32bits, + 16 + 16 + 8 + 8 + 8. Am I correct to assume that I can add another 8bit element here without causing additional memory to be used since it pads to 32bit chunks anyway?

class WFM {
  public:
    float h;
    quint16 fwhm2;
    quint16 sens2;
    quint8 fwhm;
    quint8 sens;
    quint8 nAmplitude;
    WFM(){ h=0; fwhm2=0; sens2=0; fwhm=0; sens=0; nAmplitude=0; }
};

2 - A QList holding a pointer to my metrics class (1) for each cell, thus with a size of [rows*cols]: @MattPhillips I will look into using STD container and @Angew probably forward_lists instead of QList

typedef  QList< WFM >* metricsP;
metricsP *mappedMetrics;
mappedMetrics = new metricsP[rows*cols];

This should trim it down. I will add some benchmarks later on.
Thank you everyone for your time, appreciate it!
Any additional comments are welcome.

Francesco

Francesco
  • 47
  • 1
  • 7
0

From benchmarking I somehow found out that QList< WFM* > is better than QList< WFM >.

Why is 

QList< WFM > *qlist = new QList< WFM >();
WFM wfmobj;
wfmobj.fwhm2=5;
wfmobj.sens2=6;
wfmobj.fwhm=7;
wfmobj.sens=5;
wfmobj.nAmplitude=3;
qlist->append(wfmobj);

much more memory-eating than 

QList< WFM* > *qlist = new QList< WFM* >();
WFM *wfmobj = new WFM();
wfmobj->fwhm2=5;
wfmobj->sens2=6;
wfmobj->fwhm=7;
wfmobj->sens=5;
wfmobj->nAmplitude=3;
qlist->append(wfmobj);

???? Maybe I am missing something.

Francesco
  • 47
  • 1
  • 7