Storing 200kb of data in an array?

Question

I'm pretty new to programming for embedded applications (besides some Arduino stuff) and I'm working with the CC3220SF microcontroller from Texas Instruments.

Currently I have a program that is constantly polling a device and storing the result. I would like to store 100,000 of these samples (each is 2 bytes) giving me 200kb of data to store. I'm not really sure how I should go about doing this, as trying to just make an array of size [100][1000] just crashes the device.

How should I go about storing this data for later use?

#define MAX_ARR_LENGTH                   1000
#define MAX_ARR_DEPTH                    100

// Later in the collection function:
uint16_t measurmentsArr[MAX_ARR_DEPTH][MAX_ARR_LENGTH] = {0};
unsigned int arr_length = 0;
unsigned int arr_depth = 0;

// And later, after a data point has
// been verified as useful:
if (arr_length < MAX_ARR_LENGTH){
    measurmentsArr[arr_depth][arr_length++] = angle;
} else {
    arr_length = 0;
    measurmentsArr[arr_depth++][arr_length] = angle;
}

This ^^^ way works for small arrays, but like I said I need to store 200kb... I know the CC3220SF has 512kb for use, how do I best write/read to that?

Respectfully, -James

You'll probably overflow the stack depending on how large it is. Might be safer to heap allocate, even though the size is known at compile time. — Cory Kramer, Jun 30 '17 at 14:51
If your platform supports dynamic memory allocation I would suggest using `std::vector> measurmentsArr{MAX_ARR_DEPTH};`. — NathanOliver, Jun 30 '17 at 14:53
The [datasheet](http://www.ti.com/product/CC3220/datasheet) only claims 256kB of RAM. Furthermore floats are 32 bits (4-bytes each) on this platform, yielding a 400kB array size. — doynax, Jun 30 '17 at 14:56
@NathanOliver: I would suggest sticking with static allocation as suggested by CoryKramer on an embedded platform with a tight memory budget. allowing the linker to guarantee the allocation and avoiding fragmentation concerns. Reuse it manually via unions and the like if required for other tasks, but great care should be taken with buffers requiring such a large fraction of the available memory. — doynax, Jun 30 '17 at 14:59
@doynax A `std::vector` can't fragment. `vector` is guaranteed to have contiguous storage and so is `std::array` so you are guaranteed a 2d "array" that is dynamically allocated and is one single blob. — NathanOliver, Jun 30 '17 at 15:02
@NathanOliver: Well, yes, but the entire purpose of using a dynamic allocation as opposed to a plain array is that it may be freed temporarily for other uses. The problem then is that the intervening allocations risk fragmenting the range and rendering the next use impossible. This is true in general for embedded devices on which there is no swap space available as a reserve but most especially if allocating a large fraction of RAM. Therefore I would advocate manual reuse instead in this case, or a statically allocated array dynamic use is unnecessary. Plus you get a guarantee of success. — doynax, Jun 30 '17 at 15:06
@doynax You are correct, I keep saying 512kb when I know it's 256kb... and sorry, that float should be uint16_t, I was playing around before I copy/pasted code here =/ I'll edit that now — old_dd, Jun 30 '17 at 15:19

score 3 · Accepted Answer · answered Jun 30 '17 at 15:27

Chances are that sticking static in front of that huge array makes it work.

Most compilers for embedded systems will place function-local variables on the CPU stack, but static variables have "static storage duration" and essentially behave like globals. The linker knows exactly how much memory those need, and will try to fit them in.

Of course, static will make the function non-reentrant, but you couldn't allocate two 200 kB arrays on a 256 kB device anyway.

Storing 200kb of data in an array?

1 Answers1