Extract value from .csv file

Question

I am working on STM32F429, I want to extract value from a csv file. My csv file contain more than 200.000 values, this are all in a single line. And for example I want to read the value beetween 14896th and 14897th comas. How can I do that?

USBH_Initialize (0);

while (1) {
    result = USBH_MSC_DriveMount ("U0:");
    if (result == USBH_MSC_OK) {
        e = fopen ("montest.csv", "r");
        if (e) {
            fread (fbuf, sizeof (fbuf), 1 ,e);
            fclose (e);
        }
    }
    osDelay (1000);
}

Would you please show an example of the CSV data? Which programming language do you use? — Quality Catalyst, Apr 17 '15 at 01:41
11.97;130.72;12.72;131.61;13.47;132.52;14.20;133.43;14.92;134.36;15.63;135.30;16.33;136.26;17.01;137.22;17.69;138.20;18.35;139.20;18.99;140.20;19.63;141.22;20.25;142.26;20.85;143.31;21.44;144.37;22.02;145.44;22.58;146.53;23.12;147.63;23.65;148.75;24.16;149.88;24.65;151.02;25.13;152.18;25.59;153.35;26.03;154.53;26.45;155.73;26.85;....... — Fabrice, Apr 17 '15 at 07:50
@Fabrice. Do you need all values in memory? Do you need only to find a value each time it's required? How much is the time that elapses between value requests? I don't know this MCU, but: how much is the amount of system memory its may manage? Does the MCU have something such an HD or storage device? How much is bigger such a device (if there's)? — Sir Jo Black, Apr 17 '15 at 08:45
I think it is better the file is converted into a binary file; you may supply this converted file to the MCU. Binary files guarantee a faster way to read data in it! — Sir Jo Black, Apr 17 '15 at 08:51
I put my csv file in a USB stik and connected to my STM32F429 using USB otg. Each 5 min I want to read a value — Fabrice, Apr 17 '15 at 09:34
The problem is that parsing a csv file is a very slow way, if the MCU has system memory where to load all the bytes of the file the function is not so complex! — Sir Jo Black, Apr 17 '15 at 10:22

Sir Jo Black · Accepted Answer · 2015-04-17T13:29:06.857

I think this solution is good. The idea is to convert the CSV file in a binary file (a sequence of floats). After the conversion you may supply the binary file to the MCU and use the function getVal() (below indicated) to read the data.

This code converts a file data.csv into a file data.bin:

#include <stdio.h>
#include <stdlib.h>
#include <errno.h>
#include <malloc.h>

int main(void)
{
    FILE * fin=NULL;
    FILE * fout=NULL;
    char * buff, *tmp;
    float val;
    int cnt=0;

    size_t bsize;

    fin=fopen("data.csv","r");
    if (fin==NULL) {
        perror("1 - Error");
        return errno;
    }

    fseek(fin,0,SEEK_END);
    bsize=ftell(fin);
    if (!bsize) {
        puts("The file doesn't contain data!");
        return -1;
    }

    buff=malloc(bsize+1);
    if (buff==NULL) {
        perror("2 - Error");
        return errno;
    }

    fout=fopen("data.bin","w");
    if (fout==NULL) {
        perror("3 - Error");
        return errno;
    }

    fseek(fin,0,SEEK_SET);
    fread(buff,1,bsize,fin);
    buff[bsize]=0;
    fclose(fin);

    tmp=buff;
    do {
        val=strtof(tmp,&tmp);
        printf("%12g ",val);
        if ((++cnt)%5==0)
            puts("");
        fwrite(&val,sizeof(val),1,fout);
        if (*tmp!=0)
            tmp++;
    } while(*tmp!=0);

    puts("\nEnd of conversion\n");

    fclose(fout);
    free(buff);

    return 0;
}

This function read a field from a binary file that contains float data in binary format:

float getVal(FILE *f, size_t index)
{
    float val;

    fseek(f,index*sizeof(val),SEEK_SET);
    fread(&val,1,sizeof(val),f);

    return val;
}

This main uses the function getVal() to take data from the file data.bin:

#include <stdio.h>
#include <stdlib.h>
#include <errno.h>

float getVal(FILE *f, size_t index);

int main(void)
{
    FILE * fin;
    size_t index;

    fin=fopen("data.bin","r");
    if (fin==NULL) {
        perror("1 - Error");
        return errno;
    }

    while(1) {
        printf("Insert index from 1 to n [Insert 0 to end]: ");
        scanf("%lu",&index);
        if (index==0)
            break;

        printf("%g\n",getVal(fin,index-1));
    }

    fclose(fin);
    return 0 ;

}

Comments are not for extended discussion; this conversation has been [moved to chat](http://chat.stackoverflow.com/rooms/75758/discussion-on-answer-by-sergio-formiggini-extract-value-from-csv-file). — Taryn, Apr 20 '15 at 21:40

Extract value from .csv file

1 Answers1