I have a custom type. Let's say it's a 3D point type with intensity, so it'll look like this :
struct Point{
public:
double x,y,z;
double intensity;
//Imagine a constructor or everything we need for such a class here
};
and another class that use a vector of that point.
class Whatever{
//...
public:
std::vector<Point> myPts;
//...
};
I would like to be able to create this vector from a file. It means I have a file like this:
X1 Y1 Z1 I1
X2 Y2 Z2 I2
X3 Y3 Z3 I3
....
Xn Yn Zn In
And I would like to find a fast technique to separate each lines and build a point per lines, and build my vector. Since it's an operation that I will have to do a lot, I am looking for the fastest way.
The basic solution would be to read the file line by line, convert into stringstream and create from this stringstream :
while(std::getline(file, line)){
std::istringstream iss(line);
Point pt;
iss >> pt.x >> pt.y >> pt.z >> pt.intensity;
vector.add(pt);
}
But this is too much time consuming. So The second method would be to read the file (entire, or a part of the file) in a buffer and format the buffer to create the vector from it. Using a memory mapped file, I can read fastly into a buffer,, but How to format the buffer to create my points without using stringstream which I believe is slow?
EDIT : I used this code :
std::clock_t begin = clock();
std::clock_t loadIntoFile, loadInVec;
std::ifstream file(filename);
if (file) {
std::stringstream buffer;
buffer << file.rdbuf();
file.close();
loadIntoFile = clock();
while (!buffer.eof()) {
Point pt = Point();
buffer >> pt.x >> pt.y >> pt.z >> pt.intens >> pt.r >> pt.g >> pt.b;
pts.push_back(pt);
}
loadInVec = clock();
}
std::cout << "file size" << pts.size() << std::endl;
std::cout << "load into file : " << (loadIntoFile - begin) / (double)CLOCKS_PER_SEC << std::endl;
std::cout << "load into vec : " << (loadInVec - loadIntoFile) / (double)CLOCKS_PER_SEC << std::endl;
And the result is :
file size : 7756849
load into file : 2.619
load into vec : 31.532
EDIT2 :
After removing the stringstream buffer, I had a time of 34.604s
And after changing push_back by emplace_back 34.023s
