-1

I need to get the data in .pcd format into .bin format(data is lidar point clouds). So I am trying to write the data in a row aligned order with precision. But I could able to set the precision at all. After doing the conversion, I read the .bin file. data is still not in the pre defined precision format. Here is the code:

#include <iostream>           
#include <pcl/io/pcd_io.h>      
#include <pcl/point_types.h>     
using namespace std;


int main(){
    std::string bin_path = "/home/dummy/data1/pcd_bin/bin/";
    std::string out_file = bin_path+"100.bin";
    pcl::PointCloud<pcl::PointXYZ> cloud;
    for (int v=0; v<1000; ++v)
    {
        pcl::PointXYZ newPoint;
        newPoint.x = (rand() * 100.0) / RAND_MAX;
        newPoint.y = (rand() * 100.0) / RAND_MAX;
        newPoint.z = (rand() * 100.0) / RAND_MAX;
        //newPoint.intensity = (rand() * 100.0) / RAND_MAX;
        cloud.points.push_back(newPoint);
    }
       //Create & write .bin file
      ofstream  bin_file(out_file.c_str(),ios::out|ios::binary|ios::app);
      if(!bin_file.good()) cout<<"Couldn't open "<<out_file<<endl;  


      for (size_t i = 0; i < cloud.points.size (); ++i)
      {
        bin_file<<std::setprecision(5);
        bin_file.write((char*)&cloud.points[i].x,3*sizeof(float)); 
       // bin_file.write((char*)&cloud.points[i].intensity,sizeof(float));
        //cout<<    cloud->points[i]<<endl;
      }
}

As you can see writing the values to binary files in float32(assumption, after reading some stackoverflow questions) and reading them in python code as float32 only.

Expected saved format in binary file:

[84.0187 39.438 78.309 ]

[79.844 91.164 19.755]

[33.522 76.822 27.777]

[55.396 47.739 62.887]

[36.478 51.340 95.222 ]

But actual saved format in binary file same as original format:

[84.01877 39.438293 78.30992 ]

[79.844 91.164734 19.755136]

[33.522274 76.82296 27.777472]

[55.396996 47.739704 62.887093]

[36.478447 51.34009 95.22297 ]

Python code for reading the binary file:

 raw_lidar_o_crop = np.fromfile(100.bin, dtype=np.float32).reshape((-1, 4))
    print("ouster new ")
    with open("ouster_new_intensity2.txt", "w") as text_file:
        for i in range(len(raw_lidar_o_crop)):
           print(f"ouster points cropped : {raw_lidar_o_crop[i]}", file=text_file)

PS: expected format values are dummy. I read the binary file in python and saved the values in .txt file for visualization.

PS1: One need pcl library to reproduce the result.

PS2: I have other binary files written in C++ with precision, so I am reading those files in python with type float32.

Thanks in advance

Suri
  • 21
  • 5
  • 5
    You don't need to (or should, really) set a precision when writing the raw binary data. You only set precision when you show the data to a user. Besides, the problem you seem to have is that the program converting the binary data from the file isn't using the same floating-point format as the values actually stored in the file. Please try to create a [mcve] of both the program writing the data (hard-code values if you have to) and the Python script that reads the file and present it. – Some programmer dude Jan 24 '21 at 16:01
  • Without actual data (small sample like a single line) and file content (ideally in a HEX editor that show both text and bytes), it id hard to know what you are doing wrong. Without the definition of `x` and `intensity`we don't even know if your original data is of type `float` – Phil1970 Jan 24 '21 at 16:36
  • @Someprogrammerdude now you can reproduce the result – Suri Jan 24 '21 at 16:36
  • @Phil1970 I assume the data is in float, since original data is stored in float – Suri Jan 24 '21 at 16:37
  • 1
    Seriously, are you testing your program with random numbers? You need to test with something like writing 2 rows of 0. Thecn check file size and try to read it in the other program. Once that works, try something like 1, 0, 0 on both rows. That way, you can more easily analyse the actual file content. – Phil1970 Jan 24 '21 at 16:41
  • @Suri, assuming is not enough. You have to analyse or validate data you write, file format including the endianess and the fact that the binary file contains only that in that order.I don't know how Python write binary files. – Phil1970 Jan 24 '21 at 16:45
  • @Phil1970 I did not get what is wrong with random numbers. I am generating some random numbers and saving it in a row aligned order in a binary file and reading the same random numbers in python. One thing to be clear, I do not want to analyse the data. I need the data to be in the defined precision, so I think it does not matter whether the data is random or other format? My concern is only for precision, not about values here. whether it is 0,1, 1000, 100000, 10M, but in the end binary file should have 5 digits value in it – Suri Jan 24 '21 at 16:47
  • 1
    @Phil1970, please see the code, binary file is written in type float. It is not just an assumption. More info about float in c++ https://stackoverflow.com/questions/21198948/how-to-have-both-32bit-and-64bit-float-in-c – Suri Jan 24 '21 at 16:52
  • 1
    The precision is only used whe you write as text using the `<<` operator. The "error" you percieve is onle a presentational error, where your Python script doesn't set the wanted precision when printing the values. In the file the values are stored as raw binary data, a bitwise copy of the single-precision floating point numbers, and there's nothing you can (or should!) do to prevent that. – Some programmer dude Jan 24 '21 at 18:07
  • @Someprogrammerdude I know python script is not setting the precision while writing the values to .txt file and I can do that If I need it. But my question is while writing to .bin file. Okay please let me know if you got any idea to set the precision directly in .bin file. In the meantime I set precision in .txt and then add values from .txt to .bin file – Suri Jan 24 '21 at 19:58
  • 1
    Please clarify what you mean by `"...set the precision directly in .bin file"`. Why do you think you can alter the precision over and above what is dictated by the underlying type representation -- `float` in this case? – G.M. Jan 24 '21 at 21:10
  • 2
    Since you're writing the raw binary data you *can't* set precision. Precision is a pure presentational thing. If you want to *round* the values stored in the file, you have to do it explicitly for each number before writing it. – Some programmer dude Jan 25 '21 at 00:41
  • 1
    @Suri In original question, you show unrelated numbers while in updates question expected numbers are truncated values of actual saved values. All you have to to is copy your data locally and then multiply them by 1000.0, floor then and divide by 1000.0. **You poorly write your initial question and then you almost accuse us when we try to help you!**. – Phil1970 Jan 25 '21 at 03:09
  • @Phil1970 Sorry, if I offended you. I did not mean it. I got your point now. – Suri Jan 25 '21 at 09:01
  • could someone tell me what is wrong with my question? I do not know why someone down voted it. After improving question according to commentators. – Suri Jan 29 '21 at 14:33

1 Answers1

0

As @Some programmer dude suggested in the above comments, we can use std::setprecision only when we need to present the values from binary files with precision. In order to set the precision first we need to do the store the values to .txt file with precision and store the values from .txt to .bin file.

Suri
  • 21
  • 5