1

I have code in C which reads data from a file in a binary format:

FILE *file;
int int_var;
double double_var;
file = fopen("file.dat", "r");
fread(&int_var, sizeof(int), 1, file);
fread(&double_var, sizeof(double), 1, file);

The above is a simplified but accurate version of the actual code. I have no choice over this code or the format of this file.

The data being read in C is produced using Python code. How do I write this data to a file in the same binary format? I looked into bytes and bytearrays, but they seem to only work with integers and strings. I need something like:

f = open('file.dat', 'wb')
f.write(5)
f.write(5.0)
f.close()

that will work with the above C code.

khelwood
  • 55,782
  • 14
  • 81
  • 108
jonajona
  • 13
  • 3
  • You need to know endianness and floating-point format used both in Python (which you may need to tweak) and in your C implementation. – pmg Jul 08 '22 at 11:47
  • 1
    you're going to need https://docs.python.org/3/library/struct.html#struct.pack – gog Jul 08 '22 at 11:49
  • https://stackoverflow.com/questions/11541944/behavior-of-ntohl-and-htonl-different-between-python-and-c – William Pursell Jul 08 '22 at 11:51
  • basically what others are saying is you should convert the numbers to byte-arrays first with a known number of bytes and endian before writing them to the file, to make reading them possible, as python has a very unique (and hard to predict) method of handling numbers. – Ahmed AEK Jul 08 '22 at 11:55
  • the documentation I found for byte-arrays only discusses how to convert between that and integers or strings, whereas I also need to handle floating point numbers. looking at the struct pack right now, thanks. – jonajona Jul 08 '22 at 13:27

1 Answers1

0

As mentioned in a comment, you need the struct library:

Creating file.dat with

#!/usr/bin/env python3                                                                                                                                                                                                                           
import struct
with open('file.dat', 'wb') as f:
    f.write(struct.pack('=id', 1, 5.0))

and then reading it with

#include <stdio.h>

int main(void) {
  int int_var;
  double double_var;

  FILE *file = fopen("file.dat", "rb");
  if (!file) {
    fprintf(stderr, "couldn't open file.dat!\n");
    return 1;
  }

  if (fread(&int_var, sizeof(int), 1, file) != 1) {
    fprintf(stderr, "failed to read int!\n");
    return 1;
  }

  if (fread(&double_var, sizeof(double), 1, file) != 1) {
    fprintf(stderr, "failed to read double!\n");
    return 1;
  }

  printf("int = %d\ndouble = %f\n", int_var, double_var);

  fclose(file);
  return 0;
}

will output

int = 1
double = 5.000000

Note the = in the pack format definition; that tells python not to add alignment padding bytes like you'd get in a C structure like

struct foo {
  int int_var;
  double double_Var;
};

Without that, you'll get unexpected results reading the double in this example. You also have to worry a little bit about endianess if you want the file to be portably read on any other computer.

Shawn
  • 47,241
  • 3
  • 26
  • 60
  • And if both the writing in python and the reading in C are done on the same computer, is endianess important? Are there like common types of endianess from which I can choose, or is it something set by the computer/OS/compiler...? – jonajona Jul 08 '22 at 13:20
  • If it's only going to be used on one computer, not an issue. See https://en.wikipedia.org/wiki/Endianness for details. – Shawn Jul 08 '22 at 13:29
  • Also, can you call f.write() multiple times, e.g. f.write(struct.pack('=i', 1)), f.write(struct.pack('=d', 5.0)), to make this file? – jonajona Jul 08 '22 at 13:32
  • Sure, I guess so. – Shawn Jul 08 '22 at 13:34
  • Just one more question please, can you not choose endianess in C? Is that why you used '=', as opposed to setting '<' in python for example, and using the same endinaess in C? – jonajona Jul 08 '22 at 13:53