Parsing binary data in Ruby

Question

We're doing a major rewrite of a project previously written in C into Ruby. We have a bunch of C structures, written as C typedefs:

struct my_struct {
    uint32_t foo;
    uint8_t bar;
    char baz[80];
}

Is there a quick way to load them all up in Ruby? For example, is there some way to convert these definitions into something that resembles code like

@foo = io.read(4).unpack('V')[0]
@bar = io.read(1).unpack('C')[0]
@baz = io.read(80)

There are literally tons of it, I'd rather not convert them by hand...

Can you describe where this data is stored and what format it will be in? It sounds like it might be stored as binary in a file but without more information on delimiters and file structure its hard to give a full answer. — Tyler Ferraro, Mar 11 '16 at 20:30
@TylerFerraro The data is either sent across the network (and thus resides in memory when it's time to parse it) or read/written in local disc files. — Maigio, Mar 11 '16 at 21:19
@theTinMan I've described what I've done so far — I'm convert these parsers by hand, i.e. rewriting C data structures in a way I've shown above. Of course, I've searched Google, etc, done all the basic scouting, but found no answer. Please advice how can I improve the question? — Maigio, Mar 11 '16 at 21:21
This doesn't answer your question but I wanted to mention that the code you've given could be more simply expressed as `@foo, @bar, @baz = io.read(85).unpack('VCA*')`. — Jordan Running, Mar 11 '16 at 22:46
This is not very *quick* but I would first use [cast](https://github.com/oggy/cast), then iterate over the nodes to *print* the [BinData](https://github.com/dmendel/bindata/) lines... — rdupz, Mar 24 '16 at 09:25

score 2 · Answer 1 · answered Mar 24 '16 at 08:52

class MyStruct < BinData::Record
    endian :little
    uint32 :foo
    uint8  :bar
    string :baz, read_length: 80
end

It's pretty much a 1:1 mapping between C structs and BinData Records. It should be easy to write a conversion script.

score 0 · Answer 2 · answered Mar 12 '16 at 02:09

0

If you have tons of them, you may want to write a quick program to parse the C code for you :)

Then the parsing code could create the corresponding ruby code from the C code.

answered Mar 12 '16 at 02:09

Daniel

I'm not very good at programming parsers, it's always been a major space in my CS knowledge, so I'd rather not. If I must, could you recommend some good books on writing parsers then? – Maigio Mar 12 '16 at 19:03
I wouldn't go all the way in writing a C level parser, just write enough to recognize a struct { } and convert it. The goal here is to write throw-away code that will convert something for us once. So, it doens't matter how terrible it is... to some point :) – Daniel Mar 12 '16 at 19:43

2 Answers2