-1

I have fix message file that would needed to be converted to JSON format, like below. How can I use a shell script to convert these?

From:

05/03 11:23:19.123456 << 8=FIX.4.2^9=451^35=D^49=abc^56=bcd
05/03 11:23:19.123457 << 8=FIX.4.2^9=451^35=D^49=abc1^56=bcd1
05/03 11:23:19.123458 << 8=FIX.4.2^9=451^35=D^49=abc2^56=bcd2

To:

{"time":"05/03 11:23:19.123456", "8":"FIX.4.2", "35":"D", "49":"abc", "56":"bcd"}
{"time":"05/03 11:23:19.123457", "8":"FIX.4.2", "35":"D", "49":"abc1", "56":"bcd1"}
{"time":"05/03 11:23:19.123458", "8":"FIX.4.2", "35":"D", "49":"abc2", "56":"bcd2"}
Nat Riddle
  • 928
  • 1
  • 10
  • 24
Lam Chin Fei
  • 101
  • 1
  • 13
  • Are you allowed to write a C program doing such a conversion? Are you allowed to use [JSON](http://json.org/) libraries? Did you read books (like the [Dragon book](https://en.wikipedia.org/wiki/Compilers:_Principles,_Techniques,_and_Tools)...) related to compilation and [parsing](https://en.wikipedia.org/wiki/Parsing)? Are you allowed to use [GNU bison](https://www.gnu.org/software/bison/) or [GNU gawk](https://www.gnu.org/software/gawk/) or [jq](https://stedolan.github.io/jq/)? – Basile Starynkevitch May 03 '21 at 15:58
  • Do you have a complete documentation of the FIX protocol (including its [EBNF syntax](https://en.wikipedia.org/wiki/Extended_Backus%E2%80%93Naur_form)...) ? Are you allowed to use the [QickFIX](https://en.wikipedia.org/wiki/QuickFIX) library(in C++)? You could combine it with the [JsonCPP](https://github.com/open-source-parsers/jsoncpp) C++ library... Of course you do need to learn C++, by reading [a good C++ programming book](https://www.stroustrup.com/programming.html) and the documentation of [GCC](http://gcc.gnu.org/) – Basile Starynkevitch May 03 '21 at 16:03
  • 1
    Your question has no [mre] and is opinion based.... Be aware that ***StackOverflow* is *not* a do-my-work website** – Basile Starynkevitch May 03 '21 at 16:09
  • Note that a simple search for `python parse fix message` seems to indicate that there are pre-existing modules for parsing this sort of message. – larsks May 03 '21 at 17:09

1 Answers1

1

I assume that the omission of 9=451 in the desired output was an accident.

$ awk 'BEGIN{FS="\\^|[ <]+"}{printf "{\"time\":\"%s %s\"",$1, $2; for(i=3;i<=NF;i++){split($i,a,/=/);printf ", \"%s\":\"%s\"",a[1],a[2]}; printf "\n"}' fix  
{"time":"05/03 11:23:19.123456", "8":"FIX.4.2", "9":"451", "35":"D", "49":"abc", "56":"bcd"
{"time":"05/03 11:23:19.123457", "8":"FIX.4.2", "9":"451", "35":"D", "49":"abc1", "56":"bcd1"
{"time":"05/03 11:23:19.123458", "8":"FIX.4.2", "9":"451", "35":"D", "49":"abc2", "56":"bcd2"

And here annotations to explain how it works:

# BEGIN rule(s)
BEGIN {
    FS = "\\^|[ <]+"     # define two field separators, ^ and any number or consecutive spaces or left angle brackets
}

# Rule(s)
{
    printf "{\"time\":\"%s %s\"", $1, $2  # printing the time (which we treat separately as it's internally "split" by a space.
    for (i = 3; i <= NF; i++) {           # the other fields we can iterate over, NF in awk is the 'number of fields'
            split($i, a, /=/)             # use the loop variable i to reference the fields, split them into an array on the '='
            printf ", \"%s\":\"%s\"", a[1], a[2]    # print the array elements formatted, all on one line
    }
    printf "\n"                           # add a new line when we're done
}
tink
  • 14,342
  • 4
  • 46
  • 50
  • You could call out to this [online tool](https://fixparser.chronicle.software) that parses fix messages to JSON. – Rob Austin Mar 03 '22 at 19:26
  • @RobAustin - why would I want to do that? The page seems to choke on the sample data above. – tink Mar 03 '22 at 22:12