Substitute Scientific Notation from bytes to Megabytes

Question

I Have a .xml file that has lines which look like this:

<?xml version="1.0" encoding="utf-8"?>
<!DOCTYPE rrd SYSTEM "http://oss.oetiker.ch/rrdtool/rrdtool.dtd">
<!-- Round Robin Database Dump -->
<rrd>
        <version>0003</version>
        <step>60</step> <!-- Seconds -->
        <lastupdate>1674125860</lastupdate> <!-- 2023-01-19 10:57:40 UTC -->

    <ds>
            <name> 1 </name>
            <type> GAUGE </type>
            <minimal_heartbeat>8460</minimal_heartbeat>
            <min>NaN</min>
            <max>NaN</max>

            <!-- PDP Status -->
            <last_ds>954298368</last_ds>
            <value>3.8171934720e+10</value>
            <unknown_sec> 0 </unknown_sec>
    </ds>

    <!-- Round Robin Archives -->
    <rra>
            <cf>AVERAGE</cf>
            <pdp_per_row>1</pdp_per_row> <!-- 60 seconds -->

            <params>
            <xff>5.0000000000e-01</xff>
            </params>
            <cdp_prep>
                    <ds>
                    <primary_value>8.5981579947e+08</primary_value>
                    <secondary_value>0.0000000000e+00</secondary_value>
                    <value>NaN</value>
                    <unknown_datapoints>0</unknown_datapoints>
                    </ds>
            </cdp_prep>
            <database>
                    <!-- 2023-01-17 10:58:00 UTC / 1673953080 --> <row><v>NaN</v></row>
                    <!-- 2023-01-17 10:59:00 UTC / 1673953140 --> <row><v>NaN</v></row>
                    <!-- 2023-01-17 11:00:00 UTC / 1673953200 --> <row><v>NaN</v></row>
                    <!-- 2023-01-17 11:01:00 UTC / 1673953260 --> <row><v>NaN</v></row>
                    <!-- 2023-01-17 11:02:00 UTC / 1673953320 --> <row><v>NaN</v></row>
                    <!-- 2023-01-17 11:03:00 UTC / 1673953380 --> <row><v>NaN</v></row>
                    <!-- 2023-01-18 12:00:00 UTC / 1674043200 --> <row><v>NaN</v></row>
                    <!-- 2023-01-18 18:00:00 UTC / 1674064800 --> <row><v>7.9644330667e+08</v></row>
                    <!-- 2023-01-19 00:00:00 UTC / 1674086400 --> <row><v>7.9696554667e+08</v></row>
                    <!-- 2023-01-19 06:00:00 UTC / 1674108000 --> <row><v>5.8408509440e+08</v></row>
            </database>
    </rra>

Trying to convert the scientific notation (which is a value in bytes) and convert it to a value in megabytes and back to scientific notation in Linux bash shell or script.

So far I have this lines, but i am stuck and don't know how to put them back into the file with the calculation to divide 2x by 1024:

cat Memory_mem_used.xml | grep -Eo  '[0-9]+\.[0-9]+e\+[0-9]+' | perl -ne 'printf "%d\n", $_;'

The output should look like this:

output=796443306 | output2=$(($output / 1024 / 1024)) | perl -e 'printf "%.11e\n", '$output2''
7.59000000000e+02

Can you please [edit] your question and give an example of what the number you want as the output should look like. — simbabque, Jan 19 '23 at 11:59
Please, give a sample of the XML input file including header and footer. — Pierre François, Jan 19 '23 at 12:01
`796443306.67` is not even whole bytes. Is `` some kind of mean value? If so, they could present the two values used to produce the mean as integers instead. — Ted Lyngmo, Jan 19 '23 at 12:27
Thanks for your responses, i have edited the question with the desired output and beginning and end of the .xml file. The .xml file is resumed as it has a lot of more lines. — techtuga, Jan 19 '23 at 13:50
This is really unclear. Are you simply looking for `perl -ne 'printf "%d\n", $_/1024/1024'`? — tripleee, Jan 19 '23 at 13:55
In the expected output, are you literally looking for two lines of output or is the first line supposed to somehow communicate something about how the values should be manipulated? Either way, the first line would need some additional explanation. — tripleee, Jan 19 '23 at 13:57
@tripleee thats already a great help, is there also a way to convert the result back to scientific notation within the same perl command? something like perl -ne 'printf "%d\n", $_/1024/1024, "%.11e\n' — techtuga, Jan 19 '23 at 14:13
Just replace the `%d` with whatever format you want instead. — tripleee, Jan 19 '23 at 14:15
ok, cool that worked. Can you also help to figure out how i can loop trough the file to replace this new value with the original value. Tried sed with this perl command but it didn't work. — techtuga, Jan 19 '23 at 14:21

score 0 · Accepted Answer · answered Jan 19 '23 at 15:14

Try:

#!/bin/bash
IFS=''
while read line ; do
  left=${line%%<v>*}
  rest=${line#*<v>}
  value=${rest%%</v>*}
  right=${rest#*</v>}
  if [ "$value" != "$line" ] && [ "$value" != "NaN" ] ; then # match
    num_value=$(LC_ALL=C printf '%.0f' "$value")
    new_value=$(LC_ALL=C printf '%.11e' $((num_value / 1048576)) )
    line="$left<v>$new_value</v>$right"
  fi
  echo "$line"
done < input.xml

Substitute Scientific Notation from bytes to Megabytes

1 Answers1