0

I have a function (containing numpy calculations, float32) which returns a dictionary, where the values are lists containing text and numbers. Within the function certain numbers have been rounded to three decimal places as they are added to the dictionary. Printing the returned dictionary gives an example entry (the four rounded values are in bold):

'51450049': ['ULT', 'IRU', 71254171, '30085', '11420', '20008', '23200', '22100', '46000', 'Z1', 10.132, '---', 73566310, '50069', '10413', '20001', '23100', '22000', '46001', 'Z1', '---', -51.269, 71254171, '30085', '11420', '20008', '23200', '22100', '46000', 'Z2', 17.258, '---', 73375014, '50063', '15833', '20001', '23100', '22000', '46001', 'Z2', '---', -77.339]

later in the code, values from the returned dictionary are written to a text file using f-strings. Example code

file_object.write(f'{element_id:<13}{values[0]:<12}{values[1]:<9}{values[2]:<13}{values[3]:<9}'
                  f'{values[4]:<9}{values[5]:<12}{values[6]:<18}{values[7]:<9}{values[8]:<14}'
                  f'{values[9]:<8}{values[10]:18}{values[11]:15}\n'
                  )

where for instance, the {values[10]:18} command is referencing the value 10.132 in the entry above. Similar commands are used to print the other three values on different lines in the output.

However, within the outputted text file the printed values seem to have reverted back to their pre-rounded state? e.g

rounded values in dictionary:

  • 10.132 -51.269 17.258 -77.339

printed by f-string

  • 10.131999969482422 -51.26900100708008 17.257999420166016 -77.33899688720703

I know i can 'round' the f-string but why would i need to do this twice effectively? I was rounding the number within the function to keep the size of the returned dictionary down, but am i missing something fundamental here? i.e is the whole float16 number retained? Thanks

Iain Mcleod
  • 13
  • 2
  • You're asking to print numbers with 18 digits, so that's no surprise. – Thierry Lathuille Oct 30 '20 at 13:42
  • 1
    You are storing the "rounded" values as floats, which cannot exactly store the values you are referring to. The literal `10.132` does not produce the rational number `10132/1000`, so there's no unrounding, only a misunderstanding of what is actually stored in the `dict`. – chepner Oct 30 '20 at 13:42
  • 1
    If you care about this level of precision, use the `decimal` module to store your values, not the `float` type. – chepner Oct 30 '20 at 13:43
  • @chepner that's exactly what i suspected. i.e. i was thinking that as i had added a rounded number to the dictionary then it would only literally contain, for example '10.132'. – Iain Mcleod Oct 30 '20 at 13:52
  • use decimals https://docs.python.org/3.8/library/decimal.html – ikamen Oct 30 '20 at 13:56
  • @chepner Then i could simply remove the rounding within the function and apply via f-string while writing out? Thanks, i'll check out 'decimal' module – Iain Mcleod Oct 30 '20 at 14:00

0 Answers0