0

When I read from an mdf (.mf4) file with asammdf (Python 3.7), as soon as the sample count crosses a barrier, the timestamps start counting up from close to 0 again, like an overflow :for 3 files exactly at 29045 lines, one file at 27234 lines for some reason. This means that when I use methods like resample or to_dataframe that the interpolation that occurs during these methods royally screws up and I get botched data.

I haven't found anything of the sort in the documentation, and there's hardly any resources besides the doc. I thought this might be related to chunk size or memory allocation, but I can't figure out what to do differently or why exactly this happens.

Right now I read it via the standard method

mdf = MDF(file)

I wrote up a small script to read in a file and plot the difference between the converted (interpolated) data and the original datapoints to showcase what I mean:

import tkinter as tk
from tkinter import filedialog as fd
from asammdf import MDF
from matplotlib import pyplot as plt

typeStr = '*.mf4'
root = tk.Tk()
root.wm_attributes('-topmost', 1)
root.withdraw()
files = fd.askopenfilenames(parent=root,filetypes=[("Measurement MF4 file",typeStr)])

for file in files:
    mdf = MDF(file)

    # conversion to pandas
    data = mdf.to_dataframe()
    data['Time [s]'] = list(data.index)
    columns = data.columns.tolist()
    columns.remove('Time [s]')
    columns.insert(0,'Time [s]')
    data = data[columns]
    plt.plot(data['Time [s]'], data[columns[1]],label="converted")

    # original data
    chData = mdf.get(columns[1])
    plt.plot(chData.timestamps, chData.samples, label="original")

    plt.legend()
    plt.show()

an example file can be accessed here:

example file on Dropbox

an example of a plot:

plotted comparison

wiseboar
  • 175
  • 2
  • 13

2 Answers2

0

Thank you for the example file I will have a look at it. You can always raise an issue in the github repo

Edit: the file is corrupted because the time channel is not monotonous increasing, at samples index 29045 it is reset

>>> ch.timestamps[29040:29050]
array([290.47998, 290.48998, 290.49998, 290.50998, 290.51998, 290.52998,
         0.87998,   0.88998,   0.89998,   0.90998])

what is the application that originally recorded this file?

danielhrisca
  • 665
  • 1
  • 5
  • 11
0

I found out that the trouble was actually caused by the asammdf library (somehow). I can't really point out which version caused the problem because I updated all libraries - and it worked afterwards. Currently I'm running on version 5.13.13 and there's no more trouble reading the timestamps correctly.

EDIT: unfortunately, this turned out not to be true. I'm suspecting the recording program (PUMA Testbench Software from AVL), but I'm not sure in any way. I'm using a workaround now where I'm stitching together a new global timestamp and then interpolating all channels along this. It's slower than the library function, but at least it gets rid of the error.

wiseboar
  • 175
  • 2
  • 13