Here is one very simple that has some weaknesses but some strengths as well.
In each dataset, sort the points by the time value (x-coordinate). Then sum over the Euclidean distance between each consecutive pair of points.
This works pretty well if the total spread in time values is the same for each dataset. If this is not the case, you may want to divide the sum-of-distances by the range (maximum minus minimum) of the time values. The smallest "roughness" by this measure is a straight line segment or sequence of segments on the same line. If you want the "roughness" measure to be zero for a line segment you could subtract the length of the line segment between the initial and terminal points from the sum of the distance. Other adjustments could be made for other goals.