Android: how to synchronise multiple IMU sensors(Accelerometer/Gyroscope/Magnetic)?

Question

I am currently developing an android app that collects IMU datas(Accelerometer, Gyroscope and Magnetic field) for positioning, which requires a very high level accuracy in measurement time. All three sensors are set to SENSOR.DELAY.FASTEST and this is my code.

    @Override
    public void onSensorChanged(SensorEvent sensorEvent) {
        final float alpha = 0.97f;
        //in milliseconds since last boot
        time = SystemClock.elapsedRealtime();

        switch (sensorEvent.sensor.getType()){
            case Sensor.TYPE_ACCELEROMETER:
                Log.d(TAG,"TYPE_ACCELEROMETER: "+sensorEvent.timestamp);
                //System.arraycopy(sensorEvent.values,0,LastAccReading,0,sensorEvent.values.length);
                LastAccReading[0] = alpha * LastAccReading[0] + (1-alpha) * sensorEvent.values[0];
                LastAccReading[1] = alpha * LastAccReading[1] + (1-alpha) * sensorEvent.values[1];
                LastAccReading[2] = alpha * LastAccReading[2] + (1-alpha) * sensorEvent.values[2];
                break;

            case Sensor.TYPE_MAGNETIC_FIELD:
                Log.d(TAG,"TYPE_MAGNETIC_FIELD: "+sensorEvent.timestamp);
                //System.arraycopy(sensorEvent.values,0,LastMagReading,0,sensorEvent.values.length);
                LastMagReading[0] = alpha * LastMagReading[0] + (1-alpha) * sensorEvent.values[0];
                LastMagReading[1] = alpha * LastMagReading[1] + (1-alpha) * sensorEvent.values[1];
                LastMagReading[2] = alpha * LastMagReading[2] + (1-alpha) * sensorEvent.values[2];
                break;

            case Sensor.TYPE_GYROSCOPE:
                Log.d(TAG,"TYPE_GYROSCOPE: "+sensorEvent.timestamp);
                LastGyroReading[0] = sensorEvent.values[0];
                LastGyroReading[1] = sensorEvent.values[1];
                LastGyroReading[2] = sensorEvent.values[2];
        }

        // Rotation matrix based on current readings from accelerometer and magnetometer.
        SensorManager.getRotationMatrix(rotationMatrix, inclinationMatrix,
                LastAccReading, LastMagReading);
        // Express the updated rotation matrix as three orientation angles.
        SensorManager.getOrientation(rotationMatrix, orientationAngles);

        //Log.i("OrientationTestActivity",String.format("Orientation: %f, %f, %f", orientationAngles[0], orientationAngles[1], orientationAngles[2]));
    }

By looking at the sensorEvent.timestamp enter image description here I can see that Accelerometer and Gyroscope has the same sampling rate around 420Hz and magnetic field is much slower at about 100Hz. And the accelerometer and gyroscope are not returning sensor values at the same time, which means that at a certain timestamp only one of the three sensor values are valid and the other two would have a time offset of a few milliseconds.

Is there anything I can do about that? Thanks in advance.

In the end the sampling rate of sensors is hardware limited (see any datasheet for IMU chipset). Rates are set by the [Android Compatibility Definition Document](https://source.android.com/compatibility/12/android-12-cdd#73_sensors) and for background on sensor fusion see this old Google Talks video: https://youtu.be/C7JQ7Rpwn2k — Morrison Chang, Mar 21 '22 at 00:38
@MorrisonChang Thanks for the reply. I have checked the video and it was really interesting to watch. Unfortunately they did the sensor fusion on hardware level so they didn't have to worry about different timestamp. I still have not found any solution to this problem yet, I will probably keeping searching and meanwhile try figuring out ways to compensate time offset as well. Still thank you so much for the answer! — Geng Wang, Mar 22 '22 at 08:38
Do realize that sensors have error and drift so accurate time measurements only mean so much. Most consumer applications dealing with human motion can just time average the values and/or use a low-pass filter. Without knowing why, if you truly need high accuracy values then you'll have to build your own hardware. — Morrison Chang, Mar 22 '22 at 21:00
@MorrisonChang Yes I guess its both impossible and meaningless to achieve the "perfect" timing at this stage. Please answer below so I can mark it as an accepted answer. Thank you Morrison! — Geng Wang, Mar 22 '22 at 22:48

Morrison Chang · Accepted Answer · 2022-05-19T23:25:47.457

Consumer sensors are not the most accurate and are subject to noise and drift (check any IMU datasheet for details). They are accurate enough for common use cases.

At a minimum any Android based sensor will need to conform to the Android Compatibility Definition Document section on Sensors.

As sensors in devices isn't a new subject, it may help to watch this old Google Talks video on sensor fusion: Sensor Fusion on Android Devices: A Revolution in Motion Processing

While you may be concerned about different time stamps, I believe your desired level of precision unrealistic as the sensors have noise margins and drift to their values.

As to what developers do given existing error margins, given that the use case is human motion, sensor data is usually averaged over time and/or passed through a low-pass filter. For most use cases this is enough but as noted in this article on ARCore and ARKit on those devices supporting AR, certifying the sensors in those devices will require stricter (and most costly) calibration.

Those attempting VR/AR type behavior are probably using SensorDirectChannel but be aware of the stated rate variance and hardware/OS limitations.

Android: how to synchronise multiple IMU sensors(Accelerometer/Gyroscope/Magnetic)?

1 Answers1

Linked