Google Glass how to detect head move up/down left/right events

Question

I am working on a glassware where I have a image shown in my activity. I also managed it to zoom into the image by using slide gestures on the thouchpad. What I still need to do scrolling through this zoomed image.

So my question is is their any way to get some kind of gestureevent for turning your head I have seen a google glass dart crosshair example at https://www.youtube.com/watch?v=pGhamZnj6V0 but if I understand the way this works right it uses some kind of browser/webtechnology but the way the browser gets this information could properly help me too.

Or should I switch from my imageview to a web-control in my activity and try to load my image embedded into this control? If yes how can I handle the zoom function their?

score 1 · Answer 1 · answered Jul 10 '14 at 04:25

I don’t believe there is a gesture event that is triggered by the system when you move your head. However, you can implement your own using the Accelerometer API provided in Android. Something like this might work:

double lastX = 0;
double lastY = 0;

// Set up the accelerometer
SensorManager manager = (SensorManager) getSystemService(SENSOR_SERVICE);
manager.registerListener(this, sensorManager.getDefaultSensor(Sensor.TYPE_ACCELEROMETER), SensorManager.SENSOR_DELAY_NORMAL);

@Override
public void onSensorChanged(SensorEvent event) {
    // Check for correct sensor
    if (event.sensor.getType() == Sensor.TYPE_ACCELEROMETER) {
        // Get current acceleration values
        double currentX = event.values[0];
        double currentY = event.values[1];

        // Calculate delta values
        double newX = x - lastX;
        double newY = y - lastY;

        // Set last values
        lastX = newX;
        lastY = newY;

        // Move the view!
        moveView(newX, newY);
    }
}

public void moveView(double x, double y) {
    view.setX(x);
    view.setY(y);
}

You will need to modify this code to make it work in your setup, but it should give you an idea of how to start.

Hi Thomas, may I ask a question about `arcore-for-all` and `arcore-patch`? — Andy Jazz, Apr 05 '19 at 13:25

score 1 · Answer 2 · edited Nov 19 '18 at 03:46

I used the following library.

The library only supports down/left/right as it is but I added some code to make it recognize the upwards gesture too. Also because I had to edit the code myself and wasn't able to create a updated library out of the library I linked. I made a new package in my project and just pasted all the needed code there.

This is how my HeadGestureDetector looks after I added some code

import java.util.List;

import android.content.Context;
import android.hardware.Sensor;
import android.hardware.SensorEvent;
import android.hardware.SensorEventListener;
import android.hardware.SensorManager;
import android.util.Log;

import HeadGestureDetector.*;
import HeadGestureDetector.Constants;

public class HeadGestureDetector implements SensorEventListener {
    private static final int MATRIX_SIZE = 16;
    private float[] inR = new float[MATRIX_SIZE];
    private float[] outR = new float[MATRIX_SIZE];
    private float[] I = new float[MATRIX_SIZE];

    private float[] orientationValues = new float[3];
    private float[] magneticValues = new float[3];
    private float[] accelerometerValues = new float[3];
    private float[] orientationVelocity = new float[3];

    private SensorManager mSensorManager;

    private OnHeadGestureListener mListener;

    static enum State {
        IDLE, SHAKE_TO_RIGHT, SHAKE_BACK_TO_LEFT, SHAKE_TO_LEFT, SHAKE_BACK_TO_RIGHT, GO_DOWN, BACK_UP, GO_UP, BACK_DOWN
    }

    private State mState = State.IDLE;
    private long mLastStateChanged = -1;
    private static final long STATE_TIMEOUT_NSEC = 1000 * 1000 * 1000;

    public HeadGestureDetector(Context context) {
        mSensorManager = (SensorManager) context.getSystemService(Context.SENSOR_SERVICE);
    }

    private static final int[] REQUIRED_SENSORS = { Sensor.TYPE_MAGNETIC_FIELD, Sensor.TYPE_ACCELEROMETER,
            Sensor.TYPE_GYROSCOPE };

    private static final int[] SENSOR_RATES = { SensorManager.SENSOR_DELAY_NORMAL, SensorManager.SENSOR_DELAY_NORMAL,
            SensorManager.SENSOR_DELAY_NORMAL };

    public void start() {
        for (int i = 0; i < REQUIRED_SENSORS.length; i++) {
            int sensor_type = REQUIRED_SENSORS[i];
            Sensor sensor = null;
            List<Sensor> sensors = mSensorManager.getSensorList(sensor_type);
            if (sensors.size() > 1) {
                // Google Glass has two gyroscopes: "MPL Gyroscope" and "Corrected Gyroscope Sensor". Try the later one.
                sensor = sensors.get(1);
            } else {
                sensor = sensors.get(0);
            }
            Log.d(Constants.TAG, "registered:" + sensor.getName());
            mSensorManager.registerListener(this, sensor, SENSOR_RATES[i]);
        }
    }

    public void stop() {
        mSensorManager.unregisterListener(this);
    }

    public void setOnHeadGestureListener(OnHeadGestureListener listener) {
        this.mListener = listener;
    }

    @Override
    public void onAccuracyChanged(Sensor sensor, int accuracy) {
        // TODO Auto-generated method stub
    }

    @Override
    public void onSensorChanged(SensorEvent event) {

        if (event.accuracy == SensorManager.SENSOR_STATUS_UNRELIABLE) {
            // Log.w(Constants.TAG, "Unreliable event...");
        }

        int sensorType = event.sensor.getType();

        if (sensorType == Sensor.TYPE_MAGNETIC_FIELD) {
            magneticValues = event.values.clone();
            return;
        }

        if (sensorType == Sensor.TYPE_ACCELEROMETER) {
            accelerometerValues = event.values.clone();
            SensorManager.getRotationMatrix(inR, I, accelerometerValues, magneticValues);
            SensorManager.remapCoordinateSystem(inR, SensorManager.AXIS_X, SensorManager.AXIS_Z, outR);
            SensorManager.getOrientation(outR, orientationValues);
            return;
        }

        if (sensorType == Sensor.TYPE_GYROSCOPE) {
            if (event.accuracy == SensorManager.SENSOR_STATUS_UNRELIABLE) {
                // Log.w(Constants.TAG, "Unreliable gyroscope event...");
                // return;
            }

            orientationVelocity = event.values.clone();

            // state timeout check
            if (event.timestamp - mLastStateChanged > STATE_TIMEOUT_NSEC && mState != State.IDLE) {
                Log.d(Constants.TAG, "state timeouted");
                mLastStateChanged = event.timestamp;
                mState = State.IDLE;
            }

            // Log.d(Constants.TAG, Arrays.toString(orientationValues));
            // Log.d(Constants.TAG, "V:" + Arrays.toString(orientationVelocity));

            // check if glass is put on
            if (!isPutOn(orientationValues, orientationVelocity)) {
                Log.d(Constants.TAG, "Looks like glass is off?");
            }

            int maxVelocityIndex = maxAbsIndex(orientationVelocity);
            if (!isStable(orientationValues, orientationVelocity)) {
                // Log.d(Constants.TAG, "V:" + Arrays.toString(orientationVelocity));
            }
            if (isStable(orientationValues, orientationVelocity)) {
                // Log.d(Constants.TAG, "isStable");
            } else if (maxVelocityIndex == 0) {
                if (orientationVelocity[0] < -MIN_MOVE_ANGULAR_VELOCITY) {
                    if (mState == State.IDLE) {
                        // Log.d(Constants.TAG, "isNod");
                        mState = State.GO_DOWN;
                        mLastStateChanged = event.timestamp;
                        if (mListener != null) {
                            mListener.onNod();
                        }
                    }
                }
            }

                if (orientationVelocity[0] > MIN_MOVE_ANGULAR_VELOCITY) {
                    if (mState == State.IDLE) {
                        mState = State.GO_UP;
                        mLastStateChanged = event.timestamp;
                        if (mListener != null) {
                            mListener.onHey();
                        }
                    }

            }
            else if (maxVelocityIndex == 1) {
                if (orientationVelocity[1] < -MIN_MOVE_ANGULAR_VELOCITY) {
                    if (mState == State.IDLE) {
                        // Log.d(Constants.TAG, Arrays.toString(orientationValues));
                        // Log.d(Constants.TAG, "V:" + Arrays.toString(orientationVelocity));
                        mState = State.SHAKE_TO_RIGHT;
                        mLastStateChanged = event.timestamp;
                        if (mListener != null) {
                            mListener.onShakeToRight();
                        }
                    }
                } else if (orientationVelocity[1] > MIN_MOVE_ANGULAR_VELOCITY) {
                    if (mState == State.IDLE) {
                        // Log.d(Constants.TAG, Arrays.toString(orientationValues));
                        // Log.d(Constants.TAG, "V:" + Arrays.toString(orientationVelocity));
                        mState = State.SHAKE_TO_LEFT;
                        mLastStateChanged = event.timestamp;
                        if (mListener != null) {
                            mListener.onShakeToLeft();
                        }
                    }
                }
            }
        }
    }

    private static final float MIN_MOVE_ANGULAR_VELOCITY = 1.00F;

    private static final float MAX_STABLE_RADIAN = 0.10F;

    private static final float MAX_PUT_ON_PITCH_RADIAN = 0.45F;

    private static final float MAX_PUT_ON_ROLL_RADIAN = 0.75F;

    private static final float STABLE_ANGULAR_VELOCITY = 0.10F;

    private static boolean isStable(float[] orientationValues, float[] orientationVelocity) {
        if (Math.abs(orientationValues[1]) < MAX_STABLE_RADIAN
                && Math.abs(orientationVelocity[0]) < STABLE_ANGULAR_VELOCITY
                && Math.abs(orientationVelocity[1]) < STABLE_ANGULAR_VELOCITY
                && Math.abs(orientationVelocity[2]) < STABLE_ANGULAR_VELOCITY) {
            return true;
        }
        return false;
    }

    private static boolean isPutOn(float[] orientationValues, float[] orientationVelocity) {
        if (orientationValues[1] < MAX_PUT_ON_PITCH_RADIAN && Math.abs(orientationValues[2]) < MAX_PUT_ON_ROLL_RADIAN) {
            return true;
        }
        return false;
    }

    private static int maxAbsIndex(float[] array) {
        int n = array.length;
        float maxValue = Float.MIN_VALUE;
        int maxIndex = -1;
        for (int i = 0; i < n; i++) {
            float val = Math.abs(array[i]);
            if (val > maxValue) {
                maxValue = val;
                maxIndex = i;
            }
        }
        return maxIndex;
    }
}

My OnHeadGestureListener class

public interface OnHeadGestureListener  {
    void onHey();

    void onNod();

    void onShakeToLeft();

    void onShakeToRight();
}

and my Constants class

public class Constants {
    public static final String TAG = "HeadGestureDetector";
}

Create those 3 classes in your projects and you should be able to use the head gestures

for it to work you need to add implements OnHeadGestureListener to your class

declare private GestureDetector mGestureDetector

in your onCreate()

mHeadGestureDetector = new HeadGestureDetector(this);
mHeadGestureDetector.setOnHeadGestureListener(this);

lastly in your onResume

mHeadGestureDetector.start();

and in your onPause

mHeadGestureDetector.stop();

I wrote a piece of code for this detector to trigger touchpad gestures. So when you look up glass thinks you swiped down on the touchpad. This code can be found here.

@croX glas that this was useful to someone else – NoSixties Mar 19 '15 at 11:57 — NoSixties, Mar 19 '15 at 11:57

Google Glass how to detect head move up/down left/right events

2 Answers2