Measure the time a person has gazed at multiple objects

Question

I'm trying to create an UWP app that measure's the time a person has gazed at multiple objects. There are around 300 objects and they all need to measure the time and display that in a text file. I've succesfully coded this for just one object, but I do not know how I am able to do this with multiple ones. I saw this post How to create 300 stopwatches in C# more efficiently? and the answer to that helped me quite a lot, but the code does not implement well with my code. So I like the idea of creating a list of objects and then when the person has gazed in object [o] then the corresponding stopwatch will start when the eyes have entered the object, and stop when the eyes have left the object. Problem is as I mentioned already, the solution does not work well with the code I am working with. This is the code that I used that works for just one element.

public sealed partial class BlankPage1 : Page
{
    private GazeElement gazeButtonControl;
    private GazePointer gazePointer;

    public BlankPage1()
    {
        this.InitializeComponent();
        Stopwatch Timer = new Stopwatch();
        gazePointer = GazeInput.GetGazePointer(null);
        gazeButtonControl = GazeInput.GetGazeElement(GazeBlock);
        gazeButtonControl = new GazeElement();
        GazeInput.SetGazeElement(GazeBlock, gazeButtonControl);
        gazeButtonControl.StateChanged += GazeButtonControl_StateChanged;

        void GazeButtonControl_StateChanged(object sender, StateChangedEventArgs ea)
        {
            if (ea.PointerState == PointerState.Enter)
            {
                Timer.Start();
            }

            if (ea.PointerState == PointerState.Exit)
            {
                Timer.Stop();
                CreateStatistics();
            }
        }

        void CreateStatistics()
        {
            File.WriteAllText(@"C:\Users\Vincent Korpelshoek\AppData\Local\Packages\app.a264e06e2-5084-4424-80a9-bee5f5fbb6b6_8wekyb3d8bbwe\LocalState\Resultaten.txt", Timer.Elapsed.ToString(););
        }
    }
}

The "GazeBlock" is the name of the first object that has been created in the XAML file. So long story short, I'd like to implement this solution:

static Dictionary<object, Stopwatch> stopwatchesByObject;

static void Main(string[] args)
{
    List<object> objects = new List<object>();

    // now you have to fill the objects list...

    stopwatchesByObject = new Dictionary<object, Stopwatch>();
    foreach (var o in objects)
    {
        stopwatchesByObject.Add(o, new Stopwatch());
    }
}

// Call this method when the user starts gazing at object o
static void StartGazingAt(object o)
{
    stopwatchesByObject[o].Start();
}

// Call this method when the user stops gazing at object o
static void StopGazingAt(object o)
{
    stopwatchesByObject[o].Stop();
}

static void CreateStatistics()
{
    StringBuilder sb = new StringBuilder();
    foreach (var entry in stopwatchesByObject)
    {
        sb.AppendLine($"Gazed at {entry.Key.ToString()} for {entry.Value.Elapsed.TotalSeconds} seconds.");
    }
    File.WriteAllText("c:\\temp\\statictics.txt", sb.ToString());
}

But I do not know how to 'merge' these two together so the solution not only works for just one object, but for around 300 of them. If anyone knows how to help me to make this work, thank you!

Vincent

Answer 1

It seems that for each object you create, you need to add it to the dictionary (along with a new Stopwatch), then add the GazeButtonControl_StateChanged event handler to it (you could add this same event handler to all of the controls), then in that event handler you would reference the correct stopwatch by using the sender argument: stopwatchesByObject[sender].Start();

This may not be exactly correct, but here's what I think you'd need to do to merge the two pieces of code:

public sealed partial class BlankPage1 : Page
{
    private GazeElement gazeButtonControl;
    private GazePointer gazePointer;
    static Dictionary<object, Stopwatch> stopwatchesByObject;

    public BlankPage1()
    {
        this.InitializeComponent();
        gazePointer = GazeInput.GetGazePointer(null);
        gazeButtonControl = GazeInput.GetGazeElement(GazeBlock);
        gazeButtonControl = new GazeElement();
        GazeInput.SetGazeElement(GazeBlock, gazeButtonControl);
        gazeButtonControl.StateChanged += GazeButtonControl_StateChanged;

        // Add the object to our dictionary along with a new stopwatch
        stopwatchesByObject.Add(gazeButtonControl, new Stopwatch());

        private void GazeButtonControl_StateChanged(object sender, StateChangedEventArgs ea)
        {
            // Use the sender argument to choose the correct timer
            if (ea.PointerState == PointerState.Enter)
            {
                if (stopwatchesByObject.ContainsKey(sender))
                {
                    stopwatchesByObject[sender].Start();
                }
            }
            else if (ea.PointerState == PointerState.Exit)
            {
                if (stopwatchesByObject.ContainsKey(sender))
                {
                    stopwatchesByObject[sender].Stop();
                    CreateStatistics();
                }
            }
        }

        private void CreateStatistics()
        {
            StringBuilder sb = new StringBuilder();

            foreach (var entry in stopwatchesByObject)
            {
                sb.AppendLine($"Gazed at {entry.Key} for {entry.Value.Elapsed.TotalSeconds} seconds.");
            }

            File.WriteAllText("c:\\temp\\statictics.txt", sb.ToString());
        }
    }
}

Answer 2

@Vincent perhaps it would be worth looking at this in a different way.

With eye tracking the user can only look at one location on the screen at any point in time, so unless your objects are layered overtop of each other in 2D space or potentially aligned in the line of sight in 3D space, you may only be interested in timing the length of time the users gaze point stays in the current topmost object. One stopwatche might be sufficient to keep track of the time from OnEnter to OnExit for the active object.

If on the OnExit you add the duration of time for that particular gaze interaction to lastGazedObject's cumulative time count, you probably do not need to (or want to) manage 300 stopwatches and can likely just reuse the same stopwatch each time the user's gaze enters an object then leaves the object. As it leaves one object the very next gaze point will either fall on another object or on empty space with no object.

Have you taken a look at this sample ? It has many of the pieces for determining how long the gaze remains in a single object, with some additional logic to track the lastGazedObject you might be able to accomplish what you are after without managing a whole bunch of stopwatches or timers.

However, even if the scenario that you are trying to solve for does approach the problem like a ray cast that could be intersecting more than one object at a time (due to overlay or alignment in space) it should still be easy to have one long running stop watch and just keep track of a flag property for each object of UserIsGazingAt along with when the GazeAtStarted, then calculate the duration as soon as the gaze moves away from the object and add the duration to the objects total duration.

HTH