Pixel Order for Affdex-SDK's FrameDetector?

Question

I'm working on writing a Windows emotion sensitive game in Unity3D using the affdex SDK by Affectiva. The CameraDetector finds a face consistently. When I use the FrameDetector a face is rarely found. Intense lighting seems to help, but even when a face is found it seems to be detecting a smile when I frown. I'm getting the pixels from WebCamTexture.GetPixels32. The pixels are ordered Left to right and Bottom to top (just like Windows BitMaps).

    public void ProcessFrame(Frame frame)
    {
        if (!_initialized)
        {
            Initialize(false);
        }

        byte[] bytes = new byte[frame.rgba.Length * 3];

        for(int i = 0, idx=0; i < frame.rgba.Length; i++, idx+=3)
        {

            bytes[idx] = frame.rgba[i].b;
            bytes[idx+1] = frame.rgba[i].g;
            bytes[idx+2] = frame.rgba[i].r;

        }

        nativePlatform.ProcessFrame(bytes, frame.w, frame.h, frame.timestamp);
    }

I read through the Affectiva documentation, but couldn't find anything about pixel order.

You're the devops lead at Affectiva, but when you can't find something in your own documentation you come and ask a question here? That hardly seems credible! — Mogsdad, Nov 07 '15 at 04:12
This was a legit issue I came across. I was asked by co-workers to post it here for two reasons 1) because @j4y wanted to add the affdex-sdk tag 2) so people knew we were answering questions on SO. — Forest J. Handford, Nov 07 '15 at 16:34

score 1 · Accepted Answer · answered Nov 05 '15 at 20:18

The Frame class expects the pixel order to be left-right, top to bottom. I.e the first pixel in the array is the top left corner of the image.

The FrameDetector expects the facial images to have the face upright in the image. Otherwise the face tracker will have trouble locking onto the face.

Pixel Order for Affdex-SDK's FrameDetector?

1 Answers1