2

I'm working on a project that uses the Vision Framework to detect faces in images and then uses a CoreML model to detect if the face is smiling. The problem is that the CoreML model file is nearly 500 MB. I don't want to bloat my app that much.

Since I'm already getting the VNFaceLandmarks2D observation data from the Vision framework, I thought I would try to use that to detect smiles.

Has anyone tried to use the VNFaceLandmarks2D data from the Vision framework to try to determine if the face is happy or smiling?

If so, how did you do it and how well did it work?

Thanks!

steffen
  • 2,152
  • 4
  • 19
  • 30
HansG600
  • 260
  • 4
  • 12
  • Did you get to do this? I'm trying to detect smiles and happy faces with Vision landmarks too but I'm not know how to make this. If you got it, please, share with me :) – Izabella Melo Feb 09 '22 at 20:24

2 Answers2

2

One solution is to use a smaller Core ML model file. It sounds like you're using a model that is based on VGGNet. There are several much smaller model architectures (between 4 and 16 MB) that have the same accuracy as VGGNet, and are therefore better suited for use on mobile devices.

Matthijs Hollemans
  • 7,706
  • 2
  • 16
  • 23
  • Can you tell me which models you are thinking of? Thanks! – HansG600 Dec 07 '17 at 18:54
  • Two commonly used ones are MobileNet and SqueezeNet. But you need to re-train these to make them detect faces. It sounds like your original model uses VGGNet, so you'd replace those VGGNet layers with MobileNet or SqueezeNet and then train the model from scratch again. – Matthijs Hollemans Dec 08 '17 at 09:47
0

This is better, small, fast, enjoyable:

The CIFaceFeature detection allows you to detect close eyes and smiles. https://developer.apple.com/documentation/coreimage/cifacefeature

Mikael
  • 2,355
  • 1
  • 21
  • 45