3

I'm working on an AR makeup application, using google's mediapipe technology, with attention mesh.

For most things it works really well.

However, I'm finding that the texture coordinates are really cramped together around the lips and teeth. This makes it very hard to create a lipstick effect, as the lip area seems to spill onto the teeth in many practical examples.

Here is a visualization of the mediapipe face mesh. So, you can see why it's really hard to make a lipstick effect - if you stay within the visible "lip" bounds, it basically looks like you painted the lipstick onto half of the lip, which isn't very good. Ideally, we would feather the edges of the lipstick, but you can't really do that on the part that goes over the teeth, since it's around 4px height total, the texture gets stretched over the polygons inside the teeth.

Has anyone been able to successfully achieve a lipstick effect on mediapipe attention mesh?

enter image description here

Dennis L
  • 1,713
  • 14
  • 21
  • One possible solution to improve the texturing of mediapipe 3D face mesh for your AR makeup application could be to use a technique called UV unwrapping. This technique involves unwrapping the 3D mesh into a 2D plane and creating a texture map that can be used to paint the texture onto the surface of the 3D mesh. – Noor Fahad Apr 19 '23 at 05:50
  • @NoorFahad This sounds quite suitable actually, I was wondering if such a technique existed. Do you have experience implementing this yourself? – Dennis L Apr 19 '23 at 20:02
  • I don't have much experience with that. I've only learned about it in my past free time. – Noor Fahad Apr 20 '23 at 04:16

0 Answers0