How to find out detected face is real or fake

Question

I am developing one security related project, there is need to check any face is detected or not, if face is detected then do some action, if face is not detected then close app.

Everything is perfect working, i am using SurfaceView which is implemented SurfaceHolder.Callback and in that open camera and camera have one method name is startFaceDetection using this method i detect face.

code for reference

public class SurfaceViewPreview extends SurfaceView implements SurfaceHolder.Callback {

    private SurfaceHolder mHolder;
    private Camera mCamera;

    public SurfaceViewPreview(Context context, AttributeSet attrs) {
        super(context, attrs);
        setWillNotDraw(false);
        mHolder = getHolder();
        mHolder.addCallback(this);
        mHolder.setType(SurfaceHolder.SURFACE_TYPE_PUSH_BUFFERS);
    }

    public void surfaceCreated(SurfaceHolder holder) {
        try {
            if (Camera.getNumberOfCameras() <= 0 || ContextCompat.checkSelfPermission(getContext(), Manifest.permission.WRITE_EXTERNAL_STORAGE)
                    != PackageManager.PERMISSION_GRANTED)
                return;

            mCamera = Camera.open(0);
            mCamera.setPreviewDisplay(mHolder);
        } catch (Exception e) {
            e.printStackTrace();
            if (this.mCamera != null) {
                this.mCamera.release();
                this.mCamera = null;
            }
        }
    }

    public void surfaceDestroyed(SurfaceHolder holder) {
        if (Camera.getNumberOfCameras() <= 0 || ContextCompat.checkSelfPermission(getContext(), Manifest.permission.WRITE_EXTERNAL_STORAGE)
                != PackageManager.PERMISSION_GRANTED)
            return;
        mCamera.stopPreview();
        mCamera.release();
        mCamera = null;
    }

    public void surfaceChanged(SurfaceHolder holder, int format, int w, int h) {
        if (Camera.getNumberOfCameras() <= 0 || ContextCompat.checkSelfPermission(getContext(), Manifest.permission.WRITE_EXTERNAL_STORAGE)
                != PackageManager.PERMISSION_GRANTED)
            return;

        mCamera.startPreview();
        mCamera.setFaceDetectionListener(new Camera.FaceDetectionListener() {
            @Override
            public void onFaceDetection(Camera.Face[] faces, Camera camera) {
               // face is detected.
            }
        });

        mCamera.startFaceDetection();
    }
}

Now, problem if any human post if i shown to camera then detected as human, but i want real human face detection not fake poster face.

Possible way to handle my requirement.

1) Capture 10 images periodically and check all variation is same then it means static face is there (like poster which is mounted in wall).

2) Write any proper algorithm which tell to detected face is real human or fake face.

3) Any library is available which is said human face is really available or not.

if anyone have idea please suggest, how to solve above issue (any code is available then share with me), response is appreciated !

how can use adapting learning ways to conclude real vs fake picture/video frame.

did you try any of your "possible ways"? How do you want to detect fake faces from video? — Micka, Aug 17 '16 at 13:33
how we can use adapting learning ways to conclude real vs fake picture/video frame. i dont want to detect from video, i want to detect from camera, continue camera will running and checking face is detected or not — Yogesh Rathi, Aug 17 '16 at 13:42
you could make pupil and iris detection and verify it in a classifier it is hard to fake the eyes in the face. I do not imagine putting few lines of code will solve this problem. You need a lot of knowledge of image analysis to reach this goal — Maytham Fahmi, Aug 17 '16 at 14:10
You could look at [Eulerian Video Magnification](https://people.csail.mit.edu/mrub/vidmag/). Quote: "Many seemingly static scenes contain subtle changes that are invisible to the naked human eye. However, it is possible to pull out these small changes from videos through the use of algorithms we have developed. We give a way to visualize these small changes by amplifying them and we present algorithms to pull out interesting signals from these videos, such as the human pulse, sound from vibrating objects and the motion of hot air." — Catree, Aug 17 '16 at 20:05
@Hulk maybe you don't want to detect from video, but this is how I interpret your need: If your face detection is trained to detect a single person from a single image, then your problem is that anyone could hold a picture of that person in front of your camera => right face is detected => your system failed. Now many solutions will try to integration information of several frames, so that still pictures/posters etc. wont fool the system. BUT: If the attacker holds a screen with a video in front of your camera, those methods won't work anymore. — Micka, Sep 02 '16 at 09:49
maybe you can combine your camera with different sensors like depth sensors, IR cameras etc. to find out, whether the object captured really is a living being with the shape of a head? — Micka, Sep 02 '16 at 09:50
@Micka two cameras is enought. It would be really hard to synchronize two videos of the same person from two predefined angles in front of two cameras — Erwan C., Sep 07 '16 at 15:43
If you have control on the flash you can try to do a flash and try to see if you get the "red eye effect" https://en.wikipedia.org/wiki/Red-eye_effect — Amitay Nachmani, Sep 07 '16 at 18:33
@YogeshRathi Did you ever find out any decent approach for the same problem? — Debasish Mitra, Jul 15 '17 at 18:33
no i did not found any approach thats why i stopped working on this project and waiting is any good solution found in future ! — Yogesh Rathi, Jul 17 '17 at 10:06

Paul · Answer 1 · 2016-09-08T11:20:26.620

1

You could use the parallax effect. First you take a 2 pictures from 2 different locations which are like 2cm apart. then you could compare the images and see:

*If they are very similar(almost same) then the image is 2d and it is a poster

*If they are very different then it is a 3d Face

Another way you could do this is by using the camera flash. The flash would cause a bit of reflection on photographs and this would prevent people from using a video to bypass your system as a screen would cause a lot of glare would would block the face preventing the camera from detecting the face. All you would need to do is add a flash(preferably blinking at like 100Hz so the people can't see it but it would show up in a picture)

I hope this helped :)

edited Sep 08 '16 at 11:20

answered Sep 08 '16 at 10:58

Paul

670
7
19

If you can add some references to that flash idea, I think it'll be better for the OP. (and myself :P) – Raskayu Sep 08 '16 at 11:04
1

But my camera is static , so how can capture from two locations – Yogesh Rathi Sep 08 '16 at 11:18
@Hulk Can it rotate by even the smallest amount? – Paul Sep 08 '16 at 11:19
@Hulk instead go with the camera flash if your camera can't rotate – Paul Sep 08 '16 at 11:20
I check camera given two much difference – Yogesh Rathi Sep 08 '16 at 12:10
not working, but bounty will be waste thats why i given to you thanks for response – Yogesh Rathi Sep 09 '16 at 04:54
@YogeshRathi have u solved it i am facing the same issue.. i am using google vision api. – Gaju Kollur Oct 15 '18 at 05:04
@YogeshRathi i don't know about camera api.. but what happens when u show image of person to the camera..it will detect face and may give count right..? – Gaju Kollur Oct 15 '18 at 06:01
@GajuKollur Camera API solution not works, Do not try. You can start a bounty for this question. May be any expert will give you answer. – Yogesh Rathi Oct 15 '18 at 06:02
@YogeshRathi i searched lot regarding this if u can ans here https://stackoverflow.com/questions/52549743/google-vision-api-how-to-detect-face-detected-is-either-an-image-face-or-real-l – Gaju Kollur Oct 15 '18 at 06:05
@YogeshRathi https://stackoverflow.com/questions/52776233/how-to-detect-true-faces-or-live-human-faces-using-google-vision-api-in-android – Gaju Kollur Oct 15 '18 at 06:06
1

@GajuKollur but you need to add some hack logic for that, Like capture 2 images after 5 seconds and check detection properties is same then may be its image. But its not proper logic. – Yogesh Rathi Oct 15 '18 at 06:07

score 0 · Answer 2 · answered Jun 12 '19 at 10:24

I had a challenge solving problems similiar the one @YogeshRathi. I had an algorithm with the CV2 library (Python) recognizing faces taken from a security camera. Every 5 seconds I took pictures, and the algorithm recognized faces in the poster hanged on the wall.

After test different solutions (other algorithms, train models...) what I did finally was generate a buffer, in which there were 5 pictures always, one in and another out. The one entering to the buffer do it with the list of coordinates of all rectangles that contains a face (5 faces in the pictures --> 5 rectangles), and compare with the rest of the pictures inside the buffer. The comparison of pictures consists in compare the rectangles (every rectangle has 4 coordinates) between both pictures, by the substraction of every single coordinate. If the rectangle is static (a face in a poster have almost the same rectangle in different pictures) the difference between both rectangles is simbolic, so then, unless they have different number of rectangles, if all the rectangles in both pictures have simbolic differences they are similar.

If in a picture appears a real person we will have different number of rectangles (the number of faces inside the poster and the one that belongs to the real person), o at least one of them will be different to the list of rectangles of the picture is being compared with. If the rectangles in both pictures are similar, I put a flag in a Historial field, that is 0. If there are different rectangles, the flag is 1.

You compare the picture that enters the buffer one to one to the rest of the pictures in the buffer. So when you finish, you have a list of flags (like this [0,0,0,1,1]) attached to every picture.

When the picture go out of the buffer you evaluate the Historial field. If a 0 is contained in the list it means that at least there are a picture that is identical, so then you can consider there are no faces on it you have to identify, because all it contains only fake faces from a poster.

@fluente hi, thanks for describing your method, is there a way to contact you ? Also could you please explain why you measuring the rectangles of those 5 buffer images ? If I got it right the 5 images of the life face will have different rectangle yea but fake images should have approximately same size of rectangles, am I right to say that ? but then what if you will change the distance of the printed photo by moving it forward and backward ? — hxdef, Jun 16 '19 at 12:14
Thnx @hxdef for your answer. In my case, I take pictures from a security camera, fixed without movements. So then it recognize faces from a poster hanged on a wall that in every picture, the rectangles have similar coordinates. When a pict is going to enter the buffer, you compare the coordinates of every rectangle in the pict candidate with every rectangle in the buffer. With small differences, if the coordinates are very similar, the rectangle belongs to a static face (you could say it´s press in a poster). If you need further info send me an email to fernando.delafuentemunoztech@gmail.com — ffuente, Jun 17 '19 at 15:02
Answering your question @hxdef, the buffer only works when the pictures are taken from the same point. If you change the distance moving forward and backward the coordinates of a static object will change, obviously, and you will have to look for another solution. — ffuente, Jun 17 '19 at 15:06
Oh your method applies to poster only. I thought it can be applied for real faces also( but do u know any other solutions to prevent photo Attack and video Attack ? — hxdef, Jun 18 '19 at 03:25

How to find out detected face is real or fake

2 Answers2

Linked