I'm trying to run the face effect demo to fit a mask or eyeglasses on my face. I have built a graph by command below:
bazel build -c opt --copt -DMESA_EGL_NO_X11_HEADERS --copt -DEGL_NO_X11 mediapipe/examples/desktop/_face_effect:face_effect_gpu --verbose_failures
And trying to run the graph with the command below:
GLOG_logtostderr=1 bazel-bin/mediapipe/examples/desktop/_face_effect/face_effect_gpu --calculator_graph_config_file=mediapipe/graphs/face_effect/face_effect_gpu.pbtxt
Also my BUILD is as follows:
licenses(["notice"])
package(default_visibility = ["//mediapipe/examples:__subpackages__"])
cc_binary(
name = "face_effect_gpu",
deps = [
"//mediapipe/examples/desktop:demo_run_graph_main_gpu",
"//mediapipe/graphs/face_effect:face_effect_gpu_deps",
],
)
Finally log is below:
I20220725 20:54:06.296331 358140 demo_run_graph_main_gpu.cc:52] Get calculator graph config contents: # MediaPipe graph that applies a face effect to the input video stream.
# GPU buffer. (GpuBuffer)
input_stream: "input_video"
# An integer, which indicate which effect is selected. (int)
#
# If `selected_effect_id` is `0`, the Axis effect is selected.
# If `selected_effect_id` is `1`, the Facepaint effect is selected.
# If `selected_effect_id` is `2`, the Glasses effect is selected.
#
# No other values are allowed for `selected_effect_id`.
input_stream: "selected_effect_id"
# Indicates whether to use the face detection as the input source. (bool)
#
# If `true`, the face detection pipeline will be used to produce landmarks.
# If `false`, the face landmark pipeline will be used to produce landmarks.
input_side_packet: "use_face_detection_input_source"
# Output image with rendered results. (GpuBuffer)
output_stream: "output_video"
# A list of geometry data for a single detected face.
#
# NOTE: there will not be an output packet in this stream for this particular
# timestamp if none of faces detected.
#
# (std::vector<face_geometry::FaceGeometry>)
output_stream: "multi_face_geometry"
# Throttles the images flowing downstream for flow control. It passes through
# the very first incoming image unaltered, and waits for downstream nodes
# (calculators and subgraphs) in the graph to finish their tasks before it
# passes through another image. All images that come in while waiting are
# dropped, limiting the number of in-flight images in most part of the graph to
# 1. This prevents the downstream nodes from queuing up incoming images and data
# excessively, which leads to increased latency and memory usage, unwanted in
# real-time mobile applications. It also eliminates unnecessarily computation,
# e.g., the output produced by a node may get dropped downstream if the
# subsequent nodes are still busy processing previous inputs.
node {
calculator: "FlowLimiterCalculator"
input_stream: "input_video"
input_stream: "FINISHED:output_video"
input_stream_info: {
tag_index: "FINISHED"
back_edge: true
}
output_stream: "throttled_input_video"
}
# Generates an environment that describes the current virtual scene.
node {
calculator: "FaceGeometryEnvGeneratorCalculator"
output_side_packet: "ENVIRONMENT:environment"
node_options: {
[type.googleapis.com/mediapipe.FaceGeometryEnvGeneratorCalculatorOptions] {
environment: {
origin_point_location: TOP_LEFT_CORNER
perspective_camera: {
vertical_fov_degrees: 63.0 # 63 degrees
near: 1.0 # 1cm
far: 10000.0 # 100m
}
}
}
}
}
# Computes the face geometry for a single face. The input source is defined
# through `use_face_detection_input_source`.
node {
calculator: "SwitchContainer"
input_stream: "IMAGE:throttled_input_video"
input_side_packet: "ENABLE:use_face_detection_input_source"
input_side_packet: "ENVIRONMENT:environment"
output_stream: "MULTI_FACE_GEOMETRY:multi_face_geometry"
node_options: {
[type.googleapis.com/mediapipe.SwitchContainerOptions] {
contained_node: {
calculator: "SingleFaceGeometryFromLandmarksGpu"
}
contained_node: {
calculator: "SingleFaceGeometryFromDetectionGpu"
}
}
}
}
# Renders the selected effect based on `selected_effect_id`.
node {
calculator: "SwitchContainer"
input_stream: "SELECT:selected_effect_id"
input_stream: "IMAGE_GPU:throttled_input_video"
input_stream: "MULTI_FACE_GEOMETRY:multi_face_geometry"
input_side_packet: "ENVIRONMENT:environment"
output_stream: "IMAGE_GPU:output_video"
node_options: {
[type.googleapis.com/mediapipe.SwitchContainerOptions] {
contained_node: {
calculator: "FaceGeometryEffectRendererCalculator"
node_options: {
[type.googleapis.com/mediapipe.FaceGeometryEffectRendererCalculatorOptions] {
effect_texture_path: "mediapipe/graphs/face_effect/data/axis.pngblob"
effect_mesh_3d_path: "mediapipe/graphs/face_effect/data/axis.binarypb"
}
}
}
contained_node: {
calculator: "FaceGeometryEffectRendererCalculator"
node_options: {
[type.googleapis.com/mediapipe.FaceGeometryEffectRendererCalculatorOptions] {
effect_texture_path: "mediapipe/graphs/face_effect/data/facepaint.pngblob"
}
}
}
contained_node: {
calculator: "FaceGeometryEffectRendererCalculator"
node_options: {
[type.googleapis.com/mediapipe.FaceGeometryEffectRendererCalculatorOptions] {
effect_texture_path: "mediapipe/graphs/face_effect/data/glasses.pngblob"
effect_mesh_3d_path: "mediapipe/graphs/face_effect/data/glasses.binarypb"
}
}
}
}
}
}
I20220725 20:54:06.297303 358140 demo_run_graph_main_gpu.cc:58] Initialize the calculator graph.
I20220725 20:54:06.304211 358140 demo_run_graph_main_gpu.cc:62] Initialize the GPU.
I20220725 20:54:06.317529 358140 gl_context_egl.cc:84] Successfully initialized EGL. Major : 1 Minor: 5
I20220725 20:54:06.374799 358149 gl_context.cc:335] GL version: 3.2 (OpenGL ES 3.2 NVIDIA 465.19.01)
I20220725 20:54:06.375151 358140 demo_run_graph_main_gpu.cc:68] Initialize the camera or load the video.
[ WARN:0@0.702] OpenCV | GStreamer warning: Cannot query video position: status=0, value=-1, duration=-1
I20220725 20:54:08.345331 358140 demo_run_graph_main_gpu.cc:89] Start running the calculator graph.
I20220725 20:54:08.349308 358140 demo_run_graph_main_gpu.cc:94] Start grabbing and processing frames.
F20220725 20:54:08.349412 358144 collection.h:417] Check failed: id < EndId() (1 vs. 1)
*** Check failure stack trace: ***
@ 0x55e6ef943a8e google::LogMessage::SendToLog()
@ 0x55e6ef941eea google::LogMessage::Flush()
@ 0x55e6ef943f8d google::LogMessageFatal::~LogMessageFatal()
@ 0x55e6ef86b8f7 mediapipe::internal::Collection<>::Get()
@ 0x55e6ef894348 mediapipe::InputSidePacketHandler::SetInternal()
@ 0x55e6ef894ca7 mediapipe::InputSidePacketHandler::Set()
@ 0x55e6ef8935a6 mediapipe::OutputSidePacketImpl::SetInternal()
@ 0x55e6ef8937a5 mediapipe::OutputSidePacketImpl::Set()
@ 0x55e6ef8496b2 mediapipe::(anonymous namespace)::EnvGeneratorCalculator::Open()
@ 0x55e6ef88ca8c mediapipe::CalculatorNode::OpenNode()
@ 0x55e6ef87193b mediapipe::internal::SchedulerQueue::OpenCalculatorNode()
@ 0x55e6ef871c3b mediapipe::internal::SchedulerQueue::RunNextTask()
@ 0x55e6ef8ae9e7 mediapipe::ThreadPool::RunWorker()
@ 0x55e6ef8aefc3 mediapipe::ThreadPool::WorkerThread::ThreadBody()
@ 0x7f6fd39c1450 start_thread
@ 0x7f6fd2fbcd53 clone
@ (nil) (unknown)
It looks like that execution crashes at line 99 of demo_run_graph_main_gpu.cc.
capture >> camera_frame_raw;
It looks like that the error was raised at line 417 of mediapipe::collection.h:
CHECK_LT(id, EndId());
When I comment the line, I'll get this error in the log:
I20230413 18:48:45.558230 6606 demo_run_graph_main_gpu.cc:59] Initialize the calculator graph.
I20230413 18:48:45.566534 6606 demo_run_graph_main_gpu.cc:63] Initialize the GPU.
I20230413 18:48:45.578886 6606 gl_context_egl.cc:84] Successfully initialized EGL. Major : 1 Minor: 5
I20230413 18:48:45.739595 6620 gl_context.cc:342] GL version: 3.2 (OpenGL ES 3.2 NVIDIA 525.60.11)
I20230413 18:48:45.739742 6606 demo_run_graph_main_gpu.cc:69] Initialize the camera or load the video.
I20230413 18:48:46.019868 6606 demo_run_graph_main_gpu.cc:90] Start running the calculator graph.
I20230413 18:48:46.023504 6606 demo_run_graph_main_gpu.cc:95] Start grabbing and processing frames.
E20230413 18:48:47.583186 6606 demo_run_graph_main_gpu.cc:201] Failed to run the graph: Graph has errors:
; Input side packet with id 1 was already set.
I found out that there is already an issue on GitHub.
