1

My use case is that my program programmatically captures screen recording videos from computers or attached mobile devices, e.g.:

  • ffmpeg -f avfoundation ...
  • adb shell screenrecord ...
  • xcrun simctl io booted recordVideo ...
  • any other similar command.

How can I map the very first frame of a video to the host computer time?

The reason I'm asking is because I can notice a certain delay between these two points: when the command (to record) starts, and when the video itself starts being recorded.

Maybe it is a small delay, but I need to have a precision of ±1ms. Also I'm afraid of dropped frames, and that they might ruin this precision.

I need such a precision to synchronise thousands of quick rushing application log messages (trace mode) with the screen recording, and any inaccuracy might harm the usability of this feature.

Is there any option to inject timestamps into the keyframes, so that at least, if I know the exact time difference between the computer and the remote device, I can read them and map every frame, existing or dropped, to the world clock?

noomorph
  • 837
  • 1
  • 6
  • 14

1 Answers1

0

So, I conducted an investigation while waiting for the answers, and it turns out that this mission is impossible.

Turns out that the screen recorders do not embed milliseconds into the MDPM metadata. I found a way to check that via exiftool:

exiftool -d '%H:%M:%S%3f' -s -ee screenrecording.mp4 | grep Date
FileModifyDate                  : 09:26:45.000
FileAccessDate                  : 09:26:45.000
FileInodeChangeDate             : 09:29:22.000
CreateDate                      : 07:26:33.000
ModifyDate                      : 07:26:33.000
TrackCreateDate                 : 07:26:33.000
TrackModifyDate                 : 07:26:33.000
MediaCreateDate                 : 07:26:33.000
MediaModifyDate                 : 07:26:33.000
TrackCreateDate                 : 07:26:33.000
TrackModifyDate                 : 07:26:33.000
MediaCreateDate                 : 07:26:33.000
MediaModifyDate                 : 07:26:33.000

So, unfortunately, this is not possible at the moment.

noomorph
  • 837
  • 1
  • 6
  • 14
  • To solve this you have to involve a programming language. You must get the first frame back into your app. For example FFmpeg (when running as a **process** by your code) can send its encoding output to **std in/out** (instead of a file) where you could the edit the given bytes array (by adding extra bytes of time/date) at the end of a keyframe. To edit means it is enough to just attach a **string** at end of actual frame bytes. See if [this Answer](https://stackoverflow.com/a/74311088/2057709) helps you to get some ideas – VC.One Nov 28 '22 at 14:12
  • @VC.One thanks for the tip. Generally, it is helpful, but specifically - not so much, because it's not likely I can modify Apple's command line tools or official Android emulator images to add these changes. Regardless, I think your idea can be effective in certain cases. Thanks for sharing!! – noomorph Dec 02 '22 at 13:51