How to step frame-by-frame without skipping any, with HLS playback h.264 mpeg-ts across browsers?

Question

I don't seem to be able to get to certain frames like 00:00:04:24 with

// javascript
videojs("player").currentTime(4.96); // for 25 fps video

And this is happening at every second of any video with closed GOP length 25.

All I wanted to achieve was to step frame-by-frame (without skipping any) through the video when my users feel the need to.

Things I'm using:

• videojs 5.11.9 / 5.19.1
• videojs-contrib-hls 4.0.3 / 5.4.1
• videojs-framebyframe
• Chrome 57.0.2987.133 (64-bit) / Firefox 51.0.1 (32-bit)
• Windows 7 (64-bit)

Details (Chrome)

I have the timecode overlaid on the video I'm working with. With luck, I could pause the video at the 'missing' frames, meaning the frames are there. For example, 00:00:03:24. If I call currentTime() now

videojs("player").currentTime(); // returns **4.002128**

Then if I call

videojs("player").currentTime(4.002128);

That brings me to 00:00:04:00. The same goes with all of these calls which stayed at 00:00:04:00.

videojs("player").currentTime(4);    // should be **00:00:03:23**
videojs("player").currentTime(4.04); // should be **00:00:03:24**
videojs("player").currentTime(4.08); // should be **00:00:04:00**, correct

But this

videojs("player").currentTime(3.999999999999999);

will bring me to 00:00:03:23, effectively skipping 00:00:03:24 entirely.

Curiously enough, when I changed the GOP length to 50, the same thing happens only every 2 seconds. Meaning I could jump to 00:00:02:24, 00:00:04:24, but still not 00:00:03:24.

The layout of a closed GOP length 25 (GOP M=2, N=25) looks like this for every second:

IBPBPBPBPBPBPBPBPBPBPBPBP

and for a closed GOP length 50 (GOP M=2, N=50) looks like this for every second:

IBPBPBPBPBPBPBPBPBPBPBPBPBPBPBPBPBPBPBPBPBPBPBPBPP

Which brings me to suspect the P-frame at the end of each GOP is acting up.

With limited knowledge of how truly GOP works myself, I searched for other possible layout of the I/P/B frames. Then I saw open GOP. Using ffmpeg I re-encode the video with -flags -cgop as below

open GOP M=5, N=25

IBBBBPBBBBPBBBBPBBBBPBBBB

or open GOP M=2, N=50

IBPBPBPBPBPBPBPBPBPBPBPBPBPBPBPBPBPBPBPBPBPBPBPBPB

Both of these open GOP videos have no problem jumping to any frame. With one other problem...

Problem with the open GOP 'solution'

The HLS playlist, instead of only downloading the required/playing portion of the video, now tries to download one clip after another right from the start. Maybe because now the B-frame at the end of each GOP asks for the next GOP?

Searching for more

While typing out this question, I searched for more and came across this post at bugs.chromium.org. People in that post suggested 2 things:

1. to re-encode the video w/o B-frames for Chrome
2. IE11, Safari8, Firefox34 back then seek frames accurately

Here goes testing the first suggestion.

closed GOP M=1, N=25

IPPPPPPPPPPPPPPPPPPPPPPPP

and closed GOP M=1, N=50

IPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPP

Both worked! (not so fast, doesn't work on Firefox)

Details (Firefox)

Remember the second suggestion from that post at bugs.chromium.org? At first, when I tried Firefox with any GOP length 25 video, they were all good (at seeking frame accurately). But it has similar problem as Chrome when dealing with GOP length 50 (both when M=1 and M=2) and GOP length 25 (M=1).

videojs("player").currentTime(1.96); // seeks to **00:00:01:24** correctly
videojs("player").currentTime(2.04); // seeks to **00:00:02:01** correctly

but this

videojs("player").currentTime(2); // should be **00:00:02:00**

gave me 00:00:02:00 or 00:00:02:01 randomly (biased towards the latter, while repeatedly calling only this line).

One easier way to spot this is to continuously step 25 frames ahead with videojs-framebyframe. You would get 00:00:02:01, 00:00:03:01, 00:00:04:01, and then suddenly a 00:00:05:00. Stop there and step 1-frame back and forth and you might not be able to get back to 00:00:05:00 every time.

edited portion starts

---

More searching data before the questions

Came across this other question. It's almost two years old. With a still working downloadable sample video of variable GOP length.

Its layout (first few GOPs)

IBPBPBPBPPPPPPPPPP
IPPP
IPPBPBPBPPBPPPPPPPP
IBPPPPPBPP
IPBPPPP
IBPPPPPPBPPBPPBPBPPPPBPBP

Here's what I use to show the layout of frame types

// command prompt or bash
ffprobe -show_frames -select_streams 0 video.ts | awk -f iframe.awk

and here's the small awk program (iframe.awk)

BEGIN {
    FS="=";i="";
}
/pict_type/{
    if (match($2, "I")) {
        print i;
        i=$2;
    } else {
        i=i$2;
    }
}
END {
    print i;
}

---

edited portion ends

Questions

Sorry for the long post, and thanks for reading it through.

1. Does this mean there are things needed to be fixed? (bugs?)
2. Did videojs play a part in creating this bug?
3. Is writing our own player going to help?
4. With current browsers/scripts releases, is it possible to seek frame-by-frame with HTML5 videojs HLS player across browsers (well, Chrome and Firefox at least) with one format?
5. (added question) Any agrees or dis-agrees, at all?
6. (added question) Is this a bad question as a whole? Too long?

Appreciate any sort of help and/or pointers to solve the problem! Thanks!