2

I have to highlight a word in video captions when user taps on it. So, given the coordinates of a tap the problem is to find the word index in a cue string.

Any ideas? Is it's possible at all? I am talking about the newest HTML5 touch events and WebVTT cues https://www.w3.org/TR/webvtt1/.

Agent Coop
  • 392
  • 4
  • 12
  • 1
    The first problem would be to get where this cue text is really. There doesn't seem to be any way of knowing it from the VTTCue API (a `getBoundingClientRect` would have helped a lot) The workaround I can think of would be to implement yourself the subtitle display... – Kaiido Jul 29 '17 at 09:04
  • It does not appear that a `TextTrack` supports the `click` event, but you can access `activeCues` https://developer.mozilla.org/en-US/docs/Web/API/TextTrack I don't think what you want is currently possible, but you could try combining your video with the Canvas API to get the click location, similar to this in concept: https://developer.mozilla.org/en-US/docs/Web/API/Canvas_API/Manipulating_video_using_canvas Or just use Canvas for the whole video player. – JHS Jul 29 '17 at 09:09
  • @JHS, why use the canvas to get the click event ? You can get it from the video directly, or from any visual element which could be placed on top of the video. Also, subtitles are not drawn on a canvas. – Kaiido Jul 29 '17 at 09:17

1 Answers1

3

I don't think this is possible with traditional WebVTT cues - they are pseudo-elements, which are not directly part of the DOM, so you can't bind events to them. Styling is also extremely limited for ::cues.

However, you should be able to leverage TextTrack events to accomplish something that works in a similar way. You can bind a custom function to the video track's oncuechange event, and then use the track's activeCues to generate your own captions. This custom div can then be styled and have whatever events on it that you want.

This will grab the first text track from your video, and get the text from the currently active cue every time a cue change occurs.

$('video')[0].textTracks[0].oncuechange = function() { 
    var currentCue = this.activeCues[0].text;
    // add current cue text to custom caption div
}

You will probably need to parse each word of the cue into its own span so you can add events to it, add highlight classes, etc. Then you can style/interact with each piece however you'd like.

https://developer.mozilla.org/en-US/docs/Web/API/TextTrack

Taylor Newton
  • 411
  • 7
  • 10