0

I would like to have some javascript code modify some properties of the element that is being played as part of a media overlay (smil driven). In particular, I am interested in text media overlays, which get highlighted as some audio is played. My question is two-fold:

  • Is there a way to find out whether media overlays are "active"? some call back functionality would be best, else I could create a timer and keep checking the status from the js.
  • Once I know (if possible) that there is an element being highlighted, is there a way to know which SMIL ID this one is?
XAnguera
  • 1,157
  • 1
  • 11
  • 25

1 Answers1

1

The EPUB 3 specification allows JavaScript contents embedded inside an EPUB container to "communicate" with the reading application only via the epubReadingSystem object.

Unfortunately, the specification requires the exposure of basic fields only, like the name and the version of the Reading System, and probes for its scripting capabilities (hasFeature). See: http://www.idpf.org/epub/301/spec/epub-contentdocs.html#app-epubReadingSystem

In particular, there is no IDPF-anointed way to get the state of the Media Overlays (not started/playing/pausing, which fragment is active, etc.) exposed to JS embedded inside the EPUB.

Of course, in theory one might find an "hack" for a specific reading system (especially if it relies on JS for the GUI, like Readium or iBooks), and be able to route those pieces of information to the JS code inside the EPUB (and rendered by the Reading System). But of course this approach is brittle --- as you know from my previous observations about my own rb_smil_emulator.js.

Another possibility is to just monitor the DOM, keeping track of the changes in classes applied to all the SMIL fragments, but this technique might prove problematic in terms of timing performance/accuracy.

Alberto Pettarin
  • 894
  • 6
  • 12
  • Would you mind extending a bit more on how this could be done in readium and ibooks? – XAnguera Apr 09 '15 at 14:55
  • I wrote "in theory" and "might", based on previous "reverse engineering" experiences (e.g., http://epubsecrets.com/what-ibooks-does-behind-the-scenes.php ), but I have no concrete suggestions on how to do the same for MO playback with the current implementations. Let me note that Media Overlays (in future, even with multiple granularity) are meant to be declarative, not behavioral/JS-backed, so not sure how long-term this investigation will be. Hence, playing multiple MO granularities should be an app feature (backed by proper epub:type tagging of the SMIL file), not an ebook "feature". – Alberto Pettarin Apr 10 '15 at 07:51
  • Anyway, you can lobby for richer `epubReadingSystem` APIs at the IDPF forum: http://idpf.org/forums or on the EPUB spec revision group: https://github.com/IDPF/epub-revision/issues – Alberto Pettarin Apr 10 '15 at 07:55
  • Thanks @Alberto-pettarin for your detailed answer. I am marking the answer as correct. I did not know about the Ibooks desktop web inspector. This tool definitely males things easier for the Ibooks. I wish other platforms would offer something similar. – XAnguera Apr 10 '15 at 09:37