3

I'm working on a Roku and TVOS app that is going to play HLS videos (VOD and live) as well as MP4. According to the Roku docs EIA-608 is supported on both and should also work on TVOS.

My question is, given URL to the m3u8 how can I tell what specific format (EIA-608,WebVTT etc) of closed captioning is being used in each stream?

Contents of the main m3u8 (note 1st stream says no CC, but it really does have it):

#EXTM3U
#EXT-X-STREAM-INF:PROGRAM-ID=1,BANDWIDTH=380000,RESOLUTION=400x228,CODECS="avc1.66.30, mp4a.40.2",CLOSED-CAPTIONS=NONE
http://d.com/i/video/2426832/2426832_,350,640,1000,2000,.mp4.csmil/index_0_av.m3u8
#EXT-X-STREAM-INF:PROGRAM-ID=1,BANDWIDTH=750000,RESOLUTION=640x360,CODECS="avc1.77.30, mp4a.40.2",CLOSED-CAPTIONS=NONE
http://d.com/i/video/2426832/2426832_,350,640,1000,2000,.mp4.csmil/index_1_av.m3u8
#EXT-X-STREAM-INF:PROGRAM-ID=1,BANDWIDTH=1044000,RESOLUTION=1280x720,CODECS="avc1.64001f, mp4a.40.2",CLOSED-CAPTIONS=NONE
http://d.com/i/video/2426832/2426832_,350,640,1000,2000,.mp4.csmil/index_2_av.m3u8
#EXT-X-STREAM-INF:PROGRAM-ID=1,BANDWIDTH=2127000,RESOLUTION=1280x720,CODECS="avc1.64001f, mp4a.40.2",CLOSED-CAPTIONS=NONE
http://d.com/i/video/2426832/2426832_,350,640,1000,2000,.mp4.csmil/index_3_av.m3u8

Contents of the 1st stream's m3u8

#EXTM3U
#EXT-X-TARGETDURATION:4
#EXT-X-ALLOW-CACHE:YES
#EXT-X-PLAYLIST-TYPE:VOD
#EXT-X-VERSION:3
#EXT-X-MEDIA-SEQUENCE:1
#EXTINF:4.000,
http://d.com/i/video/2426832/2426832_,350,640,1000,2000,.mp4.csmil/segment1_0_av.ts
...

I can use ffprobe -hide_banner to show the 1st program's stream has closed captioning. Ex:

Duration: 00:02:36.76, start: 0.100511, bitrate: 0 kb/s
  Program 0
    Metadata:
      variant_bitrate : 380000
    Stream #0:0: Video: h264 (Constrained Baseline) ([27][0][0][0] / 0x001B), yuv420p, 400x228 [SAR 1:1 DAR 100:57], Closed Captions, 29.97 fps, 29.97 tbr, 90k tbn, 59.94 tbc
    Metadata:
      variant_bitrate : 380000
    Stream #0:1: Audio: aac (LC) ([15][0][0][0] / 0x000F), 44100 Hz, mono, fltp, 48 kb/s
    Metadata:
      variant_bitrate : 380000

However, as you can see, Program 0 > Stream 0 just says that is has Closed captions - it does not list the type/spec of closed captioning technology being used.

How do I display the format of the Closed Captions?

rynop
  • 50,086
  • 26
  • 101
  • 112

1 Answers1

3

WebVTT is what is know as a 'side car', or 'out of band' format. Meaning captions are in a separate file that you must download. You can see the URL for this file in the m3u8. Here the caption are part of the video stream itself. The only supported format in this case is EIA-608. ffmpeg support for 608 is pretty limited. The best tool I know of to for dealing with 608 is libcaption (full disclosure, I wrote it). I recently added a ts2srt example program. Fair warning its still sorta betaish.

szatmary
  • 29,969
  • 8
  • 44
  • 57
  • Thanks. If my example above is EIA-608 why would captions show on Roku for this video but not for TVOS? TVOS claims EIA-608 compat. Any idea? – rynop Dec 06 '16 at 16:48
  • Did you set the caption format in the manifest? It's documented on apples site. – szatmary Dec 06 '16 at 16:49
  • do you have a link to the doc page ur referring to? We are not seeing anything... – rynop Dec 06 '16 at 19:12
  • Really? typing `HLS captions` into google gives it to me in the first result ?! – szatmary Dec 06 '16 at 19:15
  • 1
    Oh you were referring to the manifest of the video file its self. Thought you were referring to the tvos plist file - which is a manifest file in itself (akin to the Roku manifest). I'm not making the video so I have no control over its manifest - hence my original question - I want to determine exact caption format/spec from existing HLS URL. – rynop Dec 06 '16 at 19:22
  • I'm not sure what you mean by HLS URL. HLS has (M*N)+(N+1) urls where N is the number of renditions, and M is the duration of the video divided by segment duration. If you mean without the manifest, the answer is, You can't unless the player exposes an API. In which case, you need to ask that question separately. – szatmary Dec 06 '16 at 19:27
  • Sorry, to clarify I mean the URL to the m3u8. I've updated the orig q to clarify. – rynop Dec 06 '16 at 19:39
  • Now I'm confused. You download the m3u8, and look for `#EXT-X-MEDIA:TYPE=CLOSED-CAPTIONS` . The documentation is on apples site. – szatmary Dec 06 '16 at 19:41
  • My main m3u8 says `,CLOSED-CAPTIONS=NONE`. The 1st streams m3u8 URL within the main mainfest does not contain `#EXT-X-MEDIA:TYPE=CLOSED-CAPTIONS`. This video displays CC on Roku but not on tvOS. Q updated with m3u8 contents. – rynop Dec 06 '16 at 20:25
  • So, what we learned is that roku will ignore what the manifest said if the ts had inband captions, but tvos will not. – szatmary Dec 06 '16 at 20:44
  • yup. Is there a way to extract the format of closed captioning being used given the .ts URL? Ran ffprobe on it and it just says `Closed Captions` but not the format of the CC. Or is your ts2srt the tool to do that? – rynop Dec 06 '16 at 21:37
  • Did you read my answer? `The ONLY supported format in this case is EIA-608.` – szatmary Dec 06 '16 at 21:44