1



I have ported Cobalt8 code onto our embedded system, and it is now able to decode VP9 up to 4k quality.


However, I ran into an issue with fast forward and rewind.
Specially, when I fast forward a few times and then followed by a rewind operation, there is a chance that audio or video streaming data will stop coming into the range buffer. I am not familiar with the streaming mechanism. It would be great if someone can shed some light on where or what I can look to debug this issue.


PS: I have drawn a quick picture to show the problem.

thanks.

forward and rewind

user7160805
  • 91
  • 7

1 Answers1

0

Is it possible for you to tell me the values of the following settings in your configuration_public.h:

SB_MEDIA_SOURCE_BUFFER_STREAM_AUDIO_MEMORY_LIMIT SB_MEDIA_SOURCE_BUFFER_STREAM_VIDEO_MEMORY_LIMIT SB_MEDIA_MAIN_BUFFER_BUDGET

xiaoming
  • 101
  • 2
  • below is our setting, #define SB_MEDIA_SOURCE_BUFFER_STREAM_AUDIO_MEMORY_LIMIT (16U * 1024U * 1024U), #define SB_MEDIA_SOURCE_BUFFER_STREAM_VIDEO_MEMORY_LIMIT (128U * 1024U * 1024U), #define SB_MEDIA_MAIN_BUFFER_BUDGET (256U * 1024U * 1024U) – user7160805 Mar 17 '17 at 15:27
  • the testing content is 4K. we had traced source code, cobalt got media source but none call MediaSource::Append(). could someone help us to figure which point we can debug? – user7160805 6 hours ago – user7160805 Mar 17 '17 at 15:28
  • I will take a look and get back to you. It may take me one or two days. Let me know if you need it urgently. – xiaoming Mar 21 '17 at 02:35
  • Your settings looks right to me. I just tried to play https://www.youtube.com/watch?v=iNJdPyoqt8U with your settings and it works properly. Is it possible for you to: 1. Try to see if you can reproduce with the above video. Or give me an video id that I can use to reproduce. 2. Modify the SB_MEDIA_SOURCE_BUFFER_STREAM_VIDEO_MEMORY_LIMIT to 60MB and see if the problem remains. 3. Tell me how frequently this can be reproduced. i.e. roughly once every ten tries, etc. so I have a better idea during reproducing it. – xiaoming Mar 21 '17 at 02:47
  • after resolution up to 3840*2160, you can forward and backward some times. waiting a while, it may occur. the key step is backward. – user7160805 Mar 21 '17 at 02:51
  • we had tried cobalt(x64x11), it very hard to reproduce. it did not have vp9, so the resolution of 4k content is 1920x1080 only. – user7160805 Mar 21 '17 at 02:54
  • 1. we often test "https://www.youtube.com/watch?v=6_P8RUqGQbM". 2. SB_MEDIA_SOURCE_BUFFER_STREAM_VIDEO_MEMORY_LIMIT = 60MB still happen. 3. the frequency can be 100%, the key step is backward and wait a while. – user7160805 Mar 21 '17 at 03:01
  • I have tried the video you mentioned and cannot reproduce the issue (see https://www.youtube.com/watch?v=H0I412kuuXk). Just want to double check one thing: after such seek is SbPlayerDeallocateSampleFunc() called to release the samples used? As if samples are held by the player the seek in the media source may fail because of failing to allocate memory. Let me know if SbPlayerDeallocateSampleFunc() is called and I will try to enable 4k playback on linux-x64x11. – xiaoming Mar 21 '17 at 23:36
  • we check SbPlayerDeallocateSampleFunc() had be called. our current status is the seeking can work well. then, after seeking is done, video and audio are playing. our big problem is that none call fcn_append() in MozjsSourceBuffer.cc after a few minutes. as we know, after XMLHttpRequest::response_array_buffer(), fcn_append() should be called. the key step of our reproduce is forward one time, backward some times and wait a while. – user7160805 Mar 22 '17 at 08:33
  • This sounds more complex than I'd thought. While I am happy to help and keen to resolve this issue, it is slightly hard to put more details here. Is it possible for you to file an issue in our public issue tracker (https://issuetracker.google.com/) in component "Public Trackers > Cobalt" so we can discuss further? – xiaoming Mar 22 '17 at 17:35
  • Just wonder if there is any progress and I am looking forward to help. – xiaoming Mar 24 '17 at 18:07
  • we found a way you can reproduce this issue easy with configuration x64x11 and "6_P8RUqGQbM" . git version is " Import Cobalt 9.34413", reduce thiese two meory limitation #define SB_MEDIA_SOURCE_BUFFER_STREAM_AUDIO_MEMORY_LIMIT (2U * 1024U * 1024U), #define SB_MEDIA_SOURCE_BUFFER_STREAM_VIDEO_MEMORY_LIMIT (8U * 1024U * 1024U). could you try it please, just forward and backward as previous comment? – user7160805 Mar 28 '17 at 14:21
  • This is slightly different than the 4k case. When the video files gets parsed, the media source implementation (ChunkDemuxer) will cache certain amount of frames. Let's say the current playback time is at 10, and the app keep pushing more data into media source, the media source may contain a buffer with media from 8 to 20. Eventually, if there is too much data appended, the media source tries to eject data from [8, 10) first, then it will eject from the back. This creates a gap and pauses the playback. In the above 1080p case, 8M is too small for video buffer. (continue in next comment) – xiaoming Mar 31 '17 at 17:12
  • However, the value you set for 4k (16M and 128M respectively) is more than enough. I did test in one of our 4k platforms with a much smaller buffer size and it works properly. Again, if you can open a ticket, it would be much easier for me to offer help. But I am also ok to continue work with you here. I am pretty confident that this should be easily fixed if I can reproduce it. – xiaoming Mar 31 '17 at 17:17