While streaming Widevine encrypted, (HD/ 4K) 60 fps content on the FireTV Omni, the resulting video looks choppy due to frames being dropped. This is seen with Mediacodec being operated in Synchronous mode.
Next, upon operating the Mediacodec in Asynchronous mode, no improvement in the video quality was seen.
Continuing in Asynchronous mode, this problem was analyzed using Systrace, and it was seen that Mediacodec::queueSecureInputBuffer() takes time as follows:
ExoPlayer Demo Omni MAX AVG Name 48 ms 12.6 ms queueSecureInputBuffer OMX.MS.AVC.Decoder.secure 59 ms 11.7 ms queueSecureInputBuffer OMX.google.aac.decoder
The Systrace showed that the Audio and Video InputBuffer Enqueue threads spent a lot of that time in Lock contentions with each other, when the two threads are operating in a mutual exclusion mode.
As the next step, the MediaCodec::queueSecureInputBuffer() for the Audio and Video threads are operated in parallel. This allowed the two threads to overcome time spent waiting for lock permission to call Mediacodec::queueSecureInputBuffer(). In this case, the Systrace results showed improvement as follows:
ExoPlayer Demo Omni MAX AVG Name 56 ms 7.3 ms queueSecureInputBuffer OMX.MS.AVC.Decoder.secure 38 ms 6.4 ms queueSecureInputBuffer OMX.google.aac.decoder
This resulted in better video display as the 16.66 ms time frame is met more number of the times, less frames get skipped.
My questions are:
1. Why Mediacodec::queueSecureInputBuffer() is taking this long time on this platform ? It is seen to be taking around 3.3 ms on FireTV Cube that uses Amlogic. What can be done to improve this timing?
2. Is it safe to be operating the Mediacodec::queueSecureInputBuffer() from the Audio and Video Async Enqueque thread in parallel ?
