Camera Preview slow on Android OpenCV +possible bottleneck found

edit

So I noticed since a long time and wondered ever since why (on multiple devices) using JavaCameraView (not doing any processing just displaying preview images from camera), the smoothness of the camera preview is generally lacking compared to "regular" Android apps using the camera, or even Android OpenCV apps where instead of using JavaCameraView the camera image is manually converted to formats supported by OpenCV. (like this one: https://github.com/ikkiChung/MyRealTimeImageProcessing)

As I understand the "recommended" way to use the camera in OpenCV apps on Android (according to the tutorials and sample apps) is to use a CameraBridgeViewBase implementation (JavaCameraView or NativeCameraView), implement the CvCameraViewListener2 interface and override the onCameraFrame() method where you can process the camera frame.

I found a possible bottleneck which could be the the deliverAndDrawFrame method of the CameraBridgeViewBase.

The problem might be that in the implementation of JavaCameraView the CameraWorker (which would call deliverAndDrawFrame which calls onCameraFrame()) is running on a separate thread and while they're synchronized, the CameraWorker is not always receiving the frames from onPreviewFrame, every now and then it's skipping frames.

I verified this by making a class which extended the JavaCameraView and overrode the onPreviewFrame and logged the calls of this method:

@Override
public void onPreviewFrame(byte[] frame, Camera arg1) {
    Log.i(TAG,"onPreviewFrame called");
    super.onPreviewFrame(frame, arg1);
}

I also logged the onCameraFrame occurrences, calling Log.i(TAG,"onCameraFrame called"); in it.

If working fine this should output alternately the two logs like this: onPreviewFrame called onCameraFrame called onPreviewFrame called onCameraFrame called etc.

While this is true in my test in about 90% of the cases, I can find occurrences where "onPreviewFrame called" appears multiple times before another "onCameraFrame called" message like this: onPreviewFrame called onCameraFrame called onPreviewFrame called onPreviewFrame called onCameraFrame called

This means there are frames from the camera that NEVER make it to the onCameraFrame method. I suspect this frame skipping probably indicates that the deliverAndDrawFrame method's execution time sometimes exceeds the time between frames from the camera, which I think is really high considering it shouldn't do any hard processing work. The smoothness of other applications (not using JavaCamereView) suggests that this could be done more efficiently. Do you think there's any easy way to make it faster? Doing real-time processing is really hard if receiving frames from camera alone is already not real-time.

I only tested JavaCameraView, but since I think NativeCameraView (which also doesn't appear smooth enough) uses the same deliverAndDrawFrame method it's probably also affected.

Any help/insight is appreciated.