sometimes failed to read frame from rtsp with opencv

edit

https://stackoverflow.com/questions/5...

I got a strange issue. my code is quite simple, just read frame from RTSP and do some processing.

for (;;)
{
    cap >> frame;
    if (frame.empty()) {
        break;
    }
    // heavy_processing(frame) // opencv-dnn face detection stuff
    imshow("cam",frame);
}

the rtsp streaming is a webcam with 1920x1080P, network traffic is arround 4Mb/s

without "heavy_processing()" function, basically it runs good. CPU is 45% (i7-7700)

with "heavy_processing()" function, it will read an empty frame after couple seconds then exit. I can see CPU surge to 95%+ before it exit.

I built opencv(3.4.3) myself with mostly default option.

so far I suspect the videocap device shutdown when there are too much frames in the buffer, but it doesn't make sense.shouldn't it drop the earliest frame? or is it possible the frame is corrupted inside 'heavy_processing()'?

ffmpeg probably is 4.0(built myself), ubuntu 16.04

update:

the heavy_processing() function, suspect it is related.

void detect(Mat frame, Mat blob, Net &net, Size &inpSize, vector<cv::Rect> &face_rects) {
    Mat detectFrame;
    cv::resize(frame, detectFrame, cv::Size(300, 300), 0, 0, cv::INTER_AREA);
    dnn::blobFromImage(detectFrame, blob, 1.0, inpSize, mean, false, false);
    // if I return here is good
    net.setInput(blob);
    Mat detection = net.forward();   // as long as I add the last two line, it will get read frame empty

update:

the detect function will spread over multiple cores(mine is 8 cores i7, total cpu usage is less than 50%), so CPU resource should be enough.

just in case there is race condition, I have added some lock in detection/capture/display part, but actually the code is still in the same thread as capture/display(however net itself might spawn some threads),

the problem is still not resolved yet.

tried opencv 4.0 as well but not help. change 'break' to 'continue' not help because it will keep reading empty frame after that.