frameskipping

edit

I would do

Rect faceRect;
for(;;) {
    cap << frame; 
    if (frameno % 3 == 0) frameNo++; {
        Do all your stuff
        set the faceRect 
    }
    rectangle(frame,faceRect, ...)
    imshow

That makes sense! So this would still include the actual face detection correct, just not the face recognition processing.

Thanks a lot

I don't understand your comment but to avoid flickering you have to draw the rectangle each frame you show. Where the rectangle is placed when you skip the frame is it's up do you.

You need to apply a tracker on top of your detections that smooth your detections over time. The jitter is normal due to the pixel difference in found detection regions!