grab and process multiple frames in different threads - opencv semi-related

edit

Well as the title says I would like to grab and process multiple frames in different threads by using a circular buffer or more, I hope that you can point me to what is better. For grabbing frames I am not using the VideoCapture() class from opencv but the libfreenect2 library and the corresponding listener since I am working with the Kinect one sensor which is not compatible with the VideoCapture() + openi2 functionality yet. My intention is to have one thread grabbing frames continuously in order not to affect the framerate that I can get from the kinect sensor, also here I might add a viewer in order to have a live monitor of what is happening (I do not know though how feasible is this and how it will affect the framerate) and having another thread where I would do all the process. From the libfreenect2 listener I can obtain multiple views regarding the sensor so at the same time I can have a frame for the rgb camera, one for the ir, one for the depth and with some process I can also obtain an rgbd. My question now is how to make shareable these frames to the two threads. Having a look in the following questions Time delay in VideoCapture opencv due to capture buffer and waitKey(1) timing issues causing frame rate slow down - fix? I think that a good approach would be to go with two threads and a circular buffer. However, what is more logical? to have multiple circular buffers for each view or one circular buffer which will contain a container (e.g. an stl vector<>) with the frames from each view.

At the moment I am using the second approach with the vector and adapting @pklab 's approach from the first link I posted above. Code is below:

//! [headers]
#include <iostream>
#include <stdio.h>
#include <iomanip>
#include <tchar.h>
#include <signal.h>
#include <opencv2/opencv.hpp>

#include <thread>
#include <mutex>
#include <queue>
#include <atomic>

#include <libfreenect2/libfreenect2.hpp>
#include <libfreenect2/frame_listener_impl.h>
#include <libfreenect2/registration.h>
#include <libfreenect2/packet_pipeline.h>
#include <libfreenect2/logger.h>
//! [headers]

using namespace std;
using namespace cv;

enum Process { cl, gl, cpu };

std::queue<vector<cv::Mat> > buffer;
std::mutex mtxCam;
std::atomic<bool> grabOn; // this is lock free

void grabber()
{
    //! [context]
    libfreenect2::Freenect2 freenect2;
    libfreenect2::Freenect2Device *dev = nullptr;
    libfreenect2::PacketPipeline *pipeline = nullptr;
    //! [context]

    //! [discovery]
    if(freenect2.enumerateDevices() == 0)
    {
        std::cout << "no device connected!" << endl;
        exit(EXIT_FAILURE);
//        return -1;
    }

    string serial = freenect2.getDefaultDeviceSerialNumber();
//    string serial = "014947350647";

    std::cout << "SERIAL: " << serial << endl;

    //! [discovery]

    int depthProcessor = Process::cl;

    if(depthProcessor == Process::cpu)
    {
        if(!pipeline)
            //! [pipeline]
            pipeline = new libfreenect2::CpuPacketPipeline();
            //! [pipeline]
    } else if (depthProcessor == Process::gl) {
#ifdef LIBFREENECT2_WITH_OPENGL_SUPPORT
        if(!pipeline)
            pipeline = new libfreenect2::OpenGLPacketPipeline();
#else
        std::cout << "OpenGL pipeline is not supported!" << std::endl;
#endif
    } else if (depthProcessor == Process::cl) {
#ifdef LIBFREENECT2_WITH_OPENCL_SUPPORT
        if(!pipeline)
            pipeline = new libfreenect2::OpenCLPacketPipeline();
#else
        std::cout << "OpenCL pipeline is not supported!" << std::endl;
#endif
    }

    if(pipeline)
    {
        //! [open]
        dev = freenect2.openDevice(serial, pipeline);
        //! [open]
    } else {
        dev = freenect2 ...