You can access a specific frame in a video sequence like this:
capture.set(CV_CAP_PROP_POS_FRAMES, frame_num);
capture >> frame;
You can then access a separate frame like this:
capture.set(CV_CAP_PROP_POS_FRAMES, frame_num - 20);
capture >> frame2;
However, when you set the capture property with CV_CAP_PROP_POS_FRAMES, it actually moves the first pointer and frame and frame2 end up pointing to the same data, containing the same values.
I want a pointer to both, so I do not have to clone the data (expensive computation) but I can perform comparisons between the two frames. How can this be done with OpenCV?