Is it possible to get frame timestamps for live streaming video frames on linux?

edit

So, after many hours of digging into berek's pointer to the "cap_v4l.cpp" vs. "cap_libv4l.cpp" distinction, I find the answer is: yes ---> you can get it working...but you'll need to patch 'cap_libv4l.cpp'

You can download the modified one from: https://github.com/msandler/opencv/bl...

History and Usage Tips:

Sometime before OpenCV version 2.4, someone copied and pasted 'cap_v4l.cpp' to create 'cap_libv4l.cpp.' cap_v4l.cpp isn't even built. However the port was incomplete.

Using cap_v4l.cpp as a guide, I modified the 3.0 rc1 version of cap_v4llib.cpp, and 1st added supported for: CV_CAP_PROP_POS_MSEC, and then also for: CV_CAP_PROP_POS_FRAMES and CV_CAP_PROP_FPS.

It definitely strikes me that these changes outta be in the upstream, and, 'cap_v4l.cpp' should be removed to avoid future confusion.

Here's how to use the mod's in your code, with output examples:

 cv::VideoCapture videoCap; 
 ....
//after: 
videoCap >> image;

printf( "CV_CAP_PROP_POS_MSEC:   %ld   \n", (long) videoCap.get( CV_CAP_PROP_POS_MEC) ); 
printf( "CV_CAP_PROP_POS_FRAMES:  %ld \n", (long) videoCap.get( CV_CAP_PROP_POS_FRAMES) ));  // <-- the v4l2 'sequence' field
printf( "CV_CAP_PROP_FPS:  %f\n", videoCap.get( CV_CAP_PROP_FPS));

for a Logitech C920, this gave:

CV_CAP_PROP_POS_FRAMES: 28
CV_CAP_PROP_FPS:  30.000000
CV_CAP_PROP_POS_MSEC:   28834084     ;  diffFromLastFrame 36

where 'diffFromLastFrame' is simply the difference between two successive frames for 'CV_CAP_PROP_POS_MSEC'

Incidentally, V4L binds the timestamp to: CLOCK_MONOTONIC, which isn't unix epoch. Mostly, it's the time elapsed since the OS boot, but if your machine hibernated, it's much more complex. To simplify everything (e.g. for distributed camera processing), you'll need to add a constant offset to get it to unix time.

The following code does it -- pass in "(long) videoCap.get( CV_CAP_PROP_POS_MEC) "

#include <time.h>
long monotonicToEpochOffset_ms = -1; //<-- initialize in global scope
long convertToEpoch( long v4l_ts_ms){

    if( monotonicToEpochOffset_ms ==-1){
        struct timeval epochtime;  gettimeofday(&epochtime, NULL); 
        struct timespec  vsTime;  clock_gettime(CLOCK_MONOTONIC, &vsTime);

        long uptime_ms = vsTime.tv_sec* 1000 + (long)  round( vsTime.tv_nsec/ 1000000.0);
        long epoch_ms =  epochtime.tv_sec * 1000  + (long) round( epochtime.tv_usec/1000.0);    

        // add this quantity to the CV_CAP_PROP_POS_MEC to get unix time stamped frames
        monotonicToEpochOffset_ms = epoch_ms - uptime_ms; 
    }

    return monotonicToEpochOffset_ms + v4l_ts_ms; 

}

After this conversion, you can difference the current time from that epoch converted time to get the lag of your code from the driver's timestamp. When working with the frame data from python or java, or on a distributed system, this is particularly helpful. In my test case I'm printing:

CV_CAP_PROP_POS_MSEC - epoched:   1431173328429 ; diffFromNow: 29