Ask Your Question
0

VideoWriter fails to open inside thread

asked 2015-11-11 18:43:29 -0600

BinaryBits gravatar image

updated 2015-11-16 04:48:04 -0600

Hello,

I writing a C++ application for Windows that handles video streaming over sockets. In this program, the main thread listens for connection attempts, and creates a new thread for each client that connects. That new thread essentially takes the data stream of the socket, converts it to a Mat, and then uses a VideoWriter to write each frame to an avi file. This new thread also limits each file to a set number of seconds in length. I've tested using one client and video gets recorded just fine. But the moment I try to create another thread for a second socket, VideoWriter fails to open the output in that new thread. Unfortunately VideoWriter fails quietly (I use writer.IsOpen() to check) so it's tough to trace where exactly it is breaking. I am using x264VFW as my video encoder, with default settings. For this example, the frames being sent over the socket are in grayscale, hence the CV_8UC1.

Here is the method that is called inside each client thread:

bool Streaming::DecodeVideoStream(Socket socket, std::string output_file_path, std::string thread_id)
{
    Size S = Size({ 320, 240 });
    time_t endwait;
    time_t start = time(NULL);
    time_t seconds = 30; 
    endwait = start + seconds;

    VideoWriter writer(output_file_path, CV_FOURCC('x', '2', '6', '4'), 25, S);
    if (!writer.isOpened())
        logging.WriteLog("Failed to create video file: " + output_file_path);

    while (start < endwait)
    {
        Mat  webcamFrame = Mat::zeros(S, CV_8UC1);
        int  frameSize = webcamFrame.total()*webcamFrame.elemSize();

        char* socketData = new char[frameSize];
        memset(socketData, '\0', sizeof(char)*frameSize);

        int packetSize = socket.RecvData(socketData, frameSize);
        if (packetSize > 0)
        {
            writer.write(Mat(S, CV_8UC1, socketData));
        }
        else
        {
            delete[] socketData;
            logging.WriteLog("Recv() returned -1. ", __FUNCTION__);
            logging.WriteLog("Error code: " + std::to_string(WSAGetLastError()), __FUNCTION__);
            return false;
        }
        delete[] socketData;
        start = time(NULL);
    }

    logging.WriteLog(thread_id + " Reached clip time limit.", __FUNCTION__);
    return true;
}

I would greatly appreciate any suggestions, as I do not have much experience with running VideoWriter in multiple threads. Maybe there's something stupid I'm missing.

UPDATE (11/16/2015)

So after a lot of hair pulling, I decided to create a new project with barebones code in it to see if I could reproduce the problem with just the essentials. I have created a pastebin here with the code. It's short and should help give a better picture into what may be going on here.

When I run the code (As shown in the pastebin), I get a familiar output that indicates the first thread managed to create the output file, but the other 4 failed: image description

Still totally confused, and would appreciate any help. -BinaryBits

edit retag flag offensive close merge delete

Comments

opening and writing multiple instance of VideoWriter in different threads is allowed... are you using different output_file_path ?

pklab gravatar imagepklab ( 2015-11-12 05:34:52 -0600 )edit

output_file_path is basically "C:\Users\Test\Desktop\" with the current date and time used for the file name. For example 2015-11-13-15-47-42.avi. So it is different each time VideoWriter is opened.

Here's how I am calling this function (from a thread):

    while (GetMessage(&ThreadMessage, NULL, 0, 0))
{
    Streaming streaming = Streaming();
    if (!streaming.DecodeVideoStream(thread_socket, output_path, thread_id_string))
        break;
}
BinaryBits gravatar imageBinaryBits ( 2015-11-13 16:53:03 -0600 )edit

1 answer

Sort by ยป oldest newest most voted
2

answered 2015-11-15 13:08:15 -0600

pklab gravatar image

updated 2015-11-17 05:14:14 -0600

EDIT: I've found some interesting behaviour related to codecs and threads in OpenCV. It seems that some codecs requires that multiple object of VideoWriter have to be initialized/opened in the main thread.

After this you can use them in worker threads. My experience:

  • Using uncompressed codecfourcc=0 or XVID I can open/write VideoWriter in worker threads.
  • Using X264 I have to open many VideoWriters in the main thread but I can use them in worker threads.

May be some lock occours and could depends on codec/VFW/DSHOW implementation.

The following code works to me with X264 and OCV2.4.10 or OCV 3.0.0. if you comment out the line writers[i].open(fname, codec, FPS, image_size); you can test opening in worker thread using XVID.

#define FRAME_W 320
#define FRAME_H 240
#define FPS 25

// This thread uses a VideoWriter object opened in the main thread
void ThreadStd(cv::VideoWriter *writer, const char*fn, int threadNum)
{
    string fname(fn); 
    time_t end_time;
    time_t current_time = time(NULL);
    time_t clip_length = 10;
    end_time = current_time + clip_length;
    Size image_size = Size(FRAME_W, FRAME_H);
    // some codecs don't allow to open Videowriter in multiple tread instance
    if (!writer->isOpened())
    {
        //XVID allows open here
        writer->open(fname, CV_FOURCC('X', 'V', 'I', 'D'), FPS, image_size);

        //uncompressed fourcc=0 allows open here
        //writer->open(fname, 0, FPS, image_size);

        // X264 needs Videowriter already opened in the main thread
        //writer->open(fname, CV_FOURCC('X', '2', '6', '4'), FPS, image_size);
    }
    if (!writer->isOpened())
    {
        std::cout << "\nThe video file has not been opened: " + std::string(fname) << endl;
    }
    else while (current_time < end_time)
    {
        Mat  webcam_frame = Mat::zeros(image_size, CV_8UC3);
        cv::putText(webcam_frame, "TH#:" + to_string(threadNum), 
                    cv::Point(10, 10), FONT_HERSHEY_PLAIN, 1, Scalar(255,255,255));
        writer->write(webcam_frame);
        current_time = time(NULL);
    }
    cout << "\nTime's up! from thread " << threadNum << std::endl;
    return;
}

int _tmain(int argc, _TCHAR* argv[])
{
    std::cout << "This will test the VideoWriter class's ability to work in threads.\n";
    int numThreads = 5;
    //thread list
    vector<thread> threads(numThreads);
    // we create a VideoWriter object for each thread
    vector<cv::VideoWriter> writers(threads.size());
    for (int i = 0; i < threads.size(); i++)
    {
        std::string fname = "../img/" + to_string(i) + ".avi";
        DeleteFileA(fname.c_str()); //delete file if it exist

        Size image_size = Size(FRAME_W, FRAME_H);
        int codec;
        codec = CV_FOURCC('X', '2', '6', '4');
        // open the VideoWriter here but...
        writers[i].open(fname, codec, FPS, image_size);
        // ...we write it in the thread
        threads[i] = thread(ThreadStd, &(writers[i]), fname.c_str(), i);
        cout << "\n\nStarted test thread #" << to_string(i);
    }
    // wait for all treads
    for (int i = 0; i < threads.size(); i++)
        threads[i].join();
    // release all writers
    for (int i = 0; i<writers.size(); i++)
        writers[i].release();
}
edit flag offensive delete link more

Comments

Pklab, thank you for the reply. I added a simple:

if (fileSystem.FileExists(output_file_path))
    {
        logging->WriteLog(output_file_path + " already exists!");
        return false;
    }

Unfortunately though, it doesn't seem to have solved the problem. The output filename has now been confirmed to be unique every time.. are there any known gotchas with VideoWriter that perhaps I'm missing? Since it fails quietly, it's a real pain to figure out why it blew up. I deleted and re-downloaded my OpenCV 3.0 libraries and source in hopes that maybe something was corrupt, to no avail. I appreciate your help on this strange issue.

BinaryBits gravatar imageBinaryBits ( 2015-11-16 03:24:19 -0600 )edit

Edited the answer... try to change codec or try to open videowriters in the main thread

pklab gravatar imagepklab ( 2015-11-16 12:26:55 -0600 )edit

Thank you so much! I tried out your code and it worked perfectly on my machine. I will go ahead and implement your method into my application. I appreciate all the help!

BinaryBits gravatar imageBinaryBits ( 2015-11-16 12:56:29 -0600 )edit

Question Tools

1 follower

Stats

Asked: 2015-11-11 18:43:29 -0600

Seen: 4,667 times

Last updated: Nov 17 '15