Why is the first openCV API call so slow?

edit

I am creating a mask from an image with the size: 2000x640, and I'm running this code on a modern powerfull laptop.

I have made a function which is suppose to create binary mask from my input image, the code is quite simple and shown below:

However, when running the code it seems like doing a simple threshold takes a very long time - almost half a second.

ms_allocation: 0.001

ms_converted: 0.001

ms_thresholding: 0.383

ms_median_blur: 0.001

ms_copy_mask: 0

ms_locate_contour: 0.067

This looked very odd, so I implemented my own thresholding which ran in a few ms. However, not suddenly the median_blur became equally slow.

I then completely removed the thresholding part of the code and got this performance result

ms_allocation: 0.0008

ms_converted: 0.002

ms_thresholding: 0.000

ms_median_blur: 0.424

ms_copy_mask: 0

ms_locate_contour: 0.106

Can someone help me understand what is going on here? Why is the first call to openCV so slow, and can I prevent it somehow?

I would really prefer is this code could run in less than 0.1 s, which I think it totally reasonable.

void calculateMask(Mat & pSrc, Mat & pDst)
{

    auto ms_alloc_start = getTickCount();

    // Allocate, just to be sure it's done in advance.
    Mat mask = Mat::zeros(pSrc.size(), CV_8U);
    Mat sharp = Mat::zeros(pSrc.size(), CV_8U);
    Mat filled_mask = Mat::zeros(pSrc.size(), CV_8U);;
    Mat masks = Mat::zeros(pSrc.size(), CV_8U);;

    auto ms_alloc_end = getTickCount();
    auto ms_convert_start = getTickCount();

    pSrc.convertTo(sharp, CV_8U);

    auto ms_convert_end = getTickCount();
    auto ms_threshold_start = getTickCount();

    //threshold(sharp, mask, 20, 255, THRESH_BINARY);

    auto ms_threshold_end = getTickCount();
    auto ms_median_start = getTickCount();

    medianBlur(mask, filled_mask, 5);

    auto ms_median_end = getTickCount();
    auto ms_copy_start = getTickCount();

    filled_mask.copyTo(masks);

    auto ms_copy_end = getTickCount();

    auto ms_find_contour_start = getTickCount();

    locateLargestContour(masks, pDst);

    auto ms_find_contour_end = getTickCount();

    cout << "\n\n\tms_allocation: " << to_string((ms_alloc_end - ms_alloc_start) / getTickFrequency()) << "\n";
    cout << "\tms_converted: " << to_string((ms_convert_end - ms_convert_start) / getTickFrequency()) << "\n";
    cout << "\tms_thresholding: " << to_string((ms_threshold_end - ms_threshold_start)/ getTickFrequency()) << "\n";
    cout << "\tms_median_blur: " << to_string((ms_median_end - ms_median_start) / getTickFrequency()) << "\n";
    cout << "\tms_copy_mask: " << to_string((ms_copy_end - ms_copy_start) / getTickFrequency()) << "\n";
    cout << "\tms_locate_contour: " << to_string((ms_find_contour_end - ms_find_contour_start) / getTickFrequency()) << "\n";
}