Revision history [back]

Good day!

I am trying to learn how to manually implement stereo matching algorithms. I'm basically starting with the most basic of them all - Absolute Difference.

I found some slides online that describe how to do it. Basically, from what I understand, I should calculate the difference between the pixels in my left image and the same pixel in the right image "shifted" by a certain distance/disparity. Then among these disparities, I select the minimum, which makes sense to me since the pixel with the lowest disparity means that it is most likely the same pixel in the left image.

I've coded it in OpenCV. Here is the code:

include <opencv2\opencv.hpp>
#include "support.h"

int main(){
    cv::Mat left_img = cv::imread("..\\images\\middlebury\\tsu_left.png", cv::IMREAD_GRAYSCALE);
    cv::Mat right_img = cv::imread("..\\images\\middlebury\\tsu_right.png", cv::IMREAD_GRAYSCALE);

    int disparity_max = 64;

    std::vector<cv::Mat> cost_maps;

    for (int disparity = 0; disparity < disparity_max; disparity++){
        cv::Mat cost_map = cv::Mat(left_img.size().height, left_img.size().width, CV_8UC1);
        for (int row = 0; row < left_img.size().height; row++){
            for (int col = 0; col < left_img.size().width; col++){
                int col_disp = (col - disparity < 0) ? 0 : col - disparity;
                            //Absolute Difference
                cost_map.at<uchar>(row, col) = abs(left_img.at<uchar>(row, col) - right_img.at<uchar>(row, col_disp));
            }
        }
        cost_maps.push_back(cost_map);
        show(cost_map);
    }


    //Pick the smallest disparity
    cv::Mat final_cost_map = cv::Mat(left_img.size().height, left_img.size().width, CV_8UC1);
    for (int row = 0; row < final_cost_map.size().height; row++){
        for (int col = 0; col < final_cost_map.size().width; col++){

            int min = 65536;
            for (int disparity = 0; disparity < disparity_max; disparity++){
                min = (cost_maps[disparity].at<uchar>(row, col) < min ? cost_maps[disparity].at<uchar>(row, col) : min);
            }

            final_cost_map.at<uchar>(row, col) = min;

        }
    }

    //Show the final cost map, multiplied by 32 to show pixels with low value
    show(final_cost_map * 32);
}

Take note that I haven't implemented the variant where the differences in a certain window is summed, leading to Sum of Absolute Differences. I am only taking the difference per pixel, per disparity. The lecture slides I found online says that it should look like this (rightmost image):

https://dl.dropboxusercontent.com/u/92715312/lec.PNG

However, the result from the code above yield something like this:

https://www.dropbox.com/s/7ns7h6tqycjahuv/res2.PNG?dl=0

I can't figure out why the outputs are so different. Is there some trivial step that I am missing, or is my understanding of how the algorithm works wrong? I am also using the tsukuba image.