How can I best compare two BGR colors to determine how similar/different they are?

edit

The HSV color wheel can be used for color complementary, analogous such as color distance. The H channel from OpenCV BGR2HSV conversion is the Hue, in other words is the angle of the color on the color wheel. You might consider the difference between hues as colors distance without involve darkness or lightness.

Remember that

• hue is a angle over a circle than you have to get the shortest path between 2 hues (clockwise and counter-clockwise). For example hue=1° is RED, hue=120° is GREEN and hue=330° is ROSE. RED -> ROSE = 31° and RED -> GREEN = 119° !
• In OpenCV, 3x8bit images, the hue range 0..360deg is scaled down to 0..180 (or 0..255 if COLOR_BGR2HSV_FULL is used). Therefore GREEN is 60 (or 85 in case of _FULL conversion).

Using the conversion as @LBerger answer, this should check color distance between pt1 and pt2

Point pt1(x1,y1), p2(x2,y2);
Vec3b hsv1 = hsv.at<Vec3b>(p1); // get HSV values for the pixel. Hue is the 1st in the vector
Vec3b hsv2 = hsv.at<Vec3b>(p2);
d1 = abs(hsv1(0)-hsv2(0));  // absolute distance
d = min(d1,180-d1); // correct distance if cross 0deg . full circle at 180 because of Hue scaling

First you have to choose color space

To convert an image from one color space to another. use cvtColor function

to get pixel color use :

Mat img=imread("lena.jpg",cv::IMREAD_COLOR);
cout<<img.at<Vec3b>(lig,col);
Mat hsv;
cvtColor(img,hsv,cv::COLOR_BGR2HSV);
cout<<hsv.at<Vec3b>(lig,col);

(result [111, 144, 230][8, 132, 230])

Similarity between colors will depend of context...