When trying to use disparity map on calibrated stereo images, performance for general, un-conditioned scenes is poor. In general, I think the problem may relate to the fact that disparity maps have no global knowledge, and just try to find "matching clusters of pixels" along each epiline.
If I know that "up" is vertically up, and I know there is supposed to be a ground plane 25 centimeters below the camera center, it should be possible to do much better. For example, start by projecting the ground plane pixels out of the image, to find matches, and map all of those to a ground plane. Then look at the parts that don't match, and start matching them up using spatial heuristic reasoning based on where the "gaps" are in the ground plane, or patch-based disparity instead of epiline based disparity.
Is there anything in OpenCV that could help me implement an algorithm like this? For example, is there a function that could take as input a pair of images, and an assumed ground plane (in camera-relative coordinate space, or some other space) and figure out which parts of the image correspond to that ground plane, and which ones don't?