Is Toe-in fatal for Stereo Correspondence?

edit

I was into a similar problem (by the way, why is it called "toe-in"?). What I had to do was to get a disparity map from a stereo pair taken with two "potentially" converging cameras. I say "potentially" because the converging factor was variable from pair to pair.

I had to modify my own stereo matching algorithm from a simple "right-to-left" correspondence search to a "right-to-left & left-to-right" method.

In few words, if your method does that:

for disp = 0 to MAX_DISPARITY
     score = correspondence between left(x ; y) and right(x - disp ; y)
     if score is better than the previous
         disparity(x ; y) = disp

you have to modify it this way

for disp = 0 to MAX_DISPARITY
     score1 = correspondence between left(x ; y) and right(x - disp ; y)
     // move also to the right
     score2 = correspondence between left(x ; y) and right(x + disp ; y) 
     score = max(score1, score1)
     if score is better than the previous
         if score1 > score2
              disparity(x ; y) = +disp 
         else 
              disparity(x ; y) = -disp

In this way, you will save a disparity map where values from -MAX to 0 are referring to the object furthest than the converging point, and 0 to MAX referring to the ones closer to the observer.

The problems of this algorithm are basically two:

1. There are more operations to do
2. You need a robust correspondence metric, because it grows the numer of false positives