1 | initial version |
Matching feature points directly won't work, as you've already discovered. There are two issues that you want to solve.
1) Use a narrower field of view. You can reproject pixels from the spherical map onto an image plane with a narrower fov. This would produce regular looking images on which you can use the standard feature matching techniques, computing the fundamental matrix, etc. You're looking for the inverse spherical projection equations for this.
2) Register the two panoramas. It is likely that the two panoramas are relatively shifted. Is the object at the left-most edge of the first panorama the same as in the second one?
Hopefully this helps!