I'm trying to segment Google Street View images such that I can automatically detect where a house (or multiple houses) might be in the image. For this part I'm wanting to use something like the region proposal system used in the RCNN detector - OpenCV has implementation for the proposal system that can be found here.
This method proposes too many (~500) possible locations for objects for my algorithm; I'd like it to be around 50 or 100 at most. Are there any other methods I could use that would work better with my specific problem? I was thinking I could somehow remove things like pavement, grass, clouds/sky, etc, but maybe this is not a good idea because houses can have many different colors as well that might be very close to that of grass and sky, which vary in themselves too. I was also thinking that going off of some sort of shape based model might also work, but these don't seem to have too great success in the literature I've read. Would there be any good ideas to pursue?
Thanks