If you can determine the first border pixel accurately eg. occurs near (0,0) then you might be able to
- floodFill all black pixels starting at (0,0) or where ever, generate a binary image such that 0 is for border pixels, and 1 is for everything else
- use findContour + boundingRect to find the bounding box of the inside pixels. The findContour is very fast, it only visits pixels on the edge (not inside like a floodFill). findContour will return an array of possible contours, pick the largest one.
- crop the borders out using the boundingRect of the non-border pixels