Handwritten digit recognition from table

edit

@berak

The solution is interesting but it will work only for some of the cases because the ones I need to test my solution on are ... pretty badly written digits. Some examples of processed images are below.

Those problems may appear because of the way I chose to get the boxes:

Code:

gray = cv2.cvtColor(img, cv2.COLOR_BGR2GRAY)
thresh = cv2.threshold(gray, 200, 255, cv2.THRESH_BINARY)[1]
edges = cv2.Canny(gray, 30, 200)
contours, hierarchy = cv2.findContours(edges, cv2.RETR_EXTERNAL, cv2.CHAIN_APPROX_NONE)

# get bottom region
rect_bottom = cv2.boundingRect(contours[0])
x, y, w, h = rect_bottom
bottom_box = thresh[y: y + h, x: x+w]

# get top region
rect_top = cv2.boundingRect(contours[1])
x, y, w, h = rect_top
top_box = thresh[y: y + h, x: x + w]

if np.sum(top_box == 0) > np.sum(bottom_box == 0):
    # if more black pixels on top then the number in on top side
else:
    # else it is on bot side

because of the way I chose to get the boxes:

well you never showed us, so there cannot ever be an exact answer to your fairly vague question

@daneel95. Your code is doing wrong thing. Don't set threshold value too high. Set it between 50 to 127. And this doesn't do anything if np.sum(top_box == 0) > np.sum(bottom_box == 0):. It should be top_box > 0 and bottom_box > 0:

Snippet code:

for i, ctr in enumerate(sorted_ctrs): 
    # Get bounding box 
    x, y, w, h = cv2.boundingRect(ctr) 

    # Getting ROI 
    roi = image[y:y+h, x:x+w] 

    #cv2.imshow('segment no:'+str(i),roi) 
    cv2.rectangle(image,(x,y),( x + w, y + h ),(0,255,0),2) 

    if w > 15 and h > 15: 
        cv2.imwrite('{}.png'.format(i), roi)

cv2.imshow('Marked Numbers', image)
cv2.waitKey(0)

@berak said, you never showed us. Sadly, we can't help you.

@supra56 Thanks for the answer. The problem is not finding the boxes with the numbers or where the number is. The problem is finding the exact number without any noise (which may not even be possible).

Now, I ommited some information as I found it not necessary but now I will add here all the information I have:

1. An example of an image I start from: https://i.imgur.com/Ws9UKII.png (used imgur as the forum doesn't let me upload that file ...)
2. The full code I have now: Break the image in 3 parts, get the bottom right one (as it is the one I need to handle right now), try to get the selection: https://pastebin.com/5DyTh8Ln

It is a lot of code with a lot of things not really necessary for the question but it may help.

So you wanted to put box in 1 and box in x?

As fact as I understand. This is what you looking for multiple choice scanner. Unfortunately, I'm unable here to help, because of Xmas season. I will try if I have a time.

@Supra not really, I managed to do that part. All I need is to properly handle the digit inside the 2 boxes above the right side table :) but thanks for the link.