How to compare two handwriting characters to get similarity score
I want to compare two handwriting Chinese characters to get the similarity score(percent). We think that one is the normative writing, and the other is to evaluate the similarity with the normative one.
Handwriting with a higher similarity score should: 1 The structure is most similar to the normative one. 2 Strokes are smooth. 3 Stroke shapes are most similar to the normative one.
The two handwriting characters are in two images with the same size, and all with white background color and black foreground color.
How to get the most accurate similarity score and have a faster speed?
Should I use cvMatchTemplate? and get the similarity score(max_val) with cvMinMaxLoc?
Or use HausdorffDistanceExtractor, and getRankProportion () as the similarity score?
Or use ShapeContextDistanceExtractor, and call computeDistance (img1, img2) to get the distance as similarity? But how to convert it to a percentage form?
Any suggestions would be appreciated.
Thanks!
none of those ideas will reliably work, and you should NOT try anything with the deprecated c-api (cv* functions)
in the end, you'll need some form of machine learning here, like a training a cnn
I also thought about CNN or DNN, but I haven't figured out how to convert this problem into a neural network problem. Is it an image classification problem? Like this: I use all the normative writing words as training data, and the label corresponding to each word picture is this word. Then, rotate all the normative writing words to obtain a larger amount of data, and then the data is used to train the neural network. Finally, enter the neural network with the word to be scored. It gives the similarity to a certain word, and this is the score.
The neural network requires a lot of data, so the rotation and other transformations are used to generate more data. Does this affect the result? Can I add words of different fonts as training data?Is there a model I can transfer learning from?
I also thought about CNN or DNN, but I haven't figured out how to convert this problem into a neural network problem. Is it an image classification problem?
Well you could consider every unique character as a class. A popular example of this is the mnist dataset . This dataset is used to train a cnn to detect handwritten digits. And each digit is a class in the resulting model (0-9).
You dont always need to train from scratch / define your own dataset. Maybe you find a pre trained model which can already extract chinese characters.
And you could even evaluate this model via the opencv dnn module. I say could - not must - you can always use the api on which the model was trained(caffe, tensorflow, torch, etc.)
What i want to say - forget about similarity / template matching - find a cnn model which can extract the chinese characters into text and then just do a simple string comparison in the programming language of your choice.
Do you really need to do this using OpenCV or some CNN / DNN framework? If Holger's approach does work for you, I would suggest not to expend too much effort training your own network. A very quick and pragmatic approach would be to leverage cognitive services text analytic APIs from Google, Microsoft, etc. Just pointing this out as it can save you a lot of time and effort; however, I do appreciate you may have constraints or objectives that prevent you from using such cloud services.
If I use CNN, the question of how similar the handwriting is to the standard word is a classification problem or a regression problem? Because in fact, I really need a percentage of similarity. I also hope that there will be an AI cloud services, and it is best to use Core ML for simple calls. But I looked at the Microsoft Vision cloud service, which only has OCR for English characters. I also looked at the Google Vision cloud service. Dragging a Chinese character image into it, I got only the information like Black 95%, and I couldn't find how to get the similarity between the two word images.