3D reconstruction of Rubik's cube. Did you try it?
The task seems elementary at first glance, but try and it turns out to be tricky. Meanwhile this is just a geometric puzzle and a perfect application for various OpenCV methods. Enjoy. I don't ask about solving it or color detection. It is regarded just as a box.
As computer vision practitioners have already guessed, the pictures presented are the best. Just turn it 40 degrees and the perfect result goes bust. Detecting contours works better, but it is rather time-consuming. It produces many duplicates. I tried Canny only.
You see that many squares are lost because their contours were rejected. If I replace
approx.size()==4&&fabs(contourArea(Mat(approx)))>100&&isContourConvex(Mat(approx)))
by
fabs(contourArea(Mat(approx)))> 100)
they appear and the central knot may be detected.
Another shortcoming of contours. Probably they are the best for this concrete task, but I hope that my program will work for other boxes too. They have no squares. The typical pattern is lines of text intermingled with some pictures. I still believe that line detection has perspectives. Not only the functions used, but also their sequence matters. Initially, I converted color image to gray, then fed it to Canny. squares.cpp hints another approach. We can extract separate channels, apply Canny,
then mix results.
Almost perfect edges make it possible to reconstruct 3 necessary lines.
and question is ?
The question is, did anybody here try this particular object. It is standardized, widely available, and was designed especially for educational purposes. I see a lot of discussion possible.
i met plenty of sample source code related this. the method in your pictures seems not a robust one.
That's right. This is exactly the reason why I ask here. Would you please provide links to those samples?
take a look at this
Great. Looks like a lot of people (including you yourself) have already tried this task. I heard of a robot that solves Rubik's cube. You provided the link to some source code. I need some time to cope with this all. Maybe the next week will post here what I think.
IMO the task which you discussed with nadley is simpler. It may be designated as "understanding". At the input we have an array of equivalent pixels. At the output - quite different data. Namely, the list of symbols representing colors and their order. I don't want symbolic output for now. Just to convert 2D -> 3D. In theory, the task is elementary. Find the central knot where 3 edges come together, determine the opposite ends of these lines, and the job is done. We have 3 coordinate axes linked to the cube. Now the reality. The features which I enumerate further are typical. You will often encounter them on other objects by accident, but here they are a part of the design so may be thoroughly discussed.
1.Each edge of the cube is represented not by 1, but by 2 parallel lines (sides of squares). 2.Each line is broken. This was already addressed in OpenCV. HoughLinesP() has a parameter maxLineGap to fill gaps. 3.Canny edge detector requires gray input, but when you convert colors, they produce different intensity. As a result, white ant yellow squares are perfectly recognized, brown disappears at all, others partly disappear and contours won't be linear.