How can I detect the difference between a neutral face and a facial gesture of the user's choice?
I have a friend who, after having a stroke, now has locked-in syndrome who and can now only move his eyes and eyelids. I've been working on some C# software (written in Unity3D) to help him to communicate, the ones he has tried so far have not been successful due to the level of his disability.
I am in the process of implementing DLib for facial feature recognition. What I would like to do is to ask the user to repeatedly alternate between a neutral face and a face that is indicating. I don't want to limit the indication gesture specifically to winking/blinking/opening eyes wide, etc as I want other people to be able to freely download the app and indicate in whatever way they find easiest due to their incapacity.
I could really do with some pointers as to how to have my app learn what the user's facial indicator is by comparing the facial landmarks of the two states.
At this point I don't even know what it is I need to know. I don't know the names of any AI algorithms that might be relevant, or anything.
OpenCV would be an excellent choice as I already have a licence for a library in Unity3D, which is why I am asking here.
don't try to write an app, if you don't know the fundamentals.
I would suggest reading a book on OpenCV programming, skimming it and learning the basics you need and in-depth topics that look relevant. Like @berak, I don't think you can expect to write an app without actually learning what you need. It should become more evident as you look at capabilities, look at example code (as cookbook examples) and figure out what you need for your application. Wish I could be more help. It sounds like a worthy project.
The app is complete except for the "did the user indicate" part. He has been using it for a while but currently needs someone to click the screen whenever he indicates. I'm using DLib for facial landmark recognition too, just needed to know where to start looking for the learning part, is this part something OpenCV can help with then? If so, I will buy a book. Can you recommend any @opalmirror?
@berak writing an app is a good way to learn fundamentals.
my point is: you're asking us to do your research (e.g. about action units), and none of it is related to opencv, which makes this a bad question.
@mrpmorris although I like the fact you willing to do something for people with this syndrome, be aware that one of the main reasons why there isn't any computer vision solution out there, that performs well, is because they get heavily beated by human-computer-interaction interfaces that target the patterns of the brain waves, either through an implant or with a head mounted monitoring device. Your question will probably not be solved robustly by a PhD working 4 consecutive years on this, so I am afraid that you might be a bit out of your scope here ... that being said @berak is right, this has nothing to do with opencv programming issues, you actually need a robust literature study to start with.
Oh and BTW, if you ever want a successful application for this, you will need data, and lots of it. And that will be the first disappointing brick wall you will hit. Getting medical data collection approved is a long and administrative way with tons and tons of paperwork, licenses, costs, ... Even if the technology is available, you will have to be able to get it to the market somehow.