Detecting the status of pedestrian traffic signal.

edit

I am pretty new to open CV.My project involves identifying traffic signal status in pedestrian crossing,Its either a walk symbol in green,red or yellow.In some places its also text like "WALK" and "DONT WALK". I need to identify this using open CV .My initial thoughts are

Extracting the highest RGB value pixels. Using clustering algorithm to group the pixels to a cluster.
Using template matching to filter out other clusters.
Finally find the color the cluster that has passed filtering.

Is this an optimal way of doing? Or is there any other algorithm or approach I need to take in order to make the whole process fast and accurate.Please help