How to classify optical flows?

edit

I am working on some video classification techniques and thought that optical flows could be very useful instead of the actual video. The video contains extra features but in my case actions are pretty unique for each class means only by seeing optical flows we can classify videos. Classes could be people walking, falling of objects, throwing of objects and kicking. But I am not sure what kind of features we can use from optical flows so that we can train some simple ML algorithms.

I can definitely use some exotic CNN based methods like C3D but I feel these methods are computationally intensive so my query is:

what features I should extract from optical flows?

what classification techniques can be used over optical flows?