Max ANN_MLP size?

edit

Thanks for the links to source code and response!

From the train method here: https://github.com/opencv/opencv/blob...

Line 866: termcrit.maxCount = std::max((params.termCrit.type & CV_TERMCRIT_ITER ? params.termCrit.maxCount : MAX_ITER), 1);

It seems like there is no limit to number of epochs, it only uses a default of (MAX_ITER = 1000) if the user did not did not satisfy (params.termCrit.type & CV_TERMCRIT_ITER) which I believe is specifying to use max number of epochs and/or epsilon for termination criteria.

What you said about what my RAM can handle seems to be the issue, I was doing a benchmark and after 4000 epochs the time taken to train stays stagnant no matter how much more I increase the number of epochs.

May I ask what you're using for input data? There are some problems that can't be solved, no matter how many hidden layers you use.

Ya my job right now is to determine if this problem is solvable with an MLP, sorry I can't talk about it too much because I signed an NDA. Anyways, I had good prelim results but found more dependencies in the problem which require more nodes and layers. I tracked my memory usage while training, and looking through the source code, it doesn't seem like each epoch is increasing memory output so I don't think that is the issue for number of training sessions. The training time is still stagnant after 4000 epochs though, weird! It should definitely be increasing as I increase the number of epochs.

And it stalls even if you alter the number of layers or number of neurons per layer? You shouldn't ever really have a need for more than two hidden layers.

Where i = number of input neurons and o = number of output neurons:

For one hidden layer, the ideal number of neurons is h = sqrtf(i*o).

For two hidden layers, where r = powf(i/o, 1.0f/3.0f), the ideal number of neurons for hidden layer 1 is h1 = o*r*r, and for hidden layer 2 is h2 = o*r.

I got these equations from Practical Neural Network Recipes in C++.

I am not having any issues with number of nodes/layers. I am just using two layers with 40 nodes each. The problem is when I increase the number of epochs past 4000, it doesn't seem to be training on more epochs than 4000 because the training time is the same rather I use 4000, 5000, 10000, or a million epochs (what you call training sessions). If you use more training sessions, the training time should go up (simple math) that is true when I use between 1 and 4000 epochs, its just after 4000 epochs the training time stagnates. I am just trying to find out the limitations of using an MLP on OpenCV so I know them going forward.

Well, the best way to go ahead, if you want to use modern networks, would be to pick up a book on using TensorFlow. There may be a C++ TensorFlow book, I’m not sure. I wish you the best of luck.

Hopefully someone with more experience than I have will give you some advice too?

Ya thanks dude. I found the issue, the termination criteria is BROKEN on OpenCV, See this line: https://github.com/opencv/opencv/blob... Episilon will get set to a non-zero value if you do not specifically set it to 0. which is an issue in the training loop here: https://github.com/opencv/opencv/blob... Because even if the user specifies to terminate training with only the number of epochs, that line will terminate training when error is lower than the default epsilon value. I tried specifically setting termination criteria to be either epsilon or max_iter and setting epsilon=0 and it fixed the problem. Now I can train with any number of epochs! oops..

Glad you found out the problem. I'm not sure how to submit a bug report on GitHub.