Debugging memory leak related to instance with cv::Mat members

asked 2014-01-15 19:19:06 -0600

B. Bogart
119 ●2 ●3 ●13

updated 2014-01-23 13:40:55 -0600

Hello all.

This is related to an already posted question, but I think I've reduced the problem to some degree. I'm totally at my wit's end with this and no one I have talked to has been able to see what could be wrong.

Here is all the code. I put in both leaking and non-leaking cases to make the problem more clear.

This plot shows the difference in memory usage in both cases. The only difference in the code between the cases is lines 353 and 449 in clustering.cpp where mergePerceps() is called only in the leaking case. If I comment out mergePerceps() no leak, if I comment out line 452 (*(distIter->unit)).clone(newUnit); no leak, if I remove the cv::Mat members from the class, no leak.

I've tried valgrind memcheck without any hints to the problem, due to the total lack of any explicit allocation.

If there is another Linux user out there that could verify this is reproducible on another machine, I would really appreciate it. One only needs a sequence of appropriately named images to test, see segmentation::readFrame() line 82 of segmentation.cpp for details. I'm using OpenCV 2.4.5 on Ubuntu Precise 64bit.

It seems I'm using cv::Mats in some unexpected way, so I would be happy to hear what the proper method of doing this operation is to avoid this leak.

If this is a poor use of this system, I apologize. If there is a better venue for this type of question, I'd be happy to move it. Thanks.

Update 1

Here (~70MB) are some test images. The path needs to be changed on line 16 of segmentationClusteringTest.cpp, but the rest should work as I did not change the filenames.

Update 2

ms_print output for leaking (top) and nonleaking (bottom) cases:

"Leaking Case"

"Nonleaking Case"

edit retag flag offensive close merge delete

Comments

You forgot to include your images.

Nghia ( 2014-01-15 20:01:38 -0600 )edit

@Nghia See the update above.

B. Bogart ( 2014-01-16 13:01:31 -0600 )edit

add a comment

3 answers

Sort by » oldest newest most voted

answered 2014-01-16 22:03:14 -0600

Nghia

1000 ●5 ●16

After running the binary in segmentationClustering_noGPU_leak, valgrind reports no leak

==3848== HEAP SUMMARY:
==3848==     in use at exit: 47,332 bytes in 242 blocks
==3848==   total heap usage: 20,497,664 allocs, 20,497,422 frees, 273,052,625,181 bytes allocated
==3848== 
==3848== LEAK SUMMARY:
==3848==    definitely lost: 0 bytes in 0 blocks
==3848==    indirectly lost: 0 bytes in 0 blocks
==3848==      possibly lost: 4,676 bytes in 83 blocks
==3848==    still reachable: 42,656 bytes in 159 blocks
==3848==         suppressed: 0 bytes in 0 blocks
==3848== Rerun with --leak-check=full to see details of leaked memory

I'm using opencv-2.4.8 and valgrind 3.8.1 on Ubuntu 13.10 64bit.

edit flag offensive delete link

Comments

Thanks @Nghia. Valgrind does not show any leaks on my end either, but massif does show increasing memory usage (That is the data used to generate the plots above). Can you confirm that massif reports increasing memory usage over time?

B. Bogart ( 2014-01-20 18:54:46 -0600 )edit

I didn't run with massif (valgrind takes way too long) but just monitored the memory usage with "top" and it's sitting very steady for the duration of the run. I'm incline to say there is no memory leak from what I see.

Nghia ( 2014-01-20 23:38:47 -0600 )edit

Could you please run massif? (its faster than memcheck) Over 400 frames the increase is small, likely too small to see in top, but you can use the plots above for comparison. Over 100,000 frames the memory increase becomes very significant. (Memory should increase if you feed it any input, like from a web-cam for example). If memory is increasing on your machine (as in the plots above), then it's clearly the code, but I can't figure how to fix it.

B. Bogart ( 2014-01-21 12:14:31 -0600 )edit

First time using massif. How did you get the nice graph? I only know how to get the text graph, which is rather ugly and crude.

Nghia ( 2014-01-21 18:26:44 -0600 )edit

@Nghia Did the crappy ms_print output show a memory increase? As for the graph, I just parsed the samples from the ms_print output and used it to generate a plot for "total" in R. The crappy internal graph should still show a memory increase.

B. Bogart ( 2014-01-22 18:46:17 -0600 )edit

How is (*(distIter->unit)) freed? I don't see any destructor or any delete code.

Nghia ( 2014-01-22 22:43:30 -0600 )edit

@Nghia In this version the list contains instances (not pointers to instances via new). The clone function just assigns each class member in the existing instance ((distIter->unit)) to the values of the new instance (newUnit). Since cv::Mats are ref counted, as I understand this should work. I also tried ((distIter->unit)) = newUnit;, but memory still increases. The percepUnit destructor does not do anything, since the class does not allocate memory. I tried putting cv::Mat::release() calls in the destructor, but that did not change anything either (which is expected since Mats are ref counted). I also tried calling cv::Mat::release() in percepUnit::clone() for cv::Mat members, but that also did not make any difference.

B. Bogart ( 2014-01-23 13:52:17 -0600 )edit

@Nghia bump.

B. Bogart ( 2014-01-30 20:48:13 -0600 )edit

Forgot you there :) Just had a thought. I'm going to try and run the loop for much longer see if it really does keep eating up more memory. The other is the possibility of memory fragmentation, due to lots of new and delete.

Nghia ( 2014-01-31 07:04:53 -0600 )edit

Thanks @Nghia. I did try feeding the program the same frame over and over, and there is no leak in that case. So I expect the leak would stop when the loop repeats. Now if you feed it frames from a webcam or other video, that should show the leak better. Note that considering the results you post here, I am convinced you are reproducing my problem. As for fragmentation, is that possible with no explicit new/delete ?

B. Bogart ( 2014-01-31 10:49:10 -0600 )edit

see more comments

answered 2014-02-21 11:20:54 -0600

B. Bogart
119 ●2 ●3 ●13

It was recommended by @Nghia that I try and make the percepts a constant size. Sure enough, if I fix the dimensions and type of the cv::Mat members of percepUnit, then the leak disappears.

So it seems to me this is a bug in OpenCV that effects calling clone() and copyTo() on Mats of different sizes that are class members. So far unable to reproduce in a simple program. The leak does seem small enough that it may be the headers leaking, rather than the underlying image data.

edit flag offensive delete link

add a comment

answered 2014-01-22 22:32:10 -0600

Nghia

1000 ●5 ●16

updated 2014-01-22 22:41:11 -0600

This is the graph I get.

image description

edit flag offensive delete link

Comments

Ok, good. I put my ms_print output above.

B. Bogart ( 2014-01-23 13:33:49 -0600 )edit

add a comment

Debugging memory leak related to instance with cv::Mat members

Update 1

Update 2

Comments

3 answers

Comments

Comments

Links

Question Tools

Stats

Related questions

Debugging memory leak related to instance with cv::Mat members edit

Update 1

Update 2

Comments

3 answers

Comments

Comments

Links

Question Tools

Stats

Related questions

Debugging memory leak related to instance with cv::Mat members