There must be some ...
 
Share:

20th March 2020 - APP 1.078 has been released !

2019 November: Complete LRGB Tutorial of NGC292, The Small Magellanic Cloud by Christian Sasse, Astronomer in Charge of iTelescope.net

2019 September: Astro Pixel Processor and iTelescope.net celebrate a new Partnership!

There must be some memory leak in RGB Combine  

  RSS

(@shlomi)
Main Sequence Star Customer
Joined: 7 months ago
Posts: 35
November 23, 2019 20:56  

When I loaded large files into RGB combine, I see the RAM measure go way high up. However, after exiting the RGB Combine window, even cleaning up all files from the list, the memory never goes down. No trivial operation (such as changing black point or saturation on a small file), which must cause garbage collection does not bring the memory count down. I suspect there is some memory leak there.

This topic was modified 4 months ago by Vincent Groenewold - Moderator

ReplyQuote
Topic Tags
(@wvreeven)
Neutron Star Customer
Joined: 2 years ago
Posts: 161
November 25, 2019 11:28  

APP is written in Java. Being a Java software developer myself, I can tell you that when the Java Virtual Machine (JVM), where the code runs, allocates memory, it never frees the memory back to the operating system. It does free the memory internally but that's it. In Java terms there only is a memory leak if the internal memory of the JVM keeps growing and the JVM needs to keep on allocating more and more memory which to my best of knowledge is not the case with APP.

 

HTH, Wouter


(@vincent-mod)
Quasar Admin
Joined: 3 years ago
Posts: 1416
November 25, 2019 12:46  

I'm also becoming a developer, just not Java so thank you Wouter for that insight. 🙂


ReplyQuote
(@shlomi)
Main Sequence Star Customer
Joined: 7 months ago
Posts: 35
November 26, 2019 23:46  

@wvreeven Thanks for your response. I am a long time java developer myself. My report is referring to the APP probably maintaining some reference to memory and not releasing it, thus GC does not reclaim the space during its cycles. It was evident when I removed all frames from the list, added new ones and made a new heavy operation which resulted in an OOM exception (with a 14GiB heap). Removing everything should have released all references, as there is no way for the system to use the old data again. Regardless of whether the JVM releases the memory back to the OS or not, if there was no leak the memory manager would used that pre-allocated memory and not result in an OOM at that point in time, which it did, forcing me to restart the APPlication.

Also, your assertion that the JVM does not give memory back to the system is no correct, it does do that, and the frequency and magnitude of it can be controlled (e.g. https://stackoverflow.com/a/30464183/1027568).  

It is also a common misconception that "managed memory" languages, such as java and python have no memory leaks, but they easily do, and care should still be taken ( https://dzone.com/articles/how-memory-leaks-happen-in-java-apps ).

If there are some extra logs I could enable, I can try to recreate this. Also, using https://www.eclipse.org/mat/ can help analyze memory leaks on the jvm rather easily. 

Thanks! 

Shlomi


ReplyQuote
(@wvreeven)
Neutron Star Customer
Joined: 2 years ago
Posts: 161
November 27, 2019 10:24  

@shlomi

Thanks for pointing me to the stackoverflow comment about the JVM returning memory to the system. As is stated there, the JVM does this reluctantly because it is an expensive operation and that's probably why I never noticed it.

And thanks for your valuable comments on Java memory management. I am well aware of OOMs happening and I regularly had to use MAT to inspect heap dumps.

Your original message did not mention the OOM that you mention in your reply to my comment. That's why I replied the way I did. I agree that if there was an OOM then there is a memory leak in APP. Thanks for pointing that out.

 

Clear skies, Wouter


ReplyQuote
(@vincent-mod)
Quasar Admin
Joined: 3 years ago
Posts: 1416
November 27, 2019 12:57  

Very cool discussion, thanks both. I notified Mabula directly, I guess he can chime in. 🙂


ReplyQuote
(@shlomi)
Main Sequence Star Customer
Joined: 7 months ago
Posts: 35
November 28, 2019 18:30  

@wvreeven Oh, silly me, I forgot to mention that OOM exception in my first post!


wvreeven liked
ReplyQuote
Share: