Issue with memory usage => Mac is hanging

Hi Philippe,

I think you can check the memory APP actually uses by launching the activity monitor in macOS and looking at the memory tab. It should use the cap properly, so I think background processes are actually taking up the space as well. But good to double check ofcourse.

Furthermore, I'd advice to not go over 300-350 subs per integration, even for those amounts of memory that might cause issues. It's good though to see if we can make this better somehow.

Will check this and report. Thanks.

Noted for the current limitation in the number of subs. 

Hi @vincent-mod 

 I have been doing many other trials. In most cases, the cap is indeed respected by APP (one attached photo of the Activity Monitor shows it is not 100% of the cases).

However, I see that along the process, the "OS Memory" usage is progressively building up to its maximum. In some occurence (I can't say why), the APP suddenly hangs (rolling wheel) even it does not use all its allocated RAM. Even if I only process 200-300 subs (There is a forum message where Mabula says he has processed 400-500 with our any issue).

Check attached photos.

Despite APP hanged, I can then still open other applications with no issues, so that is not the Mac hanging but only APP.

I suppose APP is using other ressources in the Mac than its own. And maybe doesn't free them properly?    

In some cases, I see that the console log gets stuck first in an apparent hang of APP but it still works in the background to some further point (because it can write files on the disk at a time further to the last message on the console).

I just end up buying additional RAM and will see if it goes better. 

Hope this helps.

Photo 1: cap issue ? cap is set at 12Go and apparent use of RAM by APP is 13.4Go. Rare occurence though.

Photo 2: Activity monitor showing APP hanged using only 4Go while cap set at 12Go.

Photo 3: Activity monitor reporting total memory usage while hanged as in photo 3 

I use APP on a Mac with the latest Catalina on it all the time and I never experience such hangs. What happens if it hangs? Do you kill it or leave it running?

I have to kill it (and loose all the work done). I am on Mojave. 

Interesting, I'm wondering if some of these issues can be related to older MacOS releases, it wouldn't surprise me at least. Mabula does try to test for older releases as well, I'll point him to your post.

Posted by: @phfenain

Hi,

I have an iMac with 24Go of RAM and a 250Go SSD fully dedicated to APP (Mac formatted). APP is version 1.078. Mac is an i5 3.5GhZ.

While trying to process a large number of subs (400+), with APP memory usage set at 20Go or 16Go (I tried both), the iMac slow but sure sees its total RAM usage reaching more or less 100% capacity before finally hanging after hours of working hard. All other applications are closed in an intend to save memory.

I have the feeling that APP doesn't manage well the memory cap that was set and is using more memory than allowed (16 or 20 as set).

After 3 trials with 435 subs, it finally worked once. Another trial with 600 subs has failed. Since everything is lost once you have to reboot the computer, I would like to see an explanation and maybe a way to prevent these hangings. 

Thanks.

Philippe     

Hi Philippe @phfenain & @vincent-mod & @wvreeven,

Have you checked if this is somehow related to 1 of the settings in 6) Integrate?

I suspect that you might get RAM issues when turning on LNC with so many frames... the current LNC implementation will consume a lot of memory with so many frames. (So I need to fix that 😉 )

The higher the LNC degree, the bigger the memory problem will be and the longer the LNC calculation will take.... that calculation only uses 1 cpu thread at the moment so it can give the impression that nothing is happening... and that APP hangs where in fact it is still calculating.

Can you test if you can integrate that amount of frames without issues when LNC is turned off?

If you can't, then something odd is happening on your Mac I think which warrents further inverstigation definitely.

Kind regards,

Mabula

Hi Mabula,

Thanks for your answer.

I confirm that this is apparently happening with LNC on (I usually run LNC 1.1) and while the LNC treatment is going on. I have succeeded to integrate successfully quite a lot of subs with the LNC off (can’t remember how many though).
 I also noticed it is probably still running in the background, at least for a while, because, even with the wheel rolling, it manages sometimes to end successfully the integration and save the .fit file. But not always: then, however long you let the wheel running (hours), you would not end up with success.

Good to know you have identified the probable cause and are looking into it! 

Hi Vincent, I had reported a similar problem with the OS activity monitor of APP in Linux before.
But with each step of the process the OS memory usage bar slowly fills up and does not reset itself. It doesn't matter how many frames are loaded and which settings are made.

With kind regards

Hi Minusman, Yes I remember, maybe this is related then or part of the issue. Thanks for reminding! @mabula-admin