15th Feb 2024: Astro Pixel Processor 2.0.0-beta29 released - macOS native File Chooser, macOS CMD-Q fixed, read-only Fits on network fixed and other bug fixes
7th December 2023: added payment option Alipay to purchase Astro Pixel Processor from China, Hong Kong, Macau, Taiwan, Korea, Japan and other countries where Alipay is used.
2.0.0 betas seem very slow with CPU under utilised with Big! mosaic
I am currently doing a lot of trial runs on a large mosaic of the Milky Way but its taking over 12 hours per run. If I do something wrong then I have to wait until the next day to try again. I have a fast computer (AMD Ryzen 9 5900X overclocked to a maximum of 5Ghz / 64GB RAM / 1TB PCIe NVMA SSD) but the CPU seems to rarely get above 30% utilisation with APP reporting VERY low utilisation even when doing the integration stage. I would expect the CPU to be running near 100% all the time with AstroPixelProcessor using most of that performance. Its why I have a dedicated computer for this sort of work and my business work is done on another computer. This is possibly one of the biggest mosaics ever created with APP (it needs 768GB of disk space ! ) but is there something I am doing wrong or a configuration change I can check / make to get some better efficiency ? Thanks
EDIT by Mabula: changed title to better reflect the actual issue here
This topic was modified 1 year ago 2 times by Mabula-Admin
I run a very similar setup and don't see this behaviour (WIN11), I suggest that you start by removing the overclock as this can actually make your system run slower, install the latest AMD chipset drivers for your motherboard as well as AMD's power profiles for your OS.
@Mabula-Admin there seems to be a serious performance issue with Beta 16. My usual 12 hour run is nearly at 24 hours and still not finished. It also seems to be requiring more disk space than usual. This screen capture shows the stage that I have not seen go so slow before (the time leading up the the looooong "Completed 28%" stage). Beta 15 had no such problem.
@mabula-admin sorry I am cross posting in two different threads. Eventually the full panorama displayed itself after a very long wait. I realise I forgot to select the middle of Crux as the integration reference and as a result the panorama has been split up. I have done this by mistake in the past and it hasnt caused any noticeable difference in integration time but I thought I should mention it as it might help explain the huge difference in integration time of 27 hours vs previous betas 12 hours.
Beta16 has nothing changed with CPU resources or memory management with regard to beta15.
As it is now, most processing is utilising all cpu cores that are set with the top left cpu slider. But there are some algorithms still single core at the moment, like Local Normalization Correction calculation (applying is multi-core). If the mosaic calculation is difficult then the calculation can be slow to complete as well. Both issues are on our radar and are quite high now on our issue list to fix and improve.
If you process a regular dataset (100 frames, no mosaic) then APP 2.0.0-beta16 is in fact very fast as most people will confirm. With a mosaic, it is clear that we have more improvements to implement and we will 😉 for sure.
@mabula-admin sorry I am cross posting in two different threads. Eventually the full panorama displayed itself after a very long wait. I realise I forgot to select the middle of Crux as the integration reference and as a result the panorama has been split up. I have done this by mistake in the past and it hasnt caused any noticeable difference in integration time but I thought I should mention it as it might help explain the huge difference in integration time of 27 hours vs previous betas 12 hours.
Okay, I can only assume that the extra time here comes from the chosen reference here then. Are you aware that allthough you have a wrong reference in this case, that you can alter the projection of the registered mosaic? Below the start registration button, you change the projection. If you have the caibrated projective model, then you can move the projection center 😉 and check with the registered image viewer mode to see how and where the reference of your choice will be.
Beta16 has nothing changed with CPU resources or memory management with regard to beta15.
As it is now, most processing is utilising all cpu cores that are set with the top left cpu slider. But there are some algorithms still single core at the moment, like Local Normalization Correction calculation (applying is multi-core). If the mosaic calculation is difficult then the calculation can be slow to complete as well. Both issues are on our radar and are quite high now on our issue list to fix and improve.
If you process a regular dataset (100 frames, no mosaic) then APP 2.0.0-beta16 is in fact very fast as most people will confirm. With a mosaic, it is clear that we have more improvements to implement and we will 😉 for sure.
Mabula
I think the time you invest into the mosaics will be well rewarded - from what I have found there is no better software for creating large ones (way better than PTGui Pro or Pixinsight - I own both ). I have yet to register my Ha and Oiii frames that I acquired with my monochrome camera into the whole panorama - its so good that APP can do it all !
@mabula-admin even with the correct reference pan set APP still suffers from severe slowdowns. It seems to happen after the mosaic has been created and I have been looking at a few registered panes to see where they fit, how much overlap they have, are they too badly rotated etc. A single registered pane can end up taking over 15 minutes to load. At the moment to get around this I am trying to save the registered frames as JPGs so I can load them as layers in photoshop after stretching them and compare them that way. Initially they saved at a rate of about 1 per minute but as you can see from this screen capture once they got to pane 32 the time between each frame save grew to almost 30 minutes.
The main thing is that the mosaic does finish getting created - now I am trying to fine tune it by finding and replacing poorly created or aligned panels. I installed a new 2TB Samsung 990 PCIe SSD on the weekend (the biggest and fastest available to me) and it greatly improved performance cutting a 12 hour registration / integration down to 7 hours the fastest it has ever completed it. 😊 (the old disk I was working off was a 1TB Seagate Barracuda SATA cable- connected SSD)
Thank you very much for being so thorough in testing, it must have taken you a lot of time !
I realize that this is a part where we can improve a lot in terms of stability and performance at a moment that APP is still very competitive in this regard compared to other applications like you indicate 😊 . I am very confident that we can make these improvements in due time and I have added your remarks to our already existing issue on our ToDo list to improve the mosaic algorithms, improve the preview of the data and to improve performance in general.
Yes, the IO speed is simply very important here.The SATA interface is a big bottleneck here clearly. Much better performance will be experienced with SSD connections using M2/U2.
Please realize that we need to first finish work on the much anticipated and requested printable manuel and 2.0 stable featuring save/load settings and whole projects, which should also greatly help with the performance issue and big mosaic projects 😉
