1.076 beta test dri...
 
Share:

20th March 2020 - APP 1.078 has been released !

2019 November: Complete LRGB Tutorial of NGC292, The Small Magellanic Cloud by Christian Sasse, Astronomer in Charge of iTelescope.net

2019 September: Astro Pixel Processor and iTelescope.net celebrate a new Partnership!

[Solved] 1.076 beta test drive on Mac  

  RSS

(@wvreeven)
Neutron Star Customer
Joined: 2 years ago
Posts: 155
February 9, 2020 16:39  

Hi Mabula,

Last night I downloaded the 1.076 beta for MacOS and installed in on my MacBook running Catalina. Overall this beta works very well and no crashes or so happened.

The only thing is that this beta seems to consume a LOT more disk space that 1.075 when processing the same data set. I recently took 145 2 minute exposures of the Tarantula Nebula in Ha. On 1.075 this processed fine and on 1.076 beta I needed to free up several tens of Gb in order for the integration to succeed. The result is very nice if I may say so myself but I am very much worried about the disk space consumption.

My hard disk is 233 Gb of which 50 Gb were free before starting the integration. During the integration a popup was shown that I was running out of disk space so I chose the "let's try anyway" option and the integration basically failed (lots of banding in the final result). So I freed up more disk space so 75 Gb were available and started APP again. This time the integration succeeded but a pop up was shows that only 4 Gb of disk space remained.

Needless to say, I am very worried about this.

 

Wouter

This topic was modified 2 months ago by Vincent Groenewold - Moderator

ReplyQuote
Topic Tags
(@vincent-mod)
Quasar Admin
Joined: 3 years ago
Posts: 1416
February 9, 2020 16:56  

Don't worry so much Wouter. 😉 If there's an issue it will ofcourse be resolved, I'll ask Mabula directly if something changed regarding disk space consumption. Besides that, it might be wise to get an external SSD (with USB3 at least) for processing, 50 GB free is not a lot for when you want to process larger projects.

This post was modified 2 months ago 2 times by Vincent Groenewold - Moderator

ReplyQuote
(@wvreeven)
Neutron Star Customer
Joined: 2 years ago
Posts: 155
February 9, 2020 17:22  

Thanks Vincent. My main concern is the increase in disk space use.

EDIT I wonder why 75 Gb free disk space is not enough to process 2.5 Gb of images... Surely this doesn't count as one of those "larger projects", does it?

This post was modified 2 months ago by wvreeven

ReplyQuote
(@vincent-mod)
Quasar Admin
Joined: 3 years ago
Posts: 1416
February 9, 2020 17:30  

No it shouldn't, just saying that in case you want to go for larger projects in the future. I've asked Mabula about it, so will come back to you when I get an answer.


ReplyQuote
(@mabula-admin)
Quasar Admin
Joined: 3 years ago
Posts: 2364
February 9, 2020 18:16  

Hi Wouter @wvreeven & @vincent-mod,

Thank you for you feedback 😉

Nothing has changed in APP 1.076 relative to APP 1.075 in terms of space needed for an integration. Factors that affect the space needed are:

  • enabling MBB because weights will be assigned per pixel and this needs to be stored, the space needed grows with a factor of 1.5x
  • increasing the integration scale factor (so for a drizzle for instance)
  • more images in the stack
  • larger images in the stack

Off course, it will be easy to check by running 2 identical integrations in both APP 1.075 versus APP 1.076, but I would assume that either of these causes have increased the space that is needed for your integration? Could this be the case?

I will do a quick check here to make sure 😉

Mabula

Main developer of Astro Pixel Processor and owner of Aries Productions


ReplyQuote
(@wvreeven)
Neutron Star Customer
Joined: 2 years ago
Posts: 155
February 9, 2020 18:51  

Thanks for the quick reply Mabula. I have made sure that the data set and the integration settings for the runs with 1.075 and 1.076 were exactly the same EXCEPT in 1.075 I chose the integrate (average), weights (quality) and filter (lin mad wins) settings and in 1.076 I chose automatic. I’ll try again with those settings in 1.076 (as far as possible because there are differences) and see what happens.

 

Wouter


ReplyQuote
(@mabula-admin)
Quasar Admin
Joined: 3 years ago
Posts: 2364
February 9, 2020 19:03  

Hi Wouter @wvreeven,

I have tested the HDD space consumption in APP 1.075 versus the current 1.076 beta. Nothing has changed as it shouldn't as I indicated earlier.

Test: integrate 96 H-alpha frames in APP 1.075 & APP 1.076-beta

APP 1.075: average integration consumes 7.4 GB, MBB enabled consumes 11,1 GB (factor 1.5)

APP1.075 average
APP1.075 average MBB

APP 1.076 average & automatic integration consumes 7.4 GB, MBB enabled consumes 11,1 GB (factor 1.5). Scale 2.0x increases space needed 2^2 = 4 -> 44,3GB

APP1.076 average
APP1.076 automatic
APP1.076 automatic MBB
APP1.076 automatic MBB scalex2.0

I can only assume that you did not do exactly the same with regards to the mentioned settings that caused the HDD space requirement to be different. Simply said, the data to be integrated is 32bits floats, if you know the amount of pixels in your Field Of View and the number of images, you can calculate yourself how much space would be needed 😉 Nothing strange is happening here that would increase the HDD space requirement beyond this amount.

Kind regards,

Mabula

Main developer of Astro Pixel Processor and owner of Aries Productions


ReplyQuote
(@wvreeven)
Neutron Star Customer
Joined: 2 years ago
Posts: 155
February 9, 2020 20:43  

Thanks for the quick and thorough analysis Mabula. Very much appreciated. You are right, it isn’t hard to do the math myself but then I didn’t exactly know what is stored on HD by APP and what not.

I am processing the images in 1.076 in two ways: by selecting manually as close as possible the config that I normally use (and have been using in 1.075) and by selecting automatic. But with 1st degree LNC with 1 iteration and 2.0x drizzle enabled in both cases. So far no popups but then I did make sure to have over 75 Gb of disk space available.

Next I will look at the resulting images to see what the differences are.


ReplyQuote
(@mabula-admin)
Quasar Admin
Joined: 3 years ago
Posts: 2364
February 9, 2020 22:28  

Hi Wouter @wvreeven,

You are most welcome 😉 Do let me know what you experience in terms of differences. Realize that Outlier Rejection now works quite differently, by being able to reject on the low and high side. Iterations are done until no pixel is rejected per resulting pixelstack.

Kind regards,

Mabula

 

Main developer of Astro Pixel Processor and owner of Aries Productions


ReplyQuote
(@wvreeven)
Neutron Star Customer
Joined: 2 years ago
Posts: 155
February 9, 2020 23:53  

The results look very similar, though when I specify the integration configuration manually in 1.076 I think I get a slightly better SNR as compared with both 1.076 automatic and 1.075 manually. In both manual cases I made sure to select the same configuration as much as possible which, as stated before, are integrate (average), weights (quality) and filter (lin mad wins in 1.075 and adaptive in 1.076), kappa low 6.0, kappa high 2.5 (kappa 2.5 in 1.075), 1st degree LNC with 1 iteration and 2.0 drizzle.

I'll investigate a bit further but this beta version looks really good!

Thanks again for all the hard work you put into this excellent application Mabula and Vincent!


(@vincent-mod)
Quasar Admin
Joined: 3 years ago
Posts: 1416
February 10, 2020 00:13  

You're most welcome Wouter. We need as much feedback on beta versions as possible, so thank you!


ReplyQuote
 Davm
(@davm)
Main Sequence Star Customer
Joined: 2 years ago
Posts: 29
February 10, 2020 11:19  

I used 1.076 yesterday on an imac, Mavericks. All tabs on automatic and default settings. I found it faster than 1.075 on the same data set and altogether a better end result than adjusting the settings myself. This could of coarse depend on the object and the data. I also saw a large use of disc space, but I have enough anyway!

Chapeau! Mabula

Dave


(@vincent-mod)
Quasar Admin
Joined: 3 years ago
Posts: 1416
February 10, 2020 12:55  

Great to hear Dave, it should be faster for sure, I think on average about 30% and we're working on improving that further.


ReplyQuote
(@mabula-admin)
Quasar Admin
Joined: 3 years ago
Posts: 2364
February 10, 2020 13:22  
Posted by: @wvreeven

The results look very similar, though when I specify the integration configuration manually in 1.076 I think I get a slightly better SNR as compared with both 1.076 automatic and 1.075 manually. In both manual cases I made sure to select the same configuration as much as possible which, as stated before, are integrate (average), weights (quality) and filter (lin mad wins in 1.075 and adaptive in 1.076), kappa low 6.0, kappa high 2.5 (kappa 2.5 in 1.075), 1st degree LNC with 1 iteration and 2.0 drizzle.

I'll investigate a bit further but this beta version looks really good!

Thanks again for all the hard work you put into this excellent application Mabula and Vincent!

Thank you @wvreeven for your feedback, that is highly appreciated 😉

Mabula

Main developer of Astro Pixel Processor and owner of Aries Productions


ReplyQuote
(@mabula-admin)
Quasar Admin
Joined: 3 years ago
Posts: 2364
February 10, 2020 13:23  
Posted by: @davm

I used 1.076 yesterday on an imac, Mavericks. All tabs on automatic and default settings. I found it faster than 1.075 on the same data set and altogether a better end result than adjusting the settings myself. This could of coarse depend on the object and the data. I also saw a large use of disc space, but I have enough anyway!

Chapeau! Mabula

Dave

Thanks a lot Dave @davm for your feedback 😉 Highly appreciated !

Mabula

Main developer of Astro Pixel Processor and owner of Aries Productions


ReplyQuote
Share: