It did take a long time to have the work finished on this and it will have a major performance boost of 30-50% over 2.0.0-beta39 from calibration to integration. We extensively optimized many critical parts of APP. All has been tested to guarantee correct optimizations. Drizzle and image resampling is much faster for instance, those modules have been completely rewritten. Much less memory usage. LNC 2.0 will be released which works much better and faster than LNC in it's current state. And more, all will be added to the release notes in the coming weeks...
Update on the 2.0.0 release & the full manual
We are getting close to the 2.0.0 stable release and the full manual. The manual will soon become available on the website and also in PDF format. Both versions will be identical and once released, will start to follow the APP release cycle and thus will stay up-to-date to the latest APP version.
Once 2.0.0 is released, the price for APP will increase. Owner's license holders will not need to pay an upgrade fee to use 2.0.0, neither do Renter's license holders.
Would be great to know what I can do better here, it is ok for the Cocoon but for larger nebulae it is a bit of a challenge to do the LPC without affecting anything at the borders.Â
Improved, data calibration of severe Amp Glow, amp glow does not behave linearly so it's always important to not use dark scaling if your sensor has significant amp glow. You need to create darks of the same offset, gain/ISO, temperature and exposure time as your lights, to properly deal with amp glow. Now, in cases of strong amp glow, it can happen that the glow is so strong when compared to the real photon signal and noise in our data that a large part of these pixels will be calibrated to zero if we subtract a suitable dark frame. This will happen in each frame for different pixels and that will have a cumulative effect in image integration. The statistics of the pixelstacks will actually be skewed to the upside still showing residual amp glow in some cases. In 2) Calibrate, there is now a new option that will adaptively solve this for data were this happens. It's called Adaptive Data Pedestal. If you enable this, it will prevent severe clipping on the black point in calibration, and will therefore prevent skewed pixel statistics in image integration. It's an adaptive algorithm, so it takes a little bit more time in calibration. A case were you probably need to enable this is strong Amp Glow with camera's like the ZWO ASI183. For most camera's and datasets, this setting will have no effect, but in cases where severe black clipping in calibration does occur, for whatever reason, this setting will prevent it and will ensure precise calibration and no-skewed statistics in image integration.
Example of an integration of ZWO ASI183mm-c Oxygen-III data provided by Warren Landis @orangemaze, with and without the setting enabled in calibration:
In this case, the amp glow is not calibrated out with the Adaptive Data Pedestal disabled, because the amp glow simply overpowers the real OIII data on the location of the amp-glow. By enabling the Adaptive Data Pedestal, the amp glow is properly calibrated.
I'll download your data to test if the solution works on your data as well.
The other settings in the other modules (than calibration) should not affect this. It's a problem that arises especially with narrowband data where the amp glow is not covered by the photon signal and noise levels. Basically, your photon signal/noise levels are swamped by the amp glow signal.
I have checked, there is a problem with your Masterdark, it's created in APP 1.059:
Image #1 SIMPLE = T / Java FITS: Thu May 03 21:23:50 CEST 2018 BITPIX = 16 / bits per data value NAXIS = 2 / number of axes NAXIS1 = 4656 / size of the n'th axis NAXIS2 = 3520 / size of the n'th axis EXTEND = T / Extensions are permitted BSCALE = 1.0 / scaling to 16bit BZERO = 32768.0 / offset data range to that of unsigned short DATE = '2018-05-03T19:30:01' / creation date of master dark SOFTWARE= 'Astro Pixel Processor by Aries Productions' / software VERSION = '1.059 ' / Astro Pixel Processor version CALFRAME= 'master dark' / master dark frame INSTRUME= 'ZWO ASI1600MM Pro' / instrument name CFAIMAGE= 'no ' / Color Filter Array pattern GAIN = 75.0 / gain or ISO depending on instrument EXPTIME = 120.0 / exposure time (s) MEAN = ' 1,13E+01' / mean of channel MED = ' 1,10E+01' / median of channel SIGMA = ' 1,71E+01' / standard deviation of channel NOISE = ' 3,01E+00' / MRS gaussian noise estimate of channel NUMFRAME= 30 / # number of frames used in MasterDark creation
APP 1.061 had a very big upgrade of the calibration engine, and this old masterdark is not compatible since a MasterBias is already subtracted from the MasterDark in APP 1.059. In APP 1.061 or higher that will never happen again.
So you need to create a new MasterDark in APP 1.061 (or higher) of the same temperature and gain and exposure time and then all should be a fine.
I will try if I can fix the Masterdark that we have now by adding the bias pedestal to the MasterDark though 😉 will keep you posted.
And apologies for the inconvenience. The old calibration engine had some flaws and this issue was one of them, the masterdark with the masterbias already subtracted gives you less flexibility for future processing and also is a problem which has to do with black clipping in calibration.
SIMPLE = T / Java FITS: Thu May 03 21:23:50 CEST 2018 BITPIX = 16 / bits per data value NAXIS = 2 / number of axes NAXIS1 = 4656 / size of the n'th axis NAXIS2 = 3520 / size of the n'th axis EXTEND = T / Extensions are permitted BSCALE = 1.0 / scaling to 16bit BZERO = 32768.0 / offset data range to that of unsigned short DATE = '2018-05-03T19:27:36' / creation date of master bias SOFTWARE= 'Astro Pixel Processor by Aries Productions' / software VERSION = '1.059 ' / Astro Pixel Processor version CALFRAME= 'master bias' / master bias frame INSTRUME= 'ZWO ASI1600MM Pro' / instrument name CFAIMAGE= 'no ' / Color Filter Array pattern GAIN = 75.0 / gain or ISO depending on instrument EXPTIME = 0.2 / exposure time (s) MEAN = ' 7,69E+02' / mean of channel MED = ' 7,69E+02' / median of channel SIGMA = ' 3,41E+00' / standard deviation of channel NOISE = ' 2,79E+00' / MRS gaussian noise estimate of channel NUMFRAME= 98 / # number of frames used in MasterBias creation
And the MasterDark:
SIMPLE = T / Java FITS: Thu May 03 21:23:50 CEST 2018 BITPIX = 16 / bits per data value NAXIS = 2 / number of axes NAXIS1 = 4656 / size of the n'th axis NAXIS2 = 3520 / size of the n'th axis EXTEND = T / Extensions are permitted BSCALE = 1.0 / scaling to 16bit BZERO = 32768.0 / offset data range to that of unsigned short DATE = '2018-05-03T19:30:01' / creation date of master dark SOFTWARE= 'Astro Pixel Processor by Aries Productions' / software VERSION = '1.059 ' / Astro Pixel Processor version CALFRAME= 'master dark' / master dark frame INSTRUME= 'ZWO ASI1600MM Pro' / instrument name CFAIMAGE= 'no ' / Color Filter Array pattern GAIN = 75.0 / gain or ISO depending on instrument EXPTIME = 120.0 / exposure time (s) MEAN = ' 1,13E+01' / mean of channel MED = ' 1,10E+01' / median of channel SIGMA = ' 1,71E+01' / standard deviation of channel NOISE = ' 3,01E+00' / MRS gaussian noise estimate of channel NUMFRAME= 30 / # number of frames used in MasterDark creation
Check the median values:
The Masterbias has a median of 7,69E+02, which is scientific notation for 769 ADU
The Masterdark has a median of 1,10E+01, which is = 11 ADU
Clearly the bias pedestal is removed from the MasterDark and that is no longer compatible with APP's improved calirbation engine.
Perhaps, we can solve this by simply adding the bias median of 769 to the Masterdark using the Batch modify tool, simply add 769 to the masterdark. And then assing that frame as a masterdark in 1)LOAD
I am checking now if that works. If it doesn't I will add the complete masterbias to the masterdark to fix it 😉
I mistakenly removed the post that you created just now....
Oh ok that makes sense! Thanks a lot for the info. Don't worry about ...
To follow up though on
Perhaps, we can solve this by simply adding the bias median of 769 to the Masterdark using the Batch modify tool, simply add 769 to the masterdark. And then assing that frame as a masterdark in 1)LOAD
This is what I get by simply adding 769 to the masterdark:
But if you create a new MasterDark, I am sure that will improve on this further 😉
Awesome Mabula! I didn't have the time this morning to shoot new darks and did a test integration with the added ADU. Works like a charm! Very nice and flat stack, also the banding noise I faintly saw when using gain 0 is completely gone. Thanks for the help!
That Cocoon is awesome as well 😉 Excellent image and beautiful colors !
Like I said, if you would create a new masterdark, chances are that calibration is still beter, but judging from this result, it probably is quite a minor difference....
Dropbox File (Private or Invalid)
