15th Feb 2024: Astro Pixel Processor 2.0.0-beta29 released - macOS native File Chooser, macOS CMD-Q fixed, read-only Fits on network fixed and other bug fixes
7th December 2023: added payment option Alipay to purchase Astro Pixel Processor from China, Hong Kong, Macau, Taiwan, Korea, Japan and other countries where Alipay is used.
Would be great to know what I can do better here, it is ok for the Cocoon but for larger nebulae it is a bit of a challenge to do the LPC without affecting anything at the borders.
Improved, data calibration of severe Amp Glow, amp glow does not behave linearly so it's always important to not use dark scaling if your sensor has significant amp glow. You need to create darks of the same offset, gain/ISO, temperature and exposure time as your lights, to properly deal with amp glow. Now, in cases of strong amp glow, it can happen that the glow is so strong when compared to the real photon signal and noise in our data that a large part of these pixels will be calibrated to zero if we subtract a suitable dark frame. This will happen in each frame for different pixels and that will have a cumulative effect in image integration. The statistics of the pixelstacks will actually be skewed to the upside still showing residual amp glow in some cases. In 2) Calibrate, there is now a new option that will adaptively solve this for data were this happens. It's called Adaptive Data Pedestal. If you enable this, it will prevent severe clipping on the black point in calibration, and will therefore prevent skewed pixel statistics in image integration. It's an adaptive algorithm, so it takes a little bit more time in calibration. A case were you probably need to enable this is strong Amp Glow with camera's like the ZWO ASI183. For most camera's and datasets, this setting will have no effect, but in cases where severe black clipping in calibration does occur, for whatever reason, this setting will prevent it and will ensure precise calibration and no-skewed statistics in image integration.
Example of an integration of ZWO ASI183mm-c Oxygen-III data provided by Warren Landis @orangemaze, with and without the setting enabled in calibration:
In this case, the amp glow is not calibrated out with the Adaptive Data Pedestal disabled, because the amp glow simply overpowers the real OIII data on the location of the amp-glow. By enabling the Adaptive Data Pedestal, the amp glow is properly calibrated.
I'll download your data to test if the solution works on your data as well.
The other settings in the other modules (than calibration) should not affect this. It's a problem that arises especially with narrowband data where the amp glow is not covered by the photon signal and noise levels. Basically, your photon signal/noise levels are swamped by the amp glow signal.
I have checked, there is a problem with your Masterdark, it's created in APP 1.059:
Image #1 SIMPLE = T / Java FITS: Thu May 03 21:23:50 CEST 2018 BITPIX = 16 / bits per data value NAXIS = 2 / number of axes NAXIS1 = 4656 / size of the n'th axis NAXIS2 = 3520 / size of the n'th axis EXTEND = T / Extensions are permitted BSCALE = 1.0 / scaling to 16bit BZERO = 32768.0 / offset data range to that of unsigned short DATE = '2018-05-03T19:30:01' / creation date of master dark SOFTWARE= 'Astro Pixel Processor by Aries Productions' / software VERSION = '1.059 ' / Astro Pixel Processor version CALFRAME= 'master dark' / master dark frame INSTRUME= 'ZWO ASI1600MM Pro' / instrument name CFAIMAGE= 'no ' / Color Filter Array pattern GAIN = 75.0 / gain or ISO depending on instrument EXPTIME = 120.0 / exposure time (s) MEAN = ' 1,13E+01' / mean of channel MED = ' 1,10E+01' / median of channel SIGMA = ' 1,71E+01' / standard deviation of channel NOISE = ' 3,01E+00' / MRS gaussian noise estimate of channel NUMFRAME= 30 / # number of frames used in MasterDark creation
APP 1.061 had a very big upgrade of the calibration engine, and this old masterdark is not compatible since a MasterBias is already subtracted from the MasterDark in APP 1.059. In APP 1.061 or higher that will never happen again.
So you need to create a new MasterDark in APP 1.061 (or higher) of the same temperature and gain and exposure time and then all should be a fine.
I will try if I can fix the Masterdark that we have now by adding the bias pedestal to the MasterDark though 😉 will keep you posted.
And apologies for the inconvenience. The old calibration engine had some flaws and this issue was one of them, the masterdark with the masterbias already subtracted gives you less flexibility for future processing and also is a problem which has to do with black clipping in calibration.
SIMPLE = T / Java FITS: Thu May 03 21:23:50 CEST 2018 BITPIX = 16 / bits per data value NAXIS = 2 / number of axes NAXIS1 = 4656 / size of the n'th axis NAXIS2 = 3520 / size of the n'th axis EXTEND = T / Extensions are permitted BSCALE = 1.0 / scaling to 16bit BZERO = 32768.0 / offset data range to that of unsigned short DATE = '2018-05-03T19:27:36' / creation date of master bias SOFTWARE= 'Astro Pixel Processor by Aries Productions' / software VERSION = '1.059 ' / Astro Pixel Processor version CALFRAME= 'master bias' / master bias frame INSTRUME= 'ZWO ASI1600MM Pro' / instrument name CFAIMAGE= 'no ' / Color Filter Array pattern GAIN = 75.0 / gain or ISO depending on instrument EXPTIME = 0.2 / exposure time (s) MEAN = ' 7,69E+02' / mean of channel MED = ' 7,69E+02' / median of channel SIGMA = ' 3,41E+00' / standard deviation of channel NOISE = ' 2,79E+00' / MRS gaussian noise estimate of channel NUMFRAME= 98 / # number of frames used in MasterBias creation
And the MasterDark:
SIMPLE = T / Java FITS: Thu May 03 21:23:50 CEST 2018 BITPIX = 16 / bits per data value NAXIS = 2 / number of axes NAXIS1 = 4656 / size of the n'th axis NAXIS2 = 3520 / size of the n'th axis EXTEND = T / Extensions are permitted BSCALE = 1.0 / scaling to 16bit BZERO = 32768.0 / offset data range to that of unsigned short DATE = '2018-05-03T19:30:01' / creation date of master dark SOFTWARE= 'Astro Pixel Processor by Aries Productions' / software VERSION = '1.059 ' / Astro Pixel Processor version CALFRAME= 'master dark' / master dark frame INSTRUME= 'ZWO ASI1600MM Pro' / instrument name CFAIMAGE= 'no ' / Color Filter Array pattern GAIN = 75.0 / gain or ISO depending on instrument EXPTIME = 120.0 / exposure time (s) MEAN = ' 1,13E+01' / mean of channel MED = ' 1,10E+01' / median of channel SIGMA = ' 1,71E+01' / standard deviation of channel NOISE = ' 3,01E+00' / MRS gaussian noise estimate of channel NUMFRAME= 30 / # number of frames used in MasterDark creation
Check the median values:
The Masterbias has a median of 7,69E+02, which is scientific notation for 769 ADU
The Masterdark has a median of 1,10E+01, which is = 11 ADU
Clearly the bias pedestal is removed from the MasterDark and that is no longer compatible with APP's improved calirbation engine.
Perhaps, we can solve this by simply adding the bias median of 769 to the Masterdark using the Batch modify tool, simply add 769 to the masterdark. And then assing that frame as a masterdark in 1)LOAD
I am checking now if that works. If it doesn't I will add the complete masterbias to the masterdark to fix it 😉
I mistakenly removed the post that you created just now....
Oh ok that makes sense! Thanks a lot for the info. Don't worry about ...
To follow up though on
Perhaps, we can solve this by simply adding the bias median of 769 to the Masterdark using the Batch modify tool, simply add 769 to the masterdark. And then assing that frame as a masterdark in 1)LOAD
This is what I get by simply adding 769 to the masterdark:
But if you create a new MasterDark, I am sure that will improve on this further 😉
Awesome Mabula! I didn't have the time this morning to shoot new darks and did a test integration with the added ADU. Works like a charm! Very nice and flat stack, also the banding noise I faintly saw when using gain 0 is completely gone. Thanks for the help!
That Cocoon is awesome as well 😉 Excellent image and beautiful colors !
Like I said, if you would create a new masterdark, chances are that calibration is still beter, but judging from this result, it probably is quite a minor difference....
