Adaptive Pedestal creating dark artifacts

@jhart Hi,

Since they have similar shapes, they probably are dark pixels that haven't been properly removed by the flat calibration. The shape is because of drift during imaging. You should be able to see them in the raw lights as well, though they will be either the size of a single pixel or perhaps a few pixels, depending on the amount of drift during a single exposure. In the raw flats those hot pixels probably are a single pixel which is why, if they are not single pixels in the raw lights, they cannot be removed.

HTH, Wouter

Hi Wouter,

Thanks for the reply.  I can't really follow your technical explanation of how that all works.  This may be a dumb question but can the reduced amp glow part of Calibrate be applied without the use or triggering of adaptive pedestal's adverse magnification/consolidation of dark pixels?

Also, I only notice this problem when using the extract Ha and OIII algorithms on lights captured with the L-Ultimate filter?

Jeff

@jhart Hi Jeff,

The adaptive pedestal is applied by APP if it detects that it is necessary. This is usually the case if the lights aren't exposed enough. In that case, the noise level in the lights is very similar to that of the darks, resulting in pixels in the lights with zero or a negative signal when the master dark is applied in the calibration step. In order to avoid these zero or negative signal pixels, APP applies a pedestal to the lights.

The only way to fix this is by using longer exposure times. Remember that the L-eXtreme filter has very narrow bandwidths so in order to gather enough signal in all pixels, long exposure times are required. The L-Ultimate filter has even narrower bandwidths so even longer exposure times are required when using that filter.

HTH, Wouter

Hi Wouter,

Thanks for that explanation of APP's added pedestal to pixels in certain cases.  Is that process related to APP's deduction of amp glow?  Or, are those two separate things in APP?  As mentioned above, if I get those strange artifacts, I can go back and reprocess the lights with adaptive pedestal unchecked in Calibration and the artifacts will disappear in the resulting integration.  That integration without the application of the adaptive pedestal looks to be similar in quality to the one with it checked; however, it will have the amp glow star burst.  There is no amp glow if adaptive pedestal is checked.  Is there any way in APP to apply its amp glow reduction without the adaptive pedestal?

I will try to increase my exposure times in the future, but it would be nice to have the option to uncheck adaptive pedestal should those artifacts show up and still have the amp glow removed.

One last question:  Would increasing the bias or offset for the camera help in preventing APP's adaptive pedestal from creating those artifacts?  I use a bias of 30 for my ASI294MC.

Jeff

Hi Jeff,

Posted by: @jhart

Thanks for that explanation of APP's added pedestal to pixels in certain cases.  Is that process related to APP's deduction of amp glow?  Or, are those two separate things in APP?

Those are two separate things though they do happen both when the dark calibration is performed.

Posted by: @jhart

As mentioned above, if I get those strange artifacts, I can go back and reprocess the lights with adaptive pedestal unchecked in Calibration and the artifacts will disappear in the resulting integration.

As I tried to explain before, the strange artifacts happen because of the calibration with the flats, not with the darks. The dead pixels are identified in the master flat and APP attempts to remove them when applying the master flat for flat field calibration. However, if the dead pixels appear as little streaks in your lights because of drift while imaging, APP cannot remove them. The only solution for that is to minimize drift which can be achieved with better polar alignment and better guiding. If both already are as good as they can be then increasing the exposure time will be the only way to not apply the adaptive pedestal which makes them visible. They will still be there but they will drown in the background.

Posted by: @jhart

Would increasing the bias or offset for the camera help in preventing APP's adaptive pedestal from creating those artifacts?  I use a bias of 30 for my ASI294MC.

Unfortunately not. The offset is applied to both the lights and the darks (and the bias and flats too) so the net result still is that the background level of the lights will be too similar to that of the darks. In other words, you are not sky limited but dark current limited and that is why you need to increase the exposure time.

HTH, Wouter

Thanks again Wouter,

A very good and understandable explanation.

One last question:  Would it be possible to have separate enable boxes in Calibration; one for adaptive pedestal and one for amp glow removal, instead of combining them into one?

Jeff

@jhart Jeff,

If you hover with your mouse cursor over the checkbox, you'll get a popup explaining that both are correlated.

Wouter

Hi Wouter,

Would it possibly be helpful if the added pedestal had a manual setting (kind of like what I understand PixInsight has) or some sort of user adjustment?  I have no idea of the workings of the adaptive algorithm but I have read it was added in APP to deal with the amp glow problem of the cmos cameras with the 183 chip.  I don't have a 183 camera but I understand its amp glow is worse than the 294 sensor cameras.  Could it be that allowing the user to tweak the strength of that algorithm would better tailor it to deal with the particular camera's amp glow while mitigating the dark artifacts it creates?  Maybe put a strength slider or a high/medium/low choice there in that section of the Calibration panel?

Jeff

Hi @jhart,

Thank you for sharing your problem.

The dark artifacts are definitely not caused by the adaptive pedestal. The Adaptive Pedestal in this case shows you that something is missing in your data capture process plus calibration frames.

Clearly, your data needs the pedestal. The black holes will result from dark subtraction of hot pixels which begs the question if your dark frames are properly matching your light frames in terms of temperture, sensor gain+offset and exposure time?

Normally the subtraction of the hot pixels is corrected by using a Bad Pixel Map. Are you using one? And did you try to make a BPM with lower hot pixel kappa settings so more hot pixels are corrected?

Finally, the black holes are clearly clustered, which indicates no dithering or simply to small dithering in your capture process.

Now, all these things will affect your issue here, so the question is which is missing so you can improve that to solve the issue.

To be clear, the adaptive pedestal is very important and had an effect on proper processing thus resulting in excellent amp-glow correction provided the calibration data matches the lights. So the amp-glow correction in itself is a direct consequence of applying an adaptive pedestal when needed.

Providing a static pedestal/offset will not give different results provided you set it high enough. APP simply can calculate a proper value for your data and makes sure it is set high enough.

Mabula

Hi Mabula,

Thank you for the detailed info and suggestions.

Posted by: @mabula-admin

Clearly, your data needs the pedestal. The black holes will result from dark subtraction of hot pixels which begs the question if your dark frames are properly matching your light frames in terms of temperture, sensor gain+offset and exposure time?

As you point out, my data is not the best.  Imaging in my yard in a Bortle 8-9 zone creates all kinds of challenges for me.  I think my darks match the details of my lights; however, I have not created new ones in a while.  I will create a new set and try them.

Posted by: @mabula-admin

Normally the subtraction of the hot pixels is corrected by using a Bad Pixel Map. Are you using one? And did you try to make a BPM with lower hot pixel kappa settings so more hot pixels are corrected?

In the Calibration panel, I pretty much leave everything at the defaults including leaving Create BPM set to automatic.  I checked and there was a BPM created for the Ha integration excerpts above.  I have never adjusted the kappa there.  Do you recommend that I change the setting from automatic to enable and set the hot pixels kappa to something lower than 3.0? (Leave cold pixels % at 10?)

Posted by: @mabula-admin

Finally, the black holes are clearly clustered, which indicates no dithering or simply to small dithering in your capture process.

As I recall my dithering for the captures that went into that integration was 6 pixels on the imaging camera in a random pattern and every 5 or 6 frames.  From what you see, do you recommend more pixels and/or more frequent dithers?  Wouter also recommended that increasing exposure times would help, particularly on narrow band images.

Posted by: @mabula-admin

To be clear, the adaptive pedestal is very important and had an effect on proper processing thus resulting in excellent amp-glow correction provided the calibration data matches the lights. So the amp-glow correction in itself is a direct consequence of applying an adaptive pedestal when needed.

Providing a static pedestal/offset will not give different results provided you set it high enough. APP simply can calculate a proper value for your data and makes sure it is set high enough.

I understand now the effect of the added pedestal on correcting amp glow and really appreciate your recommended ways to insure APP has the best data for calculating and applying that needed pedestal.  I will definitely follow those recommendations.  Hopefully I will see significant improvement.  The question in my immediately preceding post was directed to unfortunate situation (hopefully avoided going forward) where a compromise pedestal setting less than optimal could be set to remove most amp glow while avoiding bringing out the dark artifacts.  I could then possibly make some use of that marginal data.  Again, that may be a dumb question since I have no idea of the adaptive algorithm you developed in APP.

Thanks again.  I am a big fan of APP and enjoy learning more and more about it.

Jeff