Share:
Notifications
Clear all

ADUS PEDESTAL


(@anofeles)
White Dwarf Customer
Joined: 4 years ago
Posts: 15
Topic starter  

Hello, everybody...

I know there was in previous versions but now I can't find it (1.083). Where can I enter the number of ADUs that APP has to apply to the pedestal so that it doesn't do it automatically?
Thanks in advance.


ReplyQuote
(@wvreeven)
Galaxy Admin
Joined: 4 years ago
Posts: 1651
 

@anofeles In 1.083 this option has been removed because APP will now always apply a hard coded pedestal. Do you experience any issues with 1.083?


ReplyQuote
(@anofeles)
White Dwarf Customer
Joined: 4 years ago
Posts: 15
Topic starter  

Actually, yes, I do. I don't remember if since beta 3 or 4 or before, but now I always get a message warning me that I have several clipped pixels.
I calibrate my lights with masterdark -same gain (26), offset (80), temperature (-10) and time (180")- flats and darkflats.
My lights are always to the right of the Y axis of the histogram (and with a minimum in the stats of more than 1500 adus).
In my last integration (Andromeda galaxy) the lights have approximately all a minimum value of around 1500 adus).
The masterflat has a median of 24,813, the masterdarkflat, has 1279 and the master dark, 1280 adus.
Camera: QHY268C
I don't know what I could be doing wrong...

 


ReplyQuote
(@mabula-admin)
Quasar Admin
Joined: 5 years ago
Posts: 3122
 
Posted by: @anofeles

Hello, everybody...

I know there was in previous versions but now I can't find it (1.083). Where can I enter the number of ADUs that APP has to apply to the pedestal so that it doesn't do it automatically?
Thanks in advance.

Hi @anofeles & @wvreeven,

The option to define your own pedestal was never there in fact.

And we don't use a hardcoded pedestal but it is based on the standard deviation of the pixel distribution of the masterdark or Masterbias.

The pedestal is 24 * standard deviation. If you then still have clipping occur there must be something not right hence the warning.

The pedestal that APP applies is such that if you then still get clipping, you need to investigate a data problem. The warning is there to help you produce better results.

If you want us to have a look at the data, please upload it here and I will investigate the cause:

https://upload.astropixelprocessor.com/

username and password: upload

Please make a folder with your name and let me know once it is uploaded 😉

Mabula


ReplyQuote
(@anofeles)
White Dwarf Customer
Joined: 4 years ago
Posts: 15
Topic starter  

The files are already uploaded (anofeles_adu_pedestal). There is a set of 10 lights as an example (I checked them and I got the warning again), the MD, the MF, the MDF and the BPM. If you need an example of flats or darkflats instead of the masters, let me know.
Thanks for your software and work.


ReplyQuote
(@mabula-admin)
Quasar Admin
Joined: 5 years ago
Posts: 3122
 
Posted by: @anofeles

The files are already uploaded (anofeles_adu_pedestal). There is a set of 10 lights as an example (I checked them and I got the warning again), the MD, the MF, the MDF and the BPM. If you need an example of flats or darkflats instead of the masters, let me know.
Thanks for your software and work.

Hi @anofeles,

Thank you. I have analysed your data. I think that the data in general looks good, I see no light leakage, and the sensor offset is fine as well for the calibration data versus your lights.

So your issue is in fact that you are underexposing a bit leading to the warning. There are quite a bit of pixels clipping to 0, so that means that a lot of pixels are receiving a low number of photons so the calibrated pixels are not even able to rise above the dark current floor.

You can still process the data without issues, despite the warning 😉

But in statistical terms, you need to look at the standard deviation of your masterdark versus the calibrated light frames.

The standard deviation of the 32bits masterdark is :

HDU1 - SIGMA-R = ' 1,6046E-03' / standard deviation of channel-R
HDU1 - SIGMA-G1= ' 1,6359E-03' / standard deviation of channel-G1
HDU1 - SIGMA-G2= ' 1,6125E-03' / standard deviation of channel-G2
HDU1 - SIGMA-B = ' 1,5365E-03' / standard deviation of channel-B

so roughly 0,0016 in the 0-1 normalised 32bits float range.

Now if we disable the adaptive pedestal in 2), we see that the sky background of the calibrated data after normalisation is found to be at a level of

Background levels without pedestal

0,0074 , this is only 0,0074/0,0016 = 4,6x the standard deviation... so that means that part of all pixels will always clip to 0. You can easily expose much longer.

I think your data is showing that the warning we issue is valid, but you can easily ignore the warning and process your data without issues.

I will think about if I should change the moment when the warning is triggered. In your case, I think it is good that you receive the warning though.

Did you shoot this with a certain filter?

Mabula


ReplyQuote
(@mabula-admin)
Quasar Admin
Joined: 5 years ago
Posts: 3122
 

@anofeles,

I have made some small changed in the 1.083 version so the warning is not triggered on your data 😉

If you uninstall APP, remove the downloaded installer and then download 1.083 and install it again, it should work without any issues.

I think the warning system was a bit too aggressive, so the threshold of triggering it has been adjusted to only show on really problematic data.

Mabula


ReplyQuote
(@anofeles)
White Dwarf Customer
Joined: 4 years ago
Posts: 15
Topic starter  
Posted by: @mabula-admin

Did you shoot this with a certain filter?

Yes, a standard UV/IR 2" filter.

Possibly the error is due to the fact that I thought that having 1500 ADUS as a minimum value in an image meant that there were no pixels with a value of 0 once calibrated. My fault. I guess I'll have to raise that value to 2000. It's always a tricky game trying to keep the minimum and maximum values under control at the same time.
As for APP being more "tolerant" with the warning, I really don't know what to think. On the one hand I understand that it can be "alarming", but on the other hand maybe it's good that you know that X pixels are going to be "clipped"...
In any case, thanks for the work and dedication you put into it.
Happy New Year!
Carlos.


ReplyQuote
(@mabula-admin)
Quasar Admin
Joined: 5 years ago
Posts: 3122
 
Posted by: @anofeles
Posted by: @mabula-admin

Did you shoot this with a certain filter?

Yes, a standard UV/IR 2" filter.

Possibly the error is due to the fact that I thought that having 1500 ADUS as a minimum value in an image meant that there were no pixels with a value of 0 once calibrated. My fault. I guess I'll have to raise that value to 2000. It's always a tricky game trying to keep the minimum and maximum values under control at the same time.
As for APP being more "tolerant" with the warning, I really don't know what to think. On the one hand I understand that it can be "alarming", but on the other hand maybe it's good that you know that X pixels are going to be "clipped"...
In any case, thanks for the work and dedication you put into it.
Happy New Year!
Carlos.

Dear Carlos,

Happy 2022 !

In case of your data, I saw that without the pedestal about 2000 pixels were clipping... with the adaptive pedestal it was reduced to about 200. So much better. I had the threshold for the warning at 100 pixels so it triggered a warning on your data.

I was thinking about this and decided that it makes more sense to warn if the adaptive pedestal not really improves things. In your case, it improved a lot, only 10% of the initial clipping pixels were left. If the pedestal does not improve things clearly, then the warning is needed because it is likely that there is some serious problem.

On your data I could not really see a serious problem.

So i reformulated the requirement for the warning. I now look at the ratio of:

#clipping pixels with pedestal / #clipping pixel without pedestal

if the ratio is lower than 0.5 the warning will not be triggered.

You will always be able to see how many pixels clip in the separate console window if you wish to know. If a frame is being calibrated you will see something like this in that console window:

Console Info AdaptivePedestal

Mabula

 


ReplyQuote
Share: