Strange colours when darks are used

Hi again....
I have been fighting with the dark frame colour problem all day.  I did a second experiment processing 86x30secs of M31.  Same result (see attachments).  Master flat and  master dark are both listed as RGGB (4144x2822 pix, 16b RGB). When the master dark is implemented, the colours get viered.... am I missing something crucial or is there a bug in the software here? Really frustrating, hoping for suggestions....

Stay safe!
AJ

Hi again...

I am happy to say that I found the cause of the problem described above.  The problem is linked to Astro Photography Tool (APT) the software I use for imaging.  It might be the settings I use that cause the problem OR it might be a software bug, I do not know.  When I obtained dark frames with MaxIm DL (ver 6) and integrated them, deBayering in APP worked fine.

Good to know...:)

AJ

Hi Alf @ajnilsen,

Please send me a subset of all frame types, so lights, bias, darks, flat, flatdarks so I can test what is happening.

Are you sure that all frames have matching gain and offset ?

Did you shoot all frames with the same capture software ?

There is a chance that the darks shot with APT don't match properly for gain + offset, that is my suspicion.

Kind regards,

Mabula

I have uploaded two frames of each type to your server.  I shot all frames with APT and I think they all had gain 120 offset 08, but not 100% sure.... so the error might originate here...  But again, when darks were taken with MaxIm DL, no problem and the same frames were used except for the darks... 

Many thanks for your help!

AJ

Dear Alf @ajnilsen,

thank you for sharing the data, the source of your problem is indeed in the capture process with APT. The lights and bias+darks are completely incompatible due to different sensor offsets:

I have sent you and Ivo (APT) the following conclusion:

I have looked at Alf’s data and indeed, I come to the conclusion that there is a data problem here which is created in the capture process. Either due to wrong settings in APT, or due to a bug in APT.

 

From the metadata of the frames it is hard to tell since the sensor offset can be found in the light frames header, but it is missing for some reason in the bias and darks frames.

Only the gain value of 120 is consistent between all frames.

 

In the light frames, a sensor offset is reported of 8 (which might be a bit low ? but okay):

…

HDU1 - SITELONG= '+06 32 04.000'      / The site Longitude                            

HDU1 - FOCUSPOS=                 2500 / Focuser position in steps                     

HDU1 - GAIN    =                  120 / The gain set (if supported)                   

HDU1 - OFFSET  =                    8 / The offset/black level set (if supported)     

HDU1 - END                               

 

Now the asi294 is a 14bit camera, right? So the adu offset of 8 in 14bits would translate to a value in 16bits of: 8 * 2^2 = 8*4 = 32.

 

So you would expect the darks and bias frames to show that roughly as the median value in the masters, which they clearly don’t :

 

MasterBias from APP created from the bias:

 

HDU1 - MEAN-R  = ' 1,9116E+03'        / mean of channel-R                              

HDU1 - MEAN-G1 = ' 1,9113E+03'        / mean of channel-G1                            

HDU1 - MEAN-G2 = ' 1,9113E+03'        / mean of channel-G2                            

HDU1 - MEAN-B  = ' 1,9116E+03'        / mean of channel-B                              

HDU1 - MED-R   = ' 1,9120E+03'        / median of channel-R                           

HDU1 - MED-G1  = ' 1,9120E+03'        / median of channel-G1                          

HDU1 - MED-G2  = ' 1,9120E+03'        / median of channel-G2                          

HDU1 - MED-B   = ' 1,9120E+03'        / median of channel-B

 

MasterDark from APP created from the darks:

HDU1 - MEAN-R  = ' 1,9117E+03'        / mean of channel-R                             

HDU1 - MEAN-G1 = ' 1,9114E+03'        / mean of channel-G1                             

HDU1 - MEAN-G2 = ' 1,9113E+03'        / mean of channel-G2                            

HDU1 - MEAN-B  = ' 1,9117E+03'        / mean of channel-B                             

HDU1 - MED-R   = ' 1,9110E+03'        / median of channel-R                            

HDU1 - MED-G1  = ' 1,9110E+03'        / median of channel-G1                          

HDU1 - MED-G2  = ' 1,9110E+03'        / median of channel-G2                          

HDU1 - MED-B   = ' 1,9110E+03'        / median of channel-B

 

So this indicates an offset in the bias and dark frames in 16bits of 1912 roughly. In 14bits this would be 1912 / (2^2) = 1912 / 4 = 478 … which is very high ! It is a factor of 15 higher than the sensor offset in the lights…

 

Now, to know that the lights and the darks/bias are completely incompatible is to simple look at the peak of the histogram in the light frames and the darks and bias.

 

The light frames have a peak at an ADU level of around 1400 in the weakest channel which is Red.

 

The bias/darks have a peak at ADU level of 1900… so if you subtract a bias or dark then part of the data becomes seriously negative….if the darks and bias would match the lights for gain + offset, this should not happen.

 

So either the sensor offset was set incorrectly to create either the lights or the bias + darks, or APT mistakenly used a wrong offset while capturing either lights or bias+darks?

Kind regards,

Mabula