Strange color banding in integration  

  RSS

(@geordan)
Molecular Cloud Customer
Joined: 7 months  ago
Posts: 5
September 3, 2018 10:15  

I am getting some strange color banding in my integration which is roughly in the shape of the flat master, as well as some fixed pattern noise (that I was hoping would have been removed by dithering).  What's causing this artifact?  Here is the integration, a single calibrated light, and a sample flat frame:

integration banding
light calibrated
flat

Zooming into the integration reveals what appears to be fixed pattern noise in one direction.  I was dithering in RA only (I have some pretty severe Dec backlash), but only by 2 pixels -- I discovered later that I should have been dithering by much more, but wouldn't some dithering plus calibration have removed the fixed pattern noise?  Polar alignment was off by about 5 arc minutes.

 

Also, I notice a difference when working with raw NEF files vs. FITS; when using FITS, I see more of a screen door effect when zooming into a FITS file compared to NEF:

bayer grid
nef zoomed

What is causing this?  The FITS file is not being detected as being from a color filter array so I turned on Force CFA.


ReplyQuote
(@geordan)
Molecular Cloud Customer
Joined: 7 months  ago
Posts: 5
September 3, 2018 21:09  

Update: I didn't realize 1.063 was out (was using 1.062) so I upgraded to that to see if any of the fixes in there had some effect.  Also, the screen door effect was due to the incorrect Bayer pattern being selected (should have been GBRG, not RGGB).  Between those two changes, the banding issue appears to have gone away.

1393 fixedpatternnoise

However, there is still the issue of the vertical fixed pattern noise; is that just a matter of needing to dither by more pixels, or something I could improve in calibration?

This post was modified 2 months  ago 5 times by Mabula Haverkamp - Admin

ReplyQuote
(@mabula-admin)
Quasar Admin
Joined: 2 years  ago
Posts: 1797
September 3, 2018 22:38  
Posted by: geordan

I am getting some strange color banding in my integration which is roughly in the shape of the flat master, as well as some fixed pattern noise (that I was hoping would have been removed by dithering).  What's causing this artifact?  Here is the integration, a single calibrated light, and a sample flat frame:

integration banding
light calibrated
flat

Hi @geordan,

To me at a first glance, this looks like a problem with how you created your flats. The individual flat shows an odd illumination difference at the top, which then trasnslates in a strange pattern in your integration I think. In the calibrated light, the error is already seen. Can you share how an uncalibrated lights looks?

Zooming into the integration reveals what appears to be fixed pattern noise in one direction.  I was dithering in RA only (I have some pretty severe Dec backlash), but only by 2 pixels -- I discovered later that I should have been dithering by much more, but wouldn't some dithering plus calibration have removed the fixed pattern noise?  Polar alignment was off by about 5 arc minutes.

If dithering is very small, then fixed pattern noise can still be apparent in the integration, so it could explain what you are describing. It depends on how large the fixed patterns are. On Bayer sensors, it's better to dither a bit more than 1-2 pixels. Why not dither 10-20 pixels ? That should improve things considerably and will not harm your field of view normally.

 

 

Also, I notice a difference when working with raw NEF files vs. FITS; when using FITS, I see more of a screen door effect when zooming into a FITS file compared to NEF:

bayer grid
nef zoomed

What is causing this?  The FITS file is not being detected as being from a color filter array so I turned on Force CFA.

Indeed, I fixed a problem in this department in APP 1.063:

https://www.astropixelprocessor.com/community/release-information/astro-pixel-processor-1-063-ready-for-download/

  • FIXED, Flat-field Calibration with Bayer CFA patterns other than RGGB, there was a bug in normalization of flat frames for Bayer CFA sensors. This bug manifested itself if the Bayer CFA pattern was not the default pattern of RGGB. In most cases the resulting masterflat was still okay to be used, but if you created a MasterFlat out of flats with quite different histograms/exposure times then the problem was really severe, no good flat-field calibration was possible.

Example of a MasterFlat from a Canon EOS 5D Mark II with a GBRG CFA pattern, first 2 images is with the bug, notice that the resulting MasterFlat has an odd histogram and that, when zoomed in, the pixel structure is not smooth, due to having normalized the CFA pixels with an incorrect CFA pattern :

MasterFlat GBRG BUG histogram is BAD
MasterFlat GBRG BUG zoomedIn

 

Next 2 images is with the bug fixed, notice that the histogram is now smooth and consistent with a single flat frame and, when zoomed in, the MasterFlat's pixel structure is now smooth:

MasterFlat GBRG BUG FIxed histogram is GOOD
MasterFlat GBRG BUG Fixed zoomedIn smooth

If you need to set Force CFA, do make sure, that you set the correct Bayer CFA pattern before creating the masters, this CFA pattern is stored in the metadata 😉

Kind regards,

Mabula

Main developer of Astro Pixel Processor and owner of Aries Productions


ReplyQuote
(@mabula-admin)
Quasar Admin
Joined: 2 years  ago
Posts: 1797
September 3, 2018 22:56  
Posted by: geordan

Update: I didn't realize 1.063 was out (was using 1.062) so I upgraded to that to see if any of the fixes in there had some effect.  Also, the screen door effect was due to the incorrect Bayer pattern being selected (should have been GBRG, not RGGB).  Between those two changes, the banding issue appears to have gone away.

1393 fixedpatternnoise

However, there is still the issue of the vertical fixed pattern noise; is that just a matter of needing to dither by more pixels, or something I could improve in calibration?

Hi  @geordan,

Yes I can see it in your screenshot. As in my previous answer, this could result from dithering in only 1 direction and with too small dither steps. But it's hard to tell without seeing the actual data.

How many lights were you integrating?

Do you see the same pattern in either the masterbias or the masterdark ?

Which camera is this? A Nikon D90, right?

Mabula

Main developer of Astro Pixel Processor and owner of Aries Productions


ReplyQuote
(@geordan)
Molecular Cloud Customer
Joined: 7 months  ago
Posts: 5
September 3, 2018 23:17  

Yes, the illumination difference in flats is due to the off-axis guider prism intruding a bit into the field of view.  (I can't back it out because of the way the ZWO OAG is designed; backing it out would prevent the guider camera holder from being flush against the OAG body, which would allow it to flex.)

Here's an uncalibrated flat:

Flat 001 St

I was integrating 120 lights, 20 flats, dark flats, and darks.  Bias was made from 76 frames.  It is a Nikon D90.

I don't quite see the same pattern in the master bias or dark:

masterdark
masterbias

I will definitely try out a larger dither next time.  Also, should I remake the bad pixel map using 1.063?

I did note that in 1.063 while the integration was significantly improved, there was still some minor color banding, but it was much less evident.

Thanks for all your help!


ReplyQuote
(@mabula-admin)
Quasar Admin
Joined: 2 years  ago
Posts: 1797
October 14, 2018 12:50  

Hi @geordan,

"Yes, the illumination difference in flats is due to the off-axis guider prism intruding a bit into the field of view. (I can't back it out because of the way the ZWO OAG is designed; backing it out would prevent the guider camera holder from being flush against the OAG body, which would allow it to flex.)"

Just make sure that the OAG position is not changed between shooting the lights and the flats 😉 then making good flats should be possible.

The latest version of APP, 1.068, has some improvements in Bad Pixel Mapping so you can always try to make a new Bad Pixel Map based on a nice set of flats and darks.

Kind regards,

Mabula

Main developer of Astro Pixel Processor and owner of Aries Productions


ReplyQuote
Share: