[Sticky] How to correct a stack/integration for vignetting

Posted by: Greg

When yo say to "use the third model, elliptical Kang Weiss with geometric factor" it is confusing as I see no settings for that in the vignetting tool. Do you mean to use these settings for integration again? 

Sorry about that, should have made it a bit clearer ;-), here are 3 screenshots with the setting and an example.

( just a single sub of the Cocoon Nebula, shot with a Robtics 102 ED doublet and a Nikon D5100)

 The setting "Elliptical KangWeiss with GF" is the third and most complex model. The second model is less complex, you can try that as well, you need less boxes to get a stable and good solution. The third model is really needed to get good correction in the case where your optics consist of multiple optical elements, like normal objectives with a lot of glass elements or complex optical configurations, for instance multiple element APO's or SCTs, Newtons, RCs.

The original light frame and the corrected light frame, this shows you that the "correct vignetting" tool will calibrate your background based on the supplied boxes while working in the tool, this will give you the best control over the outcome. Also notice in the tools that I have enabled saturation with a high SAT value 0.23 and a low SAT TH value of 0.12 so I saturate the background to help see possible color gradients in the data. This really helps you visually to get a good correction :

Btw, this program is rock solid in performance. Every time I've tried to integrate in DSS using drizzle, it would crash. Not APP, very smooth and impressively robust.

 That 's a great compliment 😉 thank you very much.

Hi Mabula, 

Quick question: 

When I try the correct vignetting I get a very bright background and actually even more vignetting. Any idea what I am doing wrong?

Here is the pic before and after with the selection and the boxes an settings

Thanks a lot Frank!

Edit: First picture in the attachments is the "after" pic

Hi Frank,

Indeed, something is not working correctly here,  have you tried the 2nd model: elliptical Kang Weiss without GF ?

(If you would send me the frame you are trying to correct, I'll have a look myself to see what's happening.)

If that doens't work as well, try the "remove light pollution tool", I am sure that will work to get a flat backgroud 😉

Did you use flats in the calibration of the light frames or not?

Cheers,

Mabula

Hi Mabula,

Yes I did use flats but unfortunately they were not from the same date. However I normally never change the camera position but it can be it is not 100% at 0 Degrees as I normally have the cam, hence my attempt to remove vignetting with your tool

Here is the pic I am trying to remove the vignetting:

www. dropbox.com/s/lh5l4g6odi7ideg/St-med-7800.0s-NR-x_1.0_LZ3-NS-full-eq-add-sc_BWMV_nor-AA-RL-MBB5_1stLNC_it1-mod.fits?dl=0

In case it is of interest here are the subs and Master Flat, Bias and BPM (not the best quality subs I have to admit but I wanted to see what I can get out of them):

www. dropbox.com/sh/zx0zc26jyi7hema/AAChSmkHNaNcsaykVxzh7paja?dl=0

Best Frank

Edit: Deliberately put a space in the url because somehow the forum does not take the link

Hi Frank,

Thank you, I managed to download your data 😉

I'll have a look at it, first thing tomorrow and will give you feedback...

Cheers,

Mabula

Hi Frank,

I think I know the problem.

Since you already used flats to calibrate the lights, the vignetting modelling can't work properly anymore. It's really a tool for the situation that you don't have flats at all. Having applied flats already did change the illumination profiles in your images. The illumination profile that is left after your flat calibration is a profile that can't be modelled using a Kang Weiss model. To have this corrected as it is, you will need to use the  "remove light pollution" tool.

Here is an example on your integrated stack:

That works really well as you can see.

Cheers,

Mabula

p.s. I have added this at the top of this post:

BE AWARE: the "correct vignetting" tool uses special Kang Weiss models to correct vignetting in data that isn't flat-field calibrated yet. If you have applied a masterflat already, and if it doesn't work optimally, adjust the remaining illumination problems with the "remove light pollution" tool. This tool will not work properly in that case because the illumination profile can't be modelled anymore using a Kang Weiss model.

Super! Thanks Mabula!

Posted by: Frank

Super! Thanks Mabula!

You're welcome Frank 😉

So I now understand why the remove vignetting tool doesn't work on my stacks. I've used the remove light pollution tool as you suggested but still have vignetting.

In my mind then I have a couple of options.

1)use the remove light pollution tool again,

2)not load flats when I do my integrations, just BPM and Master Bias

Am I correct? If I choose #1 do I simply keep adding areas to correct and run again. Or save a FITS file and run the remove light pollution too again on the new file?

Thom

Hi Thom,

Indeed, you can use the remove light pollution tool more than once to further correct the background.

If you don't use flats, than you can use the correct vignetting tool, that should work then, but I always advise to make flats if you can. Not all optical illumination profiles can be modeled accurately by Kang-Weiss models and dust on your sensor or optical elements won't be correctedby the model. So it's always best to focus on making good flats.

If you can show the correction that you have and/or still need to do I can advise on how to do this. If you upload the integration file or send it to me using wetransfer.com (support@astropixelprocessor.com), I can show you how I would correct it in APP ?

Kind regards,

Mabula

Hi Thom,

Thank you for sending me your files.

I have looked at your light frame integration file. This looks like flat calibration isn't working correctly. It's probably best if we first try to solve that before looking at further correction with the light pollution removal tool.

I have looked at your calibration masters as well.

• The masterbias looks fine.

• The masterdark looks a bit odd though. The darks had the masterbias subtracted. You can verify this by looking at the ADU values. The Masterbias has a histogram peak near 8000 ADU, so a masterdark without bias frame subtraction, would have a histogram peak at the same or higher ADU value. Your masterdark has a peak near 0 ADU. And it seems that the darks aren't really dark, maybe some light entered the camera, check this screenshot of a strong stretch of your masterdark, it seems some light has entered while taking the darks? It does not seem like a normal pattern for this camera and also not for something like amp glow.

• The masterflat looks fine, and is Masterbias subtracted as well. So no problem there it seems.

What I would suggest to try first, is to make a new integration.

• load your light frames
• load your masterbias
• load your masterflat
• and load your BPM.

So don't use the masterdark for now.

Process the frames up until integration and let me show the result. I think it should look much better 😉

Kind regards,

Mabula

I believe the reason the darks look funny is that there is light coming in the viewfinder. I also have patches of light on some of my other lights from other sessions. I need to cover the viewfinder when I'm shooting Astrophotos, and when making my darks, bias and flats!!!! LOL

I also found my coma corrector was loose in it's holder and was moving around. This accounts for the weird looking stars on the left hand of the frame, but OK stars elsewhere.

I'm running the new integration now and will send it to you when it is done.

Thom

Also how do I check the ADU values of the dark frames in APP??

Also can I run an integration with just the BPM and the Master Bias (no master flats or master darks) , and lights?

You said the master flat looks OK but I'm suspicious of it. It is a flat of a Canon 6D in an Orion Astrograph 8" f/3.9 with a Coma corrector. Just looks like extreme vignetting!!

Thom

Posted by: thompeters

I believe the reason the darks look funny is that there is light coming in the viewfinder. I also have patches of light on some of my other lights from other sessions. I need to cover the viewfinder when I'm shooting Astrophotos, and when making my darks, bias and flats!!!! LOL

 

I also found my coma corrector was loose in it's holder and was moving around. This accounts for the weird looking stars on the left hand of the frame, but OK stars elsewhere.

 

I'm running the new integration now and will send it to you when it is done.

 

Thom

Hi Thom,

I have checked your new integration, I am sending back to you my results 😉

Yes, some more issues it seems, the flats aren't working nicely, but this integration is at least better then the first with the masterdark in the mix.

In the following order:

• the original integration
• calibrated without masterdark, since it contains a clear light leak
• Further correction done by me using the "remove light pollution" tool
• background calibrated
• star color calibrated

So the failing flats can be compensated partially with the "remove light pollution" tool, but it's best to focus on creating good working flats for your setup 😉 I know this can be a bit tricky with fast focal ratios. That needs some experience with the particular optics I think.

Posted by: thompeters

Also how do I check the ADU values of the dark frames in APP??

Also can I run an integration with just the BPM and the Master Bias (no master flats or master darks) , and lights?

You said the master flat looks OK but I'm suspicious of it. It is a flat of a Canon 6D in an Orion Astrograph 8" f/3.9 with a Coma corrector. Just looks like extreme vignetting!!

 

Thom

Yes, I agree, it can be interesting to make an integration without the masterflat. At first sight the masterflat looks normally illuminated, you see the sensor chamber of your 6D as well, that's normal. But after seeing the result with the masterflat applied, it seems that it isn't very good. Possibly due to light leakage in creation of the flats?

If you create an integration with only lights and flats, odds are good that we can correct it better with both the "correct vignetting" and "remove light pollution" tools. My try with correcting the integration with the masterflat still shows some nasty waves in the data.

You can check the ADU value from the FITS metadata, load the master calibration frame and click on details above the image viewer:

Your MasterBias shows:

HDU1 - MEAN-R = ' 2.05E+03' / mean of channel-R
HDU1 - MEAN-G1 = ' 2.05E+03' / mean of channel-G1
HDU1 - MEAN-G2 = ' 2.05E+03' / mean of channel-G2
HDU1 - MEAN-B = ' 2.05E+03' / mean of channel-B
HDU1 - MED-R = ' 2.05E+03' / median of channel-R
HDU1 - MED-G1 = ' 2.05E+03' / median of channel-G1
HDU1 - MED-G2 = ' 2.05E+03' / median of channel-G2
HDU1 - MED-B = ' 2.05E+03' / median of channel-B
HDU1 - SIGMA-R = ' 1.36E+00' / standard deviation of channel-R
HDU1 - SIGMA-G1= ' 1.35E+00' / standard deviation of channel-G1
HDU1 - SIGMA-G2= ' 1.36E+00' / standard deviation of channel-G2
HDU1 - SIGMA-B = ' 1.34E+00' / standard deviation of channel-B
HDU1 - NOISE-R = ' 1.35E+00' / MRS gaussian noise estimate of channel-R
HDU1 - NOISE-G1= ' 1.36E+00' / MRS gaussian noise estimate of channel-G1
HDU1 - NOISE-G2= ' 1.34E+00' / MRS gaussian noise estimate of channel-G2
HDU1 - NOISE-B = ' 1.35E+00' / MRS gaussian noise estimate of channel-B

and the MasterDark:

HDU1 - MEAN-R = ' 3.65E+00' / mean of channel-R
HDU1 - MEAN-G1 = ' 4.38E+00' / mean of channel-G1
HDU1 - MEAN-G2 = ' 5.12E+00' / mean of channel-G2
HDU1 - MEAN-B = ' 3.66E+00' / mean of channel-B
HDU1 - MED-R = ' 3.00E+00' / median of channel-R
HDU1 - MED-G1 = ' 4.00E+00' / median of channel-G1
HDU1 - MED-G2 = ' 5.00E+00' / median of channel-G2
HDU1 - MED-B = ' 3.00E+00' / median of channel-B
HDU1 - SIGMA-R = ' 1.48E+01' / standard deviation of channel-R
HDU1 - SIGMA-G1= ' 1.72E+01' / standard deviation of channel-G1
HDU1 - SIGMA-G2= ' 1.32E+01' / standard deviation of channel-G2
HDU1 - SIGMA-B = ' 1.54E+01' / standard deviation of channel-B
HDU1 - NOISE-R = ' 1.39E+00' / MRS gaussian noise estimate of channel-R
HDU1 - NOISE-G1= ' 1.49E+00' / MRS gaussian noise estimate of channel-G1
HDU1 - NOISE-G2= ' 1.60E+00' / MRS gaussian noise estimate of channel-G2
HDU1 - NOISE-B = ' 1.37E+00' / MRS gaussian noise estimate of channel-B

The median value shows you (more or less) the location of the peak of the histogram. This is also called the location of the data.

Data is reported in 14bits for your camera. So the Masterbias had a peak near 2050 ADU and the masterdark near 0. That's the indication that this masterdark consist of darks that were bias subtracted.

On my priorities list is a tool that will simply show you statistics on any image, so that will soon be implemented 😉

Cheers,

Mabula

Excellent Thanks!!

I am getting similar results now. without flats in my integration. 

I've been applying the "remove light pollution" tool with many more boxes. Sometimes as many as 30 or 40 or more. Sure helps to get good sampling.

I've also adjusted the black point much more than before and this helps too. I just have to be careful not to interfere with some of the nebulosity in this grouping by moving the black point too much.

Thom

Excellent Thom 😉

Have you found the causes of the light leaks?

Mabula

Yes, Indeed!! It is the viewfinder on my camera.

When I look at other subs in the more northern sky there is no light leaks. The camera viewfinder is pointed to the west and there is no light sources in that direction.

If the subs are in the southern sky I see the light leak,  from a street light in the east.

My viewfinder is uncovered and I'm pretty confident the light is coming from the viewfinder and street light.

Okay, that's great information then. You now know were you can directly gain improvement with your data calibration 😉 and thus your end integration !

Mabula

On another topic do you have or plan to have a tutorial on how YOU feel flats should be created??

Hi Thom,

No I am afraid not, currently, I have no specific tutorial in mind regarding the actual creation of flats.

To do that properly, one needs to have examples of all sorts of optical configurations, since different optics tend to need a different approach. This is due to

• focal ratios and
• focal lengths of the optics and
• the optics themselves, (a SCT is different than a APO for example)

But I can definitely give you some pointers for the histogram, iso/gain and exposure values :

• try to illuminate the flats for as long as possible, while
• not clipping the histogram on the right side and while
• staying in the linear domain of your sensor's response. (I know this is not trivial knowledge for any sensor)

Practically, this means,

• set the gain or iso of the camera as low as possible, that enables you to make longer flat exposures. (for isos, try to not go lower than iso 100)
• try to get the peak of the linear histogram halfway or a bit past halfway, that should prevent you from having the peak in the non-linear domain of most sensors.

If you can illuminate for more than 10 seconds while not filling the histogram enough, you can increase gain/iso. The main point is that flats with exposures of fractions of a second, tend to give all sorts of problems. So make sure that the flats have exposures of at least 1 second, preferably more. You can accomplishe this by damping the light source.

Last, always make sure that the light source for the camera's focal lenght is very diffuse. (So making good flats for a wide-angle objective, focal lenght less than 50mm, is not very easy in my own experience).

I hope this will help you for now.

Mabula

OK those are good guidelines!! Thanks. I'd always read the flats needed to be shot at the same ISO as the lights. I'll try this way and see what I come up with!!

Hi Thom,

You're welcome,

You can make the flats at a different iso/gain than the lights, but then you must take into account that the masterbias or masterdark that you use to calibrate the flats (to subtract the bias pedestal for correct flat calibration) must have this same iso/gain value 😉

It's explained in detail in this post:

https://www.astropixelprocessor.com/community/tutorials-workflows/astronomical-data-calibration-priciples-must-read/

An example:

Let's suppose that we have light frames of iso 800 and we will use a masterbias /  masterdark, masterflat and BPM for the light frame calibration.

Now let's assume we shoot the flats at iso 100.

In this case the light frames need a masterbias/masterdark of iso 800.The flat frames however need a masterbias/masterdark (so called masterflatdark) of iso 100.,  to get good calibration.

Cheers,

Mabula

If I understand you correctly: Prior to my creating my stack of lights ( integration) I need to do 2 calibrations first.

1) I need to do a calibration of Bias, Dark and Flat frames at ISO 100. From that I will get a master flat(iso 100).

2) I will also need to use create master bias, and a master dark at ISO 800.

When I do my calibration and integration I need to add my lights(iso800), my master flat(iso100), my master bias(iso800), my master dark(iso800) and my BPM(iso800)

Is this correct?

Thom

Hi Thom,

Yes that should be correct 😉

The BPM can be made of any ISO value, that won't influence the results when using the BPM in calibration.

Kind regrads,

Mabula