Calibration - when is what subtracted?  

  RSS

(@schurig)
Main Sequence Star Customer
Joined:11 months  ago
Posts: 45
May 23, 2018 22:44  

Hi,

I am still struggling with the exact process of calibration. When playing around with the new version 1.061, I noticed that something probably has changed. I actually never create master calibration frames in the same run as processing lights. I create master calibration frames in a first step and use them for future lights. My workflow was like this:

1) Load all bias frames and create MB
2) Load all dark frames and MB (all darks are flagged with 'B') and create MD (which is Bias subtracted)
3) Load all flat frames and MB (all flats are flagged with 'B') and create MF (which is Bias subtracted); Flat Darks are not necessary with my cam because it has a relatively low dark current

Then I load my lights and all three master calibration frames. I assumed that from each light frame first the MB is subtracted, then second the MD is subtracted and finally this result is divided by the MF. I also assumed that there is no more calculation between the master calibration frames itselves, e.g. the MB is not subtracted from the MD  and MF again (which would be wrong, because it was already done before). So are these assumptions right? And is that still so with the new version?

I noticed in the new version, when loading my single flat frames and my MB frame in the file list, the flat frames are no longer flagged with 'B' and it seems also that indeed the MB is not subtracted during master flat creation. So unlike before I now get an non-Bias subtracted master flat!

Can you help?

Kind regards, Stephan


ReplyQuote
(@mabula-haverkamp-administrator)
Quasar Admin
Joined:1 year  ago
Posts: 1632
May 24, 2018 00:30  

Hi Stephan @schurig,

If you haven't had a look yet at the new video tutorial, please do, it will explain a lot of the changes and how you can verify what masters are assigned to the lights.

https://www.astropixelprocessor.com/fully-automatic-multi-channel-processing-in-astro-pixel-processor-by-mabula/

Major difference between APP 1.060 and the older versions, is the MasterDark. Like in your workflow, with APP 1.059 an older, the MasterBias was subtracted from the darks, from which a MasterDark was created which had the bias signal and pedestal already subtracted. This was actually a structural problem/bad implementation. In most cases this leads to black clipping of vital calibration data in the MasterDark.

So I had to change this to prevent this.

In APP 1.060 and higher, the Masterbias is never subtracted from the darks if you create the MasterDark. That will ensure no vital calibration data is lost due to black clipping after MasterBias subtraction.

Then when the light frames (or flats) are calibrated, depending on wheter a MasterBias is present, the bias pixel is subtracted from the dark pixel before the dark pixel is subtracted from the light frame pixel (after which the light pixel is divided by the flat pixel). This will ensure that calibration always works as it should whatever masters that you load.

Unfortunately, this does mean that old MasterDarks that have the MasterBias already subtracted are suboptimal and probably less effective in calibration due to the clipped calibration data and need to be replaced if that's still possible. (and otherwise, create a set of new darks and use APP's new dark scaling function 😉 )

Then I load my lights and all three master calibration frames. I assumed that from each light frame first the MB is subtracted, then second the MD is subtracted and finally this result is divided by the MF. I also assumed that there is no more calculation between the master calibration frames itselves, e.g. the MB is not subtracted from the MD  and MF again (which would be wrong, because it was already done before). So are these assumptions right? And is that still so with the new version?

So your assumptions were right for APP 1.059 and older. For APP 1.060 and later, this does not apply, since the actual calibration calculation is now dependent on which masters are loaded, ensuring optimal calibration in all cases.

I noticed in the new version, when loading my single flat frames and my MB frame in the file list, the flat frames are no longer flagged with 'B' and it seems also that indeed the MB is not subtracted during master flat creation. So unlike before I now get an non-Bias subtracted master flat!

No actually, if you load either a MasterBias, or MasterDarkFlat or both, the flats will be calibrated with those masters to create a correct MasterFlat. I you don't load these, APP will in fact warn you that Flat-field calibration will not work if you don't provide a MasterBias or MasterDarkFlat or both. Try and you will see 😉

The B mark will not show up anymore with your flats. Make sure that you load a MasterBias, or MasterDarkFlat or both with your flats and APP takes care of it properly.

The new calibration engine will issue warnings is you follow a calibration path in which flat-field correction is about to fail. So if you have created a proper Masterflat with the bias subtracted, but then load only lights with the masterflat, so no MasterBias or MasterDark for the lights, you will be warned as well 😉 since the light frames need to have the bias pedestal subtracted at least for proper flat-field correction.

Apologies for the big changes in this regard, but I am certain they are for the better 😉

With the new calibration engine, you still have all options open with the masters that you created, whereas in the old engine, you didn't and possibly had a suboptimal masterdark.

Also note, that the new engine, no longer has restrictions for exposure time and ISO/gain values to be able to assign a master to a light. And assignment is smart and automatic. Only if APP can't decide, will APP ask you which master need to be asssigned to which light.

And do check out the masterdark scaling function ;-), it needs a Masterbias /bias frames to be loaded as well to work.

Let me know if all is clear or if you have more questions.

Mabula

Main developer of Astro Pixel Processor and owner of Aries Productions


ReplyQuote
(@schurig)
Main Sequence Star Customer
Joined:11 months  ago
Posts: 45
May 24, 2018 21:10  

Hi Mabula,

Thanks for your reply. I think I got it now. I already watched the video, but the video did not explain the issue in detail.

Thanx, Stephan


ReplyQuote
(@mabula-haverkamp-administrator)
Quasar Admin
Joined:1 year  ago
Posts: 1632
May 24, 2018 21:24  

You're welcome Stephan @schurig 😉

Cheers,

Mabula

Main developer of Astro Pixel Processor and owner of Aries Productions


ReplyQuote
Share: