Flat Field Calibration pops up warning when Bias not used

I realised my problem. I get this error when I'm not using darks. The exposure times where short on the cooled cam that noise wasn't a problem, however APP didn't like this.

Posted by: AstroCookbook Kev

I read somewhere that calibration without BIAS on the ASI1600MM yeilds better results. So I don't use them.

Instead I take FLATS, and immediately cover the scope and take DARKFLATS - so all settings are the same.

APP pops up this warning though:

CRITICAL WARNING: flat-field calibration can not be performed correctly!

To be able to perform accurate flat-field calibration,
the provided flats need to be calibrated by either

- a MasterBias, of the same ISO/gain value as the flat frame, or
- a MasterDarkFlat of the same ISO/gain value and the same exposure time as the flat frame, or
- both

I'm using the second approach - DarkFlats with my Flats, so what is it complaining about? Do I need to create Master DarkFlats first, then remove the subs and then re-calibrate?

Kev

 

Hi @itarchitectkev & @astroolly,

The warning

CRITICAL WARNING: flat-field calibration can not be performed correctly!

To be able to perform accurate flat-field calibration,
the provided flats need to be calibrated by either

- a MasterBias, of the same ISO/gain value as the flat frame, or
- a MasterDarkFlat of the same ISO/gain value and the same exposure time as the flat frame, or
- both

will and should appear if you have loaded flats without bias and/or dark flats or masters of these.

If you have loaded dark flats and/or bias, together with your flats, the warning should not appear. Does it in your case ? If so, I need to investigate and solve this a.s.a.p. 😉

The warning is there to ensure to the user that flat-field calibration is working and will be applied correctly. To be able to get correct flat-field calibration, bias and/or dark flats are needed.

If you load the dark flats as darks, then they will not be used for the flats. To use them for the flats, you need to assing them explicitely as dark flats 😉

Please let me know if this clarifies the appearance of the warning and/or if there might be a bug here.

Kind regards,

Mabula

Mabula, I have loaded dark flats to match the flats, but I don’t load biases and I too get this warning. 

Are you sure that the dark flats are loaded correctly, .i.e. for the same channel ?

If so, can you show a screenshot of the warning and the loaded frames in APP ?

The warning should not appear if dark flats are loaded with lights and assigned to the same channel/session.

Mabula

Mabula, I *think* I know what may be happening. I have a new type of camera, the FLI Kepler KL 4040. It uses what’s called a Scientific CMOS (sCMOS) chip. This camera reads the chip twice for each exposure, once to generate a high gain  sub and once to generate a low gain sub. You process each set of subs separately with low and high gain darks, then use special software to merge the low and high masters into what amounts to a High Dynamic Range final master, which you then do your postprocessing on. However you don’t need to have low and high gain flats. I created my flats using just high gain. I was just doing Ha and created Hi and Low gain channels. But I assigned the flats to All Channels. Would that cause the issue? 

In order to not use biases, you need darks exactly matching lights in terms of temperature and exposure duration. If for example you have a library of 2-minute darks and you want to use them on shorter exposure lights, you need the bias to accurately scale the darks. I frequently did this for ccd images but doesn’t work so well for CMOS. 

wayne

@mabula-admin

I keep getting this warning despite having Dark Flats matched to the same gain/offset/exposure length as my Flats. I'm not taking Bias frames. I'm using Darks of the same gain/offset/exposure as my Lights. 

It doesn't seem to always throw the warning. I'll have to keep testing but I might be only seeing it when I'm using Multi Session mode or Multi Channel mode. I didn't see the error when I most recently tried just a single session, single channel integration.

Posted by: @astroolly

I don’t see why we need to use bias frames, as the bias signal is contained in the darks, so I only use darks and flats with my KAF8300 CCD camera, so as we dont need to use bias, this warining should not appear and tell us that we do...??

could someone explain why we should or need to use both darks and bias....??

Olly

Dear Olly @astroolly,

I can't agree with you here, if you only use darks and flats, your flats will not be calibrated which is a significant problem...,

you need either bias or darkflats to calibrate your flats. Darks will not be used on flats, only on your lights.

We have made a discrimination in APP between darks for lights and darkflats for flats.

To be clear, flat-field calibration can only work well if both the lights and flats have their bias pedestal/offset subtracted by either

• for lights, bias and/or darks
• for flats, bias and/or darkflats.

Kind regards,

Mabula

Posted by: @whixson

In order to not use biases, you need darks exactly matching lights in terms of temperature and exposure duration. If for example you have a library of 2-minute darks and you want to use them on shorter exposure lights, you need the bias to accurately scale the darks. I frequently did this for ccd images but doesn’t work so well for CMOS. 

wayne

Dear Wayne @whixson,

CMOS sensors can have troublesome amp-glow (non-linear pattern), therefore dark scaling is not a viable option for some cmos sensors.

A camera like the asi1600 will be calibrated ideally with

lights - need matching darks and proper flats and these flats need matching darkflats

No bias is needed and especially in this case, bias is really not wanted. The sensor can't create proper bias frames on the shortest exposure times.

Kind regards,

Mabula

Posted by: @shamantanthew

@mabula-admin

I keep getting this warning despite having Dark Flats matched to the same gain/offset/exposure length as my Flats. I'm not taking Bias frames. I'm using Darks of the same gain/offset/exposure as my Lights. 

It doesn't seem to always throw the warning. I'll have to keep testing but I might be only seeing it when I'm using Multi Session mode or Multi Channel mode. I didn't see the error when I most recently tried just a single session, single channel integration.

Hi @shamantanthew,

Thank you for reporting this.

Have you checked that with Multi-Session or Multi-Channel mode, all flats will have suitable dark flats? Maybe something is off in the Multi-Session/Filter modes with respect to assignment of darkflats to the flats. Perhaps you can share the data on which it occurs so I can have a look and posisbly fix it?

You can upload it directly to our server:

https://ariesprodstor.astropixelprocessor.com:7001/

username and password: appuser

Please create a folder with your name and let me know once uploaded 😉

Kind regards,

Mabula

Hi Mabula,

I came searching for this exact issue - I also often these days get the critical error even though my flat frames and master dark flats are the same exposure/gain/offset...I usually do multi-channel and/or multi-session integrations when this occurs.

I feel like the error shouldn't be happening, but what would also help more is if the critical error dialog pointed out which master flat was causing the issue - it is currently a generic message and I don't think possible to pinpoint which of the filters is having the issue.

I'm on v1.075. Camera is ZWO ASI1600MM Pro. One thing is I've changed capture software recently from APT to NINA - my darks/dark flats were captured with APT and my more recent flats have been captured with NINA...could that cause an incompatibility even if the exposure/gain/offsets are matched?

Many thanks

Hi all. I have had the same problems with the "critical warning" message and wondered what the cause was. Today, whilst redoing calibration, I also got an "overwrite file" query - perhaps not surprisingly because there already MFs etc from the first effort. So I cleared all loose files from the working folder - and then I didn't get any warnings at all.

Is it possible that having other APP files in the working folder may spook APP somehow into giving the critical warning message?

Tom

Posted by: @eshy76

One thing is I've changed capture software recently from APT to NINA

Yes that could definitely have influence. Software packages tend to treat data in different ways, sometimes not having correct FITS headers etc. which could confuse APP.

Posted by: @tommohawk

Is it possible that having other APP files in the working folder may spook APP somehow into giving the critical warning message?

If they have the exact name, then yes. If not it shouldn't matter.

Greetings everyone,

I am doing a test usage of this program and I am getting the same critical warning and I am not using bias frames. Part of the message is:

"... a MasterDarkFlat of the same ISO/gain value and the same exposure time as the flat frame"

This happens three times during a multisession/multichannel/filter processing. I am using Ha, SII and OIII with darks, flats and dark flats. All have gain set to 200 as I am using test data from a ZWO ASI 1600 MM. I will be testing with my new camera (QHY 16200A) once my filters arrive. I don't even know if the critical errors are critical and the program does not seem to fail. At the end of the calibration, I have a BPM, a MasterFlat for Ha, OIII and SII. I also have a MasterDark as well as MasterDarkFlats for Ha, OIII and SII. 

So this begs the question of is there an actual problem?

PS. I think there might be a problem as when I go to analyse stars, the MDF for the filter is loaded but it doesn't appear in the information after the frame type. Eg, it says Light 01 MD-1 MF-1 BPM-1 CA. That might suggest that the MDF has been created in name but it is useless information even though it has a fair sized file created (its not zero bytes).

PPS. I tried some LRGB files without any bias or dark flats and got the same error. This is interesting as DSS doesn't demand bias or dark flats to stack. Neither does ASTAP when using it to stack. So without the ability to not use bias or dark flats, flats are not used and the whole process is worthless at removing dust on the sensor. 

Thanks

Stephen

I'm somewhat new to all of this but I was also occasionally getting this error, even when using Bias frames. It occurred to me that it might be that the binning was different between my Flat Frames and my Bias Frames. I checked and saw there was a difference. I took some new Bias Frames at the same binning as the Flats and everything worked smoothly.

It might be helpful if, in the warning that pops up, in addition to warnings about ISO/gain, etc. that a warning about binning be added.

I had this error for the first time today and do have dark flats that match my flats and darks that match my lights.

I am in the middle of my processing, and will let you know how it turns out.

Posted by: @tmyers

I had this error for the first time today and do have dark flats that match my flats and darks that match my lights.

 

I am in the middle of my processing, and will let you know how it turns out.

Were you overwriting any previous files? - see my post above. I was pretty sure this may be one cause.... but maybe not

No, I cleared everything out before starting, I will watch for that though.

Tom - thanks for your question. What I did was a workflow on one day where I had the error but I proceeded anyway which creates a few APP files in the directory. The next day I had the idea about binning and created new bias frames that had the binning matching my flats, darks, and lights. I then went through the whole process again with same working directory/folder as the previous day. The presence of the files from the previous day didn't create any problems or error messages for me.

Ah well... not that then! Thanks for that. I'm not doing any DSO processing ATM so hopefully can return to this when darker skies allow. 

I'm seeing this issue as well. Flats, Dark Flats, Lights, Darks used, and no Bias.

 Any ideas on how to resolve? This is on Version 1.083 (1.083)

 Thank you,

Arvind

@base16 Please use 1.083.3 if you're on windows or 1.083.2 otherwise to see if that solves your issue.