Twilight flats with a DSRL

Mm, so with a DSLR I personally have never had an issue with the red channel being a bit noisier in flats, usually you illuminate your sensor in such a way that it's well exposed in all channels. This may also be easier using a computer to check the histogram when shooting. The usual rule is to shoot for at least 1-2 seconds, with the histogram as far right as possible (the 50% you see mentioned a lot is just a ball-park, 60-70% can be better if it's not clipping of course). You need proper dark-flats to calibrate them and then all should be well, maybe we can check some of your flats..

Twilight flats in that sense may create an issue, but do you see in your histogram that that channel is clipping to the left?

The problem with twilight flats is that they're very blue, so there is a change that the blue channel is clipping while the red channel is under-exposed. I use a laptop for acquisition, but each program that I use show flats in a different way. APP itself shows them differently between image mode and linear mode. In linear mode it matters whether I set camera white balance or not. The same for image mode. They are all different when looking at the histogram.

The forum software hates RAW's, zipped or not.

Ok, so it is indeed clipping. That is an issue, but I think not solvable with exposing the red channel separately. In that case I think you need a different light-source. In APP, you can check the raw data of your flats by loading the flat, then selecting "no stretch" on the right;

 If you want I can have a quick look at some flats. Go to https://upload.astropixelprocessor.com and use upload1 as username and upload1 as password.

Create a directory named “svdwal-twilightflats” and upload in there. Thank you!

Hi Sander @svdwal,

Yes, like Vincent says, please upload some of those flat frames, some dark flats and some lights. Like 5 of each if possible. Then we can get to the bottom of this and explain exactly what is happening 😉

Mabula

Two sets are being uploaded as I type. prefix sander, as zip files. The timed ones were done by looking at the histogram and choosing a time. These are quite dark. The Av+2.6 ones were shot using the Av method and adding 2.6 stops using EOS utilities. They are not in the folder, can't move them after upload.

I'm not a fan of flat screens. They illuminate a scope in a different way compared to the sky. For some scopes (well-baffled refractors) they work, but other scopes (non-baffled refractors x-Cassegrains, Newtonians) need modification. Light from the panel wil reflect on the walls of these scopes and will end up on the detector. Newtons can be modified easily, non-baffled refractors (like my WO Zenitstar 66D) need to be taken apart.

Day sky and twilight sky T-shirt flats have the same problem.

Mm, I must say that I've used flat-panels for all my Newtons, my current one being a F4. I have no problems there, so I wonder if it might not be a good idea to try one once? The illumination of these panels have to match the light incoming from the sky (or have the same characteristics) otherwise they wouldn't work.

My guess is that with the sky-flats you still need longer exposures, as you say they are a bit dark already. I'll have a look once uploaded. 🙂

Ok, had a look and to me the AV +2.6 flats look fine in the sense of exposure;

You may even go for a little higher exposure, but usually these work ok when calibrated with dark-flats etc.

The other ones are a bit under exposed I think, even though the red is still not clearly clipping. Can you also upload some of your master-dark-flats and/or master-bias?

Two master-darks. The 800 ISO is for the Av flats, and the 400 ISO for the timed ones. Also two sets of lights. The m13 ones are with an C8 and a weird Meade focal reducer so the starts at the edges are out-of-focus.

The problem with lights not matching the sky illumination profiles are subtle. If you're lucky the sides of the scope illuminate evenly, and that gets subtracted fine, but if it is not, you can easily loose subtle detail. When doing photometry, brightness measurements are slightly wrong. It is at the 1-5% level, or lower, but at that level are the Integrated flux nebula, molecular clouds and tidal tails of galaxies. Not that I expect to shoot that from within the city, but why settle for less if the best takes as much time as worse.

Regarding the spectral distribution, check the article by Covington at the link in the first post. I find it weird that a flat changes the color response of a light, as the correction factor per color is almost 1 in the centre of the image, and getting higher to the edges. That means that the color in the center should not change.

I asked Mabula to chime in a bit, I also did a neutralize background to see how the signals overlap and although the illumination is correct, the quality of the data is not very good for a flat (as in, the red is not matching the other histograms well, this is where flat-panels work better usually).

As I said, I'm not keen on flat panels. More stuff to buy, I would need more than one as a small one won't illuminate my C8 and a big one will be precariously to balance om my Zenitstar. Illumination differs from the night sky, which makes subtle structure harder to detect, the spectral response in a scientific filter differs from the night sky response, which makes photometry harder. 

The sole advantage would be that I can shoot flats after the lights, which is good after refocussing.

 
Regarding the flats, shooting in Av mode with the blue pushed as far to the right without clipping, while shooting enough to get a good SNR on the red, seems viable. 

 I also could let the blue clip a bit and agressively kappa-sigma clip the flats while stacking them, but then, it will be hard to access whether that works.

Hi Folks,

Interesting thread that relates to some calibration oddities I am trying to figure out.  If I understand the above correctly, the color profile of the flats when shooting with an OSC / DSLR is important ... I did not realize this was necessary ... I too typically use the twilight sky for my flats.  This becomes particularly tricky when using an OIII-Ha duo band filter causes a strong blue/green background relative to the red channel.  

A couple of questions regarding calibration frames for OSCs / DSLRs:

  - Are we talking about the R/G/B profiles with the in-camera white balance correction applied or not applied?

  - Should the R/G/B profiles of the flats roughly match each other (i.e., the histogram peaks and shapes of the individual channels are similar) or should R/G/B profiles match those of the long exposures lights?

  - If the R/G/B profiles should ideally match each other, is there a way for APP to essentially grayscale the flats / master flat and apply the same flat field correction to all channels?

  - Typically, I shoot with a custom white balance setting when using a filter and daylight white balance when I'm not.  Do the darks/bias/flats/lights all need to shot with same white balance setting?   Does the 'in camera white balance' setting in APP matter when creating/using master calibration frames?

Thanks!

-Bret

Hi Bret

My take on the matter is

1) the flat corrections should be made on the raw numbers, without any camera white balance being applied. It should not matter as these are linear corrections, but one has then to check error propagation on these corrections., as these might be of the same order as the flat correction for pixel sensitivity.

2) in a OSC/DSRL you don't get identical histograms, because the twilight sky is blue. 

3) you don't want the blue and green channels to correct for the red channel, and vice versa, because of differences in pixel sensitivities. Flats are for correcting pixel sensitivity differences, dust specs and vignetting. One should process flats as if the three colors were shot separately, like in LRGB or UBVRI shooting in a mono camera. As soon as you mix the channels you loose the differences in pixel sensitivities. This is bad if you want to do photometry. It will work quite well for vignetting and dust correction, as these happen on scales much larger than pixels.

4) custom color balance doesn't matter, as the flat correction is done before setting the color balance.

Sander

Like Sander mentions, color profiles don't matter. You're working with the raw data from the camera and process them linearly within APP. The histogram peaks do need to be similar yes, similar shape and intensity.

Hi Vincent, Sander,

Thanks for the responses. I typically shoot my flats so that the back-of-the-camera histogram is centered right around the midpoint.  Below are 4 APP histograms with different settings and a Photoshop histogram for one of my unprocessed, original flats.  

Is this particular flat OK?  Which histogram do I use and what are the requirements for a "good" flat field exposure?

Thank you!

-Bret

1) APP histo: Unstretched, In camera White Balance = Not checked

2) APP histo: Unstretched, In camera White Balance = Checked

3) APP histo: Stretched (15%, 5 sigma, 2.5%), In camera White Balance = Not checked

4) APP histo: Stretched (15%, 5 sigma, 2.5%), In camera White Balance = Checked

5) Photoshop Histogram (similar to back-of-the-camera)

If you're loading in the raw frames, the camera settings shouldn't matter. What data are you actually loading into APP?

Other than that if you load in your masterflat, click "neutralize background", you should have a histogram where the peaks are overlaying each other with roughly the same shape.

Hi Vincent,

I'm loading the raw flat frames.  I don't think there is much I can do about the shapes of the R/G/B histogram in my flats ... so, I think the take away is that when using the twilight sky with a strong blue component, I need to ensure the red is sufficiently exposed (out of the read noise) while the blue and green channels are not clipped when the data is viewed without the white balance being applied (i.e., the first histogram in my previous post).   Correct?

Thanks!

-Bret

Yes, you can always expose for longer as long as you're not clipping anywhere. The 50% you see mentioned a lot is not a hard setpoint, 60-70% may even be better for many tricky situations. I did go for a specific flat-field screen long ago and don't have too much experience with twilight flats, so I hope this will work out.