The case of overcorrecting flats, need help

My Flat Contour Plot looks pretty good too:

Fortunately mainly in the corners, again tricky to tell why, can you upload like 5 flats, 20 bias, 20 darks and/or 20 darkflats to us? Also 10 lights so I can investigate what may be happening. At least you're getting way closer to having a proper workflow, having identified the light leak before is very nice as that can be extremely frustrating.

The contour plot looks nice, it might be that the very corners in your imaging train are still being obstructed on a full frame.

Posted by: @vincent-mod

The contour plot looks nice, it might be that the very corners in your imaging train are still being obstructed on a full frame.

Files Uploaded, Lights, Flats and MasterDarkFlat

I moved to the M68 connector on the scope, because there was too much vignetting on the M54, mainly because the light cone at 55mm is around 61mm, at the M68 connector is it only around 62mm, so M68 covers this perfectly, I suspect the vignetting that is still there is due to the M54 entrance to the filterwheel...nothing I can do abut that.  BUT, it should level out with flats, just not sure why it is over correcting one side and undercorrecting on the other

After doing some PixelMath on the original flat frame, I have discovered that 27500ADU seems to be the sweet spot for the flats, so at the moment I am doing two things

1. Re-taking all of my calibration frames, including darks and dark flats

2. Taking flats at two different exposure lengths, one will be sub 10s flats, the other will be post 30s flats

The reason for number 2, is because some CMOS cameras have different download modes depending on the exposure length of the frame, I have not found any documentation on the ASI6200MMPro to say either way, but I do know that with my previous flats that worked on all my NB filters, my exposures for the NB filters were relatively high due to the max brightness of the Pegasus Astro FLatMaster.  My PrimaluceLab GIOTTO has a much more substantial brightness, so I have the luxury to play with getting the exposures to a specific level.

Here's what the results were on changing the ADU of the MasterFlat using Pixelmath did to the image:

ADU of 17500

ADU of 19000

Original ADU of 22500

ADU of 24500

ADU of 25500

ADU of 26500

So the "Overcorrection" on the left is almost gone at 26500, so 27500 will be my target

This is with the masterflat scaled to 32500 ADU, you will see there is undercorrection on both sides

So it's time to re-create all my calibration frames and go from there, unless of course Vincent finds something in the meantime 😀

That's some very nice analysis Simon! I'll still have a look but as you're now finding this out, maybe it's better to wait for those results and if they don't fully work, upload those instead?

@vincent-mod I have also found that the Primalucelab GIOTTO EL Panel I have, is very sensitive to temperature change and it could be having a negative impact on my flats, here's a graph showing how rapidly ADU falls as the external temperature falls, each frame is around 16 seconds.

The big drop in Blue is where I stopped the camera for 45 minutes to make sure it was not the camera, but the panel kept decresing in brightness!

The ADU variance across the flats for one colour is around 1800 ADU, this WILL have an impact on the stack, correct?

I think I am onto something with the long exposure for flats thing

So I am getting there, this is based on 22500ADU Flats with an exposure time of >15S for the flats, again red channel:

The above is far far better than before.  I will try again with 27500ADU Flats when I get back from Dubai (Work).  My Luminance one still looks bad though:

I would never have guessed you can have such an effect on a flat panel, again great find! I assume this is likely the issue then?

Posted by: @vincent-mod

I would never have guessed you can have such an effect on a flat panel, again great find! I assume this is likely the issue then?

All LED's have the issue, they gain brightness when they get warm, and decrease in brightness when they get cold.  I am working with Primalucelab on testing a new firmware/software that will track the external temperture by means of their ECCO2 (which I have) that will control the brightness of the LEDs depending on the rise/fall of ambient / external temp.  I have also suggested that they implement a temp sensor on the GIOTTO design to compensate for this.

Now I am back from Dubai, I am going to test 27.5k Flats 😀

So things are not going according to plan, and they seem to be giving bizarre results, and I am not sure why, tried all sorts of ADU values but what is strange is an example below.  Luminance overcorrects and undercorrects on the opposite side to the red frames

Luminance:

Red:

So right now I am not sure why this is happening, as you can see Red Overcorrects on the left, but undercorrects on the right, Luminance Undercorrects on the left but over corrects on the right.

This being in the extreme corners and you having seen an improvement by switching to a bigger adapter, I'm wondering if this is still an issue maybe? I'm happy to look at the data, but not sure I can see what may be an issue at the hardware level. The differences in which filter is used also strengthens this suspicion for me, you are using a filter-wheel right (sorry I see a lot of data 🙂 )?

@vincent-mod Yes I am using the ZWO 7x50.4mm EFW

Maybe it's interesting to have a look at how flat the filter wheel is positioning the filters maybe? I'm not 100% sure that will have this effect, but it's odd that there is a difference in between filters I'd say.

@vincent-mod Already ahead of you there, I am using the Buckeye filter masks, so the filters are pretty secure and perpendicular

Have you ever taken it off and put it on again? This may make you notice any detail possible or when re-assembling, it may point to an issue. I would maybe try to swap the filters around and see if you get the opposite result..

@vincent-mod Ok I am going to throw a spanner in the mix now, I know it is not the filterwheel, or filters not being lined up because....wait for it....here's the Luminance with 22500ADU sky-flats, and the sky-flats were taken with the scope pointed up at zenith without any cover

All the vignetting is corrected perfectly, so I am not sure why this is not working with both my PrimaluceLab GIOTTO or the Pegasus Astro Flatmaster, unless it is the Baader filters!

And for completeness, here's the red channel again with 22500ADU sky-flats

So with sky-flats it does work and with artificial ones it doesn't, is that correct? Then it's not your calibration data or anything, but must be related to what is different from the sky-flats, it's very difficult for me to judge what that might be, but I assume it's somehow related to whatever is different in between the sky-flats and the artificial ones (I hope this is not sounding like a "duhh, I know genius"). 🙂 Just trying to think out loud. I would now be very curious what would happen if you take artificial flats for everything.

@vincent-mod I have done testing on this

Ha and SII are not affected at all, OIII, L, R, G and B only seem to work with Sky-Flats, this leads me to believe that the Baader CMOS optimised filters are the issue, I mean I've tried with two different LED Panels from two vendors, both produce similar results, so I wonder if the Baader filters are blocking something.  Trying to get hold of some additional filters to test.  I am also going to email Thomas Baader too

Would surprise me as well, but there's not a lot of variables left. Just to be sure, the darks and flatdarks etc. are the same for all right?

@vincent-mod Yep, same Master Dark, dark-flats correspond with filters, but I have even resulted to using BIAS frames to test as well instead of dark flats so that I can use the exact same calibration frames to calibrate the flats, and no difference, sky-flats are the only ones that work

Hi Simon @stastro and Vincent @vincent-mod,

Let me first take away a fairly common misconception here about what a good flat is and if the peak (or mean, or median) ADU value has an imfluence on the flat-field calibration.

The ADU value has no influence whatsoever if the camera's sensor behaves linearly in the ADU range in which you shot the flats provided the illumination of the flat does not clip on both black and white levels.

Trying to shoot flats with an peak/average/median ADU value is not that relevant at all with camera's from the latest 5 years. All the new CMOS sensors are very linear over the entire ADU range from 0-65535 (if 16bit, for 14bits it would be 0-16383). So if you shoot flats with a median ADU value of either 20000 or 40000 should not give different results in terms of getting the vignetting fully corrected. The 40000 ADU value flats will have better quality though in terms of Signal To Noise Ratio, but that is the only difference.

If you can not get the vignetting fully corrected in your images, something is not okay with the way the flats are shot relative to how the lights are shot, the problem will be solved there ;-), not in getting a specific ADU value or in the processing software.

If skyflats do work and flats with the panel don't, then clearly the panel flats are not shot correctly somehow.

Mabula

@stastro Hi Simon, Mabula made a great point and something I sometimes forget indeed, for flats the ADU doesn't matter when you're in the linear part of the histogram. I was wondering though, are you also pointing the scope straight up when shooting the artificial flats? So is there any difference in how you have the setup between sky-flats and artificial ones?

@vincent-mod @mabula

This is what I sent to Thomas Baader:

I have done a lot of testing on this now, albeit from trying other filters, it would appear that my Baader filters are definitely having issues with LED Flat panels, and to also rule out one specific panel, I bought another panel from another vendor.

The issue occurs with OIII, L, R, G and B, it does not happen with Ha or SII.

The issue occurs with both a Pegasus Astro Flatmaster LED Panel, and a PrimaluceLab GIOTTO LED Light Panel

The issue occurs no matter what ADU Value of the flats are, I have tried 17500, 22500, 27500, 32500, 37500, 42500 and 45000

The issue does not occur with sky-flats (Taken at zenith with the scope uncovered) at an ADU Value of 22500

The issue occurs whether the flatpanel is sat directly on the scope, at the end of the dew shield, or 2ft away from the scope, same results

I get Over-correction and Under-correction within the same image, and I get opposite side effects between say L and R, where the overcorrection is on the opposite side and the same for the undercorrection.

Posted by: @stastro

@vincent-mod @mabula

This is what I sent to Thomas Baader:

I have done a lot of testing on this now, albeit from trying other filters, it would appear that my Baader filters are definitely having issues with LED Flat panels, and to also rule out one specific panel, I bought another panel from another vendor.

 

The issue occurs with OIII, L, R, G and B, it does not happen with Ha or SII.

The issue occurs with both a Pegasus Astro Flatmaster LED Panel, and a PrimaluceLab GIOTTO LED Light Panel

The issue occurs no matter what ADU Value of the flats are, I have tried 17500, 22500, 27500, 32500, 37500, 42500 and 45000

The issue does not occur with sky-flats (Taken at zenith with the scope uncovered) at an ADU Value of 22500

The issue occurs whether the flatpanel is sat directly on the scope, at the end of the dew shield, or 2ft away from the scope, same results

I get Over-correction and Under-correction within the same image, and I get opposite side effects between say L and R, where the overcorrection is on the opposite side and the same for the undercorrection.

Hi Simon @stastro,

To me this sounds like an issue with your optical train, rather than an issue with your filters. From my own experience, what you are experiencing might be caused by a very minor light leak in your optical train. The sky flats are least affected by that and can give you the illusion that they are ok, while also not as good as wanted. I would be curious though about Thomas Baader's reply as well.

Did you thoroughly examine your optical train? How do long exposure darks look if you shoot them with the telescope closed during daytime (or with the lights on)?

Mabula

@mabula-admin 

Already one step ahead of you on that one as Light Leak was the first thing I looked at.  But that has been eliminated too.

Darks and Flats are both taken with the observatory roof closed, and the observatory is in darkness, the scope is pointed directly up, and either the flat panel is resting on top of the dew shield (for flats) or the telescope cover is resting on top of the dew shield (for darks).  And with the darks, the Mean ADU level across all exposures is 11, even with dark flats.  So there's no light leakage possible.

In fact now I come to think of it, I have even taken dark flats with the LED panel still covering the scope but powered off, and still the mean ADU value is 11, and I examined the dark flats for any gradients and there was none, which confirms no light leak

Oh now it just gets crazy.  Finally got my Chroma filters, and took some flats using my Giotto Panel, the strange thing is, in all the filters, I see artifacts in the flats that are not there in the lights, and I have checked the panel too.  Here's a Blue Flat (and yes it's there in all flats) with STF applied:

Here's the Calibrated Light Stack:

And here's the stack without the master flat applied where the artifacts do not exist:

So not sure what the hell is going on any more 🙁

That doughnut seems like it would be an out of focus dust speckle or something like that. If you overstretch your light stack, there is not even a vague sign of it?

Posted by: @vincent-mod

That doughnut seems like it would be an out of focus dust speckle or something like that. If you overstretch your light stack, there is not even a vague sign of it?

Nope, Nothing, I have a theory as to the cause I am just going to do some testing