Flat problem - unab...
 
Share:
Notifications
Clear all

2023-09-28: APP 2.0.0-beta24 has been released !

 

Improved application startup, fixed application startup issues, upgraded development platform to Oracle GraalVM JDK21

We are very close now to  releasing APP 2.0.0 stable with a complete printable manual...

Astro Pixel Processor Windows 64-bit

Astro Pixel Processor macOS Intel 64-bit

Astro Pixel Processor macOS Apple M Silicon 64-bit

Astro Pixel Processor Linux DEB 64-bit

Astro Pixel Processor Linux RPM 64-bit

Flat problem - unable to remove all dust

18 Posts
5 Users
2 Likes
199 Views
(@itarchitectkev)
Neutron Star
Joined: 6 years ago
Posts: 108
Topic starter  

I have two QHYCCD268M cameras. One is attached to a 107PH with Reducer and Chroma LRGB filters in a filter wheel; the other is attached to a 76EDPH with Reducer and Baader LRGB filters wheel.

I have never had an issue of calibration with my QHY268M on the 107PH.

Weirdly I've always had problems of calibration of LRGB on my other scope - I previously had a 294M on here and it too had the same issues calibrating.

 

The issue I have is that vignetting is removed, but spots still remain. This occurs across all LRGB filters. For SHO it probably exists, but not noticable - or lucky to be more dust free.

The attached image below shows the uncalibrated sub showing a level of vignetting. This is L at 360s, Gain 26 Offset 30.

Screenshot 2023 05 14 at 11.15.49

The next image shows the calibration (Darks + Flats + DarkFlats).

Screenshot 2023 05 14 at 11.16.02

It appears to be ok but here is the flat which will make you then notice a couple of spots near top right quarter that are affected, but over-corrected.

Screenshot 2023 05 14 at 11.23.55

I create my flats using a flat panel/paper using NINA, ADU around 32k. For L, this is around 1-1.5sec in length. This has served me very well for my cameras, and also my first 268M which this telescope sits next to and images the same target, just using a different scope.

I use the same settings for Flats and DarkFlats as I do Lights: Mode 4 (2CSM-0), Offset 30, Gain 26 for LRGB.

 

I have tried the following:

- Use bias and not. No difference (as one might expect)

- Use Darks and no Darks. No discernible difference apart from APP not complaining about calibration when Darks used. 268M is very low on noise but worth doing.

- Tried 20% ADU, 30% ADU, 40% ADU, 50% ADU, 60% ADU and 70% ADU when creating Flats. No difference.

- Tried ScalingMasterDark Flat and not. No difference (in how I process anyway)

- Turned off Adaptive Pedestal (doesn't impact this camera, but doesn't make a difference anyway)

- Experimented with Flat settings from Automatic to Maximum. Didn't appear to make any difference to the calibrated sub.

I had this problem with my 294M, so I'm assuming the vignetting caused by something in my image chain (logically, the reducer), which also corrects for vignetting effect, is causing a calibration problem where the spots get over-corrected.

 

Any suggestions on what to do next?


   
ReplyQuote
(@wvreeven)
Quasar
Joined: 5 years ago
Posts: 2119
 

@itarchitectkev Hey Kev, are you sure that the filter rotated back to exactly the same position? A slight offset may already give the results you see.


   
Mabula-Admin reacted
ReplyQuote
(@itarchitectkev)
Neutron Star
Joined: 6 years ago
Posts: 108
Topic starter  

No. I've had that before. It leaves a 3D looking artefact.

I've just overlayed in Photoshop and the two match.

I also used the Flat/DarkFlats created from last month (as I don't touch the scopes or cameras in any way it's a good little test) and it removes that main dust mark, but the other one has moved slightly since last month - giving that 3D effect.

I'm suspecting it's the way it was created but I've lost about 4 hours today trying to get Flats that work. Which is painful as it's the same process I've used for years, and especially painful as two of these same cameras so doesn't make sense I have to change anything, but the problem seems to be just this configuration of scope - albeit I didn't have this problem last month (the last time I was able to do any AP, hence why using last month's Flat worked (enough to prove a point)).

The problem looks data related, but everything matches. My hunch is that the Flat that I created last month "works". Nothing has changed since creating that Flat. No software updates. No changes in hardware. But the Flat created is brighter - as if the gain is different.

This post was modified 5 months ago by AstroCookbook Kev

   
ReplyQuote
(@wvreeven)
Quasar
Joined: 5 years ago
Posts: 2119
 

@itarchitectkev Did you try to make fainter flats by decreasing the exposure time? And did you check the gain of the flats of last month to verify that it indeed is the same?


   
ReplyQuote
(@itarchitectkev)
Neutron Star
Joined: 6 years ago
Posts: 108
Topic starter  

Yes, according to the metadata the two gains are correct. I checked the sequence and the specific gains and offset are put in the sequence.

I've just tried another flat panel (thinking my older panel has a fault, certainly the power adapter was loose which allowed me to see brightness changes when I touched it - which made me not trust it) but no joy there (and it worked for when I used it for my other 268M immediately before creating the flats for this set up).

I've tried really short flats (0.3s and 20% ADU) to long flats (6 second 70% ADU) and no difference is seen in the calibration. Which itself is weird.


   
ReplyQuote
(@itarchitectkev)
Neutron Star
Joined: 6 years ago
Posts: 108
Topic starter  

I have noticed something very subtle and weird which is possibly related - will need to test if this has an effect when I have time. I use NINA for everything related to APP.  Lights, Flats, Darks, etc.
I run two scopes on the same mount which means I use the same compute to control the 2 different QHY268Ms.
I've just noticed in NINA's Camera info page that depending on which camera gets loaded first, it appears to have randomly load a slightly different USB driver version.

Loading just the secondary, the one with the problem, (I call slave as it's related to how the sync mount call works) and the USB Driver is 22.8.12.0.

Screenshot 2023 05 15 at 10.08.28

On loading the primary (master), same camera, loads USB Driver 22.7.6.0. <-- This works fine, but wondering if there's enough difference to cause the behaviour seen.

Screenshot 2023 05 15 at 10.05.25

But I can change this behaviour by loading a different one before the other and they both report the latest 22.8.12.0.


   
ReplyQuote
(@itarchitectkev)
Neutron Star
Joined: 6 years ago
Posts: 108
Topic starter  

Even stranger - highlighting this potential issue...

Screenshot 2023 05 15 at 10.19.06

 


   
ReplyQuote
(@itarchitectkev)
Neutron Star
Joined: 6 years ago
Posts: 108
Topic starter  

And it gets even stranger. Windows sees this as the same driver file, yet reporting different vendors and dates.
I'll contact QHY to see what's going on - but given that this felt either a processing problem, e.g. over/under-correcting because of the dust present in the vignetting (hence reachingh ou here) or a data problem (suspecions are high!)

Screenshot 2023 05 15 at 10.21.43

 As I said - same make/model of camera (bought 2 years apart) connected to same computer, Both USB 3.0. I've never seen this before.


   
ReplyQuote
(@itarchitectkev)
Neutron Star
Joined: 6 years ago
Posts: 108
Topic starter  

My last post on this as I've been trying for a few hours more today.

Updated USB Driver and it didn't affect the calibration. Same result.

Made Flats using Gain 56. No difference, same result. Slight movement on histogram.

Made Flats using Gain 0. No difference, same result.

Made Flats using Gain 26, ADU 10%. No difference, same result.

 

I would have expected some changes. I appreciate I'm looking visually but I can't seen any changes at all - no movement to think/explain what's going on.

 

I should point out versions: APP 2.0-Beta 17 on Mac OS Monterey 12.6.5. 

Capture: Windows 10, APP 2.2. QHYCCD Latest Beta 20230412, LRGB shot using Mode 4 2CMS-0.


   
ReplyQuote
(@wvreeven)
Quasar
Joined: 5 years ago
Posts: 2119
 

Is there a way to install the drivers for your camera and filter wheel on macOS and try to make flats using that? Maybe that way you do see a difference?


   
ReplyQuote
(@larry-1969)
Main Sequence Star
Joined: 2 years ago
Posts: 24
 

I used to have some issues with my flats overcorrecting before I started using bias frames. They are easy to take, and you can reuse a master for quite a while.

 

Larry


   
ReplyQuote
(@itarchitectkev)
Neutron Star
Joined: 6 years ago
Posts: 108
Topic starter  

Thanks - already tried Bias and non-Bias. No difference. In fac that was also odd - there were no discernible differences - only subtlely noticed in the histogram.

I also tried Affinity Photo (it also supports Astro stacking and calibration). The problem existed there too, although Affinity allows you to adjust the background so you can get to a stage where this is "removed"... But, it suggests there is a problem with the data.

 I think it is to do with the brightness of the vignetting/reflection which I suspect is the reducer - and the dust spot being on the edge as the image appears flat (i.e. vignetting removing) but this then becomes prominent. I should point out that other dust spots get removed. So it's a calculation problem, but possibly remediated through altering the input (data/imaging chain).

Given the next few months here are not suited for my set up, and full darkness has disappeared, I'll tackle this with some changes in hardware for August time when real dark skies return again in the North of the UK. 


   
ReplyQuote
(@mabula-admin)
Universe Admin
Joined: 6 years ago
Posts: 3875
 

Hi @itarchitectkev,

I ahve read the entire topic and this definitely points then to a problem in your imaging train. Have you excluded even the most miniscule light leak by making a dark of 30-60 minutes with the telescope and camera in daylight illumination? IF that is still dark, then there must be something off with the coatings of the internals of your imaging train causing problems maybe. Hope you can solve this soon !

Mabula


   
ReplyQuote
(@itarchitectkev)
Neutron Star
Joined: 6 years ago
Posts: 108
Topic starter  

I have strong suspicions it was the imaging chain.

After spending a few hours of various tests, inspecting and pondering over the orientation of my filters (a potental of causing reflection), and finally doing some dark tests - I did manage to tape up a part of my imaging chain that impacted histogram when comparing darks. The net results is a Flat that visually looks like the older Flat that I showed actually calibrated well albeit a dust mark moved slightly making it redundant. I appreciate visually looking that isn't scientific but because I've altered far too much in my imaging chain - adjusting/inspecting filters, rotating camera etc - it has completely ruined any chance of actually testing this on my current Lights.

Whilst these are screenshots only, the first image shows the improvement and is a Flat after my dark tests and taping up a connection to the camera, the second image is before taping up which didn't calibrate correctly. Interestingly (or disappointingly) it is at the camera body end that includes an in-built tilt adjustment on the QHYCCD 268M - or it's the adapters connecting into the ZWO filter wheel. Either way, there is tape on there now and it adjusted things enough to show visual changes in Darks and Flats. It was not a huge problem, very subtle - but present.

My other QHYCCD 268M is screwed into a QHYCCD filter wheel and doesn't exhibit this behaviour.

Screenshot 2023 05 20 at 11.39.14
Screenshot 2023 05 20 at 11.39.02

So for now I'm highly suspecting it was this problem, but I don't think I'll be able to test for a while now.

Also notes that despite cleaning my filters and sensor, I need to try harder next time.

This post was modified 4 months ago by AstroCookbook Kev

   
ReplyQuote
(@mabula-admin)
Universe Admin
Joined: 6 years ago
Posts: 3875
 

Hi @itarchitectkev,

Okay, thanks for the feedback, it does indicate a slight light leakage which was causing the issues, hope next time it is gone now 😉

Mabula


   
ReplyQuote
(@itarchitectkev)
Neutron Star
Joined: 6 years ago
Posts: 108
Topic starter  

SOLVED
Just to close this out. As suspected, it was light leak.
I successfully took new lights and flats (and dark flats) after putting some tape in some choice places around the camera and fixings and the calibrated results were now as expected.


   
ReplyQuote
(@cdugger)
Main Sequence Star
Joined: 2 years ago
Posts: 14
 

 

NGC6523 flat problem

I am having the same problem, and have tried many of the tests mentioned above, e.g. different numbers of flats for integration, reduced ADU's, darkflats, bias... no help.  The image below is blown out to make the donuts obvious.  Flats resolve vignetting, but I have these frustration artifacts.

ASI071MC Pro.  Filter drawer with UV/AR Cut.  RC6.

Don't understand why a light leak, which I don't think I have, would cause this if that is the conclusion from he thread above.

Thanks

 

 

This post was modified 3 months ago by Chris Dugger

   
ReplyQuote
(@mabula-admin)
Universe Admin
Joined: 6 years ago
Posts: 3875
 

Posted by: @cdugger

 

NGC6523 flat problem

I am having the same problem, and have tried many of the tests mentioned above, e.g. different numbers of flats for integration, reduced ADU's, darkflats, bias... no help.  The image below is blown out to make the donuts obvious.  Flats resolve vignetting, but I have these frustration artifacts.

ASI071MC Pro.  Filter drawer with UV/AR Cut.  RC6.

Don't understand why a light leak, which I don't think I have, would cause this if that is the conclusion from he thread above.

Thanks

Hi Chris, @cdugger

A problem like this can occur due to a slight leak, barely noticable. But also bu your duse having moved slightly between shooting the lights and ad the flats 😉 If vignetting is solved cleary then the other possible causes like incorrect darkflats, darks, bias for the lights and flats in terms of gain/offset/temperature/exposure can be excluded, at least likely.

Mabula

 

 


   
ReplyQuote
Share: