Another residual am...
 
Share:
Notifications
Clear all

2022-05-29: APP 2.0.0-beta2 has been released !

Release notes

Download links per platform:

windows 2.0.0-beta2

macOS x86_64 2.0.0-beta2

macOS arm64 M1 2.0.0-beta2

Linux DEB 2.0.0-beta2

Linux RPM 2.0.0-beta2

Another residual amp-glow issue

Page 1 / 2

(@francismoreau)
Brown Dwarf Customer
Joined: 4 years ago
Posts: 14
Topic starter  

Hi,

This is my first post here and I would like to highlight the great quality of this software.

I use a cooled  QHY178M camera.
This camera, like others, is affected by a large amp-glow.

Before posting here, I read all the topics dealing with amp-glow including the recent one on the IMX183.
I also processed the same subs with PixInsight. There is no amp-glow left on the PixInsight processing.
All subs were taken at the same temperature (-20°C) as well as at the same gain (5) and offset (250).

Although I use "Adaptative pedestal/reduce amp-glow" and BPM, there is still a residual amp-glow that does not allow for quality integration.

I have posted 5 views of each DOF and light in a dropbox

I am finishing this post in order to open another one allowing the insertion of images and link

Francis

 

EDIT by MABULA:

Beta versions with a fix are available here:

APP 1.071-beta for Windows 64bits

APP 1.071-beta for MacOS 64bits

APP 1.071-beta for Linux DEB 64bits

APP 1.071-beta for Linux RPM 64bits

This topic was modified 4 years ago 6 times by Francis Moreau
This topic was modified 4 years ago by Mabula-Admin

ReplyQuote
(@francismoreau)
Brown Dwarf Customer
Joined: 4 years ago
Posts: 14
Topic starter  

This is comparaison between calibrated frame, APP on the left, Pix on the right

Comparaison Calibrated

Here is the comparaison between integration, APP on the left, Pix on the right

Comparaison Integration
residual amp glow
This post was modified 4 years ago 4 times by Francis Moreau

ReplyQuote
(@vincent-mod)
Quasar Admin
Joined: 5 years ago
Posts: 4910
 

First of all sorry for your trouble with the forum, I think we need to make it more clear that when starting, links are removed automatically. This is a measure to avoid spam.

Since you already took many of the options there are to reduce this, I think we have to wait for an answer from Mabula on this (@mabula-admin), he's back from a holiday next week.


ReplyQuote
(@kijja)
Red Giant Customer
Joined: 5 years ago
Posts: 150
 

My friend who own QHY10 told me that his ccd never has much amp grow like mine when he used with another astro-processing application. However, after I used longer exposure from 300s to 600s ,the amp grow was much better. It was quite contrast with my first hypothesis on too long exposure was the cause of it. 


ReplyQuote
(@francismoreau)
Brown Dwarf Customer
Joined: 4 years ago
Posts: 14
Topic starter  

I have made progress in solving my problem.
The cause of the problem seems to be bias.
If we remove the bias from the treatment, the ampglow disappears

Capture 1

For the sake of awareness, I'm going to do a new batch of bias acquisition.


ReplyQuote
(@francismoreau)
Brown Dwarf Customer
Joined: 4 years ago
Posts: 14
Topic starter  

by making comparisons with the process I use under PixInsight, I think I have identified the cause of the problem.
It seems that there is a clipping when generating the MasterDark.
I got the confirmation by subtracting a small offset (-100) from the MasterBias before starting the calibration. Under PixInsight, this corresponds to the Pedestal during calibration 
The ampglow has completely disappeared

Capture 2

 

This post was modified 4 years ago by Francis Moreau

ReplyQuote
(@vincent-mod)
Quasar Admin
Joined: 5 years ago
Posts: 4910
 

In tab 2, scroll all the way down. Did you select the adaptive pedestal? That might do the trick in APP.

 

edit: woops, you already tried that. 😉 Well that is interesting, I'll tag Mabula (@mabula-admin) and pm him as he seems very busy somehow.


ReplyQuote
(@gotak)
White Dwarf Customer
Joined: 4 years ago
Posts: 17
 

Was this ever looked at and a solution found? Seeing similar problems with a QHY163C.


ReplyQuote
(@mabula-admin)
Quasar Admin
Joined: 5 years ago
Posts: 3180
 

Dear @francismoreau, @gotak and @vincent-mod,

I am downloading Francis' data right now and will check the issue this afternoon. I will report back later 😉

Thank you Francis for making the data available.

Mabula


ReplyQuote
(@mabula-admin)
Quasar Admin
Joined: 5 years ago
Posts: 3180
 

Dear @francismoreau,

I am looking at your data now, first thing that I notice is the the bias frames and dark frames are not consistent/compatible technically. The resulting Dark Master has a lower median/mean value than the MasterBias which should never occur, since a dark normally is the bias + dark current/amp glow signals.

MasterBias has

HDU1 - MEAN = ' 1,00E+03' / mean of channel
HDU1 - MED = ' 1,00E+03' / median of channel

MasterDark has

HDU1 - MEAN = ' 9,91E+02' / mean of channel
HDU1 - MED = ' 9,88E+02' / median of channel

This is not good and either indicates a camera issue or that the data was captured with a different offset. I do see that gain values are the same.

Do you have control over the cmos offset in your camera?( I can't find a reference to sensor offset in the metadata.) If not, then this issue should at least be reported to QHY I think.

Metadata of the masters:

FITS HDUs: 1
HDU1 - SIMPLE = T / Java FITS: Tue Jan 29 1317 CET 2019
HDU1 - BITPIX = 16 / bits per data value
HDU1 - NAXIS = 2 / number of axes
HDU1 - NAXIS1 = 3056 / size of the n'th axis
HDU1 - NAXIS2 = 2048 / size of the n'th axis
HDU1 - EXTEND = T / Extensions are permitted
HDU1 - BSCALE = 1.0 / scaling to 16bit
HDU1 - BZERO = 32768.0 / offset data range to that of unsigned short
HDU1 - DATE = '2019-01-29T1240' / creation date of MasterBias
HDU1 - SOFTWARE= 'Astro Pixel Processor by Aries Productions' / software
HDU1 - VERSION = '1.071 ' / Astro Pixel Processor version
HDU1 - CALFRAME= 'MasterBias' / master bias frame
HDU1 - INSTRUME= 'QHYCCD-Cameras-Capture' / instrument name
HDU1 - CFAIMAGE= 'no ' / Color Filter Array pattern
HDU1 - GAIN = 5.0 / gain or ISO depending on instrument
HDU1 - EXPTIME = 0.001 / exposure time (s)
HDU1 - MEAN = ' 1,00E+03' / mean of channel
HDU1 - MED = ' 1,00E+03' / median of channel
HDU1 - SIGMA = ' 6,74E+00' / standard deviation of channel
HDU1 - NOISE = ' 6,47E+00' / MRS gaussian noise estimate of channel
HDU1 - NUMFRAME= 5 / # number of frames used in MasterBias creation
HDU1 - END

FITS HDUs: 1
HDU1 - SIMPLE = T / Java FITS: Tue Jan 29 1317 CET 2019
HDU1 - BITPIX = 16 / bits per data value
HDU1 - NAXIS = 2 / number of axes
HDU1 - NAXIS1 = 3056 / size of the n'th axis
HDU1 - NAXIS2 = 2048 / size of the n'th axis
HDU1 - EXTEND = T / Extensions are permitted
HDU1 - BSCALE = 1.0 / scaling to 16bit
HDU1 - BZERO = 32768.0 / offset data range to that of unsigned short
HDU1 - DATE = '2019-01-29T1250' / creation date of MasterDark
HDU1 - SOFTWARE= 'Astro Pixel Processor by Aries Productions' / software
HDU1 - VERSION = '1.071 ' / Astro Pixel Processor version
HDU1 - CALFRAME= 'MasterDark' / master dark frame
HDU1 - INSTRUME= 'QHYCCD-Cameras-Capture' / instrument name
HDU1 - CFAIMAGE= 'no ' / Color Filter Array pattern
HDU1 - GAIN = 5.0 / gain or ISO depending on instrument
HDU1 - EXPTIME = 300.0 / exposure time (s)
HDU1 - MEAN = ' 9,91E+02' / mean of channel
HDU1 - MED = ' 9,88E+02' / median of channel
HDU1 - SIGMA = ' 7,93E+01' / standard deviation of channel
HDU1 - NOISE = ' 9,47E+00' / MRS gaussian noise estimate of channel
HDU1 - NUMFRAME= 5 / # number of frames used in MasterDark creation
HDU1 - END

I will do some more analysis and testing to see if I can improve the calibration results and possibly find a solution for the bad offset values of the masters 😉

Kind regards,

Mabula

 


ReplyQuote
(@mabula-admin)
Quasar Admin
Joined: 5 years ago
Posts: 3180
 
Posted by: Francis Moreau

by making comparisons with the process I use under PixInsight, I think I have identified the cause of the problem.
It seems that there is a clipping when generating the MasterDark.
I got the confirmation by subtracting a small offset (-100) from the MasterBias before starting the calibration. Under PixInsight, this corresponds to the Pedestal during calibration 
The ampglow has completely disappeared

Capture 2

 

Hi @francismoreau,

There is no clipping here when the masterdark is created, since the MasterBias is not subtracted from the darks to create the MasterDark (App will never do this, very old versions used to do it though...).

It actually is the camera that has made incompatible bias and dark frames it seems. The darks clearly have a lower offset. By subtracting a pedestal of 100 adu from the MasterBias you compensate this effect and than all goes well 😉

I think Pixinsight internally detects this and makes the compensation automatically or did you do this yourself in Pixinsight as well?

I will try to automate this in APP to see if all goes well then automatically.

Mabula


ReplyQuote
(@francismoreau)
Brown Dwarf Customer
Joined: 4 years ago
Posts: 14
Topic starter  

Hi Mabula,

thank you for watching.

I think there's no issue with this camera. Offset is set to 250 for every frame.

I read somewhere that some CMOS sensors have this behaviour, i. e. average bias values slightly higher than dark and that's what pedestal is for.

However, I will ask QHY for confirmation.

 

Francis

This post was modified 4 years ago by Francis Moreau

ReplyQuote
(@francismoreau)
Brown Dwarf Customer
Joined: 4 years ago
Posts: 14
Topic starter  

It actually is the camera that has made incompatible bias and dark frames it seems. The darks clearly have a lower offset. By subtracting a pedestal of 100 adu from the MasterBias you compensate this effect and than all goes well 😉

I think Pixinsight internally detects this and makes the compensation automatically or did you do this yourself in Pixinsight as well?

I will try to automate this in APP to see if all goes well then automatically.

Mabula

ImageCalibration

PixInsight has a Pedestal field. If I set it to 0, the result is the same as APP


ReplyQuote
(@mabula-admin)
Quasar Admin
Joined: 5 years ago
Posts: 3180
 

Hi @francismoreau,

Yes the 1000 ADU value measured by APP is in 16bits. So the 12bit offset would be 250.

Definitely strange that it drops when dark current gets involved, but I can imagine this has to do with cmos technology in the camera somehow.

I will try to make an automatic adjustment in APP for data with this issue to assure proper calibration 😉

Thanks,

Mabula


ReplyQuote
(@mabula-admin)
Quasar Admin
Joined: 5 years ago
Posts: 3180
 
Posted by: Francis Moreau

It actually is the camera that has made incompatible bias and dark frames it seems. The darks clearly have a lower offset. By subtracting a pedestal of 100 adu from the MasterBias you compensate this effect and than all goes well 😉

I think Pixinsight internally detects this and makes the compensation automatically or did you do this yourself in Pixinsight as well?

I will try to automate this in APP to see if all goes well then automatically.

Mabula

ImageCalibration

PixInsight has a Pedestal field. If I set it to 0, the result is the same as APP

Thanks, yes I am aware. I will try solve this in a more automatic sense. Basically the Adaptive Pedestal should solve this, I need to add a minor tweak probably for the instance when the masterbias median is higer than the MasterDark Median 😉

I will update you on my findings.

Mabula


ReplyQuote
(@gotak)
White Dwarf Customer
Joined: 4 years ago
Posts: 17
 

If you need more example data, I'd be happy to supply some to make sure this all works as expected.

 


ReplyQuote
(@mabula-admin)
Quasar Admin
Joined: 5 years ago
Posts: 3180
 

Hi @francismoreau & @gotak,

Thank you very much Francis for sharing this issue with your data.

You were totally right, there actually was a black clipping occuring in the calibration code which would only happen when, like in your case, the bias signal had higher values than the dark signal. Normally this would not happen, but with your particular QHY camera it certainly does and this now shows the bug. I have corrected all code for all affected calibration paths, so it should not happen again 😉

I will try to release APP 1.071 with this fix as soon as possible.

You will still need to enable the Adaptive Data Pedestal in this case to make sure no residual amp-glow remains from biased integration to the upside in the amp-glow areas.

https://www.astropixelprocessor.com/community/release-information/astro-pixel-processor-1-071-preparing-next-release/

  • FIXED, Calibration AMP-GLOW, a specific calibration issue was mentioned in topic : Another residual amp-glow issue, https://www.astropixelprocessor.com/community/main-forum/another-residual-amp-glow-issue/ This issue was made apparent by a specific sensor characteristic of a QHY178M camera. The median of the MasterBias had a higher value than the median of the MasterDark which caused a clipping problem in the calibration code for certain calibration paths. Not all paths were affected. Calibration with only a masterdark worked fine, but not with a Masterbias, MasterDark and Masterflat loaded. All code for all affected calibration paths have been corrected now. Data from a camera with such clear amp-glow still needs to be calibrated with the Adaptive Data Pedestal enabled to prevent skewed integration in the amp-glow areas.

Without-Adaptive-Data-Pedestal-Without-Fix

Without Adaptive Data Pedestal Without Fix

Without-Adaptive-Data-Pedestal-With-Fix

Without Adaptive Data Pedestal With Fix

With-Adaptive-Data-Pedestal-Without-Fix

With Adaptive Data Pedestal Without Fix

With-Adaptive-Data-Pedestal-With-Fix

With Adaptive Data Pedestal With Fix

 

Kind regards,

Mabula

 

 

This post was modified 4 years ago 2 times by Mabula-Admin

ReplyQuote
(@mabula-admin)
Quasar Admin
Joined: 5 years ago
Posts: 3180
 
Posted by: gotak

If you need more example data, I'd be happy to supply some to make sure this all works as expected.

 

Hi @gotak,

Feel free to send me a small subset of your data, then I will test this immediately to report if my solution works on your data as well 😉

5x light, 5x bias, 5x darks, 5x flats is fine.

Please send it to support@astropixelprocessor.com using wetransfer.com or dropbox.

Thanks,

Mabula


ReplyQuote
(@gotak)
White Dwarf Customer
Joined: 4 years ago
Posts: 17
 
Posted by: Mabula Haverkamp - Admin
Posted by: gotak

If you need more example data, I'd be happy to supply some to make sure this all works as expected.

 

Hi @gotak,

Feel free to send me a small subset of your data, then I will test this immediately to report if my solution works on your data as well 😉

5x light, 5x bias, 5x darks, 5x flats is fine.

Please send it to support@astropixelprocessor.com using wetransfer.com or dropbox.

Thanks,

Mabula

I'll do that later today when home.

Experimented with not using bias and yes it seems without bias there is no issue.


ReplyQuote
(@mabula-admin)
Quasar Admin
Joined: 5 years ago
Posts: 3180
 

Hi @francismoreau & @gotak,

The fix for your amp-glow issues is contained in this beta release, which you are invited to test.

The Windows installer for APP 1.071-beta is available here:

https://s3.eu-central-1.amazonaws.com/apastropixelprocessordl/AstroPixelProcessor-1.071-beta-Windows-64bits.zip

I will try to create the Linux and MacOS installers this evening.

Please let me know if this version solves your problems 😉 and if not, please share the problematic data with me so I can perform further testing.

Kind regards,

Mabula


ReplyQuote
Page 1 / 2
Share: