[Solved] Integration - Weights

Hi Steve,

Yes, in the most cases, the quality setting for weights will give you the best integrations for noise and sharpness combined.

Can you replicate this behaviour?

A completely black integration would mean that no data was actually integrated. So one cause could be that the quality scores were zero or infinite perhaps? Can you check the quality scores of the frames after registration and try to integrate again with quality weights?

Cheers,

Mabula

Hi Mabula

Yes.  It is completely reproducible.  I am working with a stack of files that have already been calibrated, registered and saved.  I don't see anything odd in the quality scores - but here is a screenshot with the stack loaded (all of the 'numbers' for the calibrated files can be seen in the bottom window):

If I leave it as 'Equal' then I get a normal looking result. 

Incidentally, I have just checked a few other images.  Every time I Integrate with 'Quality' I get an image that is completely black.  'Equal' gives a reasonable result, but if I should be using 'Quality' ....   My workflow is: calibrate all subs and save them; register all subs together and save them; load in the subs for one channel (eg Ha); and integrate with registration mode set to 'no registration'.

Ok Mabula - a bit more progress with testing.  If I try integrating the 'Registered Lights' that I have saved, I get a black screen.  If, however, I go back to the saved calibrated lights, then I get a good result.  So, for 'Quality' to work, APP seems to need to perform a registration before integrating.  Why 'Equal' would work with my pre-registered subs whilst 'Quality' will not is a mystery to me.

I hope this helps.

Steve

Hi Steve,

Thank you for testing. Yes this confirms my suspicion.

The problem  then most likely is due to a bug I suspect with the quality calculation in case of using already registered frames... I'll test this myself tomorrow and will fix it.

Thank you for being so thorough 😉

Mabula

I just used a bunch of of registered frames that I had saved. I ran them through the integration module using Quality as the setting for weight and it worked fine. Maybe its a combination of settings?

 I don't know if this will help, but perhaps another data point - I happened to do an integration last night using quality weighting and it worked fine.  One thing that I always do after calibration is to save my calibrated lights.  So, my basic flow was:

1)  Load lights and calibration files.

2)  Calibrate

3)  Save Calibrated frames

4)  Normalize (I only did this so that I could see the quality scores before doing an integration - normally I just go straight to integrate).

5)  Integrate.

As I said, I didn't have any problem when I did this.  One other difference I noticed was that Tim's quality scores were a lot higher than mine.  His were around 1500 - 2500, whereas I recall my highest being around 95.

Hope this helps,

Scott - http://www.astronomersdoitinthedark.com/

Posted by: Tim

I just used a bunch of of registered frames that I had saved. I ran them through the integration module using Quality as the setting for weight and it worked fine. Maybe its a combination of settings?

Okay, thanks Steve, will test this 😉

Mabula

Posted by: Scott Rosen

 I don't know if this will help, but perhaps another data point - I happened to do an integration last night using quality weighting and it worked fine.  One thing that I always do after calibration is to save my calibrated lights.  So, my basic flow was:

1)  Load lights and calibration files.

2)  Calibrate

3)  Save Calibrated frames

4)  Normalize (I only did this so that I could see the quality scores before doing an integration - normally I just go straight to integrate).

5)  Integrate.

As I said, I didn't have any problem when I did this.  One other difference I noticed was that Tim's quality scores were a lot higher than mine.  His were around 1500 - 2500, whereas I recall my highest being around 95.

Hope this helps,

Scott - http://www.astronomersdoitinthedark.com/

Thank you for your feedback Scott 😉

The problem that Steve encountered happens with loading already registered frames, and then using the "none" registration mode. It's a bug having to do with quality calculations internally, which I will fix.

I tend to do the same as you, I always save my calibrated frames, because I feel it's very easy for later use of those frames when I add more data or if I just want to reprocess the data.

Regarding the quality scores: they are based on the star density, noise and FWHM values in pixels of the reference frame of the frames that you loaded. If you have low quality scores, you probably have bigger stars in pixels or a lower star density in the field of view?

cheers,

Mabula

Hi Mabula - yes, you're probably correct about the quality scores.  I was integrating a 1200mm image of the galaxy NGC 772, so it was significantly less dense in stars than Tim's Propeller Nebula.  My FWHM were higher, too - typically around 2 to 3. 

Thanks for letting us know about the factors involved with the quality scores.

Scott - http://www.astronomersdoitinthedark.com/

Hi Scott,

The quality scores are based on your reference frame like I indicated. If you combine data of different image scales and different field of views, the scores will be corrected for the different image scales and field of views 😉 That's the reason for showing the star density (instead of just star count) and relative FWHM values. So these scores are image scale and field of view corrected relative to the reference frame.

Mabula

Hi Steve and Scott,

The problem is solved 😉 and is fixed in the next APP release... (will release it within a couple of days now...)

https://www.astropixelprocessor.com/community/release-information/astro-pixel-processor-1-055-preparing-next-release/#post-1195

all the way to the bottom...

In 4) REGISTER, the "no registration mode" had a bug. The star density wasn't calculated correctly resulting in a wrong quality score. After normalization this error further extended since new quality scores are calculated after that step. When you started integration with quality weights, this caused the integration to integrate with weights of zero, giving you a totally black result... this shouldn't happen anymore with APP 1.055.

I have double checked in testing, all the settings for integration weights now produce correct integrations when processing already registered frames.

Kind regards,

Mabula

Great stuff Mabula.  Looking forward to the fixed version.

As a user with a large number of lights ( 8" f/3.9 reflector, unmod Canon 6D) I too am looking forward to the new release!!

Any potential issue date yet?

Hi thompeters,

I am planning to release it within a couple of days time now. I am finishing work on a new star color calibration module as well and want that to be part of the next release 😉

Kind regards,

Mabula

Mabula,

Do I understand correctly that an integration using “quality” rather than “star shape” will result in a lower noise image?

Thom

Hi Thom,

It can indeed. The quality weight setting optimizes the integration for

• star shape (roundness + size)
• noise
• star density

So it optimizes for several things and the result will be better for those factors combined. On the other hand, if you have images for which the star shape varies significantly between the frames (so some frames with less guiding for isntance) then the star shape weights can be very beneficial, because the integration result will be much sharper. In the FITS header of the integrations you can find the MRS gaussian noise value of the integration. You can use that as s guidance to verify which integrations are best for noise. If you want to verify for FWHM, load the different integrations as lights, and process them up until 5) normalize, then you'll see the FWHM and noise values in the frame list window. For some datasets, the integrations with star shape weights can be more than 10% sharper (FWHM value) 😉 which can clearly be seen.

Mabula

Thanks for the great explanation.

Would you please explain what MRS Gaussian noise value is? Is that the same as RMS (root mean square)? I know a lower value is better but is there a “better/best” target value?

LOVE LOVE your program sir! It is so much easier and straightforward than any other program expect perhaps DSS! But obtaining a usable image and the ability to tune that image is far superior to DSS!

Thom

Hi Thom,

You're most welcome, thank you for your nice compliments 😉

MRS gaussian noise is an abbrevation of Multi-Resolution Support Gaussian noise.

Simply said, precise noise calculation is not a trivial task, it's pretty difficult, especially if you have data that is full of stars and nebulosity.

the MRS gaussian noise is discussed in the following paper if you're interested:

Automatic Noise Estimation from the Multiresolution Support

by Jean-Luc Starck and Fionn Murtaugh

Simply said, a simple noise calculation (using easier to implement methods) will estimate the noise with a precision of about 10% only. That is not very precise.

The MRS gaussian noise, if implemented correctly, will have a precision of only 1%. So the error in the estimate is only 1% of the found value. That's 10 times better 😉 than those simpler methods.

Even more simply said, the MRS gaussian noise metric is probably the most precise noise metric available.

Let me know if you have more questions 😉

Mabula