Calibration Tab: SAVE Calibrated: Remove Light Pollution

Dear Mike,

Thank you very much for your question on this.

The batch LP removal has been on my Todo for a long time indeed. The reason it is not there yet, is because it is much more complicated to do this correctly. It is not a trivial task.

A batch mode would allow you to place area select boxes on 1 frame, the reference frame, after which all images would be corrected with the same area selectboxes creating a correction model per image. The correction models per image need to be calculated  after they have been registered to the reference frame ideally. If you do it before registration and the images have a clearly different orientation than the reference, due to agressive dithering for example things will not work well and might even complicate data processing a lot...

Theoretically, data calibration should include LP removal per frame, if you would ask me 😉 so that is the reason I provided this issue several years ago already when APP was just released.

If you would shoot data at different sessions, then yes, the LP correction model of the stack is very complicated when compared to the models of a single session, let alone of a single sub. The models per sub will be the least complicated of course.

I guess the number and size of the sampling boxes has a major bearing on the result?

Well no, placing more boxes when the correction is already top notch, will do little and might even cause the model to surpress faint signals. Do not place too many area select boxes ! The results will be best with little area select boxes when that leads to a good result already.

The size of the boxes are important in the sense, that a larger box will sample the sky background better than a small box. It has more data, so better statistics. A bigger box will be better to ignore stars in the boxes as well.

If there are grounds for anticipating noticeably improvement then I would propose for consideration the following approach which seems to me would require only a modest change to APP. At present it is possible to save subs following star analysis and registration. If it were possible then to go back to Calibration with the saved subs and then placed sampling boxes for LP correction onto a single frame and then to use the same set of sampling boxes for all subsequent frames taken with the same scope/camera/filter etc combination then this I think would be a huge time-saver compared with the current approach which I believe necessitates putting sampling boxes on each individual frame. I am a monochrome imager soI am thinking that following registration the sampling boxes are placed in positions of the lowest expected luminosity in the first frame taken with a given filter would also apply to all the other frames taken with the same filter. A refinement might be to slightly dither the positions of the sample boxes between frames - excuse my ignrance of statistical sampling!

I would propose to remove the Remove Light Pollution option in 2)Calibrate and have it replaced by an option in 5)Normalize to Batch remove Light Pollution on the registered frames after which the normalize parameters are calculated as well. That is the most logical and smartest solution I think.

In normal registration mode, each registered image will be corrected with it's own model using the same Area select boxes as supplied once only on the reference frame.

In mosaic registration mode, you will be able to correct LP on all the mosaic panels of course.

I will be happy to add this to our ToDo and place it high, since the LP removal in calirbation really should be removed and replaced soon by a much better option I feel.

What do you think?

Mabula

Hi Mabula,

Thanks so much for taking the time and trouble to respond to my email.

I like your suggestion of transfering the Remove Light Pollution option from 2) Calibrate to 5) Normalize. It seems to me that this would achieve exactly what I was suggesting and seems to me to better fit with the existing APP workflow.  Also it seems you are simply(?) relocating the trigger point for existing APP capabilities from one point to another so hopefully the impact of the change proposal would not be that great.

I fully appreciate the need to correct LP post registration otherwise the area select boxes would need to be positioned on each frame individually for the reasons you highlighted. 

What is the processing overhead of calculating an LP correction model? The elapsed time from pressing the Calculate button to having the resulting corrections displayed seems quite modest when the process is invoked from the Tools/Remove LP menu but clearly the time taken would rapidly climb if the process was applied at a per frame level for an integration with e.g. many tens of frames taken with a large format, miniscule pixel camera.

Assuming corrections are made a per frame level, I assume it is infeasible to calculate/recalculate the corresponding correction model on the fly, so therefore it would be necessary to save the resulting corrected image for subsequent use? In saying this I am assuming that the correction model would have significant size (i.e. of similar dimensions and pixel depth as its corresponding image file)? Or is the correction model something that can be losslessly compressed to a quite modest size? I do not have an issue with the idea of saving the corrected image files and then to reload them as Lights for a final integration. For users struggling for disk space they would have the option of dealing with LP post integration as at present.

If I've understood you correctly I have a minor concern with your idea of positioning area select boxes solely on a single reference frame for a multiple channel integration. As you say the objective of LPC is to remove unwanted signal while not obliterating faint nebulosity from the real target objects. My concern is with NB and BB integrations.  I'm thinking that faint nebulosity in say an NB channel may not be at all apparent in a given reference (typically L channel) frame. In voicing this concern I can see however that if a user thinks this is a real issue a workaround would be to process the subs channel by channel prior to performing a final multi-channel integration.

I'm not personally into mosaics: I find it difficult enough to acquire sufficient data in BB to achieve a reasonably decent SNR level in the period that most objects are well positioned  let alone to acquire sufficient data for overlapping views of the same object. Notwithstanding, how do you envisage that area select boxes would be positioned on a multipanel mosaic without also having a multiframe reference image on which to work? An obvious workaround I guess would be to LP correct the frames for each panel individually before performing a final collective integration.

Summary

Your reply suggests that you are already mindful of some scope for more refined approach to light pollution removal in APP. No doubt you have some further ideas that go beyond my current thoughts.

I hope this response is useful to you. I would appreciate a heads-up when you have decided how you wish to proceed.

Best Regards

Mike

Hi Mabula,

As above I have often thought that performing LPC individually on source images prior to integration ought to yield a superior result to later application, especially for users who do not benefit from near pristine skies.

However I am seeing some artifacts in my integrations that I ameliorate by means of the LPC tools I am by no means certain are actually caused by widely varying gradients in the source images.

My most recent project has been M94 for which I have collected some 260 source images in LRGB.

Here are two screenshots of the L frame selected as the reference by APP using the Linear and Calibrated viewing options:

a)   Linear

b)   Calibrated

32-bit MB, MD, MF and BPM are being used for calibration.

Apart from a number of cold pixels and elevated brightness due to LP these frames look reasonably decent to my eyes.

c)   Full frame view of reference frame

d)   L Channel Integration

 At this point all looks acceptable.  However after applying a Crop I see what seem rather excessive artifacts emerging during LPC.

e)  Artifacts in background areas

I appreciate this is a highly stretched view of the L channel integration but I feel the luminance variation in the background areas is considerably greater than it really should be. I do not understand the cause of this variation nor if there is some setting I might use to reduce the impact.  

 f)   Final Result

By reloading my channel integrations as lights and using the Remove LP option in Calibration and again after RGB Combine, I do eventually manage to get (in my view at least) a reasonable result.

g)  Application of LPC to eg the Reference L frame

If LPC was applied individually to the original source frames prior to integration I feel I would have a better starting for integration and subsequent processing in comparison to Image c).

I would appreciate your thoughts on what might be causing the the luminosity variations in Image e).  Is this an avoidable problem by means of using different settings? Or is there an issue somewhere with APP or my calibration frames?

Regards

Mike