APP Ha OIII extraction from OSC with Duoband filter - Unmixing Bayer colour leakage from duo band filters

@williamshaw I'm not an expert on APP, so take what I say with a grain of salt... However, I can't imagine how APP could deal with light leakage when it uses the Ha/OIII color, mono, extract Ha, extract OIII algorithms because 1) the relative amounts to subtract to get pure Ha and OIII depends upon the relative QE of the chip, which can vary considerably, and 2) although you say it's independent of the bandwidth, in fact whether the bandwidth of the OIII band is broad enough to include the Hb signal would have a major effect on the amount that needs to be subtracted to get a pure OIII signal. Without having inputs for your chip and filter, it can't imagine how it would be possible for APP to come up with proper scaling for "leakage". As you say, you can always determine your own values for Ha and OIII leakage and subtract using something like PixelMath but given that the amount of leakage is generally nearly a magnitude lower than the signal, I expect most would just ignore it. 

But those with more intimate knowledge of the algorithms, feel free to correct me. I'm interested in the answer as well.

Keith

@microastro Thanks for the feedback - Don't have an argument with any of that! I did not actually think that the details would be completely independent of the filter bandwidth, it was more than I was trying to make sure people understood that this matter would exist even if the filter bandwidths were tiny. Truth is, I do not know what that tab 0 option does at all, and would quite like to know! I did speculate that, on top of doing a basic job without leakage correction, it might do a generic remapping that might allow for a typical level of leakage, without necessarily being spot on for any particular filter and sensor QE combination. Another thought was that there could be an optimisation algorithm in there to adjust that typical level to something more optimal based on a fitness function - image contrast? - who knows? One final observation is that although the leakage indicated by the QE plot is small, the Ha coming in is often quite strong compared to the Oiii and Sii so the levels on the graph I showed could have uneven multipliers. Any further thoughts very welcome.

Good points. Hopefully someone with more detailed knowledge of the algorithms can enlighten us.

Keith

I would argue to use the extraction methods always as these are specifically meant for this (they also make it possible to retain the resolution of the separated channels), but to get a more detail answer I asked Mabula to chime in, he is busy till at least tomorrow, but I hope he can get back to you soon.

@vincent-mod thanks. I have mostly been using the tab0 Ha Oiii extraction setting as I realised it gave slightly cleaner data than eg just B and R. But this discussion of leakage made me realise I do not know how the extraction works in terms of using the RGB raw data. It would at least be good to know that as a starting point for perhaps refining a method to correct for Bayer leakage. Will check back for further insights. Ta

Hi William @williamshaw, Keith @microastro & Vincent @vincent-mod,

It is a very good question to ask this, thanks!

Extracting the pure H-alpha and Oxygen-III signals while compensating for bayer filter leakage is simply impossible if you would ask me. You can not determine per pixel how the ratio of incoming photons is distributed  from the varying signals...including noise as well. So I would not even attempt to go there 😉

APP's Ha & O3 extract debayer algorithms are created to give the user a very simply way splitting the channels while preserving the native sensor resolution. Depending on which extract algorithm you would use, APP uses the best possible debayer solution for the appropriate signal (Ha or O3).

For instance, if you would debayer with Adaptive Airy disc to get a RGB result that contains both Ha and O3, you will get a worse result than debayering with the Ha-O3 color algortihm.

If you debayer with Ha-O3 color and the split the channels of the RGB result, the red channel is the H-alpha signal, and the green and blue combined is the O3 signal. APP makes it easy, with the extract algorithms, to immediately provide you with these monochrome Ha & O3 signals without loss of resolution which has great benefits of course.

Let me know if this clarifies things 😉

Mabula

@mabula-admin thanks for this information. Very useful clarification. I did imagine that it would be hard work at the very least to clean up any leakage automatically but thought it useful to check on this. 

You did not specify exactly how Ha Oiii extract works and what is meant by "best possible debayer solution" for it.  No problem if that is commercially confidential but I remain curious!

I think the one thing I can try is to try to calibrate an unmixing of the leakage myself for my particular filters. For example I have an Halpha filter from my older HaRGB days, so could actually shoot with the Ha filter in line with my duo and see exactly how much Ha leaks into the Oiii channel that your algo extracts, then explore subtracting it. That assumes a certain amount of linearity and might depend on the bandwidth of my Ha filter, but would at least give me a starting point for a simple one-sided unmixing. If that works I could then attempt a full matrix calibration. Thanks again.

Hi @williamshaw,

"You did not specify exactly how Ha Oiii extract works and what is meant by "best possible debayer solution" for it. No problem if that is commercially confidential but I remain curious!"

Well, simply said, a normal advanced debayer algorithm uses information from the green channel to reconstruct both red and blue. That is far better for regular for broadband RGB data. The key here is that for the H-alpha channel reconstruction, that is not the best method of course. The Red channel can only be reconstructed using information in the red channel 😉 For O3, the green channel which has more information can again be used for reconstruction of both green and blue channels after which APP directly created the monochrome O3 signal from the green and blue channels.

I think the one thing I can try is to try to calibrate an unmixing of the leakage myself for my particular filters. For example I have an Halpha filter from my older HaRGB days, so could actually shoot with the Ha filter in line with my duo and see exactly how much Ha leaks into the Oiii channel that your algo extracts, then explore subtracting it. That assumes a certain amount of linearity and might depend on the bandwidth of my Ha filter, but would at least give me a starting point for a simple one-sided unmixing. If that works I could then attempt a full matrix calibration. Thanks again.

Do let us know if you are able to get some interesting results 😉 I do plan to make it possible that camera matrices can be used and programmed even for processing...

Cheers,

Mabula

Will do. Could be a while as I am doing galaxies right now but will attempt it when I go back to a suitable emission nebula with a good mix of H and O.

@williamshaw This has been a very interesting thread. I have a filter wheel on my ASI2600MC camera and currently have Ha and OIII filters in the last two spots. I had been thinking of replacing them with a duo-band filter to speed up data collection, but this thread has caused me to reconsider that strategy. Especially your point about the impact of stronger Ha emissions leaking into the G & B channels and overwhelming the weaker OIII signal. I'm thinking now that my current setup with separate Ha and OIII might be the better strategy. Never would have thought about this if you hadn't brought it up. Thanks for posting.

To be clear it wasn’t me who spotted the issue, it was Jean Dean, an exceptional astrophotographer who queried my use of pairs of dual narrowband instead of the individual band filters in a Facebook group.  I think it can be fixed by doing some calibration but I need to look into that. 

@microastro Keith, the risk of Ha emission leaking into G and B very much depends on the filter used and may also happen when using an imperfect OIII filter. As can be seen in the diagram posted by William in his original post, the filters don't fully block Ha in G and B and OIII in R. But that simply applies to the filter he used. Other filters may very well have much better isolated pass bands and shouldn't run this risk. Many people on this forum use L-eNhance and L-Xtreme filters which perform very well in that respect.

@wvreeven I think you have really misunderstood the source of the problem, and also my graph. Those channel sensitivities are of the sensor colour channels, not of badly wide filter transmission bands. The issue is leakage caused by the Bayer matrix. The narrowness of the band pass in a Duoband type filter will affect the total transmission into each channel but the relative effect is a persistent problem.  Even even if the filters were a perfect pair of delta functions at the emission bands you would have the problem we are discussing here. The underlying issue is the non zero level of the Bayer matrix sensitivity of, e.g.  the blue and green channels to light at the Ha frequency. The narrower bands of something like an L-Extreme do a good job of blocking out background LP but do nothing to mitigate the problem here. I drew my graph with the emission lines as delta-functions rather to make that point. 

The real source of the problem is the fact that the colour sensitivities of the Bayer structure are not confined to the red green and blue zones the way R,G,B filters are. I believe this is because colour cameras are designed to mimic the properties of the colour receptors of the human eye in having wide sensitivities, rather than being designed for scientific purposes like our astrophotos. If the Bayer matrix filtered light as perfectly as R,G,B filters this problem would not arise and it is the fact that our Bayers do not that is the problem, not the details of the filter. 

@williamshaw Thanks for the clarification William, I indeed misunderstood that. Still, this issue will also have effect on images taken with a Ha or an OIII filter so that's no reason to not use a dual-band filter.

@wvreeven My initial reaction to hearing about this was the same as yours in fact! What would be really cool (and solve it) is an OSC camera with perfect RGB separation. Or even one with emission band filters on the sensor. Imagine OSC Hubble shots. 

I also agree it is not a reason to stop using dual band with tools like the APP Ha-Oiii extract. I do think the images might be improved by cleaning up the leakage but I need to work on the numerics of that. But I also think the single band filters do a cleaner job - after all with a pure Oiii filter there would be no Ha coming in anyway to leak over to the B and G from which you would extract the Oiii. But with the duos you are using the Bayer at higher efficiency and get a double dose of Oiii from something like the NB2,3 combo (or L-Extreme+NB3), so there are very good reasons to persist with them. 

@williamshaw You're right of course that narrow band images will provide cleaner results. One might even try to compensate for the leakage by discarding the R channel in case of an OIII filter and discarding the G and B channels in case of Ha. On the other hand, many astro photographers suffer from lots of clouds and can save a lot of time by using dual-band filters. So whether or not to use a dual-band filter needs to be reviewed on a case by case basis and not discarded up front because of sensor leakage.

@wvreeven Well that's my view too! I'm rather minded to pursue the issue of how to squeeze the best out of one or more duos. And living in cloud-covered England I have a very strong incentive.

Hei everyone,

Thanks for all the valuable input. It was a very interesting discussion. Recently I started using a cooled OSC camera (QHY268c) and an Optolong L-extreme filter. That is why I was interested in how to extract Ha and OIII from an RGB images. I tried to compare the Ha and OIII extraction in APP with a manual method in PI. For the manual method I calculated the weights as described here. Curves were sampled from the datasheets provided by QHY and Optolong. Not 100% accurate.

After doing the math, I came up with the following values to enter in PI's Pixel Math:

I have to admit that I'm very new to APP and PI and I did not pay too much attention of what I'm doing. Therefore, the images are drizzled slightly different. So please take this into account when comparing the following images. Furthermore, the quality isn't the best. I took these images with a small MSM tracker which allowed me only to take 1min subs which is not really enough when working with the L-extreme. I didn't collect a lot of data either.

No DBE applied in both cases. Just the pure extraction of Ha and OIII:

...and a crop on Orion and the Horsehead. Note that I didn't drizzle my APP data (or maybe not x2). Stars look a little squarish in APP ... don't know why. Noise looks quite similar.

Then I stretched all images and combined them into a HOO image.

... and another crop.

Stars are still smaller in APP. Maybe I messed something up in PI. But the other all look and noise are quite comparable. So, in my opinion, both methods work quite well. What do you think?!

Posted by: @mabula-admin

If you debayer with Ha-O3 color and the split the channels of the RGB result, the red channel is the H-alpha signal, and the green and blue combined is the O3 signal. APP makes it easy, with the extract algorithms, to immediately provide you with these monochrome Ha & O3 signals without loss of resolution which has great benefits of course.

Let me know if this clarifies things

Hello Mabula

Recently, I have been a lot of testing to check the difference on the Oiii results after an OSC capture with dual narrow band filter, an Idas NBZ in my case.

Even if the Green channel is the main "input channel" for the Oiii wavelength, there is still some SNR improvement to get from the blue channel.

So I tried and compare the 4 following channels from the exact same set of datas:

- the Green channel obtained after the "Ha-Oiii colour" algorithm

- the Blue channel obtained after the "Ha-Oiii colour" algorithm

- the Oiii obtained after the "Ha_Oiii extract Oiii" algorithm

- the B_G obtained after an integration of Blue and Green channels with a ponderation based on quality (the 2 channel are linear fitted before the action) 

Here is waht I saw from these :

1- the worst SNR is the blue channel, 

2- then Green channel, Green has a better SNR than blue

3- B_G is better than Green (and blue obviously) , 

4- Oiii is better than Green 

5- not a big difference but I would say that B_G is slightly better than Oiii in terms of SNR; always hard to compare, you have to make a strict comparison of the histogram and evaluate the "width of the sky background" for example and be sure that there are no offset that disturbs the analysis, and if there is a difference, in all cases, it is a very very small difference

So my question to understand the "Ha-Oiii colour" compared to "Ha-Oii extract Oiii" algorithm :

- why Green & Blue are differents with "Ha-Oiii colour" ? after all, we want to get 2 channels : Oiii & Halpha, why leaving the Green and Blue "separated" ? Obviously the blue capture from the camera has a worse SNR that the green but why leaving thes channels in the Ha-Oiii colour algotithm ?

- Am i correct in the analysis of Green / Blue channels versus Oiii ? If yes; why didn't you provide an algorithm that provides the results of Halpha and Oiii in an HOO RGB file ?

I hope that I am clear, I am a little disturbed because what you implemented seems the best way to extract a dual narrowband acquisition but still, two operations needs to be done : either play the datas with "Ha_Oiii extract Oiii" then "Ha_Oiii extract Halpha" algorithm or play the data with "Ha_Oiii colour" then recombined Green and Blue channels

thanks in advance ... and thansk for your work 🙂

Jean-Baptiste 

(from France)