Frustrating times trying to process NGC6888 LRGB image files

Hi Mike,

Sure, just upload the relevant files (including calibration subs and preferably in their raw format) to the APP server: server, log in with username and password: appuser and please create a folder with your name first. I'll then try to check the data within a week. Please send me a selection of the raw subs, not the stacks.

@vincent-mod

Hi Vincent,

I have made a mistake logging onto your server - now getting a message that my IP has been locked owing to failed logon attempts. 

I thought this might clear after waiting a while but no luck so far.  Do you have instructions for unblocking?  Not sure if action is needed at my end or yours.

And can you also clarify the username / password that I should be using?

Uname 'appuser',  Pword 'appuser'?

Apologies

Mike

No worries Mike, I'll ask Mabula to unlock it. Yes, it should be appuser for both.

And it's unlocked. A lock is placed when you log in wrong 5x within 5 minutes. Be aware it's case-sensitive so all lower-case.

Hi Vincent,

I've uploaded a number of files as requested.  Hopefully you will not need more but I have plenty more if needed!

Hoping that you find something.

Mike

Hi Mike,

Thank you, I'm downloading them now and will let you know within a few days.

Hi Mike,

So I analyzed the data and I'm having the same problem you are seeing. Even just combining R, G and B I get a very purple crescent. Changing the R with the B and after color calibration, I got something a bit more like what I would expect, but the graphs show me that the colors are a bit "all over the place", which to me indicates you might still have had mixed frames or mixed channels somehow.

These are the color calibration graphs on the R, G, B data as you provided. There's a very clear skew instead of a more normal distribution.

Going back to the subs and the signal in them, I thought it looked a bit like the G is actually B, the B is actually R and the R is actually G. Ofcourse I'm absolutely not sure, but combining this combo did produce a more sensible color for the crescent and the color calibration curves look less skewed and more normal. I think there is indeed some mix-up going on.

Here a crop I made of the above picture, I took the green out of the yellow using HSL selective color which makes it look even more "natural". I think the filter wheel needs some repair or there's an issue in software controlling the wheel.

Hi Vincent,

At the moment I am even more confused:

a)  I just opened my FW and the filters are located in the expected position and when I look at the sky through eg the R filter position I see very red coloured clouds.

b)  I connected to the FW from SGP and when changing the filters using the FW control I see the filter wheel move to the expected position.

c)  I look at the SGP Sequence and Profile and everything there looks correct.

:  so at the moment I can't think what might be going wrong if your suggestion is correct

I will try a new RGB Combine assigning the stacks as per your suggestion and see what results.  In support of your idea I had wondered why the Blue and Ha stacks looked very similar - I had assumed that the Blue filter was capturing H-beta light in the same regions as H-alpha but perhaps that is not right conclusion.

Thanks for taking a look.  I'll keep you posted.

Regards

Mike

PS - I like your result, it looks far more natural than mine.

Like I said, I'm not 100% sure this is what is going on, but it does make a lot of sense given your statement about the issues you previously had with the filter-wheel and that the data looks much better when I change the colors around. It's just a matter of taking out 1 possible cause at a time to see if we can find the issue. Could you post a snapshot of the settings in SGP? And could you check the hardware configuration of SGP as well to make sure all is set well there?

Also, if my assumption is correct, would it make some sense in how the filters are put in your wheel that all 3 colours could have been reversed if e.g. the wheel turned the wrong way around? Just thinking out loud. 🙂

Hi Vincent,

I quite understand what you are saying about trying to eliminate one thing at a time.

I am thinking this must be either a systemic issue eg filter positions not correctly matched to SGP config files (or possibly an APP fault), or an intermittent issue, eg faulty USB connection.

As far as I can tell the filters are all correctly mounted as per the config file but I am thinking I will email a picture of the wheel with  the filters marked to SX and ask them to confirm that everything is mounted correctly.  I can also ask them to confirm the position sensing magnets are installed correctly, but as I mentioned when I look at the wheel turning under instruction from SGP it does seem to move and stop as expected.

I am wondering if I might spot something by examining the star counts identified by APP.  As a rule of thumb I think I should see the highest star counts in Lum, followed Green/Red, Blue and Ha owing to CCD sensitivity and filter band pass.  Would you agree?  By looking at the ranges in my Crescent subs I am wondering if I might spot an incorrect sequence or possibly a few outliers.  I have looked at all the subs on several occasions and I have not noticed any obvious outliers which is making me think this is a systemic issue.

A couple of small things I can mention.  First, I was imaging a couple of times during the recent very hot weather and discovered that the TEC cooling was close to its max rating and this resulted in some faint Moire like patterns on some subs - most noticealbly on Ha.  I have mostly replaced these subs so I don't think this is the cause   which I think would be no worse than a new dust bunny.  Second I noticed I had a couple of Green subs with rather high Registration RMS numbers which showed up as a one sided green fringe on a few very bright stars when combined.  I think I will eliminate these but again I am doubt ful this would cause the problem I am seeing.

Mike

Hi Vincent,

Just to update you I ran Analyse Stars with Automatic Stars Target set to 10000.

Sorting the results on Star Density,  I see a strong grouping of results with very little overlap between subs taken with the same filter and binning level.

L 1x1 120s:  Apart from 2 exceptions (2456 and 4221), all L subs scored better than 8620 stars.

R 1x1 120s:   Seventeen subs between 5661 and 7843, fifteen of which above 6423. One outlier at 2841. Remaining 3 at 4221, 5313 and 6060.

Green 1x1 120s:  Seventeen subs between 4375 and 6254. Outliers at 2792, 2841, 3026 and 3720.

Blue 1x1 180s:   Seven subs between 1901 and 2075.   Because I was seeing such low star counts on my Blue subs I deliberately increased the exposeure on these B subs to 180s

Ha 1x1 300s:  Fifteen between 2134 and 2626. Outliers at 1098, 1109, 1194 and 1616.

Red 2x2 120s:  Twenty-two subs between 1338 to 2036

Green 2x2 120s:  Twenty subs between 363 and 954

Blue 2x2 120s:  Thirteen between 133 and 711. Outliers at 91 and 116.

This strong stratification of results makes me think this is a systemic issue rather than a few anomalous subs skewing the results. Would you somewhat agree?

I propose now to drop the obvious outliers and redo the integration.

Given the very low star scores achieved by the 2x2 binned subs I am wondering about the benefits of capturing binned subs in the future!

One question I have is what happens when, as I normally do,  weight on quality during Integration.  Is the weighting done on a global basis or within a single Filter / Bin level?  If it is global it seems to me the Blue subs will be heavily penalised.

Regards

Mike

I'll have to give those things a long thought, I'm analyzing the subs in a few extra ways so maybe I can see something interesting. I kind of like some detective work. Not sure it will give me any answers though as I'm not sitting next to you to check the entire setup, which always helps. I'm pretty sure it isn't APP at fault though, although I never rule things like that out, but if it was we would have seen this issue a lot already.

If you choose to load in the subs based on the filter, it's quality stacking per filter.

@mestutters & @vincent-mod,

With regard to star counts in binned frames, i would not worry about this. This is just logical, if you bin, the star FWHM sized are expected to be twice as small in pixels. So a lot of the weak stars in the unbinned data, will not be detected as stars in the binned data because these stars won't even occupy 4 pixels which is needed for detection.

I would try to first work with integrations with equal weights applied. If that gives much different results than it might have to do with the quality weights. If it does not, I would expect a problem with the data like Vincent is suggesting, but this is merely guessing at this point...

Please let us know what the results are with equal weight integrations 😉

Kind regards,

Mabula

@mestutters & @vincent-mod,

I received the 5 integrations from Mike, thanks 😉

By looking at the 5 results per channel, I am very sure something is mixed with the filter assignments 😉

The Blue integration is almost identical to the H-alpha integration which I would never expect on this object:

Luminance:

Red:

Green:

Blue:

Hydrogen-alpha:

So, this just looks very odd. I think that in data capture frames were assigned wrong filters, maybe all frames, or perhaps only some... My advice, would be to first study your individual frames and make sure you group all frames correctly for a later to be determined filter name 😉

Then we try to determine which set is for which filter and proceed from there.

Kind regards,

Mabula

Yes, when I looked at the subs seperately that was exactly my thought as well and why I started changing them around, with a better result as seen above. So, must be a case of switching then.

 As seen in these graphs when switched around the distribution looks better.

Hi Mabula, Vincent,

I have read your notes above and will think about this further overnight.

As previously:

a)  I have examined my filter wheel setup and watched the filters move under control from SGP and all looks good on the set-up front.  I will however try to speak wih SX to make sure I've not misunderstod their documentation in any way.

b)  Having had a couple of issues of filter wheel disconnection I have been watching my imaging sessions closely and have not notced any problem.

c)  When I examine the raw frames through SGP viewer I do not see any single frames that look anomalous to others recorded with the same filter so if there is a problem I think it must apply to every frame within a group.

d)  When I looked at the Analyse Stars result from SGP I see a stratification of the results corresponding to the filter settings with frequencies which I think are logical ie Lum > Red > Green > Ha > Blue

e)  I've redone the integrations after dropping a small number of the lowest scoring frames and using only 1x1 binned frames.  I still get a similar result using assignments in line with the recorded filter (ie rather blue Cresent and background nebulosity) but the carbon star RS Cygni comes up showing Red.  If I swap the channels as suggested by Vincent (G > B, B > R, R > G), the Crescent and background look much better but the RS Cygni shows bright Green.

 I have had one further thought.   In front of my filter wheel I have a Baader Neodymium (Moon & Skyglow) Filter.  I've had this since I started imaging with DSLR to counter LP.  I kept it in the imaging train when I moved to mono CCD thinking it would keep dust out of the filter wheel etc when not attched to the scope.  I've never noticed a problem when imaging eg galaxies, but I'm now wondering if is perhaps harmfully affecting frequency response for targets with a high Ha background.   If you think this might be the case, might it be possible to play with the strength / background sliders in RGB Combine to counteract?

Mappings as recorded at data capture - Blue nebulosity but RS Cygni about right

Adjusted mappings - Much redder background but very green RS Cygni

Regards

Mike

Hi,

Mabula said:

I would try to first work with integrations with equal weights applied. If that gives much different results than it might have to do with the quality weights. If it does not, I would expect a problem with the data like Vincent is suggesting, but this is merely guessing at this point.

Here are the resulting screen-shots:

And an unvarnished RGB Combine  using the above - just add and calculate, no Ha

Regards

Mike

Yes, it's clear something causes the data to be mixed in a wrong way. The extra filter you have in front wouldn't cause dramatic differences, but that's just guessing on my part as I don't know the specifics around that. Maybe it does cause Cygni to be green after calibration, or it's because not all frames are switched and you get some cross contamination in the signal.

The give-away in the data that something is wrong with either the filters or maybe even the assigning of the R,G,B flags to the filenames is the fact that the blue channel is so bright, it shouldn't be on this object. And when I switch it around it looks more normal, so the simplest explanation is that the data has the wrong assigned labels.

Hi Vincent,

I've taking the liberty of uploading all my Crescent Lights to your server. Sadly some of them are not of the highest quality owing to light clouds and as mentioned earlier I was caught out by the recent heatwave which raised overnight temperatures to >20°C, causing the camera cooler to struggle and leaving water-silk like patterns on some of the subs.   Credit to APP in that the integrations all look clean to my eye despite these problems

As you will notice the subs were captured over several nights but when I look at the subs taken with each filter they do all seem to be reasonably consistent in appearance.   This makes me think this is not some intermittent fault.

If the subs are being consistently labelled wrongly then I guess we should be able to reassign the channels and get a decent result. Sadly I have nothing else to use as a comparison to decide which are which.

It being almost full moon I won't be able to get further data for at least a while

I'm pleased you think the Neodymium filter is not the issue but I think I will take it out just to satisfy myself it is really not the culprit here.

Hoping we can pin this issue down soon

Best Regards

Mike

Well I'm not sure it isn't causing issues, I would simply be surprised it would have this specific effect. As far as I understand a skyglow filter, it doesn't filter out R, G, B completely and when red comes through, having a red filter after that would work I think. It probably will diminish the signal a bit, but I wouldn't expect it to completely change the color, that makes no sense in my mind. I'm not 100% sure though if I'm missing something obvious here, but taking it out is always a good idea as having extra glass in between diminishes an already relatively dim signal. You will have less contrast if light pollution is high, but it depends then if you still can get more signal out of that soup and correcting the pollution with the LPC tool in APP or not.

Anyway, thanks for the subs, I'll have a look at them and see. If I simply do my conversion like posted above I'm guessing the result will be similar.

Hi,

Just to let you know that by using the integrations produced with mode Equal and by using L(L+Ha), R(Ha), G and B in RGB Combine, I have managed to produce an image that is I think a considerable improvement.  I think there is perhaps a little residual LP that could be removed and I would like to improve the colour of the larger blue stars. Something to work on.

I don't know if you or Mabula have had any further thoughts?

Regards

Mike

No other thoughts than we already gave at the moment. Your new processing looks better, but is still more purple/pink than red. Also the stars lack color and are pink, this still points to switched labels. I'll have a go at your lights again, with my switching around scheme. 🙂

So I did an entire integration with all your lights (only 1x1 bin to keep it simple), just using bias and with my scheme I mentioned a few posts back. After the combination, the yellow stars indeed look green so that to me indicates contamination of certain channels with others. However, it still gives the best starting point and after color calibration I get them to turn orange like expected and blue-ish for the blue stars. Tweaking the colors a tiny bit, adding some saturation, contrast and sharpness (all in APP) and it looks as good as I can get it with this data. Honestly, to me this proves the labels are somehow mixed/switched as using the normal labels I can't even get close to this result.

ps. I also decided not to use the luminance. It has almost no signal for the nebula. I have no experience in taking luminance though so maybe I'm mistaken here, but I still decided to leave it out as I didn't see it would add a lot.

Decided to stretch it a bit more carefully, did a double saturation on it (maybe a bit overdone) and saved again. Does pop out the blue better and the more careful stretching got the surrounding dust out more while trying not to clip the bright crescent.

Hi Vincent,

I took a good look at my previous image in natural light and see what you mean about the stars being pinkish. I've had another shot and the result is below.  Possibly a little too pink for your liking but as I was after catching the faint nebulosity in region around the Crescent I am now reasonably happy with the result.   Looking at Astrobin there are numerous interpretations of this object so I'm not sure which colour balance is definitively the most correct: nearly all the pictures awarded Image/Pick of the Day are made with Ha/O3 which yields a very different result anyway. Either way, having this dialogue with you has made me look more closely at what I was achieving.

I've just spotted Diego's video of Wolf Nebula so think I'm not alone in having some difficulties in colour balancing images with significant Ha backround content!

I'm still not convinced there is anything wrong with my filter wheel configuration but I'm thinking of setting up a long sequence in SGP and imaging a known colour test target so it should be very obvious if an error is occuring.  If I include a large number of filter changes then hopefully this will also show if I have an intermittent fault.

Regards

Mike

Well, in the end it's ofcourse what you like best yourself. 🙂 And it's true that in "reality" hydrogen has a more pinkish glow to it. But the stars should be orange/white/blue if you look at what their spectra look like visually and that's not the case in the processing done with the labels as they are. However, let's not keep on processing as that could take up a few pages more. In the end it's about your taste and what you're happy with and I'm happy you got some more insight from it either way!