[Solved] Alternating Bayer pattern within a single imaging session

You can check the FITS header itself in APP (the window left of the image viewer (if not visible, click on the tiny arrow-sign shown in the screenshot below). In there check for the pattern and if you indeed see differences, is that the case?

If it is, the next APP version will have an easier way to change this in the header. For now you'd need other software. If it is the case, it's also important to figure out why, what capture software are you using and could a setting in there be giving the problem..

Thanks Vincent,

I know about these aspects of APP, of which I am a happy user. I use APT, and this problem exists regardless of option changes. I fear it may be an ASCOM issue, or the way APT writes to file perhaps. 

At the moment, I took one project and checked every sub to suffix them with GRBG or BGGR. I will process tow 'sessions': one with subs from different night (identical setup and acquisition setting/software) and process the GRBG together, and a second session using BGGR images, and then combine both stacks into a single master light. And see what happens.

My worry is that the darks have a similar issue, and it seems they do although much smaller compared to the green wash and strange maze-shaped pixelation in the GRBG vs BGGR debayer issue with this chipset/software.

Using the native software and driver always works fine, but does not integrate with PHD2 for dithering, so I am stuck. I wonder if others have this issue, which is separate form the ROWORDER problem with Top-DOWN or BOTTOM-UP reading of the FITs header. With everything unchanged, including any setting, approx. 50% are either BGGR or GRBG for this chip.

I'll reply once I know if it improves a project.

Second query: with such a debayer issue, how do I tackle the Ha and OIII extraction for my subs with OSC camera and quad band filter? I have not checked whether Ha and OIII extraction from filter OSC requires me for force CFA? If not, life is easier for those subs. If so, I assume I need to extract Ha from my GRBG stacks and again for the BGGR stacks, and twice again for OIII....ooof.

Mmm, yes I have seen driver issues (or supposed driver issues) with a few camera models. Very frustrating, something to take up with the manufacturer and the makers of APT I think.

Regarding extracting the Ha and OIII; good question, I have no experience with a mosaic pattern being incorrect and extracting data. I would assume that may be wrong then as well as APP likely uses the pattern to determine where the signals for Ha etc should be. I have no good solution to correct that, other then changing the mosaic pattern to be correct yourself, maybe with the next APP version.

Posted by: @vincent-mod

Mmm, yes I have seen driver issues (or supposed driver issues) with a few camera models, where the fits goes to an all-green tint, with the next image being fine again. Wouter uses an ASI2600MC which seems to have that on the driver level.

I am not sure at all that the issues I have with the ASI2600MC are related to what this thread is about and I am still investigating what's going on so let's not drag that into this discussion please.

Sorry, that wasn't my point but I removed it not to confuse the thread. Good call.

Some more information:

opening up every single sub in APP and forcing either BGGR or GRBG allows me to see which one is which. It is almost perfectly 50% for an even number of frames in a single session. It does not alternate every other frame, which is interesting especially since it works out as 50% with one pattern and 50% with another.

I labelled each of the 112 subs for M101, and integrated each batch separately, and integrated the two outputs together and it looks much better. Stacking stacks requires a similar reference frame and can introduce some noisy edges it seems, but easily cropped.

I will post something on APT forum, but I also notice it when using ASCOM driver with Sharpcap, for other targets when I used to use it. Seems to be related to camera and/or driver. Also I recall some discussion about ASI 1600 MC and ASI 1600 MM with MN34320 chip that had a similar confusion about bayer pattern. 

Finding out it gives 50% correct and 50% incorrect in a single session where nothing changes was important but very annoying to process, especially if I try to split RGB and merge with extracted Ha and OIII. 

I wonder if there are many users of various camera that view the green tint as a typical result of having a matrix with two G components, but have this underlying issue on top of that? The typical greenish tint should be very slight. The large green wash with unusual maze-like pattern seems to be an issue of alternating file writing within a session for some cameras, and there might be many out there with excellent data, and less than excellent final stacks images due to many subs being forced to debayer from a single session (as we would expect to be OK), that are adding green and more difficult noise floor and patterns to process out. I doubt it happens for all cameras, and will try to find out if uniquely ASCOM related. When I mix them all and force a debayer with GRBG only or BGGR only, the noise in the background undoes all my 30 pixel smooth dithering. When I carefully process each batch of correct debayering, its comes back beautifully.

I've put a few screen shots of a single 240s sub of M101 showing a typical example of a single frame within a single session, using BGGR and GRBG. Of course, the same difference is found using these bayer patterns in the opposite way for the other 50% within this session.

An update and possibly advice and solution to this issue.

For users with ASCOM cameras using APT (maybe extends to native driver versions also, I dont know), the debayer preview does seem to influence the final output file, at least within the FITS header.

When I choose the debayer pattern for onscreen APT preview, that is written to the FITS header. I tested this with all patterns and in APP, only one will show correct colors (using an image containing a brown fence, red door and a blue basketball ring!)

Shift+click on the camer connect will allow tuning of ascom driver settings, but also a choice of debayer preview. Select you option there. Then go to APT setting and do the same for the debayer preview option in the CCD tab. This does not always save and can default to RGGB. This is likely part of the problem before, since some of the FITS headers have RGGB, requiring a GRBG pattern to display correct colors for an BGGR camera!

Ultimately, by having your correct debayering as a preview in the camera settings in both instances (shift+click for camera connection and APT settings CCD tab), then the BAYERPAT keyword written in the FITS header is correct, and it renders correctly in APP under forced CFA with that pattern, or indeed as 'supported' without forced debayering.

Others have notice this 50% incorrect debayering effect after I looked deeper through APT forums, so I hope this solution helps. I still believe that APT users who are not carefully checking the debayer preview option are processing subs that have odd colors within their stacks, at least it was the case for me and others. APT states the preview does not affect that data- that is true at pixel level, but it does write to the header and that is important for many processing softwares.

Clear skies,

Oeh that is both good to hear and not very nice from a workflow standpoint. Can you also ask the APT developers to maybe change this?

It is quite a annoying as an extra step, especially if you have hundreds of short exposures. The alternating change is random, but 50:50!.

To be fair to APT, I can now report the same issue with Sharpcap. Even when force BGGR preview is enabled.

One more note: testing during the day with objects of known color shows something interesting. Short (several ms) exposures with the correct preview and this fits header writing, works perfectly. When the camera is cooled and/or exposures are tens of seconds or more, this issue returns.

It is happening for quite a few models of cameras it seems. And may people have stacks that are below the quality I believe they could be, because of this debayering oddity.

Still no obvious solution. I do note that my camera is absolutely BGGR, but the ascom driver lists it as an RGGB-type camera. That RGGB is meant as a general statement, i.e. it is not a CMYK type. I wonder if programs see this as being the actual bayer pattern? 

Sharpcap reports automatically as GRBG, APT is whatever if preview it to be. BGGR and GRBG are a pair, so....

All is well if I split all my subs into two GRBG and BGGR integration sessions. APP works wonders still...cannot wait for the new version.

UPDATE and TEMPORARY SOLUTION

For anyone that comes upon this thread...

Problem still persists, and I must integrate two sets of subs, one BGGR and one GRBG and then integrate those two 'master' lights.

I now know that it also affects dark frames, since their duration is longer than a few second (several minutes sometimes)

Flats and darkflats (and biases too) are immune, the duration of the light frame is directly correlated somehow.

I came across this thread (in german) where there is a solution and a software called F4W2HDU that will batch rewrite FITS headers so I can process everything in a single step now. Credit to the people in that forum for this solution that I came across after another search after I discovered the problem exists with darks (grr)

https://www.astrotreff.de/forum/index.php?thread/254604-wechselndes-bayer-pattern-in-einer-sequenz/

To mods: I dont know if posting a link from another forum is allowed? If not, can I post a link to the useful software?

Clear skies all,

No go ahead, if that is helping with this specific instrument. Thanks for sharing!

Hi Colm @cwm2col,

I have fixed this issue now by adding support for the FITS keyword ROWORDER so if you load your SharpCap images in 1.083 stable you will not need to use FORCE CFA nor set the correct Bayer Pattern, APP will detect it correctly now.

Before the ROWORDER keyword was introduced we were able to correctly read the pattern from sharpcap images, because we knew they were using BOTTOM-UP until then.. Since the keyword was introduced however, they changed things leading to your issue.

I will be releasing 1.083 stable very soon !

Mabula

Thanks @mabula-admin.

That change/update will be very useful when we want to mix not only data form different scopes as APP can do, but from different cameras and different capture software without worry.

The issue I identified above turned out to be a driver problem, although the observations I listed were real. In the end, the oldest touptek driver (touptek being the parent company for the rebadged Es, rising star, omegon etc. cameras) fixed the problem and I could integrate all subs captured from then onwards using the force debayer and specifying BGGR. On the ASI camera with RGGB, I only use force debayer as it reads the rggb just fine when captures in sharpcap. With pixinsight, things can be different, so this ROWORDER implementation will be useful I think.

For anyone reading this and having the same RGGB vs BGGR issue, remove the driver that came with your camera (explore scientific ones specifically) and install the following touptek driver instead. the ASCOM toutpteksetup file, 3rd one down the list.

http://www.touptek.com/download/showdownload.php?lang=en&id=2

You are most welcome Colm @cwm2col,

Please note that Force Bayer is only needed when there is no Bayer Pattern anywhere in the FITS header. Your images did have the info in the header from both camera's.

The issue really was caused by SharpCap changing things internally. We already supported SharpCap's method of writing FITS, but when the ROWORDER was introduced in SharpCap they started writing FITS like we do... I guess because of that ROWORDER change. So our old work around for SharpCap was in fact no longer needed. Anyway that keyword should fix any issues with this going forward 🙂 !

Mabula

@cwm2col fair play & thanks man for all the work you've done in helping resolve this problem. I have the Explore Scientific 16MP and have had this issue for years and seems its an issue facing many other astrophotographers with a wide range of cameras with the same sensor. Will try this new ascom driver and hopefully will fix the issue too with my camera.