[Sticky] Be aware of capture programs and what they write to the FITS headers

Update:

I tired unchecking multi-filter and multi-session and loaded all the Ha subs as a single session. This time normalization got to 54% before giving the same error as before but this time with no cause.

Strange. The fact you have pre-calibrated data is interesting though, I'm wondering if one of those is causing an issue for some reason. Could you check if it works with combining just 2 (in the hope you miss the possible wrong one)?

I tried combining 2 subs (one from each set) and got the same issue. I just noticed that one set is showing a bayer pattern in the CFA column while the other isnt. This is weird because both sets are from a mono camera. Could this have something to do with it?

Might be, as the crash happens in the normalization this usually is the case when combining broadband data with mono. This may also have been done by the one who precalibrated the data, can you ask him for his un-calibrated mono and his calibration frames so you can test with at least the exact same settings?

@vincent-mod

So I use Astrophotography Tool to capture my data. There is a setting that debayers the captured fits files so that you can see OSC subs in color on the preview screen. Turns out by leaving that setting on while capturing mono data, it appears to add the Bayer matrix in anyway.

I tested this by plugging in my Asi183mm and taking single sub with Bayer preview enabled. I loaded that sub into APP and it shows RGGB under the CFA tab no matter what setting I changed in tab 0. I deselected Bayer preview in APP and took another sub, loaded it into APP, no CFA reports "None". So I think APP was seeing my subs as color even though they weren't because of the ATP setting. I won't be able to verify this with actual data for a few more nights when I get clear skies but I'm pretty sure that was the problem. 

Ah yes, that might be the issue then. APP does look at the FITS headers to see what type of data was taken. Nice one you thought of that!

That stinks because that means 11 hours of data is now unusable 😩. It's OK though I'm just glad to have figured it out, we've been working on figuring this out all week. I didn't think that ATP setting mattered because it's called "Bayer preview" implying it only effects the preview and not the actual fits file. Lesson learned. When I verify this with actual data I'll report back, maybe this information can be stickied or something so that other ATP users are aware. 

Well, maaaaaaybe there is a way to remove that pattern from the headers? Never done that myself, but there must be something to do that... I asked Mabula.

Yeah, my next step is to see if I can somehow wipe the Bayer pattern off the files. I tried doing an extract Ha on them in APP but that didn't work 

Just made you a sticky, I think it's a good general warning.

Ok, so talked with Mabula. Both him and me never saw this before, so there's no super easy solution yet (unless you can find a meta data editor). He's thinking about making this possible in the batch modify tool, so don't throw away the data and don't recapture just yet (unless you want even more data). In batch modify there is a way to edit the header, but not to switch off a bayer pattern, it's likely never needed but will solve your issue. He added it to his to-do list.

Oh and he offered to fix this in between for you to speed things up, so if you want you can upload the data to our server, he'll fix it and you can download it again.

Go to https://upload.astropixelprocessor.com and use upload1 (or upload2 till 5) as username and password will then be the same as the username.

Create a directory named “regpratt-bayerpatterninmono” and upload in there. Thank you!

@vincent-mod

I really appreciate the help. The files are uploading now but its 9gb of data so its going to take a few hours 🤣 

Oh no worries. 🙂 So there is maybe another way, you can also select "no interpolation" as option. That might work as well, can you try?

I tried that last night and it didn't work. I'm actually going to cancel the upload when I get home because I found an Windows application that allows you to strip the Bayer info out of the header. I tested it on a single sub and after loading it in APP it was no longer showing anything under CFA so worse case scenario that will work I just have to change them one by one. 

Ok we can do that in 1 go if you want, but whatever you find easiest.

Did you reload the data after you changed to the "no interpolation" setting? I'm not talking about the "force cfa" option, to be clear.

Posted by: @vincent-mod

Ok we can do that in 1 go if you want, but whatever you find easiest.

Did you reload the data after you changed to the "no interpolation" setting? I'm not talking about the "force cfa" option, to be clear.

That would be great, I just got home and noticed the upload was done anyway. Thanks!

Greetings all. I stumbled upon this same error. In my case, I have data that was captured with an OSC camera, 1600MC, and a Luminance filter on a 1600MM at a later time.  I load each of these into a session and get to the normalization and then get the exact error above.

I have multi-session and multi-filter checked, and all the defaults on the normalize tab. On trying to re-run it, I can also get this error:

Encountered error in module:
OverlapBetweenTwoImageObjectsCreatorWorker

Error message:
java.lang.ArrayIndexOutOfBoundsException

Cause:
null
no trace

I did have same camera and optics selected though, so I cleared this and ran it again but had the same error.

@xsnrg

You need to convert the 1600MC data to mono before combining with the 1600mm data. This error happens when you try and combine color and mono data. Use extract Ha or extract Ha-Oiii mono if you want lum then combine that with the 1600mm data. 

I was hoping to add the MC data as color to the MM data for Lum. Perhaps this is not a thing though.

@xsnrg

So to get around the issue of not being able to register OSC and mono data together, you can load the OSC data, split it into RGB data which would give you 3 sets of data (red, blue, and green) those 3 sets of data would be mono.

Take one set, and register the 1600mm data to the red channel set of the OSC data. Now you'll have R G B and L with the same alignment.

How you proceed from there depends on how you want the process the data. You can combine it all as LRGB in APP, Pi, Ps or whatever you use for post processing. 

Thanks Reginald, that is the path I was going down next.

Jim