Mosaic Integration Issue

Is it correct that I see "no offset" in the Fits headers? You normally do need an offset when calibrating frames. It may be that something is going wrong because of variations in the background of the panels or even black background because of that. Can you check if precious results had no offset as well?

@vincent-mod

I just checked the fits header of the sub frames and there is an offset available! Same for my calibration files....but that's not that kind of offset you mean or?

In all of the files produced by APP, the offset is always zero...I have checked this on some good pictures!

Ah ok, got it. So then it might be that the frames, even though correcting nicely on their own.,still show a different background signal to each other. Does it help when you use light pollution correction on the mosaic? This is normally a good idea as well. I do agree the differences look a bit extreme here.

No, I tried several times to use the light pollution correction with different settings for the LNC, without success!

But on my last attempt, I turned off the LNC, just out of frustration, and it worked... 🤣 

Will now use the LPC to do a final adjustment and I really hope that the SCC will work properly on that mosaic now....

PS: I saved all FITS before as a 16-bit integer FITS

You now what, that is actually a nice find. I'll ask Mabula if this is normal behaviour for APP as well and what might make it so extreme. Regarding bit-depth, especially for calibration data, 32-bit is best to use.

Ok, got answer from Mabula; the problem lies in the fact you used 2nd degree LNC. For mosaics, 1st degree is better for the panels and mosaic (algorithm wizardry that I also don't understand completely yet). 🙂

@vincent-mod

Ok, thanks for the fast reply! I will try to use the 1st degree LNC now to combine my panels to a mosaic....will also use the 32-bit FITS Files for the integration!

Will share the result with you asap 🙂 

@vincent-mod

Hi Vincent, here is the result of the 1st degree LNC intergration. I started with the standard 3 iterations, but the result was not really good, so I increased the number of iterations to 10...

Not really that what I expected...should I send you the four panels for some tests? (Fits, 32bit, LPC, CBG)

Mmmm, yeah not the result I was hoping for. Yes, you can share your data and I will have a look (bed time now and a busy week ahead, so please allow for a few days);

Go to https://upload.astropixelprocessor.com and for the username and password, use: upload

Create a directory named “gogonfa79-mosaicIssue” and upload in there. Thank you!

@vincent-modHi Vincent, any news regarding the mosaic issue? Let me know if you found a time slot to make some tests...

Sorry Michael, I missed that you uploaded it. I'll have to ask to notify me when uploaded in the future, apologies!

I'll work on it either this evening or tomorrow.

So yes, I checked the files as well and have the same issue. I still think something is wrong in the background, could you upload a few single subs and your master calibration files as well? Thanks!

I wonder if a preview option were possible to check the differences in the Mosaic settings and see which work. I have set of 100+ images to integrate from four sessions. If the program used a subset of the Lights from each session you could more quickly test what worked best. I'm using an iMac Pro and the processing for the last integration takes many hours. 

@vincent-mod

hello Vincent, sorry for my last reply! Will upload my master frames and some lights for each panel tomorrow!

In the meantime I had the chance to test PixInsight, and this tool did not had comparable problems with the mosaic…the result is also not perfect due to some registration artifacts (APP is the best for such things 😉) but it seems that the background normalization is much better… 😅

I used the same files for the mosaic as uploaded (the single stacks made with APP)

Nice result! Yes, I think you used some of the HDR capabilities in PI. This is not implemented as such yet in APP. So for those specific cases it can definitely help. Please notify me when your upload is finished. 🙂

@vincent-mod I uploaded all files, really curious to know what went wrong with my image processing!

regarding the Mosaic with PixInsight, yes I used as a last step the HDR capabilities…but the mosaic and background normalization worked very well and I didn’t had comparable issues with PI

I'll have a look at the data today. Thanks for uploading more frames!

One thing I notice is that your flats are taken at 0.1 seconds and such. They don't look wrong to me, but it's usually advised to aim for 1-2. I used all frames and put them into their own session, with flats for that session applied.

At each subsequent step things seemed normal and I integrated all frames into a mosaic and indeed, LNC seems to still go wrong here. So next up is me trying without any calibration file, just to see if anything there might cause it.

Ok and now without calibration and the effect is the same! How odd, I think there must be something in the lights. I'll notify Mabula about it.

@vincent-mod

Hello Vincent, that is exactly what I have seen during my image processing of this specific session. During the Mosaic integration (based on a previous integrated stack for each panel) something seems to be going wrong in neutralize background function or in the LNC.

I do not think that my calibration frames are the reason for that behavior, I used APP for other Mosaics without any problems! There is only one thing, which is different to all my previous work! It is the large dynamic range of the single panels, which is also not the same for all frames! In the center the galaxy is quite bright and towards the edge, the brightness decreases strongly (this counts for the images, diagonally above and below M31)! In addition to that, the images vary alternately in the brightness strongly! While the other two images (along the galaxy axis) are much brighter, it could be maybe difficult to neutralize and adapt the background with the LNC algorithm correctly in this scenario. 

Regarding my Flats:

In general, I use a dedicated cooled astro camera (the QHY268C) for all my images. I make my flats automatically with N.I.N.A (Nighttime Imaging 'N' Astronomy). This tool has a flat wizard that allows you to specify how the flats should be made. I always specify a histogram mean target of 32768 (for my 16-bit camera) with a mean tolerance of 5%. So, I never had any problems with my flats! But I know, that some CMOS cameras do have a strange behavior when it comes to short exposure times and the sensor is not working in a linear mode! I know that my camera works fine above 0.1s (Sensor linearity test in Sharp Cap) but for the future, I will take care of longer exposures around 1-2s. If the flat correction is wrong, you would see it in an under- or over-correction in the stacked single frames.

Let me know if you need more pictures, I really love to work with APP and I support you as good I can to make that tool better 🙂

@vincent-mod

what was the problem in this thread https://www.astropixelprocessor.com/community/main-forum/normalization-failure-in-mosaic-mode/#post-13298

Posted by: @gogonfa79

@vincent-mod

what was the problem in this thread https://www.astropixelprocessor.com/community/main-forum/normalization-failure-in-mosaic-mode/#post-13298

 

No idea as of yet, I asked Mabula to have a look at it as I've not seen such an issue before. I'm suspecting something in the lights or a weird combo of data and the LNC that goes wrong.

edit: Doh, scratch that. 🙂 That does look similar, but he never answered I believe. So your data can be very helpful, it doesn't happen often so it is interesting to see.

Hi Michael @gogonfa79,

Thank you very much for sharing your issue and your data. I will have a good look myself this evening and will let you know my findings later today.

I suspect the issue could be that you apply Light Pollution Correction on the individual mosaic frames. It seems logical to do so, but in fact it is not the best workflow in my experience, because it will only work if you do Light Pollution Correction perfectly. And on your High Dynamic Range object M31, chances are big you did not do it perfectly end thereby introduced a complication for the LNC algorithm in the mosaic integration proces. Anyway, only speculating now... I will try to see if I can get a better result and give you an explanation why it did not work well for you in APP.

Mabula

Hi Michael @gogonfa79,

The best result I get is with using 5) advanced normalization, 6) MBB set to 10% and no LNC at all...

If you turn on LNC on your mosaic, things start to look bad indeed. The reason is the little amount of overlap that you have between the frames... The current LNC algorithms work very well if you would use more overlap when you plan a mosaic. Your mosaic only has 10% overlap and this is just very little in general. If you would use 15-20% things should work perfectly in my experience. Of course, in due time we will release an updated LNC algorithm that should work also with so little overlap.

The fact that M31 is an object with a high dynamic range is clearly a bit too difficult for LNC with so little overlap. On objects with smaller dynamic range, it would still work normally though.

Mabula

@mabula-admin

Hello Mabula,

thanks for your time and your fast reply! I understood, that it is much better to have a more overlap between the individual frames in a mosaic. Does it mean that all frames have to overlap with at least >15-20% or would it also helpful to add additional data e.g. an image frame where the M31 core is centered in the middle (like that picture below)? 

 Thanks 🙂

Michael

Yes that should work I think, then the mosaic has plenty of overlap. I hope this is the issue, nevertheless it's going to be improved upon so your data is definitely helping us there. Thanks!

Hi @gogonfa79,

Indeed, adding a mosaic panel in the center would definitely help a lot for sure !