[Solved] Integration issues

So would you want to share your workflow, settings and maybe some of the panels to investigate? We ofcourse are very keen to investigate.

Hi Chris @chrispeace & @vincent-mod,

Results like this normally only happen when you enable distortion correction on data where it is not needed. Have you tried to combine the data without distortion correction?

I think I need to have a look at your data and also your workflow to be able to understand why you are getting results like this.

Can you share the data? Can you upload it to our server?

https://ariesprodstor.astropixelprocessor.com:7001/

login and password : appuser

Please make a folder with your name and let me know once uploaded, I will have a look as soon as possible.

Cheers,

Mabula

@mabula-admin

Mabula, I will take you up on your offer to look at my data. Most of it worked eventually burt we are talking hours of slog here not a couple of attempts. It took many hours and I wasn't really happy with the quality of the output regarding light pollution correction. I tried doing LP correction  both before integration as a mosaic and afterwards but importing files that had been 'worked' in a previous integration of say a single image for a panel seemed to fail the registration or integration more often. 

I only ever used dynamic distortion correction as advised by your tutorial to stitch the mosaic panels together. Nowhere did I find any reference to not using it and indeed when I failed to select it by mistake the integration went wrong.

I would typically but not always, have one stacked image (one corner) out of four making up one colour panel distort. Other times I'd just get four rectangles or triangles as registered images.

I also had star analysis issues where APP would only detect 180 stars in a colour stacked image that had been picking up 1700 the attempt before. Clearly, it failed registration on low star count. Typically I was selecting 500 for i=mage stacking and 2000 for mosaic stitching. 

I tried too many combinations to list here but if you can think of it, I bet I tried it. I've been at it for three days!

Finally, I'd still like an explanation as to why its necessary to show animal images when a guy's hard work goes wrong and anticipation is high. Nobody is in the mood for a bit of coder fun at that point believe me. 

Chris

Hi Chris,

Thank you for uploading your data. I will have a look at it today and will let you know my findings 😉

Finally, I'd still like an explanation as to why its necessary to show animal images when a guy's hard work goes wrong and anticipation is high. Nobody is in the mood for a bit of coder fun at that point believe me.

In this case, I can assure you that there is no coder fun involved here... it's a bit hard for me to understand why you would think that would be the case. The weird shapes that you get are in no way the result of me adding strange behaviour when something fails. I don't think that is funny at all. The weird shapes are the result of the distortion correction algorithm going crazy for some reason and it's a mathematical problem on which I am working to get fixed 😉 Probably hard to imagine, but those weird shapes are very distorted patterns that are mathematically constructed with the same formula and parameters as correct optical distortion patterns like pincushion/barrel distortion.

Anyway, please know that I will do whatever I can to help you with your data, I will let you know what I see in your data and if we can solve your issue here. Maybe APP is failing here for some reason and that could then help me to further improve APP's mosaic capability.

Kind regards,

Mabula

@mabula-admin

Extremely appreciative of your help Mabula and thanks for your reply. I guess it's similar to why only religious people see images of the Virgin Mary in burnt toast!  Or astronomers see a baby in the Soul Nebula. 

Chris

Hi Chris @chrispeace,

I am working on your data now. Is it correct that you only uploaded luminance light frames? And that the frames shot in 2020 don't have flats provided?

Cheers,

Mabula

Hi Chris @chrispeace,

The luminance mosiac is created without any issues. So i can only assume that the error is somehow in your workflow?

This is the mosaic without MultiBand Blending (MBB) and Local Normalization Correction (LNC) enabled:

I will now create the mosaic panels with both MBB and LNC enabled and then a mosaic with MBB and LNC enabled...

Mabula

@chrispeace,

To create the individual mosaic panels:

1)Load all the light frames of that panel (per channel) and add all relevant calibration frames.

2) make the calibration masters if still needed, otherwise load the masters in 1)Load

3) keep max star count at 500

4) keep scale stop at 5, enable dynamic distortion correct and enable same camera and optics, registration mode normal

5) defaults

6) enable MBB to 5% and LNC first degree 3x, rest of settings at defaults

This is panel 1 then:

Will post panels 2,3,4 in next post...

Maybe your problem occured because of

1. settings star count very high (more than 1000) on the individual panels
2. enable distortion correction while
3. disabling same camera and optics (that could trigger the mathematical problem, because there is hardly any distortion)

Kind regards,

Mabula

Hi @chrispeace,

panels

2,3 and 4 look like this with MBB 10% and LNC1st 3x

 Now I will create the mosaic, again with MBB and LNC...

@mabula-admin

No,  I zipped and uploaded LRGB with associated Bias, Flats and Darks. The Lum frames all had the same Flats. The RGB all had their own Flats shot this year. I'll check what has happened during upload. I had the most trouble with Green 

@chrispeace

Creating the mosaic:

1) load the mosaic panels that you have created from the individual frames, I am loading now luminance panel 1 to 4

2) not applicable

3) increase star count to 2500

4) scale stop 10, enable dynamic distortion correction, disable same camera and optics now ! , mosaic registration mode

5) set to advanced normalization mode

6) again, MBB 10% & LNC 1st 3x

This is the result:

It is clear that we need to perform light pollution removal to be able to see the IFN now (will try to this as well)... but the mosaic has been created with a precision of 0,12 pixels using 1131 starpairs in the overlapping areas 😉

The same will hold for the other filters, R,G,B

If you create the mosaic, you will need to load the 4 panels in L,R,G,B, giving you 16 panels, right? Make sure that you load them while assigning them to the correct filters.

Then you will end up with 4 mosaics for L,R,G and B, which you can then combine to a composite with the RGB Combine Tool.

Let me know if this is clear 😉

Perhaps you can indicate where your workflow was different to get such odd results?

Kind regards,

Mabula

@mabula-admin

I cannot log back in using the details you previously sent me so I cannot check what has uploaded and cannot upload any more. Are you saying all you have are Lum files and no RGB? 

Hi @chrispeace,

Maybe somebody else was logged in with the public account. You should be able to login now though 😉

Yes, there is only luminance light data at the moment...

This is the mosaic with Light Pollution Correction. With careful corrections, the result will be awesome I think 😉

Mabula

@mabula-admin

Have you got the green data and associated calibration files? These just would not respond to the work flow you have just outlined.  I got the same results as you have using the exact same workflow for making the individual panels except for where you say above " 4) keep scale stop at 5, enable dynamic distortion correct and enable same camera and optics, registration mode normal". here I did not select dynamic distortion correction when making individual panels - only selected DDC when making the four panel mosaic. 

Chris

@mabula-admin

Have you got the green data and associated calibration files? These just would not respond to the work flow you have just outlined.  I got the same results as you have using the exact same workflow for making the individual panels except for where you say above " 4) keep scale stop at 5, enable dynamic distortion correct and enable same camera and optics, registration mode normal". here I did not select dynamic distortion correction when making individual panels - only selected DDC when making the four panel mosaic. 

Just thinking about what you also said here: 

Maybe your problem occured because of

1. settings star count very high (more than 1000) on the individual panels
2. enable distortion correction while
3. disabling same camera and optics (that could trigger the mathematical problem, because there is hardly any distortion)

So 2 and 3 are interesting as their use seems to be linked.  I thought you never had both set at the same time.  I never at any stage had both DDC and same Camera same optics selected together. But I did manage to make panels and get a Lum mosaic as you have done.

Regarding only these two settings, to be clear my workflow was to use same camera same optics for panel integration only,  and then only DDC for Mosaic integration. Is this not correct?

If you don't have the other data I'll keep trying to log in and send only the green data as this is what gave me animals etc.

Chris

You should have the RGB now. Focus please on the Green if you are limited on time.

Chris

I have just followed your guidance to the letter whilst stacking the individual green panels and then making the mosaic. This was the result complete with distortions apparent in all panels and particularly nasty in the bottom right hand corner. Honestly, I am not messing about here. 

Chris

 Note arrow in the top RH corner, the swirl in the bottom RH corner and various edge distortions on the other two panels.

Just re-stacked the whole of the green data files into fresh panels once again following your guidance to the letter and then here in mosaic mode it's at it again. APP cannot get past analyse stars despite having 2500 star set. Panel three this time won't play ball, yet the same data let me get all the way to Integrate as above earlier but had distortions upon completion.

The second image above is panel 3. it looks okay to me with no obvious issues visually. The Panel stack went without hitch.

Chris

I do notice that refence frame has a far lower quality score than the others. Mmm, I'm also downloading your data to have a better look. Will let you know later, seems there is something going on still with this specific data.

@vincent-mod

Yes Vincent, The ref frame this time has a very low star count. That same frame has produced much more compatible counts (in the region of 1400) in previous attempts especially if I force a change in the Ref Frame. I don't know enough to understand why this would vary but appreciate your help especially on a Sunday!

Chris

No problem, to me every day is similar to the previous one. 😉 I'm downloading still, I likely will struggle with it as well, but maybe I can find a better workflow somehow.

@vincent-mod

Well you understand the heartbeat of APP better that I.

Maybe, but I also do it by experimenting a lot and you already did that. So, not keeping my hopes up high, but it'll for sure be great to use for improvement.

Ok, everything downloaded. Massive data project, so I'll have a go at it a bit later today or tomorrow if you don't mind. 😉

For more details on one way of tackling this, see https://www.astropixelprocessor.com/community/tutorials-workflows/green-panels-wont-respond-to-the-mosaic-workflow/

Yes, I've made that into a sticky already. Thanks for that!