Problems with creating a mosaic 🙁

Okay I have a question
Why are the registered files all 3gb big when just 5% of the image is the actual image and the rest is black..?

When the images are not saved they are also not that big but it still works when you integrate the images right after registration without saving...

The main problem now is that even the normalization takes ages 😀 Hours of waiting just for 20 images to normalize. I cant beliebe how long the integration would take haha.

I think to register and save the images all at once is the wrong way... there has to be a way to speed up this intire process! 🙁

Maybe I will find a other solution myself, I will let you know!

@jkbsahner There is no compression in the form of "123456 black pixels" which gets stored as two values so this is stored as 123456 black pixel values. That grows to very large files quadartically if the dimensions of the image get larger.

@jkbsahner Well yes, by creating bigger panels and then integrating those, that should also be faster. Fact is that it's a huge mosaic, that is what is causing a bit of an issue regarding the amount of processing needed.

@vincent-mod Okay so I actually did another try by doing all at once and after 2 days of calculation stuff I got this!

It worked and I see no errors in the mosaic 🙂

Now its time to edit this beauty...

Well, I applaud your computer. 🙂 I do however see a tiny dark patch on the lower right. What kind of projection did you choose in the end?

@vincent-mod I used the equirectangular projection 🙂 The hole in the lower right part is okay, thats fixable in photoshop 😀

Ok, well very happy it worked out in the end! Using LP correction now in a careful manner will likely help a lot with it overall. Looks great!

@vincent-mod This is just a crop of the final image but it looks amazing in my opinion! 

Damn, that indeed is amazing! Glad it worked out so well!

Hi APP team,

I think that my question is good to this thread, especially that I am already correspond with Jakob. 
I already asked you Vincent(around 1,5 months ago) on other thread is it possible to stitch a few GigaPixel Pano and you said yes, but I need a really powerful computer. I think I found at least a good one to try: Mac Pro, 12cores/24 threads 3,3Ghz Intel Xeon, 256GB RAM, 1TB SSD(it could be more if I need it at integration, but its not a crucial parameter at the moment - Registration - I think). 

What I would like to do:
I have 282 already stacked and calibrated panels from my two cameras (Canon Ra and Canon R mod - the same sensor) taken on two 135mm canon lenses - same field of view. Its approximately 200h of data taken in May in Namibia. Its cover around 1/3 of Milky Way. The goal is to stitch it into one, few gigapixels pano.

I already made few tests with 20-25 panels and it worked really well. These tests I run on the same setting what I tried with 1 attempt, described below.

Now I am trying to stitch it everything together(282 panels into one pano).
But I encountered a few problems and I am looking for the possible reasons because every another attempt takes many hours, so it is better to do it consciously. Let me describe already done attempts( I will describe only a registration step because its my current problem):

1. Attempt
5000 stars, 
Pentagons, 
scale 1-10, 
disabled - "flip descriptors in x/y"
enabled - DDC
disabled - same camera and optics
registrartion mode - mosaic
registration model - calibrated projective (135mm, pixel size 5,36)

This attempt failed but I am not sure why and did the APP was a problem. Computer restarted during the night and on the morning I just found the starting desktop. So I suppose that I could be just an update of the system, not APP error. 

2. Attempt
5000 stars, 
Pentagons, 
scale 1-10, 
enabled - "flip descriptors in x/y"
enabled - DDC
disabled - same camera and optics
registrartion mode - mosaic
registration model - projective

This attempt after around 68h run into RMS calculations but couldn't lowered RMS below ~30(APP try to do iterations on RMS by around 24h), so I decided to cancel it by my self and try with quadrilaterals and calibrated projective after I spoke with Jakob. 

3. Attempt
5500 stars, 
quadrilaterals
scale 1-10, 
enabled - "flip descriptors in x/y"
enabled - DDC
disabled - same camera and optics
registrartion mode - mosaic
registration model - calibrated projective (135mm, pixel size 5,36)

After around 44h when APP calculated every "2-view... combination", APP shown the Error "Solution contains uncountable numbers"

Do you have any ideas what can caused this problem? Do you have any ideas about settings what I should use with so large data set? 

My conclusions are that I should try two thing:
1. Run once again the same setting what I used on 1 attempt, only with enabled "flip descriptors in x/y", and maybe try to increase the scale stop to 12. Probably setting were good but computer restarted from a different reason.
2. Run once again the same setting what I used on 3 attempt, but this time lowered the stars number to 4000 and increase the scale stop to 12-15

I will kindly appreciate all your ideas 🙂

Regards, 
Michał

Hi Michal,

Thanks for the explanation and huge tests. 😉 So, did you try to make panels first? When I do a mosaic I always start with;

- Scale start of 5, stop to 10

- Advanced normalization

- 2000 stars to analyze.

- Same camera/optics disabled and DDC on.

With a very big mosaic I always make panels first.

Hi Vincent, 

Thank you for so fast answer. 
Panels are ready and calibrated now everything what I am tying to do, it is on 32bits fits. But I did not tried to stitch for example a 47(282/6=47) panels into bigger panel and then make them into one final panorama, its my last strategy to try(6x47->6->1). Do you recommend to do it already, not try to stitch every 282 at once(282->1)?

So your idea is to lower the stars number and combinations needed?
Should I left it on quadrilaterals recognition?

I recommend not to do all at once no. Calibrated projective is a good idea, it may speed it up a bit. I always try to set a mosaic up like I mentioned and not change anything else. If that doesn't work, only then I'll try other strategies. Let's try do this step by step, if you try this first then post your progress and we can see what the next best action might be. I would maybe even try to make one half of the mosaic from panels (not all subs again) and then the other half...

Ok, thank you for your ideas. I will try to make it in few steps and I will be back with results 😊

Hi @Vincent, 
A few updates and new questions. 

First of all, after 2 weeks of really intensive I found working registration settings for all 282 panels at once. Right at the moment I am waiting for first final integration of my panorama. But I found a few follow questions listed by numbers and letters:

1A. I asked you here and in other mentioned thread about possible of stitching so big panorama(a few gigapixels panorama in resolution) and you said that it is possible, but I also know about the file size limit 715mpx for RGB image noticed in Quick Reference Guide. At the final step on Integration I got an notification that APP has to downscale an image to maximum 715mpx. What is a little troublesome because in this resolution I will lose a lot of details at this focal view or I have to go back and make 5 or 6 panels each for these 715mpx and later stitch them together in different panorama program but the then problem could be a normalization of panels. Also stitching in different program than APP could be problematic (I mostly think about not perfect connections), so if I already managed to register all panels at once It will be it will be very nice if I can do it at once in APP 🙂
1B. Does this limit is based on technical aspects of FITS file or Integration engine of APP which caused that limit can't be enabled?
1C. If it's not any technological limit is it possible to get an installation file of APP version with enabled limit. I really care about it an in Quick Reference Guide, Mabula noticed that it shouldn't be a thing in coming updates of APP.

During whole registration processes I did a lot of tests with different input sets, different parameters and I learned a lot about all the algorithms behind the registration engine. I also made these tests on different computers with different CPUs. At the end I noticed that registration algorithm canceled in most attempts at step RMS 11 and displayed error "MultipleViewCameraCalibrationSimpleDistortionCorrectionNotSameCameraIncomplete"...Solution contains uncountable numbers

2A. I also noticed that I can successfully end the registration in not every attempt (but I think its mostly caused that RANSAC is non deterministic and I had so big data set at input that my parameters sometimes were "countable" and sometimes they didn't depends on random set which was chosen by RANSAC - is it correct way of think? 

Is the RANSAC depends on number of available CPU threads? Because when I analyzed usage of CPU during the registration APP working on as many threads as I gave available to APP by the slider, but speed of achieving final result of registration in general don't change. I could even say that APP gives me final registration faster when I gave only 4-8 CPU threads available to the process. When I set for example 24 threads, whole process slow down and mostly fails. 

2B. If there is no optimized RANSAC multithread algorithm, so what calculated all these available threads of CPU?

I will be very thankful for answers. Mostly about my question of file size limit. It's very important for me and whole my project. 
When I will end it I will also share all my thoughts about taking panoramas like this because I think that I came to a few very important conclusions during this project. 

Regards, 
Michał

Great questions and you sure are testing the limits of any system. 😉 I'll ask Mabula to chime in..

Hi, 
Do you know something new about my questions?
I am mostly interested is the FITS file format does have this size limit?

Theoretically it shouldn't. I'll ping Mabula again to see if he can answer, sorry about the delay.

@mostaszewski FITS files get used by professional astronomers for LOTS of data. So the only limit to FITS file size is set by the operating system. Not sure if there is a limit to the size that APP can handle, again apart from the limits set by the operating system and the memory of the computer processing the data.

Posted by: @mostaszewski

Hi @Vincent, 
A few updates and new questions. 

First of all, after 2 weeks of really intensive I found working registration settings for all 282 panels at once. Right at the moment I am waiting for first final integration of my panorama. But I found a few follow questions listed by numbers and letters:

1A. I asked you here and in other mentioned thread about possible of stitching so big panorama(a few gigapixels panorama in resolution) and you said that it is possible, but I also know about the file size limit 715mpx for RGB image noticed in Quick Reference Guide. At the final step on Integration I got an notification that APP has to downscale an image to maximum 715mpx. What is a little troublesome because in this resolution I will lose a lot of details at this focal view or I have to go back and make 5 or 6 panels each for these 715mpx and later stitch them together in different panorama program but the then problem could be a normalization of panels. Also stitching in different program than APP could be problematic (I mostly think about not perfect connections), so if I already managed to register all panels at once It will be it will be very nice if I can do it at once in APP 🙂
1B. Does this limit is based on technical aspects of FITS file or Integration engine of APP which caused that limit can't be enabled?
1C. If it's not any technological limit is it possible to get an installation file of APP version with enabled limit. I really care about it an in Quick Reference Guide, Mabula noticed that it shouldn't be a thing in coming updates of APP.

During whole registration processes I did a lot of tests with different input sets, different parameters and I learned a lot about all the algorithms behind the registration engine. I also made these tests on different computers with different CPUs. At the end I noticed that registration algorithm canceled in most attempts at step RMS 11 and displayed error "MultipleViewCameraCalibrationSimpleDistortionCorrectionNotSameCameraIncomplete"...Solution contains uncountable numbers

2A. I also noticed that I can successfully end the registration in not every attempt (but I think its mostly caused that RANSAC is non deterministic and I had so big data set at input that my parameters sometimes were "countable" and sometimes they didn't depends on random set which was chosen by RANSAC - is it correct way of think? 

Is the RANSAC depends on number of available CPU threads? Because when I analyzed usage of CPU during the registration APP working on as many threads as I gave available to APP by the slider, but speed of achieving final result of registration in general don't change. I could even say that APP gives me final registration faster when I gave only 4-8 CPU threads available to the process. When I set for example 24 threads, whole process slow down and mostly fails. 

2B. If there is no optimized RANSAC multithread algorithm, so what calculated all these available threads of CPU?

I will be very thankful for answers. Mostly about my question of file size limit. It's very important for me and whole my project. 
When I will end it I will also share all my thoughts about taking panoramas like this because I think that I came to a few very important conclusions during this project. 

Regards, 
Michał

Hi @mostaszewski,

Congratulations on your work to create such a large mosaic of the sky!

Let me address your questions now...

1A: The pixelsize limit is there and is caused by the image size limit in Java that we currently use. We can't have more pixels than 2^(32-1) in an image. So for a monochrom image, you can have 2.1 Gigapixel, but for an RGB this is 2.1/3= 715MegaPixels. It is definitely on our ToDo list to make much larger images, but for now it is not yet possible. This is the reason that the mosaic needs to be downscaled.

1B: answered in 1A

1C: answered in 1A

2A: The incountable solution occurs when not all frames were successfully registered in the mosaic, a bug then actually occurs which is on the list to fix. Ransac needs to use a random number generator, so yes, that causes it to sometimes get a slightly different result.

Is the RANSAC depends on number of available CPU threads? Because when I analyzed usage of CPU during the registration APP working on as many threads as I gave available to APP by the slider, but speed of achieving final result of registration in general don't change. I could even say that APP gives me final registration faster when I gave only 4-8 CPU threads available to the process. When I set for example 24 threads, whole process slow down and mostly fails. 

Okay, that is a very interesting observation of which I am not aware myself. If it is like you say, then maybe there is a concurrency issue. I will investigate as soon as possible.

2B: At the moment, the current registration engine can perform multiple Ransac calculation for different images at the same time. But those Ransac calculations per image are single threaded. I just had a good look at our code, and I think I can easily make it multi-threaded which should really speed things up immediately. I have made a big note of this to implement quickly 😉

Thank you very much Michał for sharing your questions, experience and thoughts about this.

Mabula

Hi Michał @mostaszewski, @wvreeven, @vincent-mod,

Thank you very much for bringing the strange behaviour to my attention regarding the use of many cores and then slow down of the registration of your big mosaic...

I am very happy to say that I have located the problem that caused the slowdown and i have fixed it. Initial tests on small mosaics give a very big speed increase now! and you will see that the cpu usage stays much higher while performing the mosaic calculation 🙂

The performance increase will be included in the next version for sure.

Mabula