Error when trying to integrate a mosaic (downscaling by factor 0)

I've been able to overcome the issue by choosing Regular instead of Advanced normalization mode.

However, later another error occurs: in the Lanzcos module, independently of the type 3, 4 or 5 or even other methods selected.

I managed to overcome that error by changing integrate mode from Interpolation to Drizzle (Just to remind that I'm not really integrating, not drizzling - I'm stitching a mosaic from already processed panels, but well...it worked this way).

But now a new error pops out, when it tries saving.

It just says Found exception while saving the file...which is ridiculous, because I already managed to get finished and saved the result within the same session, when experimenting tweaking options and successfully stitched and saved two attempts in which I unselected several panels of the mosaic, just to check a lighter version...

I understand that my mosaic may be somewhat challenging, because of different optics, but this last error seems to be completely unrelated to that.

This quantity of errors feels frustrating tbh...

Clear skies

Adding the log info, from the moment of succsessfull integration and trying to save:

--------------------

10:53:52 - 6) INTEGRATE: integrate light frames: integration task has completed
10:53:52 - 6) INTEGRATE: integrate light frames: integration task finished
10:53:53 - DATA ANALYSER TOOLS: re-instantiated multi-core analytical memory blocks, size 511 MBs
10:54:00 - GENERAL FRAME SAVER: file name: Mosaic_jun4-RGB-session_1
10:54:00 - GENERAL FRAME SAVER: directory: C:\APP
10:54:00 - GENERAL FRAME SAVER: file type: FITS
10:54:00 - GENERAL FRAME SAVER: bit depth: no Bitdepth Conversion
10:54:00 - GENERAL FRAME SAVER: DPI (Dots Per Inch): 72
10:54:00 - GENERAL FRAME SAVER: ICC profiles: no profile/linear data
10:54:00 - GENERAL FRAME SAVER: C:\APP\Mosaic_jun4-RGB-session_1.fits : starting...
10:54:00 - GENERAL FRAME SAVER: performing bitdepth conversion if required...
10:54:00 - GENERAL FRAME SAVER: C:\APP\Mosaic_jun4-RGB-session_1.fits : starting FITS FRAME SAVER
10:54:00 - FITS FRAME SAVER: C:\APP\Mosaic_jun4-RGB-session_1.fits : starting...
10:54:00 - FITS FRAME SAVER: C:\APP\Mosaic_jun4-RGB-session_1.fits : constructing 32-bits FLOAT databuffer..
10:54:01 - FITS FRAME SAVER: C:\APP\Mosaic_jun4-RGB-session_1.fits : creating FITS HEADER...
11:03:58 - FITS FRAME SAVER: C:\APP\Mosaic_jun4-RGB-session_1.fits : was cancelled
11:03:58 - GENERAL FRAME SAVER: null : was cancelled

Posted by: @vorobiev

↑

Hello there!

I've been trying to integrate a mosaic which I've been growing adding new panels. 

In this mosaic I combine different optics and cameras, so I know it's a challenging task, but it was integrating successfully in the past versions (after tinckering with many settings).

Then recently I added a few new panels, which look very similar in quality and have overlap (which is proven, I guess, by the fact that Registration completes successfully with all the new panels). 

But then when I try to Integrate, this error pops:

-------------------------

The image will be downscaled

with a factor of 0.00000

Since the amount of pixels would otherwise be more than

715.828 MegaPixels

The current technical limit of the image dimensions has been reached

--------------------------

 

All the images are already stretched versions in 16 bit TIFF format, which I know is not the best format, but again: previous integrations were succsessful and the registration with the new ones is also succsessfull.

 

Could you please point me in the right direction to solve this?

 

Thank you and clear skies,

Pavel

 

 

Hi Pavel,

Thank you very much for reporting this issue.

The warning message says downscale with a factor of 0.0000, indicates to me that a serious registration error does occur, the image seems to blow up to infinity... What do you see in the registration RMS column once the registration finishes? What is reported for the RMS error?

Are you using a calibrated projective registration model? If not, try that and set the projection to anything other than rectilinear and try again.

To be clear, 16bit TIFF stretched data will have physically uncorrect (because intensity stretched) star profiles so calculating accurate star centroids will be less precise when compared to linear data. But, in most cases it should still work fine for registration I would think.

Mabula

Posted by: @vorobiev

↑

I've been able to overcome the issue by choosing Regular instead of Advanced normalization mode.

However, later another error occurs: in the Lanzcos module, independently of the type 3, 4 or 5 or even other methods selected.

Hi Pavel @vorobiev,

It clearly does not solve this issue then. All subsequent errors must be related to the initial issue with registering your data properly. We first must focus on getting all panels properly registered I would think, because at the moment a downscale of 0.000x will give nothing meaningfull, it is a mathematical blackhole.

I see from your screenshot that you have star analysis set to detect a very high amount of stars. That usually will create issues/instabilities when there are indeed stars everywhere in your images.,. Better to set star analyis to detect 2000-3000 stars, did you try that? What happens then?

Mabula

@mabula-admin 

Hi, Mabula,

Thanx for your answers. I will try lowering the star numbers (I was bumping it up thinking it would help registering, until I discovered that "regular instead of advanced" trick) and trying Calibrated as you suggest.

How would I know if the panels are properly registered?

I mean, in my described process, the registering works..I mean it finishes normally, if I run registration only.

Where errors seem to start appearing is with subsequent steps: normalization or integration (composition in this case).

So, from a finished registration process how would I know if it is a proper result or a bad one to continue with?

Meanwhile answering your question:

RMS - # stars column report the value of: 312.17 - 9149

About the suggestion of using Calibrated Projection:

is asks for the initial parameters of focal length and pixel size.

As I mentiond, for this project I combine several optics and cameras.

So could I still use Calibrated? What starting parameters should I give it then? Like, the shortest of longest focal and smallest or biggest pixel?

My gear mix in this mosaic is:

Samyang 135mm with asi2400mc (5.9um)

Samyang 135mm with Nikon D750 (5.9um)

Nikkor 50mm with Nikon D5300 (3.91um)

Takahashi fs60 @ 255mm with qhy268m (3.76um)

This is probably some crazy mix, and before knowing APP I uldn't even try to blend this, but what motivates me to continue trying is precisely the amazing capability of your software and the successfull previous integration of slightly fewer panels, keeping pretty much the same crazy mix 🙂

Regards,

Pavel

Hello,

I'd like to report that I patiently went to save the unstretched version of every panel and with those files the whole process worked without any errors.

So, I guess the matter is solved, while the fact that some portion of the mosaics was integrating well with stretched tiffs was probably a miracle 🙂

Cheers,

Pavel

Hi Pavel, @vorobiev,

Thank you very much for all the feedback, sorry for my late response but I will try to answer everything now.

I'd like to report that I patiently went to save the unstretched version of every panel and with those files the whole process worked without any errors.

So, I guess the matter is solved, while the fact that some portion of the mosaics was integrating well with stretched tiffs was probably a miracle 🙂

I am glad to read that all worked fine with the unstretched data, it really shows that stretched data is much harder to analyse properly. The linearity is gone and many algorithms assume that the data is linear. This concerns analysing stars, but it also affects in a significant way the normalization process, correcting the images for background levels and dispersion/noise/gradients.

How would I know if the panels are properly registered?

I mean, in my described process, the registering works..I mean it finishes normally, if I run registration only.

Where errors seem to start appearing is with subsequent steps: normalization or integration (composition in this case).

So, from a finished registration process how would I know if it is a proper result or a bad one to continue with?

 

Meanwhile answering your question:

RMS - # stars column report the value of: 312.17 - 9149

Frames are properly registered if you see that in the RMS= # stars column, the first value, the RMS error value is below 1. It means that all stars that are found and used in the registration process are aligned with a precision of less than 1 pixel. The 2nd number in the column represents the number of star pairs used for the registration between the frame and the reference. So a RMS - # stars value of 312.16 -9149 means that 9149 star pairs are matched with an alignment error of 312 pixels... that means it totally failed... the value needs to be below 1.

About the suggestion of using Calibrated Projection:

is asks for the initial parameters of focal length and pixel size.

As I mentiond, for this project I combine several optics and cameras.

So could I still use Calibrated? What starting parameters should I give it then? Like, the shortest of longest focal and smallest or biggest pixel?

 

My gear mix in this mosaic is:

Samyang 135mm with asi2400mc (5.9um)

Samyang 135mm with Nikon D750 (5.9um)

Nikkor 50mm with Nikon D5300 (3.91um)

Takahashi fs60 @ 255mm with qhy268m (3.76um)

 

This is probably some crazy mix, and before knowing APP I uldn't even try to blend this, but what motivates me to continue trying is precisely the amazing capability of your software and the successfull previous integration of slightly fewer panels, keeping pretty much the same crazy mix 🙂

APP should be able to process that. The focal length and pixel size asked are starting points in the whole calculation. So it really helps to get the result faster and more stable. In your case, since it is a mosaic, you can actually try to use a mean/average value So if you use focal length = 125-150 it should be fine for the pixel size, set it at 4.5-5 I think. If the data is good and if the images an be aligned without much trouble, then the calculation will quickly converge to a robust solution no matter the exact input of focal length and pixel size, as long as they are close and realistic.

I am actually working now intensively on the registration engine, particularly the mosaic algorithms. I have removed many bugs and inefficiencies in the past couple of days, especially for the calibrated projective model. So 2.0.0-beta39 should have this all quite a bit improved 😉

https://www.astropixelprocessor.com/community/release-information/astro-pixel-processor-2-0-0-beta39-release-notes-work-in-progress/

• IMPROVED & FIXED 4)REGISTER, calibrated projective model for mosaics, much more robust now !

  We have fixed several bugs that could manifest in mosaic registration mode with the calibrated projective registration model. The first bug that is fixed would throw and ArrayIndexOutOfBounds exception in module MultipleViewFullCameraCalibrationSimpleDistortionCorrectionNotSameCameraIncomplete. We fixed an error in MultipleViewDistortionAndCalibrationTask that throws a NullPointerException: Cannot load from object array because "simpleResults" is null error. We fixed an error where APP would say that we have infinite solution because a matrix could not be inverted. We fixed another error in MultipleViewDistortionAndCalibrationTask that would also throw a NullPointerException: Cannot load from object array because "camerasSim[i]" is null. All these errors could manifest in different occasions even when all frames could be registered in 2-view. By fixing all these bugs, the calibrated projective model for mosaic will process much more datasets successfully where previously it would break down. An example is this 18 panel mosaic shot with a 24mm lens with horizon and milky way visible and with a very large field of view. This mosaic would not work at all with beta38 or earlier. It now works flawlessly with standard scale stop 10 and 2000 stars, both with same camera and optics on or off. Data is courtesy of Bikram Ghosh @bikramghosh which helped us fixing/improving this by providing his data. The left result is a full processing of the 18 panel wide field mosaic in APP by Mabula.

Mabula

That's great news, Mabula. I was experimenting with different panels combinations and the actual version of APP leaded to similar errors again, even with unstretched panels. I was going to report that, but then your answer came out, so I better wait for the new version and try again.

By the way, that you for explaining how to interpret the RMS column!

Greetings,

Pavel

You are most welcome Pavel,

Please checkout the work in progress for beta39 in the release notes, more improvements are coming... I have fixed many flaws and implemented several good improvements for mosaics.

https://www.astropixelprocessor.com/community/release-information/astro-pixel-processor-2-0-0-beta39-release-notes-work-in-progress/#post-33139

Mabula

Hi, Mabula,

I am growing the mosaic previously mentioned in this thread and now new errors arrive 🙂

As you can see on the provided screenshot the registering went great (0.52 error RMS). 

Normalization also went smoothly, but when I integrate I get this error.

I've tried changing several integration options but it remains.

Any ideas of what I should do?

I'm using beta38 version now.

Thanx in advance and greetings,

Pavel