How much memory for mosaics?

I would recommend doing each panel individualy with Master calibration frames and than combine all ready panles into a mosaic.

I tried to "dump-it-all-at-once" method, which I thought I was happy with...
A couple of days ago I did it the "proper" way, like in the milky way 7 pt. tutorial, and not only did work like a charm, the output was a much better registered mosaic, and even a lot more usable space for the crop later... So bigger, better file.

Haven't done 100 panes, only max 9 (also modified DSLR), and all works great (running on Z440 workstation with XEON cpu, 32gb RAM etc...)

Yes, using all frames in 1 go is not recommended, this will make the statistics really complex and really huge, which is why the memory runs out. It's best to create panels first from a smaller data set and then create the mosaic in the end with those panels.

OK, I will take your word for it and try it that way.

It certainly does not have to be that way.   I guess I had assumed that APP was smarter than it is.

I had assumed that the "star analysis" step was plate solving.  The first blind plate solve might take a little bit of time, but plate solvers in other contexts are very fast - a few seconds generally.

After the first plate solve, the incremental ones are not blind and ought to be much faster, because it is very likely to be adjacent.  

Once you have plate solved each shot, assembly into stacks should be very fast.  You probably still want to run comparisons to fine tune the placement and run a distortion model.  But you do not need to do very many comparisons.  

Is there a reason this would not work?

This also begs the question - what does the star analysis step actually do?   

It takes a long time, and apart from reporting star sizes it doesn't seem that critical to the workflow.   Is it optional?   

One possible use for star analysis is to pick the best frames.   I tried to find a way to use the star analysis to omit bad frames but I did not see a way to do that.  Is that possible?

Of course I recognize there might be more to this, just eager to learn.

One more thing - the YouTube video by Sara Wagner on APP explicitly shows doing all of the lights in one go - both stacks per location and multiple locations.   That's why I tried it.  You might want to comment on the video to say it only works for a small number of frames.

Sorry, I mean Sara Wager - I misspelled here name inadvertantly

I didn't program the algorithms, but I do know they are always improved regarding memory use and speed. So that will improve, but the calculations behind it are advanced and the fact remains that the calculations become extremely complex when everything is done in 1 go. That's what makes it use more and more memory just because of that, we can't really make that much simpler (and thus faster and less intensive) as that's where the power of the calibration process comes from. Splitting the data up into panels first is the best way to go for a perfect result. It may take a few extra steps, but still, getting seamless mosaics in the end almost automatically is something no other package does with this kind of precision.

I'll tag Mabula, he may give some extra info about this if he has time; @mabula-admin

It is true that doing the mosaic alignment the way you are apparently doing it is complex.

But that isn't the only way to do it.

If you use a program like SGP, NINA, APT, Voyager to execute a sequence of Go-To moves and capture a mosaic, it is commonplace to use plate solving frequently - indeed you could do it after every change of position (panel).

These programs allow you to use a different plate solving algorithm and software package for blind plate solving (when you don't know where you are) and near plate solving where do you.   

In this context, plate solving is considered to be a fast, reliable way to get closed loop control and be sure that you are actually capturing an image where you think you are.  

So if plate solving is fast enough and reliable enough for capturing the images, why would it not be fast and reliable enough to stack and stitch them?

You can still run the pairwise comparisons you currently use for fine adjustment and for distortion models.  However the large scale pairwise approach you currently use would never have to run.

There are cases like stacking planetary images where plate  solving will not work but for the majority of deep sky images it should work very well.

--------------

Of course if you are using SGP, NINA or others, you can program them to put each panel in a separate directory.  Unfortunately, it does not seem that APP has a good way to batch process many panels across multiple directories automatically - or anyway, I have not found it. 

So it is a ton of manual work to stack and stitch a large mosaic with APP.

---------------

If you use SGP or others you could also program these capture integration programs to save the plate solved location of each panel.   

Something like this is done by commercial panorama heads like Gigapan or Roundshot - they produce what amounts to a map which tells the panorama stitching software which frame connects to which other frame.    They still run the fine scale alignment algorithms, but the connectivity of one frame to another is entirely decided.

But it does not seem that APP has a way to accept the plate solved position.

---------------

So, consider these all suggestions for the future.

1.  Use plate solving to allow "all in 1 go" stacking and stitching.

2.  If you still feel that panel at a time is preferable, or for cases where the capture software puts files into directories, or names them to identify panels, then APP should support this by allowing the user to batch operations across directories, so you don't have to manually select each panel.

3.  If you know the exposure locations, allow this to used as a hint to stacking and stitching.

Plate solving is fast because it only requires some stars to get a position from a database. It won't do any complex distortion correction or normalization corrections on the images. This is what APP does, because without all that extra information the panels won't be stitched with subpixel precision.

Yes, I said you still need to do the fine level matching, but only on frames that actually overlap.   

Knowing whether they overlap is trivial if you know the coordinates of each frame.

---------------

Consider this:  you recommend stacking by panel, then stitching the stacked panels as two steps.  

If you had plate solving, then doing that first would tell you which images sort into each panel.  So it could stack those first, then stitch the results.

So your own recommendation shows that this would be a vast improvement.

An efficient algorithm would be not just sort and stack, because you also want to know which panels overlap which other panels.

----------------

Currently APP appears to use a needlessly intensive brute force search.  That is clear from the log of actions that is in my first post.     

The completely naïve comparison algorithm for frame overlap checks every frame against every frame.  For N frames that takes time proportional to N^2.   

APP appears to do this, with some small optimizations, but still N^2.

If you stack the panels first, then instead of N being the number of frames, you get N as the number of panels, but even that smaller number grows to be large when squared.

The point of plate solving as a first step is that plate solving N frames (or panels) takes time proportional to N.  That entirely answers the "which frame overlaps which other frame" question.   

Once you know which images overlap, then you can apply the fine scale matching and distortion.

Ahhh, I misunderstood your intention for the use of plate solving. Yes, that might actually work and make that process more efficient and automatic. Pretty sure Mabula has thought about this, there are some similar discussions I remember. Let me get back to that tomorrow. Thanks

Any update on this?   

As a concrete example, ASTAP http://www.hnsky.org/astap.htm is an app that uses plate solving to resolve which images are in which stack.  It's often used in image acquisition as a plate solver.

This might be an easy thing to add to APP, or to use it as a stand alone pre-processing step that uses ASTAP to organize files into stacks.

Just for clarity, the role of plate solving in this case is to determine which images are part of the same panel (tile), and also what the relative position of the panels are to each other.    That is the part where the existing APP algorithm is problematic.

You would still use the existing APP algorithm to do fine scale alignment.