Share:
Notifications
Clear all

Black Friday & Cyber Monday Sales on Renter's & Owner's licenses - sale will end on the 1st of December 2020 at 12:00 UTC

16 November 2020 - Wouter van Reeven has officially joined the Astro Pixel Processor Team as a moderator on our forum, welcome Wouter 🙂 !

31 July 2020 - Comet Registration video tutorial using APP 1.083-beta1 released.

30 July 2020 - APP 1.083-beta1 has been released introducing Comet processing! This 1st beta has comet registration. The stable release will also include special comet integration modes.

9 July 2020 - New and updated video tutorial using APP 1.081: Complete LRGB Tutorial of NGC292, The Small Magellanic Cloud by Christian Sasse (iTelescope.net) and Mabula Haverkamp

2019 September: Astro Pixel Processor and iTelescope.net celebrate a new Partnership!

[Sticky] Tricky Mosaic workflow, alignment problem  

Page 2 / 2
  RSS

(@ralph)
Red Giant Customer
Joined: 2 years ago
Posts: 78
May 10, 2020 22:12  

OK, I tried a different approach. I took the liberty of downloading the files you uploaded to the APP server and did my own analysis.

Very simply put, I dumped all the exposures per panel on one big heap (of course taking care to use the proper bias, dark and flats for calibration) and did the integration using the default settings of APP. Since all exposures overlap to large degree this worked without any issue.

Then I took the 4 panels (with all LRGB data dumped into a "masterluminosity" integration) and did the mosaic out of this. No cropping of edges, not worrying too much about visible edges yet. The registration RMS of this mosaic ended up at 0.09 pixels! That's awesome. In total 243 common stars were needed. Visual inspection confirms the very good registration RMS value, I could find no registration errors.

So now there is a master alignment reference that can be used to align all other individual exposures in normal registration mode. Before integrating, deselect the master alignment reference in the list of lights, so only the LRGB original frames are used in the final integration. I haven't done this step yet but will set things up to do it now.

M81 M82 reference

 

 


ReplyQuote
(@chrispeace)
Main Sequence Star Customer
Joined: 2 years ago
Posts: 40
May 10, 2020 22:55  

Okay, that's really out there. Just so I understand you:

1. Stake the whole of each filter data set in turn in one process using default APP settings with associated Flats, Darks, and Bias to make each of the filter Mosaic's. No LP fixing or stretch adjustment.

2. Then, with the four filter mosaics you combined them in normal mode into a single 'SuperLum' mosaic. Still in rough.

3. Then you went back and registered each filter panel, again in one go, into a filter mosaic but registered with the 'SuperLum' file. The SL file was then deselected/removed for that filter's integration to make that filter's mosaic.

4. We now have four mosaics, all registered to a common 'SuperLum' file which can be tidied up for LP and then be finally registered together, ready for colour processing etc as required. 

Have I got this right. I'm wondering if the four mosaics, once tidied up need a final registration together or will they already be good to layer and work directly in Photoshop?

Chris


ReplyQuote
(@ralph)
Red Giant Customer
Joined: 2 years ago
Posts: 78
May 11, 2020 01:08  

Almost right. There is only one mosaic in the whole process.

I did the following procedure:

  1. Generate the masterflats, masterdarks, masterbias.
  2. Select all data for what you identified as Panel 1 in your directory naming. This was the luminosity data (from two different dates), the red data, green data, and blue data. Using multi-channel and multi-session processing I assigned the correct masters to the lights. I used 2 different dates as 2 different sessions.
  3. Analyse stars, 500 was enough; register with the default settings; normalise with the default settings; integrate with "integrate all" for both multi-channel and multi-session options. This generated one SuperLum frame for Panel 1.
  4. Repeat step 3 for the 3 other panels. We now have in total 4 SuperLum panels, covering your entire intended field.
  5. Remove multi-channel, multi-session form the settings. Load the 4 SuperLum panels.
  6. Analyse stars, default settings with 500 stars.
  7. Register in mosaic mode with dynamic distortion correction, not the same camera and optics, scale start 5, scale stop 10 (I don't think the settings are that critical as long as the mosaic mode is selected). This worked like a charm with beautiful 0.09 pixel RMS alignment.
  8. Integrate with 2nd degree LNC and 10 iterations (not that critical). This generates what I call the Alignment Reference, it's the picture in my previous answer, its size is about 5500 x 4500 pixels in my case.
  9. Re-activate multi-channel and multi-session. This will clean all your input files in the list.
  10. Load masters, and assign to the correct channels and sessions.
  11. Load all lights, yes, that's 221 of them. Make sure to assign the proper channels and sessions.
  12. Now the important step: load the Alignment Reference. I just assigned it to its own channel and session 1, but that shouldn't matter I think.
  13. Analyse stars, I selected 5000 stars here. With 500 stars quite some frames failed during registration due to too few star pairs in common.
  14. Register, make sure to use normal registration mode otherwise it's going to take forever. Also make sure that the Alignment Reference is indeed selected as reference frame for the alignment. Scale start of 1, scale stop of 10, dynamic distortion correction on. This is going to take quite a bit of time, half an hour in my case. One registration had an RMS of over 100 pixels and had clear distortion, so I removed that one. There were a few others that had an RMS of well over 1 pixel, but the majority of the images had a very decent RMS value.
  15. Normalise, I set the advanced mode here.
  16. Uncheck the Alignment Reference.
  17. Integrate with your favourite settings.
  18. Combine RGB.

At the time of writing step 15 is still running, but I don't expect anything strange to happen from there on.


ReplyQuote
(@chrispeace)
Main Sequence Star Customer
Joined: 2 years ago
Posts: 40
May 11, 2020 01:14  

@ralph  

What can I say but a huge thank you. It looks like a very comprehensive guide. I guess we should both get some sleep now and re-attack this in the morning.

kind regards

Chris


ReplyQuote
(@minusman)
Neutron Star Customer
Joined: 3 years ago
Posts: 174
May 11, 2020 02:45  

https://www.astropixelprocessor.com/community/tutorials-workflows/astronomical-data-calibration-priciples-must-read/#post-2516

Here is the link to the forum post about data calibration. The reason why I noticed this is that the gradient in the panels has a strange shape. Top and bottom right, dark corners.


ReplyQuote
(@ralph)
Red Giant Customer
Joined: 2 years ago
Posts: 78
May 11, 2020 08:48  

Very nice set of data, looking at the combined result!

I've done just a little bit of light pollution removal on the final image. It is something that requires a lot of manual labour, but something that also pays off. Take some time to finish it properly. Tips: enable saturation in the tickbox on the right, select 30% BG, 2 sigma, 0,0% base, set SAT to 0.50, SAT.TH to 0.00. This will make everything in your data horribly visible 😉 and allow you to attack the unwanted gradients in your image. 

combine RGB LRGB image St

ReplyQuote
(@vincent-mod)
Universe Admin
Joined: 3 years ago
Posts: 2665
May 11, 2020 10:04  

You're way ahead of me Ralph, thanks so much! Excellent result I think.


ReplyQuote
(@chrispeace)
Main Sequence Star Customer
Joined: 2 years ago
Posts: 40
May 11, 2020 10:19  
Posted by: @vincent-mod

You're way ahead of me Ralph, thanks so much! Excellent result I think.

Yes, a massive thank you to you all for helping me unpick this problem. I’ll be running through all the superb workflow advice today and hopefully will have first hand sight of my IFN mosaic soon. 

Clear skies and good health to you all.

Chris


ReplyQuote
(@vincent-mod)
Universe Admin
Joined: 3 years ago
Posts: 2665
May 11, 2020 10:25  

Lately there have been some great posts, I'm making a sticky of this one as well.


ReplyQuote
(@vincent-mod)
Universe Admin
Joined: 3 years ago
Posts: 2665
May 12, 2020 15:40  

So I started on the big dataset as well as the 4 panels didn't work for me either, likely because indeed some settings should've been tweaked during the making of the panels. Apologies if some of my findings are already mentioned above, I just went on first without looking at the other nice result to see if I can make it work. There are other workflows to do this as well, this is just mine.

My findings and some recommendations for the Luminance;

- Per panel;
  Star amount: 500
  Scales: Start 1, Stop 5
  No dynamic distortion correction, same camera and optics
  1st degree LNC, 3 iterations, 5% MBB
- I decided to leave the flats out as they didn't correct the luminance correctly (without them, the overall background illumination looked better, so the flats need work).
- I processed each panel fully, correcting with the light pollution tool (judging where the nubilosity is by overstretching the preview) then saved that result.
- Seems some data was taken with a different capture program? Panel 2, Luminance from 04-21 has a slightly different resolution (by 1 pixel) and different description of the instrument used. I'd advise to keep the capturing exactly the same, as different capture programs might influence the data differently (preferably they don't influence it at all, but most important is to keep this the same). APP can handle all kinds of resolutions just fine, but calibration files need to fit exactly with the light frames.

RGB;

- Same settings as Luminance
- Didn't have the flats for these
- I loaded the R, G and B per panel (assigning them manually to the correct channels) and made a combined RGB result per panel.
- I did the removal of light pollution and color calibration.

Mosaic;

- Restarted APP
- Loaded the Luminance panels as Luminance
- Star amount: 2000, Scales: start 4, stop 12, dynamic correction, not the same camera/optics
- Advanced mode in normalize, MBB to 10%, 1st degree LNC, 3 iterations

That gave this, beautiful, result. This is without any extra correction and very much stretched but the Luminance lends itself for that I think:

Screenshot from 2020 05 12 15 27 05

And then I had to give up for today as the RGB panels didn't want to register properly with each other. Was the overlap on the RGB panels 10% as well? Apparently it can work as Ralph shows, but it might need some extra steps like he did with all the frames. Shouldn't be necessary though.. I also get low (100-300) star counts on the RGB panels, so that might be a bit of a data issue. Will continue tomorrow.

I think I'll reprocess the RGB to see if I can make that better.


ReplyQuote
(@chrispeace)
Main Sequence Star Customer
Joined: 2 years ago
Posts: 40
May 12, 2020 15:52  

Ralph

My pixel alignment for the Alignment Mosaic came out at 0.11  for 257 star pairs so its consistent with yours. 

In step 8 you didn't mention an MBB setting. I set 10% as it seemed sensible to smooth the panel edges a little in the Alignment SuperLum Mosaic.  Also, your image of the Alignment Reference Mosaic looked like you had removed a lot of the LP but you didn't mention this. Was that just for appearances or would you LP correct the Alignment Reference Mosaic before using it in step 12?  I didn't and it didn't seem to impact the workflow. 

Having gone through your workflow above I too had a 6 outlier subs that produced eye catching RMS numbers.  Here are a couple of screen shots only because the worst (57.29) was badly distorted after registration and had three satellite trails through it as you can see.  @vincent-mod  Any thoughts or reasons that might explain this?

IFN Mosaic Failed registration Red
IFN Mosaic Failed registration Blue

Regards

Chris


ReplyQuote
(@chrispeace)
Main Sequence Star Customer
Joined: 2 years ago
Posts: 40
May 12, 2020 16:08  

@vincent-mod

Good spot Vincent. The early subs were taken with APT and the later with NINA. Would not have done that if I'd have thought it would have made a difference. Not best practice though as you say so noted.

You say the Flats need work. I could try going higher than full well (20K) on the sensor or get a dimmer flat panel to make the exposure longer. Is that the kind of 'work' that you meant or am I off the point?


ReplyQuote
(@vincent-mod)
Universe Admin
Joined: 3 years ago
Posts: 2665
May 12, 2020 16:15  

@chrispeace Well, the "work" is that it seems the light gradient produced by the flats, didn't quite match those of the subs. So when I checked the correction of a frame with the flats (by selecting L-calibrated on top of the viewer), I saw a nice sub without much gradient going to one with a gradient. Which should be the other way around.


ReplyQuote
(@chrispeace)
Main Sequence Star Customer
Joined: 2 years ago
Posts: 40
May 12, 2020 16:19  

@vincent-mod

 

Got it thanks. I'll start with my light source. It's a electro-luminescent panel rather than LED so I cannot dim it and it could have a gradient I cannot see visually.

Chris

This post was modified 7 months ago by ChrisPeace

ReplyQuote
(@vincent-mod)
Universe Admin
Joined: 3 years ago
Posts: 2665
May 12, 2020 16:20  

Having gone through your workflow above I too had a 6 outlier subs that produced eye catching RMS numbers.  Here are a couple of screen shots only because the worst (57.29) was badly distorted after registration and had three satellite trails through it as you can see.  @vincent-mod  Any thoughts or reasons that might explain this?

If you select "L-c-registered" after the registering, you can go through your frames to see how they registered (zoom in 100%). This is also nice when you registered using the mosaic-mode, you can then check already if the panels are correctly placed and don't have weird distortion. That's way faster then waiting on the entire registration. I didn't leave out any frames actually, seemed to work fine for the luminance.

I've send Mabula 2 frames I couldn't register, maybe he sees way faster what the issue was then me. 🙂


ReplyQuote
(@vincent-mod)
Universe Admin
Joined: 3 years ago
Posts: 2665
May 13, 2020 13:36  

Ah so, according to Mabula the issue I'm having with the RGB is due to the undersampling. The main problem is that the stars in the 2x2 are so small it's going to be very tricky for him to get that to work in the algorithm. The boundary between stars and hot-pixels becomes so vague then that the registration engine might then cause more issues than it solves. So I think the best would be to keep RGB also at 1x1 for your setup, that should work.


ReplyQuote
(@kajouman)
Hydrogen Atom Customer
Joined: 3 years ago
Posts: 2
June 27, 2020 12:07  

hello all,little question about  mosaics.

 

What is the max  APP can handle when combining panels ?

i havve a 7 panels of the soul nebula,all more then 1 gig,when combining to all APP crash.

 

ps its not my pc,16 gig ram   etc...

 

greetings.  for the rest APP performse very well for me,the pollution tool is very great.

 

groet John https://www.astrobin.com/users/kajouman/


ReplyQuote
(@vincent-mod)
Universe Admin
Joined: 3 years ago
Posts: 2665
June 28, 2020 10:57  

That will depend on memory used by both your system and APP. So even with 16GB things can go wrong, depending on your system settings. In theory there is no hard limit, but the system will have one.


kajouman liked
ReplyQuote
(@chrispeace)
Main Sequence Star Customer
Joined: 2 years ago
Posts: 40
June 28, 2020 11:21  

@kajouman

Hi John

Whilst I’m not an expert on APP capabilities I would suggest you try building the mosaic in stages rather than trying to put all 9 panels together in one go. This was a suggestion I got when I was having my difficulties. So try connecting just two panels at a time and do that 4 times to give you 4 new (double) panels and one single panel. Then combine two double panels together to give you 2 new panels plus one single frame. Continue To grow your mosaic until you get every panel combined. 
Also,  look to see how much memory you have allowed APP to use from your total 16gb. It’s configurable and you may not have allocated sufficient memory priority to APP to get the job done.

Make sure your laptop is plugged into the mains as many run more slowly on battery power.

Finally if you have a solid state drive I’d work from that as it will be a lot quicker.

Good luck and do post me an image when you get this all together. 

Chris

This post was modified 5 months ago 2 times by ChrisPeace

ReplyQuote
(@kajouman)
Hydrogen Atom Customer
Joined: 3 years ago
Posts: 2
June 28, 2020 23:10  

@chrispeace

 

thanks,did this also.  in pixinsight i can build  all into one. very well merged.

 


ReplyQuote
Page 2 / 2
Share: