15th Feb 2024: Astro Pixel Processor 2.0.0-beta29 released - macOS native File Chooser, macOS CMD-Q fixed, read-only Fits on network fixed and other bug fixes
7th December 2023: added payment option Alipay to purchase Astro Pixel Processor from China, Hong Kong, Macau, Taiwan, Korea, Japan and other countries where Alipay is used.
I am also using my Desktop as the workspace for constructing mosaics so everything is linked directly to the internal SSD. Each of my individual panels are also previous integrations of large numbers of light frames, stacked, then cropped and light pollution corrected. The original light frames are not involved, just the mosaic panel stacks, and about 20-30 of these.
I still only see the problem when APP has to choose to downsize the task by 0.79650. If I manually downsize to a greater degree the issue is not present.
I will give Walters 're-integration' approach a go....
With "images" I mean mosaic panels - stacked, cropped and light pollution removed, no further processing. It is a Mac Studio, there are no spinning discs inside. I am 51 years old, I am used to this term, sorry.
Interesting, whenever I create my Mosaics, the mosaic panels are completely virgin from when APP created them, I do not perform any manipulation on them at all, no cropping, no LP removal, I just allow APP to create each panel, and then I re-load those files to create my mosaic. Never seen the need to perform any cropping or LP removal before creating the mosaic.
Any edge inference in each panel caused by stacking frames would usually be eliminated during the multi band blending, and LNC would normalise the panels too during the mosaic integration.
Can you try with the frames untouched after APP created them and see what the result is?
I still only see the problem when APP has to choose to downsize the task by 0.79650. If I manually downsize to a greater degree the issue is not present.
I will run some more tests with Jon's data and that scale factor. I will also use the desktop as the working directory.
I mentioned some time ago that my mosaik has growing holes/gaps with the increasing number of panels. I now figured out that this can be corrected by decreasing the MBB-value. I have been using a MBB-value of 50%, now at 25% all gaps and holes are gone.
I mentioned some time ago that my mosaik has growing holes/gaps with the increasing number of panels. I now figured out that this can be corrected by decreasing the MBB-value. I have been using a MBB-value of 50%, now at 25% all gaps and holes are gone.
What is your overlap in your frames? I presume much lower than 50%. I have a 10% overlap in my frames, so I set my MBB to 12%, I would never go any higher, your MBB percentage should be very close to your actual overlap, if you have varying degrees of overlap, I'd be inclined to mosaic those panels together and then crate the larger mosaic from your smaller mosaic panels.
With "images" I mean mosaic panels - stacked, cropped and light pollution removed, no further processing. It is a Mac Studio, there are no spinning discs inside. I am 51 years old, I am used to this term, sorry.
Interesting, whenever I create my Mosaics, the mosaic panels are completely virgin from when APP created them, I do not perform any manipulation on them at all, no cropping, no LP removal, I just allow APP to create each panel, and then I re-load those files to create my mosaic. Never seen the need to perform any cropping or LP removal before creating the mosaic.
Any edge inference in each panel caused by stacking frames would usually be eliminated during the multi band blending, and LNC would normalise the panels too during the mosaic integration.
Can you try with the frames untouched after APP created them and see what the result is?
I did this in the beginning, this resulted in strange gradients that were not there before mosaicing. Since I crop all the edges after stacking, these gradients no longer appear.
I mentioned some time ago that my mosaik has growing holes/gaps with the increasing number of panels. I now figured out that this can be corrected by decreasing the MBB-value. I have been using a MBB-value of 50%, now at 25% all gaps and holes are gone.
What is your overlap in your frames? I presume much lower than 50%. I have a 10% overlap in my frames, so I set my MBB to 12%, I would never go any higher, your MBB percentage should be very close to your actual overlap, if you have varying degrees of overlap, I'd be inclined to mosaic those panels together and then crate the larger mosaic from your smaller mosaic panels.
I have varying overlaps. Yes, I thought about doing some smaller mosaics for one large mosaic too late. I have now 93 panels, most of them just numbered just in the order I shot them. It is now nearly impossible to sort out which one is located where in the mosaic. But I will try anyways.
I mentioned some time ago that my mosaik has growing holes/gaps with the increasing number of panels. I now figured out that this can be corrected by decreasing the MBB-value. I have been using a MBB-value of 50%, now at 25% all gaps and holes are gone.
I mentioned some time ago that my mosaik has growing holes/gaps with the increasing number of panels. I now figured out that this can be corrected by decreasing the MBB-value. I have been using a MBB-value of 50%, now at 25% all gaps and holes are gone.
Okay, so maybe the MBB 50% setting has been causing this then? I will test immediately and fix a.s.a.p. when I have found this issue.
Mabula
Yes, this parameter fixes THIS problem. Still havimg the stripes after one image-integration of all panels. These go away after re-integrating three or four times (depends on the number of the panels).
Okay, but now I am confused a bit. What problem is gone with setting MBB lower than 50%? Are those the mentioned holes and gaps ? Do you have a screenshot of this then, I do not think I know how that looks because have not seen this myself yet I think?
The horizontal stripes issues is not affected by the MBB % then, right?
Okay, but now I am confused a bit. What problem is gone with setting MBB lower than 50%? Are those the mentioned holes and gaps ? Do you have a screenshot of this then, I do not think I know how that looks because have not seen this myself yet I think?
The horizontal stripes issues is not affected by the MBB % then, right?
Mabula
Right, the MBB does not affect the horizontal stripes. It is just so that MBB eats away the overlapping spaces of the panels if set to a high value. This creates gaps, they keep growing with the increasing number of panels. But this can easily be "fixed" by decreasing the MBB-value.
Ah, that is it indeed, thank you very much for the clarification, this will be fixed immediately ! I was not aware it could do this, I never set MBB higher than 25-30% when testing... Now I know where to fix this 😉
The other issue is of course still on my issue list as well.
Okay, but now I am confused a bit. What problem is gone with setting MBB lower than 50%? Are those the mentioned holes and gaps ? Do you have a screenshot of this then, I do not think I know how that looks because have not seen this myself yet I think?
The horizontal stripes issues is not affected by the MBB % then, right?
Mabula
Right, the MBB does not affect the horizontal stripes. It is just so that MBB eats away the overlapping spaces of the panels if set to a high value. This creates gaps, they keep growing with the increasing number of panels. But this can easily be "fixed" by decreasing the MBB-value.
Hi all @walsc, @stastro, @jdwood, @mountainair, I have found and fixed this particular problem with high MBB %, I will soon release 2.0.0-beta26 with this fix. Only issue left now are those horizontal stripes..
As reported here : https://www.astropixelprocessor.com/community/main-forum/strange-horizontal-lines-in-mosaic-integrations/paged/2/#post-29618 MBB could remove data in overlap areas when the MBB % was set to a high level. We have found the cause of this. MBB was always removing 1% of the image borders relative to the set MBB percentage. This is completely fixed now. All data will always be used, so these strange artefacts should no longer happen. In addition, the MBB % was relative to the size of the entire field of view, which is not ideal for mosaics. Preferrably, the MBB % is relative for each image in the stack to the area where data is not 0 in that image, so we have implemented this improvement. This should improve Multi-Band Blending in all situations, especially when combining data from different setups and or mosaics.
This post was modified 7 months ago by Mabula-Admin
Looks very promising, thank you for your work! Tomorrow I will test if reducing the number of threads has any influence on the stripes. Until now I am using all 20 available CPU-threads, noticing that not everybody is doing this as well.
Another update, the MBB process in a mosaic like yours is now about 5x faster as well, so that reduces the integration time also 😉 I have made an image together with the MBB weight map, in beta25 and earlier, you could see the gaps there 😉
So this fully shows now that all data is preserved and those gaps/holes are a thing of the past now.
If you enable the MBB map in 6) integrate, the resulting file will have the same name as the integration file but with a small postfix -wm added. I guess that is what you are asking, or is it something else?
If you enable the MBB map in 6) integrate, the resulting file will have the same name as the integration file but with a small postfix -wm added. I guess that is what you are asking, or is it something else?
Mabula
I meant the filenames of the panels written on the map. This would be nice, but not that important yet.
Oh yes indeed, it would be great to get an output that actually shows which panel is which 🙂 that also helps with quick adjustment of the reference frame is needed/wanted. I have added this to our ToDo list !\
I agree it is not the most important thing, but I will make it possible down the road because it is helpful and simply nice for sure.
Oh yes indeed, it would be great to get an output that actually shows which panel is which 🙂 that also helps with quick adjustment of the reference frame is needed/wanted. I have added this to our ToDo list !\
I agree it is not the most important thing, but I will make it possible down the road because it is helpful and simply nice for sure.
Sorry for the late reply. I did circle back to try this again on another Mac (instead of the Apple Silicon M2 Max, this is an Intel Core-i9). I get the same behavior. I also tried integrating other panels from the same target (RGB instead of RGBSHO) and had the same behavior. LNC and MBB on and off, the lines remain. I also tried higher LNC iterations and even levels, as Walter suggested, to no avail.
Unfortunately I don't have a Windows or Linux PC to test with. I could try it in Parallels running Windows 11, but I'd need to build a new VM. I'll try this week.
So these are the outstanding horizontal lines that show up, right?
Then please provide a dataset and exact settings for all menus from 0) -6) apart from defaults so I can try to duplicate this. I have not had this issue once yet... and I have tested at least 100x times now with different datasets on all my computers including 3 different macs. I have analysed the lines that Walter @walsc got and the data in these lines make little sense to me at the moment.
Just confirming previous observations but now with beta 26. Using the exact same dataset and operations prior to the tab 6) 'integrate' panel but just modifying what I do on this tab. If I let APP do the automatic scaling to 0.79655 during integration then I see lines in the final mosaic.
I have now also tried Walters @walsc suggestion of then pressing 'integrate' again. For me this reduces the thickness of the lines but does not eliminate them.
Finally, if I manually set scaling to 0.7 and integrate then the lines disappear. Three images shown below in that order.
BTW @mabula-admin I think that you only have a small portion of the data going into my mosaic (was very large to upload)so maybe this doesn't reach the same scaling threshold on your Mac that I reach?
Thank you very much for your feedback here, yes, it seems that the issue is related to such a detailed rescaling value of 0.79655. I will look more closely in our code to see if I can solve this.
I only see the final resuilt. Maybe it will help if you can upload the mosaic panels there as well and then I will try to process it with that scale value to see if I can finally see the error myself happening.
