Problem with satell...
 
Share:
Notifications
Clear all

2022-05-29: APP 2.0.0-beta2 has been released !

Release notes

Download links per platform:

windows 2.0.0-beta2

macOS x86_64 2.0.0-beta2

macOS arm64 M1 2.0.0-beta2

Linux DEB 2.0.0-beta2

Linux RPM 2.0.0-beta2

Problem with satellite trails

Page 1 / 2

(@aurora74)
Molecular Cloud Customer
Joined: 3 years ago
Posts: 5
Topic starter  

Hi all!

Since 1.083 I seem to have problems removing satellite trails.

I tried every possible integration setting sigma, windsor, MAD, adaptive and also high kappa down to 1.0.

The satellite trail(s) are still visible.

This happens with 5 lights and also with 30 lights.

It it also happening with DSLR raw files as well as CCD fits files.

Any suggestion or maybe a known bug?

Thanks for any information.

Regards,

 

 Ulrich


ReplyQuote
(@vincent-mod)
Quasar Admin
Joined: 5 years ago
Posts: 4854
 

What version are you using at the moment?


ReplyQuote
(@aurora74)
Molecular Cloud Customer
Joined: 3 years ago
Posts: 5
Topic starter  

1.083.4


ReplyQuote
(@vincent-mod)
Quasar Admin
Joined: 5 years ago
Posts: 4854
 

Ok, that shouldn't be a problem. The standard settings (set when you start APP) already are enough for most data. If you have enough frames (like 30) and proper calibration files, that should work very well, if you have trails from geo-stationary satellites, it may be trickier. Is that the case? Could you post a few screenshots of some lights?


ReplyQuote
(@connor231)
Molecular Cloud Customer
Joined: 3 years ago
Posts: 4
 

I also noticed this with 1083.4.

Earlier versions of APP did such a good job on satellites that I usually ignored them, but the other night I had two that were visible in my final integration using 1083.4.

I had 42xred, 42xgreen and 42xblue. There were two red frames with bad satellite trails and some fainter trails in other frames. The two worst were easily visible in the final integration. I used standard settings for all tabs.

In the end I inspected every frame (takes a while - I wish there was a blink tool) and deleted the two worst. Fixed the problem, but I was surprised it was an issue at all. Maybe I've just never had trails as bad as this. It was a longer focal length than I usually use.

 

By the way - congrats on APP. Its a terrific program. Mabula has done an amazing job.

JC

 

This post was modified 2 months ago by John Connor

ReplyQuote
(@aurora74)
Molecular Cloud Customer
Joined: 3 years ago
Posts: 5
Topic starter  

@vincent-mod

Here are two screenshots of M51.

First one with the sat trail.

M51 Sat Trail

Second one without the trail.

M51 clean

Camera was ZWO ASI 2600 MC Pro.

Telescope was Meade 14-inch ACF.

Exposure time 300s each.

I first extracted the RGB channels, because I took H Alpha frames on another telescope parallel with that one.

If I don't use the frame with the sat trail, the finale result is nice as always with APP.

Maybe, it has something to do with the 3.5m focal length?

Thanks for looking into my case.

Regards,

 

 Ulrich


ReplyQuote
(@vincent-mod)
Quasar Admin
Joined: 5 years ago
Posts: 4854
 

Really shouldn't be an issue. I think I need the data to be able to check out what's going on, if you want you can share it to our server;


ReplyQuote
(@aurora74)
Molecular Cloud Customer
Joined: 3 years ago
Posts: 5
Topic starter  

@vincent-mod 

Dear Vincent!

I am just uploading the files.

You will find a folder aurora74-outlier-calibration

In that folder I uploaded the original (unseparated) RGB files from ASI2600MC plus masterbias, masterdark and BPM.

In the subfolder HAlpha you will find the HAlpha files from a ASI1600MM plus masterbias, masterdark and BPM.

I hope you can use the files for your evaluation of the problem.

Thanks.

Regards,

 Ulrich


ReplyQuote
(@vincent-mod)
Quasar Admin
Joined: 5 years ago
Posts: 4854
 

Hi Ulrich, sorry it's been a bit busy. I'll have a look today!


ReplyQuote
(@vincent-mod)
Quasar Admin
Joined: 5 years ago
Posts: 4854
 

So, I think your masterdark has some stars in it. Is that possible? It's best not to use calibration frames which have any kind of pollution in them.

The satellite though is the biggest issue, I think it wasn't rejected well due to how big it appears with your FOV. To be able to reject it better, higher dithering steps may help. I managed to get it somewhat better with setting integration to median, switching to adaptive rejection and a kappa low of 2.2, kappa high of 4.0. Not completely gone, but reduced a great amount.

image

ReplyQuote
(@jochen-scharmann)
Main Sequence Star Customer
Joined: 3 years ago
Posts: 46
 

Hi,

I also had seen this effect, stacking 400+ files while just one strong plane trail remained clearlyvisible (though dimmed). Outlier rejection should have handled this. The only way out was manual frame deletion (Which is a bit boring with 100s of files)

I have 2  questions here:

1)Once APP fidentified an Outlier Pixel, i.e plane/sat trail, what does it do about it?

a) frame gets discarded

b) Pixel gets discarded

c) Pixel value gets replaced, e.g by median of that very pixel (DSS has that option which works pretty well for loose Kappa andmultiple iterations)

 

2) To avoid tight kappa values, one would require more iterations of the outlier rejection process. With which control can this be achieved?

thanks,

Jochen

 


ReplyQuote
(@connor231)
Molecular Cloud Customer
Joined: 3 years ago
Posts: 4
 

Vincent

I don't understand your suggestion -

    "To be able to reject it better, higher dithering steps may help."

I dither fairly aggressively now, but as it is extremely unlikely that a satellite trail will be in exactly the same position in different frames surely dithering is irrelevant to this issue. Have I misunderstood something here ?

Your other comment about the size of the satellite trail in the FOV sounds right to me. I'm starting to think that I may need to manage satellite trails better when shooting at longer focal lengths, either by deleting more frames or by playing with the rejection settings.

 

JC

 


ReplyQuote
(@vincent-mod)
Quasar Admin
Joined: 5 years ago
Posts: 4854
 

Yes, then dithering won't have an effect, it does depend on the type of satellites though, geo-stationary I could imagine benefiting from it.

In this case it seems the size of the trail and mainly the amount of signal it produced, was to blame. The outlier rejections aren't very aggressive as APP does want to preserve as much of faint signals as possible. So that's why you would need to experiment a bit when there indeed is an issue in the end. A better way to quickly screen for very big trails and such would indeed be nice.


ReplyQuote
(@jochen-scharmann)
Main Sequence Star Customer
Joined: 3 years ago
Posts: 46
 

Do you have any answers to my above questions, please?

thanks,

Jochen


ReplyQuote
(@vincent-mod)
Quasar Admin
Joined: 5 years ago
Posts: 4854
 
Posted by: @jochen-scharmann

I have 2  questions here:

1)Once APP fidentified an Outlier Pixel, i.e plane/sat trail, what does it do about it?

a) frame gets discarded

b) Pixel gets discarded

c) Pixel value gets replaced, e.g by median of that very pixel (DSS has that option which works pretty well for loose Kappa andmultiple iterations)

I have to say I'm not an expert on the actual algorithm used here, but the frame definitely isn't discarded. Mabula always mentions not to just throw away frames with satellite trails as you should be able to deal with that and the rest of the image can still contribute a lot to the signal. It does, normally, work very well with the standard settings and enough data though, so when you do run into the fact it wasn't fully rejected, you have to tweak the kappa parameters. If that is needed to be set to very tight parameters, then it's better to remove the offending frame.

2) To avoid tight kappa values, one would require more iterations of the outlier rejection process. With which control can this be achieved?

This is done internally by APP itself, you can choose different algorithms as well. We don't have an iteration setting to be set.


ReplyQuote
(@jochen-scharmann)
Main Sequence Star Customer
Joined: 3 years ago
Posts: 46
 

@vincent-mod understood.. but with 100's of files, any plane trial should be recognized as outlier and ironed out, no matter what kappa, but obviously it doesnt work. as Aurora74 mentiond, that started with 1.083, so maybe this breaks up after the first iteration. I'd rather use the conservative APP kappa values with 1 iteration more than to turn the kappa srcrew, so if APP does that automatically it should recognize the trail remainder or just provide manual control.


ReplyQuote
(@vincent-mod)
Quasar Admin
Joined: 5 years ago
Posts: 4854
 

Well, it may definitely be improved upon, but we haven't had many issues regarding this so it doesn't seem to be a wide-spread issue, but more towards specific data sets. I'll ask Mabula to give a more in depth reply on the algorithm used here, he's sick at the moment, so please allow for a few days.


ReplyQuote
(@mabula-admin)
Quasar Admin
Joined: 5 years ago
Posts: 3159
 

Hi all @aurora74 @connor231 @jochen-scharmann,

I think I need to see this  for myself, i never have outlier rejection issues like you describe myself, not with 20-30 frames and not with 100+ frames.

Could it be that you use the automatic integration setting in the mosaic integration mode? If so, that explains why no rejection occurs. In mosaic mode, you don't want that to be enabled and thus it's disabled in mosaic mode.

When the integration starts, the generale console and the process console will show you in details what outlier rejection settings are being used if any. And the settings are stored in the FITS header of the integration, so can you check what happens/happened?

1)Once APP fidentified an Outlier Pixel, i.e plane/sat trail, what does it do about it?

a) frame gets discarded

b) Pixel gets discarded

c) Pixel value gets replaced, e.g by median of that very pixel (DSS has that option which works pretty well for loose Kappa andmultiple iterations)

 

2) To avoid tight kappa values, one would require more iterations of the outlier rejection process. With which control can this be achieved?

1) A whole frame is never discarded, that would be very bad and a huge waste of good data... Now depending on the algorithm.... for each pixel stack, a median and dispersion/scale is calculated. With winsor rejection for example, the pixels that lie outside of the bounds (central value +- kappa*dispersion) are replaced with the bounds, then winsored median and dispersion/scale is calculated. That is the essence of Winsor rejection..

2) Iteration control has been made redundant with smart implementation that APP now uses. It will continue iterating always until all pixels outside of the +- kappa*dispersion bounds are removed. So once no pixels are removed after an iteration, it will stop for that pixel stack and the median/average(/max) of the survived pixels becomes that integrated pixel.

Mabula


ReplyQuote
(@minusman)
Red Giant Customer
Joined: 5 years ago
Posts: 204
 

Hello Mabula, I've also recently had problems with the rejection of satellite tracks in a dataset. The data set now includes 460 frames and if I now integrate everything in automatic mode, there are always remnants of satellite tracks. Even with aggressive rejection settings, trace resters still remained. I found out more about what could be wrong. As far as I understand it, my data set is due to the strongly changing recording conditions. If images are now brightened by moonlight, you have very noisy frames. And this noise creates a high dispersion of the kappa central value, so the rest of the satellite track can lie within the central value. Which are then reintroduced into the pixel table after rejection and thus become visible again in the final integration result. I tried again to reprocess the data set, but this time I didn't use the best frame quality as a reference, but the picture with the lowest background brightness. Now all traces could be rejected. I have to investigate it further with other datasets to see if it works there too. I can also upload the dataset once. maybe helps to better understand the problem. I think the thing with the satellite tracks will increase in the near future (Starlink etc.)

With kind regards Henry


ReplyQuote
(@jochen-scharmann)
Main Sequence Star Customer
Joined: 3 years ago
Posts: 46
 

Thanks @Mabula for the detailed explanation!

"With winsor rejection for example, the pixels that lie outside of the bounds (central value +- kappa*dispersion) are replaced with the bounds, then winsored median and dispersion/scale is calculated. That is the essence of Winsor rejection.."

This might explain the reults, as I do not see weak satelite trails but just weak remnants of previously strong plane trails. as they are replaced by the bounds instead of a median, these might still survive. 

Usually for my 200+ frames stack I select average / adaptive rejection. I do not do mosaics, so I might be slightly displaced in that thread

Thanks again,

Jochen


ReplyQuote
Page 1 / 2
Share: