15th Feb 2024: Astro Pixel Processor 2.0.0-beta29 released - macOS native File Chooser, macOS CMD-Q fixed, read-only Fits on network fixed and other bug fixes
7th December 2023: added payment option Alipay to purchase Astro Pixel Processor from China, Hong Kong, Macau, Taiwan, Korea, Japan and other countries where Alipay is used.
Looking for some help on diagnosing a poor result. I'm still learning APP, but today I tried creating a set of masters by loading my flats (ISO1000 at 1/80), dark flats (same gain & exp) and darks (ISO1000 at 30s taken at -10C ... same as lights). I had about 30 flats and dark flats and 55 darks.
Imaging was done with an RGB Canon mirrorless full-frame sensor.
Prior to processing, I noticed that my flats were much bluer than usual. This was the first time that I had taken flats the next morning by pointing at the sky (still with white t-shirt).
After creating the masters, I loaded in 137 lights and started processing. The integration recently finished and there was a huge background gradient that surprised me and the image seemed much noisier than I anticipated.
I went back and looked at the masters and noticed that the master flat had almost no red data and all of the calibrated and normalized lights were very green-ish.
How do I go about diagnosing what went wrong here?
Thanks you!
I think this might be a problem with sky flats. The light is bluish, not full spectrum. Some tutorials say to use a cloudy sky, being grey which is full spectrum. But I think the stacking software should not use each channel separately but normalize them and blend the three into a monochrome flat. I don't know if APP is doing this... if not, then why?
I agree with astrogee. For me flats taken against the blue sky don't work either for R, G and B images (I shoot mono, not color). What does work is pointing the telescope at a white surface (door, wall, towel) instead.
Thank you @astrogee and @Wouter-Moderator. I suspected that this might be the case, and I have never taken sky flats before (always used some form of white background with a device screen - tablet or laptop, etc).
Is this an unrecoverable mistake? I have broken down my setup already. I do have flats taken from similar setups (same camera, telescope configuration, different night).
I have read up on calibration frames and understand that I need to take darks every session and bias can last for months, but I’m still a bit unclear on whether flats can be used across sessions if the camera/telescope configuration is the same.
Thanks again!
@tbarn Tom, both darks and bias last for months provided you make sure that all lights, darks and bias have the same gain, offset and temperature. Flats you should take every session but can be used over several sessions if necessary.
Flats correct vignetting and dust devils and are used to identify dark pixels (pixels that generate little or no signal even with long exposure times). If your lights don't show dust devils then you can setup your gear again at (more or less) the same focus point and take new flats.
But that may not be necessary. Can you post a screenshot of what your integration result looks like. In my experience with OSC cameras the result will look green which can be corrected with a background neutralization.
Will do @Wouter-Moderator. I'm re-running with flats (and bias) frames I took earlier this month. Again I noticed strange amounts of green in my master frames. The Master Bias frame that was created is very very green and the calibrated lights look green. It is possible that I didn't properly reset the session, but I did delete all master frames and BPM from the previous attempt.
In any case, I just kicked off normalization and will post a screen shot after integration using the "non-sky" flats from a couple of weeks ago.
@tbarn Bias frames of course don't depend on light so if they look different from what you're used to then something may be wrong. Are the cables and power supply of the camera functioning well? Again, please include screen shots of the normal and the current situation so we can see the difference.
Okay, final integration using the older flats and bias looks better, but with significant vignetting. I'll try tools to work on that.
I'm wondering if I goofed up the master frame creation step. Here is the Master Bias that was created from a bunch of bias images that are black.
Between this and the heavily vignetted final image, surely I screwed something up.
@tbarn How did you create the master bias? How many bias frames did you use? What version of APP? Did you use the default values or did you modify something? Did you use an external drive or the internal HDD?
Okay, I'm making some progress on this. I went back and looked at all of my calibration frames. If I open one of the flat frames in the APP image viewer there is virtually no blue data. However, if I open the same raw file (.CR3) in Bridge or Photoshop, the overall color is taupe with reasonably balanced blue, green and red.
Is there some APP setting that would be ignoring the blue data in my bias frames?
In the image viewer, I noticed that some kind of stretch had been applied. If I change the dropdown box to "no stretch" I get some blue data back, but it still doesn't look as taupe as what I see in Photoshop.
Is there some APP setting that would be ignoring the blue data in my bias frames?
No there is not. Can you please answer the questions in my previous comment?
@tbarn How did you create the master bias? How many bias frames did you use? What version of APP? Did you use the default values or did you modify something? Did you use an external drive or the internal HDD?
Sorry, I missed those while troubleshooting on my end.
I loaded 56 flat frames, 54 dark frames and 30 bias frames.
APP v1.083
I believe I used the default values (I did not intentionally change anything).
Files are located on the internal SSD.
@tbarn Thanks and no worries. It is easy to get caught in the flow 🙂 In any case, it looks like all is fine with the APP version and the way you use it.
What happens if you use the new flats and darks but use the old bias?
@tbarn Thanks and no worries. It is easy to get caught in the flow 🙂 In any case, it looks like all is fine with the APP version and the way you use it.
What happens if you use the new flats and darks but use the old bias?
I’ll give that a try. I was looking around a bit more last night and the thing that seems most surprising is that when I look at my flats using APP image viewer with no stretch applied (data as-is), they don’t look anything like what I see in Photoshop.
I’m getting ready to travel for a few days, so it might take me some time to run more experiments.
I was looking around a bit more last night and the thing that seems most surprising is that when I look at my flats using APP image viewer with no stretch applied (data as-is), they don’t look anything like what I see in Photoshop.
This may be related to the way the applications display images on your computer screen. Again, please add screenshots to let us see what the differences are. So please add a screenshot of what APP shows and one of what Photoshop shows so we may be able to explain the difference.
I’m getting ready to travel for a few days, so it might take me some time to run more experiments.
No problem and thanks for letting us know. Safe travels!
Okay, here is a screen shot of an APP flat frame and the same frame in Photoshop:
If you look at the screenshot I shared previously, the master flat was very green - this flat would look similarly if I used the same stretch drop-down, but the above screen shot is "as-is".
@tbarn Raw images from a OSC (one shot color) camera need to be debayered before they can be used. APP debayers the images (which is why they show color) and Photoshop doesn't (it interprets them as mono data).
Also seems that photoshop is doing some kind of normalization, looking at both histograms. APP isn't at this stage, hence the green color.
Okay, thanks to you both. So I'm back to my initial diagnosis problem. If you look at that first screen shot of mine, you'll see significant vignetting. I used this same set of flats and bias frames a few weeks ago to generate this:
This processed output from APP definitely had some light pollution gradient, but I was able to easily remove that with the LP tool in APP. The image I have at the top of this thread looks very different. Same optical train, different nights.
That then would either be an issue with the flats (did you use different ones as well then?) or the light pollution you had there caused such a gradient perhaps. My guess would be the flats there.
Sorry I missed reading you did. Might still be an issue, but you could give it a try to see what Light Pollution correction does there as well.
Yeah, I tried light pollution correction. After an analysis, I would remove the red rectangle(s) and then run again. Each run generated new red rectangles. I may try another session where I use the Master Flat and Master Bias generator by the photo session earlier in the month, but then use the new darks (proper ISO, exposure and temp) and then re-run processing to see what happens.
I'm just surprised that using the earlier month flat and bias frames plus the new lights and darks didn't result in something similar to the previous result. Same subject (Orion), but I had comments about too much exposure when I did it the first time, so I reduced exposure this time (30s at ISO1000 vs the 90s at ISO1000 I did first time around).
You don't have to remove the red boxes. it doesn't mean it's totally wrong, just that it doesn't fit the model that APP thinks it is at that moment. But if you see it correcting ok, I would leave them.
