2019 September: Astro Pixel Processor and iTelescope.net celebrate a new Partnership!
[Sticky] Creating a Bad Pixel Map
It is very interesting though, it should be very straight forward and just work. You load in your longest darks, your flats, check the "create BPM" and press "create Masters". You keep the darks and flats selected in the list I assume?
I do wonder what you mean by pressing "Calibrate" now that I think of it. It should be the button in the above screenshot.
I think I've just been following Sara's video by rote instead of using my brain and working out that I should be pressing 'create masters' instead of pressing 'calibrate' at the top - will try that and engage my brain this time, am sure that will work as you say it's quite straightforward!
Thanks for the response
Have now been able to create a BPM and master calibration files - what a relief! Although APP creates a BPM just using darks and flats it obviously won't calibrate the flats without a master bias. Does this affect the quality of the BPM? Sorry for being a bit dim I can be a bit slow to understand some fundamentals though normally get there in the end!
Oh no problem Pam, we're here to help and it's always better to ask a question then to spend many hours on a simple misunderstanding. 🙂
The BPM is made with darks and flats and it won't create masters when "create bpm" is selected if I'm not mistaken (been a while). It's only creating a masterdark and masterflat for the purpose of the BPM and this is done without any outlier rejections. A proper masterdark and masterflat might benefit from a slight correction so this is why it's not created during this bpm process. The BPM will only consist of hot- and cold pixels which can be found in the darks and flats.
Sorry guys to come back on the BPM creation. We have on the forum videos and explanations showing different processes and settings.
Sara and Mabula use darks and flats. Sara is setting: hot pixels kappa 2, cold pixels 25%. Mabula is setting: kappa 2.5 - 50% hot pixels. Bad columns settings didn't exist at the time of the video and there is therefore no indication on these.
Then nitesky here in the forum uses darks and darkflats (no flats), plus hot pixels kappa 2.4, cold pixels 25%. He does not check the bad columns settings. He mentions his hot pixels kappa is determined by the percentage of bad pixels he wants to limit to around 3%.
I have tried with flats or darkflats and I don't see (visually) big changes and numbers of bad pixels seem similar. What appears to be the most discriminating is the detection of bad columns. If not selected, I can reach the 3% bad pixels with bad pixels kappa set at 6 (seems high for me, is it?) and 50% cold pixels (attached file1). Now when in addition, I check the bad columns detection, and leaving the bad pixels kappa at 6, the bad pixels % surges at 13% (attached file2)!! I then need to set the bad column kappa at 6 as well to come back to 4% bad pixels (attached file3). I would have to increase more the bad pixels kappa to lower the bad pixels to 3%.
So, 3 questions:
1) Should we use flats? and/or darkflats in creating the BPM?
2) What is the rational for choosing the kappa (1-3% of bad pixels ?) and the cold pixels % (same?).
3) Should we check the Bad Column? (I understand from Mabula's comment that we should do this if we have bad columns... but I don't know if I have some). And what is the rational for choosing the kappa? Should we tweak these columns kappa independantly of the bad pixels kappa or together?
Sorry for this long post and thanks in advance for clarifying!
Also here, @mabula-admin will probably give an in-depth answer. But to answer quickly;
1. Yes, but they are only used for cold-pixel detection, which modern sensors rarely have anymore. So that won't affect the BPM much. Dark-flats are essentially bias subs, but with a slightly longer exposure. Might be some camera's need this. The longer exposure probably doesn't have many hot pixels, just the bias signal mainly which is definitely what you need.
2. Here you set what you think, based on results usually, is the threshold for APP to look for hot pixels in the darks and cold pixels in the flats (based on a percentage of the median signal in the flats). If you see that the BPM is correcting pixels that are not hot-pixels, you might want to change this setting. It'll vary per sensor, standard settings tend to be ok though.
3. Same here, the kappa will depend on your sensor and data, a lower value will detect (or if set too low, maybe detect columns that aren't actual bad columns) more columns, higher less. It's going to be an experiment for your data to see what works and what not. Again, usually the default is fine. If you're not sure you have bad columns, you probably don't have them. 🙂 I don't think it'll harm when you select it.
1) as you say, we have to set the kappa level to the point where no good pixels are wrongly removed and (reversely) no bad pixels stay in the final image. I am not sure though of being able to say how to check this (this is a question from a relatively newbie and seasonned users probably can notice that easily!). As you mentionned, if I have not noticed them I probably don't have them!
2) In my BPM (whichever version above), the amp glow from the ASI294MC is identified as bad pixels. That is probably why I have to raise the kappa level much higher than the recommanded 2 to 3 because the pixels forming the ampglow are many and with these values it announces 11-15% of bad pixels! Shouldn't the Amp Glow be corrected through the darks? Should I then keep the kappa level lower (2 to 3) even if it announces a high level of bad pixels (we are now back to point 1 I believe)?
1) Well, it's just a matter of looking at your data at 100% zoom and looking around. If the calibration worked well, there should be no or extremely little left of hot-pixels (bright red or bright blue). If you don't see them, it's fine. You should ideally see a nice, random noise pattern that remains even after all the calibration, no weird regular things line stuck pixels or entire columns.
2) Same here, just use the BPM with the bad pixel columns and without and test both. I have no experience with amp glow in this case so I'm not sure if the BPM can better be used without, but as far as I heard it should work. It's basically also bad pixels as it's fixed noise added to your subs.
Try it first without changing the kappa, they should be fine in most cases.