[Sticky] Creating a Bad Pixel Map

WOW what a great function !!!! I have always wondered what that is good .. I always tried it on the stacked picture and in full view .. but never in 100% view and on individual lights.
Great thank you!

It is a really nice feature yes, you can play around and check the effect before stacking. Same with flats applied or not.

Yes, so you can see if the flats are "working"
Just experimenting with it, which raised the question of what happens when I select "L-C-Registered" ?? Registering means aligning several lights with each other exactly, or am I confusing something? But I can only display 1 picture at a time. And I can see no changes between "L-Calibratet" and "L-C-Registered".
And what does "Normalized" or how do I normalize a light? I have already changed everything under 5) Normalize but I see no difference between Registered and Normalized.

L-C-registered is useful after registration is done. You can then go through the frames 1 by 1 to check if that worked. Or after registering a mosaic, checking what it did, change that if required and check again.

@vincent-mod

Madness! Thanks for your help. You helped me a lot.

Is it useful to create multiple BPM for different iso settings of a DSLR?

I have two session of imaging with a canon EOS-6d, one at 3200iso, one at 6400iso

For each one i have acquired flat, bias, dark, so i create two set of masters. I use multi-session processing and APP let me to assign each master to a specific session, but not for BPM. APP seems to accept and use only one BPM: when i open a BPM file APP assign this file to all lights, and does not ask for which session.

Is that correct?

Good question, I don't think so no. A BPM will never be harmful for your data, so take the longest dark with the most noise to make it and it should work.

I have a question regarding the creation of the BPM. I've read that it requires the use of Dark frames and Flat frames, and that once created, it can be used many times (~ 1 year) before needing to create a new one. My question...Dark frames are typically taken after the Lights using the same exposure parameters under the same temperature conditions. But if the BPM file can be used over and over again, this would imply that exposure length and temperature condition is not important when creating the Dark frames needed for creation of the BPM. Is this correct? If so, it would also seem appropriate to use a very long exposure in order to heat up the sensor, making bad pixels more apparent - does that make sense?

Thanks

The creation of a BPM isn't depended really on the conditions indeed. Ideally you take a few really long darks to get all the possible noise that can occur on your sensor. As a BPM is never destructive this will get you a better BPM.

Hello ,

is it correct that in version 1.082 the BPM is always created and automatically applied?

CS Karl

Yes, a BPM is now created automatically if the correct masters are available. You can copy/move that BPM to a folder where you also store your darks if you want, darks and a BPM can be used for multiple sessions. A BPM even for at least 6 months.

Thanks !

unfortunately I cannot load and see a finished stack result fits another session to test this BPM?

still V8.02..... do I have to delete all flats and darks first to load a finished stack?

You can load the BPM and just another single light, zoom in to the pixels on the BPM, then double-click the single light to load it in and switch between the normal view and "L-Calibrated" from the drop-down menu on top of APP. L-calibrated will show you what your calibration masters will do with that single light. It's a fast way to check if your calibration is working nicely.

sorry Vincent , I made a mistake, it works fine

But I have again the question: does it really make a difference to use BPM and darks, bias and flats?

Yes, it does. It will depend a little on the sensor, but when you zoom in just a little and then apply all those calibration files you'll see that difference yourself with the method I described above. It will remove a lot of the fixed pattern noise in your data. Flats will probably have the biggest effect of them all (when created properly which can be a bit of a challenge sometimes). You can correct for vignetting and dust with those. When properly calibrated you'll be able to stretch a bit more, get a nice signal out of the data etc.

Thanks Vincent !

I will try it !

Hello again,

Now I have one more question:

if the BPM is always created automatically anyway Is it still worthwhile to create a special one with, for example, a longer Darks exposure time? Does this then have to be exchanged for the automatically generated one?

CS karl

Best is to use a BPM with something like a long exposed dark and even a good flat (if you have any cold pixels) and loading that in before starting to process. No BPM will be created then but the one you provided is used.

I am just starting up the learning curve for Astrophotography and APP and started by making a bad pixel map for my Canon EOS R.  I was a bit surprised to see 2.9% cold pixels given this should qualify as a recent sensor.  I have 1.8% hot pixels which seems fine.  I made a white T-shirt flat at ISO 100, 1/250 s (50x), bias at ISO100 1/8000s (50x), and black at ISO100, 300s (50x).  I loaded into APP and created masters using the default settings (hot pixel kappa 3 and cold pixels 10%).  I think what I am getting may be some kind of CR3 artifact since the image size for APP is 6888x4546 which is quite a bit larger than nominal 6741 x 4494.  The bad pizel map seems to have cold pixels along the top and left hand side which roughly account for the 2.9% as can bee seen in the screen shot.  It appears the BMP is masking out some part of the sensor that isn't functional?  Since I am new to this I want to be sure since the BMP will be used for every image going forward and I'd hate to have messed it up somehow.

FYI screen shot was from a second attempt using cold fraction 50% just to see if it made a difference (it didn't).

Canon cameras have a region to the top and left of the sensor that don’t collect light and serve as a way for the camera to determine the black point of the sensor. Normally these regions are not read out and in your case account for the larger image size.

APP processes the images as they are and doesn’t introduce any additional pixels. What software did you use to take the images? I’d suggest you talk to the manufacturer of that software to discuss this issue.

APP indeed does take the entire sensor data, this shouldn't cause issues normally though. If you see it does, we'd definitely like to see. 🙂 Also, a BPM will not be destructive to the data, so it shouldn't add artefacts (like create black pixels because the BPM has a hot pixel and the light not). Best is to zoom into a light with pixels, hot or cold, and switch to "l-calibrated" on top of the preview window with a BPM loaded. This way you can check if the BPM is working properly.

Hello everyone,

I am currently struggling to prepare a BPM for my ZWO ASI294MC pro camera. I tried following the instructions by Mabula in the first post, but couldn't achieve a satisfying result yet.

An issue seems to be the amp glow - the ASI294MC is known to suffer from it quite a bit. When I prepared the BPM using 5min darks (which nicely show the amp glow pattern), the BPM will essentially include the amp glow pattern as well, which leads to a very high density of hot pixels there (the "main beam" of the amp glow, which is about 5 - 10 pixels in width, is 50% hot pixels, it is actually interesting to see that the BPM has columns of hot pixels alternating with columns of regular pixels along the beam). Since APP will replace hot pixels by a mean value from the surronding pixels, the light image suffers severly in the amp glow region (I will attach an image later when I have the rights to do so (first post)).

I tried using shorter exposure darks while heating the camera to 40°C, so that dark current noise is maximized. I got subs with amazingly many hot pixels (about 10% at kappa=3) and without noticeable amp glow, which looked good to me. However, when I create the BPM using these darks, the main aim of the BPM (removing essentially all hot pixels in light images) is still not achieved, and I don't understand why.

For testing whether or not the BPM works correctly, it is suggested to use the "l-calibrated" image viewer mode. I would assume that if I loaded one of the darks used for creating the BPM as a light and then check it, most of the hot pixels in that image should be earased when viewed in "l-calibrated" mode. Indeed, some hot pixels are cancelled, but so many remain that I doubt this is what can be achieved. The same holds when I check using a "normal" light frame. 

I am wondering if I am doing something wrong in checking if the BPM works correctly. What I currently do is that I load a light frame and have the created BPM assigned to it. I deselect all the darks/flat frames and only leave the light and the BPM selected, then switch between "image" viewing mode and "l-calibrated" viewing mode (with DDP turned of) to see if the hot pixels get cancelled. Is this the correct workflow or am I missing something??

I will continue to try some different settings for creating the BPM, but in the meantime I would be happy for any advise you guys can give me on this topic. It would be specially interesting to hear what other users with cameras having amp glow experienced when creating the BPM and/or how they dealt with it.

Cheers,

Marcel

Thanks for the detailed explanation. If I'm not mistaken, a BPM shouldn't harm the data. So if there are pixels in it that are not in the light, it shouldn't matter as far as I understand it. So just to clarify, you used standard settings in APP and you have uncalibrated darks to create the BPM right?

Oh and one other thing, to increase the number of hot pixels detected by APP for the BPM, you can play around with the Kappa values. In tab 2 (APP version 1.082 and further) you can switch "create BPM" to "enable" and choose your own kappa values (try to lower it). Hover over the buttons to see more information on what to do there. I would use slightly higher temperatures and shorter exposures first in that case. This takes a bit of experimenting, but when you have a good one you can use this BPM for at least a year probably.

Hello Vincent,

thank you for your replies. I played a bit more around and it seems to be actually working not bad.

As promised, I provide below an image showing the problem due to amp glow. Looking at the BPM in the amp glow region, I am not surprised that the calibrated image suffers, considering that the pixels which are removed based on the BPM need to be populated some way. 

Avoiding the amp glow issue:

Using shorter exposures while at the same time heating the camera to high temperatures, I obtained dark frames that do not really show an amp glow signal. The following image shows an example of a BPM created using those darks. I believe the result is actually good enough to work with (settings for creating the BPM were kappa=3, coldpix%=10, resulting in close to 6% hot pixels). As is shown in the test using a dark frame, it actually removes most of hot pixel succesfully (compare bottom left/right images). What's also interesting to note is that the BPM appears to have a "frame" of around 10px width where there is no hot pixel data (the effect of this can be seen in the calibrated dark frame (bottom right), which shows many residual hot pixels on the top and right edges).

Finally, I used the BPM on a number of normal light frames and found that, while removing hot pixels, some artifacts are created in some of the light frames (see image below). I don't believe this is a problem that would actually show up in an integration of several light frames. Still, it may be worth considering what causes these artifacts and if they can be avoided by changing the algorithm that fills up the deleted pixels.

I hope this information is helpful for other people having trouble in creating BPM's.

Cheers,

Marcel

Thanks a lot for that, this is really valuable to us as well. I'll ask Mabula as to why this last effect might be as my understanding indeed is this shouldn't happen with a BPM. But I hope it does work out for you now in the integration! Thanks again.

Sorry if this has been asked and answered but:

• Does it matter which flats you use, as they would be potentially taken with different filters?
• What if you have amp glow, which I've seen the BPM pick up as hot pixels to some degree? Amp glow is generally stronger with longer darks, so it would seem the longer darks you toss is would flag more pixels as hot (that aren't really) and not be great if used in an integration with exposure times that didn't have as much of it. I see this is recently discussed.
  

I'm seeing something similar with my amp-glow infested BPM. The left image is uncalibrated, the middle is MD/MF/MDF only, and the last is also adding in the BPM. The smudge-like artifacts are greater in the image using the BPM.

Posted by: @neverfox

Sorry if this has been asked and answered but:

• Does it matter which flats you use, as they would be potentially taken with different filters?

Yes that matters a lot. If you take, for instance, flats with an R filter and use them on B lights then the response of the sensor will be different which will introduce all kinds of artifacts in the calibrated image. Also the properties of at least a part of the optical train may differ when using different filters, especially small band versus broad band. In short: always take flats with the same filter as the lights that you want to calibrate.

I wonder.. If you shoot light frames with a cmos, in 2 x 2 binning mode, must darks and flats so be taken with the same binning to create a BPM?

- John