19 May 2020 - APP 1.080 will soon be released with full Fujifilm RAF support, so that will include SuperCCD & X-Trans camera's 🙂 !
2019 September: Astro Pixel Processor and iTelescope.net celebrate a new Partnership!
[Sticky] Creating a Bad Pixel Map
Mabula - some food for thought. I think there have been other people that didn't realize you can load the BPM by choosing it as one of the "darks" in 1) Load. Perhaps it would be helpful to rename the "Darks" button to "Darks + Bad Pixel Map"?
The tooltips for the load buttons for light, flat, dark, and bias currently explain this behaviour:
Would this be clear enough ?
In version 1.045 I have corrected a lot of tooltips and added a lot more documentation in APP using tooltips 😉
Hi Steve, as long as the fits header isn't changed it really shouldn't affect APP from being able to recognize it as a BPM.
If a FITS file is loaded, using any button, the FITS header is examined and if the following similar header line is found:
CALFRAME= 'bad pixel map' / bad pixel map for instrument CANON EOS 1200D
Then it should always be assigned as a BPM by APP.
I have just double checked, the same BPM is loaded three times, after having made 2 copies and changed the names, one with -kopie.fits and the other just randomly named 2.fits. All three are loaded as a BPM for the same instrument/camera as you can see in the screenshot below:
So just changing the filename shouldn't be a problem, if the original BPM was converted to another FITS, then most likely the fits header line is gone and it won't be recognized by APP as a BPM. Could this be the case? If you changed it to a TIF, then it won't work either.
I have created it in such a way that the filename has no influence, the information needed is in the metadata. Maybe you can double check the metadata of the BPM file you refer to ?
OK - I just checked it and you are correct. (Same goes for Bias, Darks and Flats.) I must have done something wrong last time.
Okay, confirmation is always good, thanks for your feedback Steve 😉
Hi Mabula -
I guess what I'm saying is that you have buttons for each of the types of calibration frames, but none of the buttons say BPM. As such, it's unclear how you would load a BPM. So, it doesn't necessarily need to be Darks + BPM - even a separate BPM button would be useful. I just think that something that indicated you can click it and load a BPM would help new users to know that's how you load a BPM.
Hope this helps clarify,
Thanks, yes you may have a valid point here, I'll see if I can add a button for loading of master calibration frames, together with the all/non togglebutton and clean button. That would make it definitely more user friendly. I agree 😉
I have added it to one of my direct priorities, because it's fast to implement.
+1 for BPM button
Thank you so much for version 1.060. What would your recommended workflow be for creating a BPM from 1.060 onwards?
The Bad Pixel Map can be created from one set of darks and flats.
- darks, make sure that you have at least 20 darks (more is better) and that their exposure time is several minutes. The longer the dark current can build up the better the defects can be detected. So for a camera that you normally cool, you could even leave cooling disabled to get good darks for the BPM creation.
- flats, 5-10 flats is fine.
The Bad Pixel Map has no dependency on exposure time and temperture, so it can be used on all your data once you have a good one. You can use a Bad Pixel Map for more than 1 year in msot cases. It depends on the sensor's quality and age. Some sensor's will deteriorate quicker than others.
If you have a sensor that has bad or defect columns, enable the bad column detection.
Let me know if this covers all your questions about a Bad Pixel Map.
Thank you for answering my question. The question was constructed very badly. What I meant to ask was:
Do you recommend that we redo our "old" BPM with 1.061 and if there is a new or different workflow with 1.061? I presume it would be better to redo BPM? For the sensor ageing sake as well as the improved methodology used?
Apologies if the next two questions are really stupid. Over reading, a lack of understanding and poor memory creates loads of confusion:
1. What about gain for the darks and the flats used in BPM?
2. How do I know if I have defect columns in my camera?
Only recreate a new BPM if you think the current one doesn't correct all bad pixels/columns. I ahev been using a single BPM for more than 3 years on Nikon DSLRs. For older and lower quality CCD camera's I think you will need to update the BPM more often though, because they can deteriorate rather clearly.
Workflow for applying the BPM in calibration hasn't changed. APP does a better job of correction the bad pixels though, but will do that equally on older Bad Pixel Maps.
- gain for darks and flats in creating the BPM does not matter significantly. But simply use gain values that ar emore or less the same as you would normally use for the darks and flats. Same applies for the exposure time. For the darks, I do recommend to use darks of several minutes instead of several seconds. The idea is to properly sample deviations of pixels in the dark current as it builds up over time 😉
- You can visually see it in your light frames, this is an example with hot and cold columns:
Let me know if this clarifies things 😉
Thank you so much. All good
does anyone know why this causes a problem? I just createtd a BPM which will not be included in further processing as seen here. what am I blindly overseeing? (APP 1.061, never had this before)
Hi Stefan @elgol,
The BPM is created from the initial raw darks and/or flats, right? And I think this is DSLR data?
The lights that you are trying to calibrate are already cropped from their raw borders most likely and you will need to calibrate the raw data straight from the camera.
For instance, a Canon CR2 (or Nikon NEF) file has raw image dimensions of 7380x4928. You actually only see 7360x4912 since the raw borders of the sensor are cropped away. This always happens with DSLR data.
So if you save a CR2 to a fits, you lose the raw border data. And if you then try to calibrate the fits data with a BPM created from CR2's or NEFs, you will get this problem.
So if you make calirbation masters using DLSR raw frames, you will need to apply them on raw CR2/NEF frames, not on FITS frames.
Let me know if this clarifies your problem.
What happened with the "details" checkbox? I am making a new BPM and want to see the FITS header info, but i can not find an option to read the %hot pixels etc. (I now use Pixinsight to read the FITS header)
Yes, so that's a bit hidden in the new releases, probably needs to be changed as more seem to miss it. You can read the FITS header by clicking on the tiny arrows.
Thanks Vincent, "a bit hidden" is an understatement, let's call it a feature that is now an Easter Egg.... And, why? There is still all this room where the checkbox was, the tutorials refer to the checkbox etc, so why?
Hi, I´m writing from Mexico, I recently bough my first DSLR camera, is an used Nikon D5100 and I´ve noticed after using it on a trip that it has some hot pixels,is an horizontal straigth red line on the left of the camera sensor and it is only visible on some exposures. I´m new on this camera things so I hope that someone hera can tell me if I can solve this problem whit the program and how to do it, I have the factory version firmware.
I can´t find the way to paste images in the comment, so is you can taell me how, I can post the photos here. thanks.
It is not allowed to post attachments on your first post on the forum, but now it should be possible 😉 Below the window where you type your message, there is a button Attach Files. You will need to use that to upload images with your posts.
is an used Nikon D5100 and I´ve noticed after using it on a trip that it has some hot pixels,is an horizontal straigth red line on the left of the camera sensor and it is only visible on some exposures. I´m new on this camera things so I hope that someone hera can tell me if I can solve this problem whit the program and how to do it, I have the factory version firmware.
Okay, do you know how to mark Dark frames ? If so, create a couple and try to create a Bad Pixel Map using those darks and please share how it looks together with one of the dark frames.
Hello Mabula, I am just trying to get a feel for APP and have chosen to create a BPM following your instructions. I have loaded 5 dark and 5 Flats.
I then scrolled down to create Bad pixel Map and selected the box. I am obviously doing something wrong since I do not get the Calibrate option.
This is my first attempt to create Flat frames so could this be the problem with quality. I use a cannon 550 D.
New to APP having just downloaded the trial version. It all looks quite straightforward so I think I must be in 'dim' mode as I c an't persuade it to create a BPM or master calibration files! For the BPM have loaded some 300 sec darks and some flats - lum only. After loading I press calibrate and under the BPM sub menu have hot pixels kappa set to 2 and cold pixels at 50%. I make sure 'BPM' is checked and then click on calibrate again but nothing happens! It must be something very obvious but it's beginning to drive me mad that I can't spot it!!
Hi Pam, no worries we should be able to get you going. Did you have a look at this short video from Sara Wager yet? https://www.astropixelprocessor.com/how-to-create-a-bad-pixel-map-bpm-by-sara-wager/
It might help you. If not, ask away.
Yes I've seen that . The interface is slightly different to the one I have just downloaded. I assume the video refers to an older version - I wonder if this is the problem. Unless I've missed something very obvious - eminently possible knowing me - have followed it carefully. Will have another look though. Have also tried following Sara's video on creating master calibration files and can't persuade this to work either. Am going to have an other look tomorrow and post again if I'm still stuck. Thanks for a quick response.