The BPM is created from the initial raw darks and/or flats, right? And I think this is DSLR data?
The lights that you are trying to calibrate are already cropped from their raw borders most likely and you will need to calibrate the raw data straight from the camera.
For instance, a Canon CR2 (or Nikon NEF) file has raw image dimensions of 7380x4928. You actually only see 7360x4912 since the raw borders of the sensor are cropped away. This always happens with DSLR data.
So if you save a CR2 to a fits, you lose the raw border data. And if you then try to calibrate the fits data with a BPM created from CR2's or NEFs, you will get this problem.
So if you make calirbation masters using DLSR raw frames, you will need to apply them on raw CR2/NEF frames, not on FITS frames.
What happened with the "details" checkbox? I am making a new BPM and want to see the FITS header info, but i can not find an option to read the %hot pixels etc. (I now use Pixinsight to read the FITS header)
Yes, so that's a bit hidden in the new releases, probably needs to be changed as more seem to miss it. You can read the FITS header by clicking on the tiny arrows.
Thanks Vincent, "a bit hidden" is an understatement, let's call it a feature that is now an Easter Egg.... And, why? There is still all this room where the checkbox was, the tutorials refer to the checkbox etc, so why?
This post was modified 4 years ago 3 times by Kees Scherer
Hi, I´m writing from Mexico, I recently bough my first DSLR camera, is an used Nikon D5100 and I´ve noticed after using it on a trip that it has some hot pixels,is an horizontal straigth red line on the left of the camera sensor and it is only visible on some exposures. I´m new on this camera things so I hope that someone hera can tell me if I can solve this problem whit the program and how to do it, I have the factory version firmware.
I can´t find the way to paste images in the comment, so is you can taell me how, I can post the photos here. thanks.
It is not allowed to post attachments on your first post on the forum, but now it should be possible 😉 Below the window where you type your message, there is a button Attach Files. You will need to use that to upload images with your posts.
is an used Nikon D5100 and I´ve noticed after using it on a trip that it has some hot pixels,is an horizontal straigth red line on the left of the camera sensor and it is only visible on some exposures. I´m new on this camera things so I hope that someone hera can tell me if I can solve this problem whit the program and how to do it, I have the factory version firmware.
Okay, do you know how to mark Dark frames ? If so, create a couple and try to create a Bad Pixel Map using those darks and please share how it looks together with one of the dark frames.
New to APP having just downloaded the trial version. It all looks quite straightforward so I think I must be in 'dim' mode as I c an't persuade it to create a BPM or master calibration files! For the BPM have loaded some 300 sec darks and some flats - lum only. After loading I press calibrate and under the BPM sub menu have hot pixels kappa set to 2 and cold pixels at 50%. I make sure 'BPM' is checked and then click on calibrate again but nothing happens! It must be something very obvious but it's beginning to drive me mad that I can't spot it!!
Yes I've seen that . The interface is slightly different to the one I have just downloaded. I assume the video refers to an older version - I wonder if this is the problem. Unless I've missed something very obvious - eminently possible knowing me - have followed it carefully. Will have another look though. Have also tried following Sara's video on creating master calibration files and can't persuade this to work either. Am going to have an other look tomorrow and post again if I'm still stuck. Thanks for a quick response.
It is very interesting though, it should be very straight forward and just work. You load in your longest darks, your flats, check the "create BPM" and press "create Masters". You keep the darks and flats selected in the list I assume?
I do wonder what you mean by pressing "Calibrate" now that I think of it. It should be the button in the above screenshot.
I think I've just been following Sara's video by rote instead of using my brain and working out that I should be pressing 'create masters' instead of pressing 'calibrate' at the top - will try that and engage my brain this time, am sure that will work as you say it's quite straightforward!
Have now been able to create a BPM and master calibration files - what a relief! Although APP creates a BPM just using darks and flats it obviously won't calibrate the flats without a master bias. Does this affect the quality of the BPM? Sorry for being a bit dim I can be a bit slow to understand some fundamentals though normally get there in the end!
Oh no problem Pam, we're here to help and it's always better to ask a question then to spend many hours on a simple misunderstanding. 🙂
The BPM is made with darks and flats and it won't create masters when "create bpm" is selected if I'm not mistaken (been a while). It's only creating a masterdark and masterflat for the purpose of the BPM and this is done without any outlier rejections. A proper masterdark and masterflat might benefit from a slight correction so this is why it's not created during this bpm process. The BPM will only consist of hot- and cold pixels which can be found in the darks and flats.
I think it does use them to calibrate the darks and/or flats during the process so I would include them yes (not 100% sure sorry, it's been a while since I created my BPM). Either that or bias frames.
Sorry guys to come back on the BPM creation. We have on the forum videos and explanations showing different processes and settings.
Sara and Mabula use darks and flats. Sara is setting: hot pixels kappa 2, cold pixels 25%. Mabula is setting: kappa 2.5 - 50% hot pixels. Bad columns settings didn't exist at the time of the video and there is therefore no indication on these.
Then niteskyhere in the forum uses darks and darkflats (no flats), plus hot pixels kappa 2.4, cold pixels 25%. He does not check the bad columns settings. He mentions his hot pixels kappa is determined by the percentage of bad pixels he wants to limit to around 3%.
I have tried with flats or darkflats and I don't see (visually) big changes and numbers of bad pixels seem similar. What appears to be the most discriminating is the detection of bad columns. If not selected, I can reach the 3% bad pixels with bad pixels kappa set at 6 (seems high for me, is it?) and 50% cold pixels (attached file1). Now when in addition, I check the bad columns detection, and leaving the bad pixels kappa at 6, the bad pixels % surges at 13% (attached file2)!! I then need to set the bad column kappa at 6 as well to come back to 4% bad pixels (attached file3). I would have to increase more the bad pixels kappa to lower the bad pixels to 3%.
So, 3 questions:
1) Should we use flats? and/or darkflats in creating the BPM?
2) What is the rational for choosing the kappa (1-3% of bad pixels ?) and the cold pixels % (same?).
3) Should we check the Bad Column? (I understand from Mabula's comment that we should do this if we have bad columns... but I don't know if I have some). And what is the rational for choosing the kappa? Should we tweak these columns kappa independantly of the bad pixels kappa or together?
Sorry for this long post and thanks in advance for clarifying!
Also here, @mabula-admin will probably give an in-depth answer. But to answer quickly;
1. Yes, but they are only used for cold-pixel detection, which modern sensors rarely have anymore. So that won't affect the BPM much. Dark-flats are essentially bias subs, but with a slightly longer exposure. Might be some camera's need this. The longer exposure probably doesn't have many hot pixels, just the bias signal mainly which is definitely what you need.
2. Here you set what you think, based on results usually, is the threshold for APP to look for hot pixels in the darks and cold pixels in the flats (based on a percentage of the median signal in the flats). If you see that the BPM is correcting pixels that are not hot-pixels, you might want to change this setting. It'll vary per sensor, standard settings tend to be ok though.
3. Same here, the kappa will depend on your sensor and data, a lower value will detect (or if set too low, maybe detect columns that aren't actual bad columns) more columns, higher less. It's going to be an experiment for your data to see what works and what not. Again, usually the default is fine. If you're not sure you have bad columns, you probably don't have them. 🙂 I don't think it'll harm when you select it.
1) as you say, we have to set the kappa level to the point where no good pixels are wrongly removed and (reversely) no bad pixels stay in the final image. I am not sure though of being able to say how to check this (this is a question from a relatively newbie and seasonned users probably can notice that easily!). As you mentionned, if I have not noticed them I probably don't have them!
2) In my BPM (whichever version above), the amp glow from the ASI294MC is identified as bad pixels. That is probably why I have to raise the kappa level much higher than the recommanded 2 to 3 because the pixels forming the ampglow are many and with these values it announces 11-15% of bad pixels! Shouldn't the Amp Glow be corrected through the darks? Should I then keep the kappa level lower (2 to 3) even if it announces a high level of bad pixels (we are now back to point 1 I believe)?
1) Well, it's just a matter of looking at your data at 100% zoom and looking around. If the calibration worked well, there should be no or extremely little left of hot-pixels (bright red or bright blue). If you don't see them, it's fine. You should ideally see a nice, random noise pattern that remains even after all the calibration, no weird regular things line stuck pixels or entire columns.
2) Same here, just use the BPM with the bad pixel columns and without and test both. I have no experience with amp glow in this case so I'm not sure if the BPM can better be used without, but as far as I heard it should work. It's basically also bad pixels as it's fixed noise added to your subs.
Try it first without changing the kappa, they should be fine in most cases.
Sorry if I overread it, but I understand the BPM as following:
I´ll take today some darks (30 - 50) and flats (30 - 50) with my modded Canon 1200Da and my Samyang 135mm. In the next step I´ll create a BPM with your wonderful software and in the future I do not have to take any more darks/flats?
With further integrations I´ll only proceed with those steps:
1.Creating the MasterBias
2.Load lights, add Masterbias and BPM
3.chill?
Is this restricted to some camera lenses or telescopes?
I have just tried this, 5 Dark and 5 Flat but I always get this message. What am I doing wrong or is it "normal" and does not need to be taken into account? I have the latest APP version 1.079 Thanks and compliments great software
I have just tried this, 5 Dark and 5 Flat but I always get this message. What am I doing wrong or is it "normal" and does not need to be taken into account? I have the latest APP version 1.079 Thanks and compliments great software
As the warning indicates, you need a bias or dark flat to properly calibrate flats.
Sorry, I missed your question. If your camera has no amp-glow, you can use the BPM in place of the darks. Flats however are always needed as they also correct dust and any changes in light across the frame, the flats used in a BPM are just there for detecting any cold pixels (if you have them at all).
Hello Vincent, Thank you for your answer. Yes, but I just want to create a BMP and not calibrate flats. In Mabula's instructions, he only used darks and flats.
It should be. Best practice is to load a dark with as high of an exposure as possible. You can check what the BPM is doing by loading in a light, zooming (100%) to a field with hot pixels, and switching on "l-calibrated" on top.
