Black pixel dots in calibrated image

My guess would be that you don't have a high enough offset in your camera settings. Depending on your gain, you need to have enough offset to make sure you are not blocking - cutting off the bottom of the histogram - your image or calibration frames - i.e. your offset must be high enough to keep all camera output data, including noise above 0 ADUs.

Hi @andersstange,

That looks very weird indeed and should not happy normally. Maybe @astrogee has a point, the sensor offset could be slightlhy too low, so you clip pixels to zero in the calibration process.

To be sure, I think it will be best if I have a look at your data as well.

Can you upload it here:

https://upload.astropixelprocessor.com/

username: uploadData

password: uploadTestData

Please make a folder with your name and issue like Anders-black-pixel-holes

and upload your data there and let me know once done. Thanks!

Mabula

Great input! I agree, it is not unlikely that my offsets could have changed for some reason and are now too low. I will look in to that. I have also uploaded the integrated LUM and a subset of the subs together with the calibration masters.

Thanks.

Anders

I have not been able to solve my problem in APP. The problem does not seem to be offset related. The histogram is not clipped as far as I can see and the offset has not changed since I last used this camera setup, one half year ago. My camera in this setup is binned 2x2. To me it seems that there is something wrong with the bad pixelmap generation. I have tried to play around with the hot pixel calibration settings for the BPM and in some cases the black dots has been corrected but far from everywhere.

Also I ran the same set of files through PixInsights Fast Batch Preprocessing and this proiblem did not show up there. No black dots in the integrated images.

Hi Anders @andersstange,

Okay, thank you very much for testing more and checking the offset.

I just checked your uploaded data, but I think that only already calibrated lights (-cal) are uploaded? Can you double check? I would need the original lights and also the individual calibration subs to properly solve this I think.

Thanks,

Mabula

Hi Mabula,

Sorry for the late response, but there was some problem logging in to the forum. To reply I had to login and then I was prompeted to verify my email with a 4 digit code that was going to be mailed to me but I never received anything. I tried several times. Eventually I discoverd that by reloading the APP forum site I could access the reply function without this verification. 

Anyway, I have created a new directory "/AstroPixelProcessorUpload/Anders-NEW-black-pixel-holes" with a set of red channel subs together with the master dark and master bias files.  I also uploaded the generated BPM and the resulting Red image I get when integrating these calibrated subs. In the integrated image(created using the default APP settings), black dots are clearly visible at high zoom eg 225x.

Let me know if you also need the individual Darks and Bias subs(25 of each).

Thanks,

Anders

Hi @andersstange,

Okay, thank you very much.

I have checked the email verification issue, I think that is solved now. It should not happen again.

I will test your data and will get back to you 😉

Mabula

Thanks Mabula

AnderS

Hi Anders @andersstange,

I have solved your issue. It is caused by 2 factors actually.

1. The main issue is that your sensor has clusters of hot pixel groups. The current implementation of the Bad Pixel Map creation is allowing for hot pixels to be together vertically, but not horizontally because of an old issue with CCD sensors. So in the areas where you see the black holes in your integration, several horizontal pixels seems to be hot but the hottest one is not corrected at the moment.
2. Another reason for the black holes in your result is that you did not really dither between your exposures. The dither steps are really small. If you would dither a couple of pixels, the integration would have still looked good allthough the calibration per frame was not correct.

Obviously 1) needs to be corrected to fundamentally solve this issue. I can already create an integration that has no black holes and good calibration per frame:

I will have this fixed in 2.0.0-beta38 😉 Thank you very much for mentioning this problem !

Mabula

Hi Mabula,

Great that you were able to identify the problem. I’m looking forward to the fix😀. Thanks also for your dither input. I will increase my dither size setting right away.

Anders

@andersstange, you are most welcome! Will try to release beta38 in the coming week with the fix.

Hi Mabula,

I just verified the fix in beta 38 with my full data and it works beautifully again.

Thanks,

Anders

Hi Anders @andersstange,

Great, thank you very much for the feedback, that is highly appreciated.

I am glad all is okay now 😉 with the BPM

Mabula