Adaptive pedestal removal needed for bias frames

Mmm, it would occur strange to me if that should be a thing with bias frames and Mabula missing that. With which program do you take the images? And can you maybe show the masterbias and effect it has on the image? That might give a clue.

The images are taken manually with a trigger cable or pushing the button. I don't have capture software at the moment - The Pentax is a good camera but a pain in the a**

Attachements are working now...

Below is a normal sub and then with the master bias applied. As you can see its clipped. I used Nebulosity to generate the fits from DNG. Neb is working consistently. With it I was able to see that the bias noise (directly from the Bulb generated DNG) at about 0-20 but there was a pedestal of about 500 for the manual exposure, so bias noise was at about 500-520.

Ok, thanks for showing. So using Nebulosity is that required to get the images loaded in APP? If so it could potentially be that using a different program causes an issue. It then all depends on how that program changed the data.

Yes I use Nebulosity to batch convert the DNG files to FITS. I doubt Nebulosity is causing the bias - it’s one of my most trusted software 🙂 but I’ll have to verify later using rawtherapee. 

Ah assumption is... 😉 It's definitely a nice program but the best is to let APP do all the work as it could influence the specific calibration calculations its doing.

Posted by: @vincent-mod

Ah assumption is... 😉 It's definitely a nice program but the best is to let APP do all the work as it could influence the specific calibration calculations its doing.

I too have a Pentax that I use astroimaging, (K-1), unfortunately APP does not yet have support for .DNG files nor Pentx RAW (.PEF)  Pentax can save RAW in either format. 

Posted by: @vincent-mod

Ah assumption is... 😉 It's definitely a nice program but the best is to let APP do all the work as it could influence the specific calibration calculations its doing.

You may be right. It looks like the manual mode files at 1/8000s are better than the bulb mode according to these RawDigger images. Nebulosity was the only software I could find to convert DNG to fits. It would be great if APP would import DNG but since it doesn't, can you recommend a converter?

Excellent! So DNG should be supported in the next version of APP. In the mean time, not sure which converter would be best. Maybe Pixinsight, but that's not free..

Posted by: @vincent-mod

Excellent! So DNG should be supported in the next version of APP. In the mean time, not sure which converter would be best. Maybe Pixinsight, but that's not free..

Any Idea when the next version is due? I can hardly wait! 😉

I don't have that knowledge unfortunately, but it shouldn't be months I'm guessing. 🙂

I knew Nebulosity had to be OK. I looked again with RawDigger and found out it was auto-subtracting black levels. I disabled this feature and you can see clearly the frame taken with high speed bursts in manual mode have a black pedestal. I have no idea why Pentax would do that but it is there. Attached is bulb mode shot and then burst mode with black level removal disabled:

Well that's very interesting! No idea if that would be controllable in the camera settings somehow? I'll ask @mabula-admin his opinion, thanks a lot for this detective work!

@vincent-mod

No, I can't control it in the camera. But I'm writing again because this issue is more complicated. When doing flats, I cannot use bulb. I need to use manual or preferably auto exposure. Here, once again I have a black level pedestal of 500.

So when the images are processed, they really need to be preprocessed to take care of these kind of issues. Hopefully, when DNG processing is supported, this stuff will be automatic.

Attached is screen shot showing a flat with the offset:

I've messaged Mabula to take a closer look, it's a very specific case and needs an extra pair of eyes. 😉

Posted by: @vincent-mod

I've messaged Mabula to take a closer look, it's a very specific case and needs an extra pair of eyes. 😉

Thanks Vincent! yes I hope this can be fixed in the next updated because its starting to be difficult to make flats. I'll have to take them and get the exposure right in bulb mode - I have no software that will remove the pedestal.  With DNG support and using the DNG exif data properly a lot of DSLR issues could be resolved to the majority of DSLR users satisfaction - this goes for the Colour Conversion Matrix (CCM) too!

Hi Vincent,

Sorry to keep beating this horse but I've discovered that the black level is not exposure mode dependent - just seemed that way. I seems to be exposure dependent and it not always the same value. Whatever the reason for the black level, this issue is now as show stopper for me with a Pentax and DNG files. Since Nebulosity doesn't remove the black level when converting to FITS, I'm stuck without a way of doing flats now that work with APP. (makes sense to me now that calibration has been inconsistent.)

I hope DNG processing can be put higher in APP priorities. Thanks.

Posted by: @astrogee

Hi Vincent,

Sorry to keep beating this horse

No worries, I'll update our spam filter. 😉

but I've discovered that the black level is not exposure mode dependent - just seemed that way. I seems to be exposure dependent and it not always the same value. Whatever the reason for the black level, this issue is now as show stopper for me with a Pentax and DNG files. Since Nebulosity doesn't remove the black level when converting to FITS, I'm stuck without a way of doing flats now that work with APP. (makes sense to me now that calibration has been inconsistent.)

I hope DNG processing can be put higher in APP priorities. Thanks.

It already is. A camera someone owns and isn't supported yet always needs to be put above all else. 😉 That's totally understandable, but it's always a code dance Mabula has to perform around the most pressing features and issues. DNG support though is coming in the next version as far as I know as it's a much requested feature and also something Mabula wanted to incorporate asap.

Posted by: @astrogee

Hi Vincent,

Sorry to keep beating this horse but I've discovered that the black level is not exposure mode dependent - just seemed that way. I seems to be exposure dependent and it not always the same value. Whatever the reason for the black level, this issue is now as show stopper for me with a Pentax and DNG files. Since Nebulosity doesn't remove the black level when converting to FITS, I'm stuck without a way of doing flats now that work with APP. (makes sense to me now that calibration has been inconsistent.)

I hope DNG processing can be put higher in APP priorities. Thanks.

Hi @astrogee & @vincent-mod & @craig_astronomy4you ,

A camera that will not give a consistent bias pedestal / black level, but varies this depending on exposure will give all sorts of issues when you want to use that camera for astrophotography including the very vital step of data-calibration.

Having said that, I would be surprised if this actually is the case with your camera. How do you know that the black level depends on exposure time ?

Have you created darks with different exposure times? Do they show the histogram peak at clearly different values? Assuming Nebulosity does the converting to FITS and is not subtracting anything.

I will work to have DNG support included in the next release 😉

Cheers,

Mabula