Problem with Integrate function: limited dynamic range results
Hello, everything seems to be working fine on the steps leading up to 6)Integrate. On previous steps, I have the option to save the data in my Atik 414EX (CCD) native 16 bit type. However, on integrate, the image result is confined to a range of 12 pixel values- which I can't do much with.
black= .07, white = .99
I'm mostly using the default values and experimenting with the settings doesn't get me past this problem. My example uses 12 frames(5 minutes each), calibrated with time and temperature matched dark frames and bias frames only. I've also tried with no calibration frames, just to see if there was a problem in those.
I appreciate a little clue to get me moving forward again- thanks!
Out of curiosity on 2 Calibrate did you check the "32 bit masters" checkbox?
@bsturges Sorry, but I don't understand the problem. APP clearly displays 32 bit data in your second image (the one with the red lines) and not 12 bit. What is the issue exactly?
@wvreeven The issue is that the image is only being represented across a range of 12 pixel values after the frames have been integrated. So there is not enough information there to form a usable image. My 16-bit camera produces images which have a range across most of zero through 65,535. The results are correct for steps up to, and including normalize.
@wvreeven I think the issue is that I don't have a good understanding of what is happening when the data is being converted to a "scheme" I didn't select and where bit depth is defined as .96 bits/pixel. I found that after integrating a color image, I can use the option to save the output as 16-bit FITS. I don't see the advantage of applying data conversion in the processing steps when it cannot add any information to the native format.
@bsturges Could you please upload, say, 5 raw lights and calibration files plus the integration result to our file server? See the upload instructions at the top right of any forum page and create a folder called bsturges_bit_depth and place the files in there. Let me know when the upload is done and I will have a look at what's going on. Thanks!
@wvreeven The file "m63-luminance.fits" is the result after integration. I tried it with the Create 32-bit Masters unchecked also. I get the same results running without the calibration files. The next challenge I want to work on is adjusting the curves in the final color image to show detail without blowing out the nucleus and also dealing with overexposed bright stars. Scale is 1.11 arc second/pixel with my Meade 10" ACF. Thanks for checking this out!
@bsturges Thanks for the upload. I see that the master dark and master bias were created with AIP4Win and not with APP. This in general will not work. Please make sure that your masters are created with APP.
@wvreeven I've removed those two and also the resulting integrated frame and replaced them with MD and MB made with APP. Also used those to make a new 5 frame integration called "M63-luminence-1.FITS".
@bsturges Thanks. Would it be possible for you to upload 5 raw calibration files of each type as I originally requested?
@wvreeven That makes sense- sorry for the confusion! I've uploaded 5 original fits of each type. The "60deg.." is the F ambient temp for the darks, while the setpoint is -15C. The subject was taken at -10C setpoint since my calibration library is outdated. However, the issue is not related to calibration.
Another question I have is about darks. In the past, I've made all frames to match my standard 5 minute integrations. I saw a mention in the tutorials about dark frame matching and the need to make darks a longer time period to give room for the software to match. Can you suggest what time period specifically for APP I should be using when I rebuild an ideal library of calibration frames? Thanks!
@bsturges Thanks for the upload. I processed the 5 files and got perfectly OK results. The master dark, the master bias and the integration result are stored as a 32 bit floating point FITS files as can be seen in the FITS headers of all three. The BITPIX header item is set to -32 which according to section 5.3 of the FITS standard is the correct value for 32 bit floating point values. The values of M therefore are represented to be floating point values between 0 and 1, with 0 completely black and 1 completely white.
The issue is that the image is only being represented across a range of 12 pixel values after the frames have been integrated.
I still don't understand how you came to that conclusion. 12 bit have a range from 0 to 2-to-the-power-of-12-minus-one or 4095 (4096 values including 0). Even in the images that you posted in your comments in this thread I nowhere see values in that range. Can you please explain how you got to this conclusion?
Regarding the darks, there are two options. Either you make sure that the darks match the gain, offset, temperature and exposure time of the lights or you make sure that the darks match the gain, offset and temperature of the lights and that they have a different, standard, exposure time chosen by you. In the first case you will not need bias frames. In the second case you do because APP will then extrapolate the dark signal of your camera to the exposure time of your lights assuming that your camera has a linear sensor (which most cameras do have nowadays).
There is no need to take dark frames with very long exposure times. This may only be necessary if, and only if, you notice that the bad pixel map created by APP doesn't remove all bad pixels. In that case taking very long dark exposures will allow APP to better identify bad pixels that it may otherwise miss. But this is an exceptional case which you can ignore if you don't see any residual bad pixels in the integration result.
@wvreeven I appreciate you taking the time to check this out and explain the results. The reason I came to the conclusion that this image was being displayed across 12 pixel values is because I've been very accustomed to only working with 16 bit images from all three of the CCD cameras I've owned over the years. It seems much more intuitive to understand the histograms over such a wide range of zero to 65,535. I still use AIP4WIN to do some analysis of the images. Even a Google search on 32 bit images left me thinking there was something wrong with the processing. I'll have to spend some more time with the color tools to get better results from my LRGB sessions.