2023-01-19: APP 2.0.0-beta13 has been released !
!!! Big performance increase due to optimizations in integration !!!
and upgraded development platform to GraalVM 22.3 based on openJDK19
We are very close now to releasing APP 2.0.0 stable with a complete printable manual...
Scaling of 12 bit data in 16 bit fits files
I have just started a trial of APP, and I have some data which I will be stacking. This is a mix of narrowband subs, mostly at 300 seconds unity gain, but some at high gain and some at different sub lengths; all have appropriate temperature matched darks and matching flats and flat darks. I love that APP lets me specify both a filter and a session, and I am hopeful that it's going to be able to stack everything correctly (this is going to take a while, it's at 12% complete on calibration so far, so I have plenty of time to write this post ;-)).
I have a question which I can't seem to find an answer to on the forum. My data was collected over about six days with a mix of capture software, including SharpCap and N.I.N.A. My camera has a 12 bit ADC, so the values of each pixel are in the range 0-4096; I always save as FITS. Sharpcap by default scales the images up to 16bit values (effectively bitshift right four, or multiply by 16), so each pixel is in the range of 0-65535 (obviously with a step of 16 between each value). NINA didn't used to scale, so you'd get a FITS file which was a 16 bit FITS but each pixel was in the value 0-4096 on a scale of 0-65535. For some stacking software this doesn't seem to make a difference, for example the excellent ASTAP seems to automatically handle the differences, even say a calibration frame is scaled and a light frame not scaled, or vice versa. However other stacking software (notably Deep Sky Stacker) mandates that all files be either scaled or unscaled (and there is an option in its FITS settings to multiply by 16, which makes a dramatic difference to the stacked output if you are using unscaled FITS).
So, my question is -- how does APP behave? Do I have to scale all of my images up? Will it automagically notice that some images are in the range of 0-4096? Is it looking at FITS headers and doing the appropriate thing? (Actually I'm not even sure whether NINA sets those headers right for unscaled images -- I'm just looking at a file which has values in the range 61-4094 but it has BZERO set to 32768 and no BSCALE header). What should I be doing to make this lot behave correctly with APP?
Hmm...well I messed something up somewhere, I must have missed attaching one set of darkflats. Any chance that APP could tell me which flats are actually missing the matching dark flat so I can see where I screwed up? I clicked OK and assumed it would point it out, but it just carried on calibrating! 🙂
Hi Simon, welcome to the forum and thanks for trying APP!
So regarding bit-depth, I don't think APP minds as long as you have the proper calibration frames of each session. Since you're using different capture software, this is basically the same as "different sensors" funny enough, but that's how you should think about it and you can select the option to reflect this. It's usually best practice to just get the data from the camera, without any upscaling or other processing going on if possible, just to make sure the software isn't changing the data (which happens, depending on the conversion software used, many change the noise levels for instance).
And yes, would be nice to have more warnings beforehand I agree. Nice tip for @mabula-admin.
Oooh, telling it that I have "different sensors" and using the functionality already in place to stack from various setups sounds like exactly what I need -- thank you. I'll need to change some of the darks (I took them in 16-bit scaled) but I can batch process them with fitsworks4 and just have two sets so that each session matches.
In terms of whether or not to upscale -- the vast majority of 12 or 14 bit astro cameras upscale in their drivers, e.g. if you have a 12 bit ZWO or ASI camera then the driver actually gives you a 16 bit image. This actually makes sense, as the FITS file format doesn't actually specify a 12 or 14 bit file format, so you're storing in a 16 bit file anyway. That said, my camera is an Altair, and their drivers give you 12bit data dumped inside a 16bit FITS file -- which is fine if the program processing it knows that. SharpCap for example has an option to leave the raw data as is, "for photometry purposes" (but Robin argue's against doing so here https://forums.sharpcap.co.uk/viewtopic.php?t=644), and NINA used to leave the raw data as is but accepted a change request I made to allow bit scaling like this. It's a lossless operation and simply moves Altair cameras into line with other manufacturers, and means that processing software doesn't go crazy 😉