Stacking pre-calibrated images loses almost all signal

I'll have a look, but want to already say that doing the calibration by APP will be the best course of action. APP is known to be best in class when it comes to that and any issues you now have could potentially also be because of the external process. I'll have a look at the data to see if there is anything obvious at fault.

So the example subs are the raw, calibrated files right? I'll give it a go at stacking and report back.

Ok so I did a quick integration with LNC and MBB, no rejection so that's why the trail shows up. Looks like there is no correct flat calibration actually as I see dust particles in the center.

But, at least it's a lot of signal, I didn't have the problem with complete blackness. One sub versus integration:

I guess I can hop on this topic because I encountered a comparable problem. I collected some first-light data with Ekos and my brand-new ASI294MC (OSC, RGGB) and of course a lot of things went wrong during this very first session (worst mistake that I captured in 8bit only).

However, even if the data are not great I wanted to stack it at least so, took just a few calibration frames to give it a try. The problem is, when I throw my lights, flats, darks and bias into APP I get completely black pre-calibrated images. I used the same settings as when I worked with my Sony raw files what worked fine so far. I do get a master-flat, master-dark and a master bias. The bias frames seem to contain no signal but guess thats a ASI294MC specific issue. I also tried without bias frames and only darks but get the same result - black pre-calibrated lights. Thus all following steps fail because stars etc are lost.

I know the dataset is not really worth it to put more work into it and I basically can discard it but what worries me is that I don´t get any calibration nor stacking done with it and it would be good to know what I´m doing wrong here.

Is it possible to check that so that I can avoid any potential future mistakes? I uploaded all files to my OneDrive: https://1drv.ms/u/s!Ajmg7RJXTfSN7yc3JgHte8PJrXTN?e=SmJE7j  

Thanks in advance and greetings,

Nico

PS: the lights and flats were shot with a L-eNhance Dual-Narrowband filter - just in case that matters?

Well, having no signal in the bias could be an issue, it is sometimes the case that you need to take slightly longer exposures for certain sensors. You can probably find more info online for that. I'll give your data a shot as well.

@vincent-mod

Thanks for the fast reply! I just recovered the setting "create 32-bit Masters" and when I disable that I do get a (very ugly) pre-calibrated image. Can this be the issue? As mentioned above, I accidentally shot in 8-bit OSC only what may causes this problems?

PS: don´t get shocked by the poor quality - it´s my worse dataset ever 😉

Ok, I went through the frames and indeed end up with a black screen as soon as I add calibration files. So this seems to be an issue with those. I took away each master 1 by 1 and it turned out that the masterdark is to blame. Without that I at least get a signal, though not very strong (using your filter you  do have to set "force CFA" and check the algorithm to be Ha-OIII and then any of the combinations you wish to use). I chose Ha-OIII color to keep it simple and without darks I get this calibrated single sub;

Which seems to show the flat isn't working properly either or there is a lot of stray light in the subs.
Integrating all of it I get this;

Now what is wrong with the calibration data is hard for me to say, at least they are 16-bit versus 8 of your lights, no idea what effect that might have though. Maybe the problem lies in different noise levels for your lights versus the 16-bit calibration files, usually though APP warns about that as far as I know.

The masterflat looks overexposed, I see no dust for instance and a wrong masterflat will influence the end-result as no proper calculation can be done with it, reflecting the situation in the lights.

So I took those out as well and got this;

Which is better. Trying to do some light pollution correction on it, gives this;

Still not great, but this lies in the difficulties in the data itself. The light is really affecting the signal a lot.

Little star color calibration and I get this as the end result;

So to sum up;

1. Flats need work to get those right
2. Darks seem to have an issue as well (how did you take those and were you sure they were taken in absolute darkness?)

@vincent-mod

Many thanks for checking on this and pointing out that the darks are the (main) problem! That´s surprising though because I took them at the exactly same settings as the lights and the scope was perfectly closed. The strong glow was expected and is known from these cameras. Basically  I did my dark frames the same way as I did it with my Sony a6000. I have to check that. I already started a (hopefully proper) dark library with 16-bit darks - as soon as I have clear skies I will try a session at proper settings and hopefully get better lights (in 16-bit as well).

That the flats are overexposed... well, that might be. I took advices from some forum discussions about ADU values etc... there are just too many opinions and its confusing when coming from DSLR. So far, I just took my flats against a dimmed iPad showing a grey picture and the only thing I cared was that the histogram doesn´t exceed the lower third.

So, I have to learn a lot about my new camera and do some homework to get used to the ASI294MC Pro 😉

Thank you very much again!

@vincent-mod

PS: I even have to re-think my APP skills... Can´t even get a stacking result that looks anything near to yours. I know that this particular  dataset is (carefully spoken) sub-optimal but my result (without darks and flats) looks extremely bad - compared to your result :-0

cropped screenshot:

Did you do "Force CFA" and pick the Ha-OIII algorithm? Also, the signal looks really weird, I think that is the 8-bit constrained data space coming into play. I made it better by doing light pollution correction in tab-9 as well as color correction.

It's definitely a good idea to read-up on these camera's a lot as their workflow and use can be quite different from DSLR. And you're most welcome.

Posted by: @vincent-mod

Did you do "Force CFA" and pick the Ha-OIII algorithm? Also, the signal looks really weird, I think that is the 8-bit constrained data space coming into play. I made it better by doing light pollution correction in tab-9 as well as color correction.

It's definitely a good idea to read-up on these camera's a lot as their workflow and use can be quite different from DSLR. And you're most welcome.

Yepp, I did the force CFA and also picked the "Ha-OIII color" option... yes, I think the 8-bit was the biggest fail during this night (amongst some others). In the meantime I checked some more settings in Ekos, set them more carefully and did safe some default settings (incl. 16-bit raw)

I guess I just discard this dataset and put it in my "failed-but-learned-folder". I just hope for clear skies and some chance to try again soon. Not that easy here in northern Germany. But it must be possible to get nice results with this camera as it is clearly shown on Astrobin 😀

Cheers!

Oh how I sympathize with that. I started this hobby in the Netherlands and boy did that test my patience during a 3-month non-stop cloud period. My trip to NZ where I lived for a year didn't help either. 😉 But, never give up, the most fun I have when I keep on learning.

@vincent-mod

Thanks for trying Vincent.  I've done some testing, and it seems to be at the 'Normalisation' stage that I see this happening.  I've tried

- Normal and Advanced mode

- add-scale and none

- with MAD and BWMV for scale

- with neutralise background turned off or on

If I save the lights after registration but before normalisation, then the data is still there.  If I save the lights after the normalisation step (even with method set to 'none'), then the files have just a few stars in them and all black (zero pixel values) for the background.

I'm getting this on all images at the moment, so it feels like something odd in my setup.  This definitely used to work fine, and it works OK if I load the uncalibrated frames in and let APP calibrate them.  The reason I'm using calibrated frames is largely that I'm using ACP to capture, and get it to automatically calibrate after taking each sub - saves remembering what flats correspond to what night.

Do you have any suggestions on things I can do to track this down?  It's happening with both the Window and Mac versions (I'm on 1.074.1).

Thanks once more in advance for your help!!

Stewart.

Hmm. OK, worked this one out. I think this may be an APP issue, though one I can now work around :).

It seems to be a bug in normalisation.

-  If I use APP and register the frames, and save them, then loading the file into Pixinsight, and look at statistics, then pixel values are in the range 9.9E-1 to 1.1E-2  (i.e. .011 to .99).  All good.

- If I then Normalize the frames in APP (even with Method set to none, in an attempt to actually disable normalisation) and save, then if I load that file into Pixinsight, pixel values are in the range 2e-5 to 2e-8. 

Pixinsight's default range is 10e-4 for readout of RGB/K image, as it assumes normalisation happens between 0 and 1.  This also causes the screen transfer function to fail other than for the brightest stars.  If you tell PI to rescale the data, then it's still there.

Sorry for the delay here. Great investigative skills! 😉 You do mention that the calibration subs basically cause this issue right, something I also saw a few posts back, so why would this then be a bug in APP itself? What happens if you stack using PI? Rescaling might also point to one of the calibrations frames having an issue with the data as normally that shouldn't be required. As you see the data is still there, I think it has to do with APP calibrating the lights with a wrong noise level background... or something like that.