APP 2.0 cannot register anymore long focal images (cosmic rays are taken in place of stars)

Hi Philippe,

So it did work fine before you say with this data? Yes, we would like to check the data to see if we can improve on that.

@vincent-mod ok, ZIP file is inside folder

thanks

Thanks! I'll have a look and also forward it to Mabula.

just in case of you ask... CMOS Gpixel Gsense4040 must have a specific calibration (dark, flats and bias) with high and low gain amplifiers. This change nothing to the issue but APP cannot do this calibration itself (we have written a script in another software)

The images I sent are already calibrated.

Ok, so Mabula is going to change the algoritm a bit to see if that may solve this data as well.

Hello CCD2048, with this setting in Tab 3 I could register the images. The rest are standard settings.

If you look at the images individually, I get the impression that the darks do not fit properly. There are two dark spots left at the bottom left. Maybe this causes additional problems because of too many hotpixels in the images. Just a thought. Are the images dithered? Because the integrated image shows banding.

With best regards.

Posted by: @minusman

Hello CCD2048, with this setting in Tab 3 I could register the images. The rest are standard settings.

If you look at the images individually, I get the impression that the darks do not fit properly. There are two dark spots left at the bottom left. Maybe this causes additional problems because of too many hotpixels in the images. Just a thought. Are the images dithered? Because the integrated image shows banding.

With best regards.

Hello

Thanks, I will try with the SII Ha, OIII entire stack together. I changed many parameters but didn't have the idea to lower the number of stars 😪

The dark is fine but the dark spot is result of glow on CMOS sensor in this part. There is also horizontal banding on sensor which is removed by Pixinsight script at the end. Image are dithered (you can see it by blinking all images).

Best,

Hello, I would choose a larger offset for the dithering. Then the rest of the banding is suppressed even better. So if there is still so much sensor glow left after calibration, that is not yet optimal. Does the above script also take into account a pedestal if too many pixels are cut to zero?
Maybe Mabula can add the camera to the list of supported cameras?

With best regards.

Gsense4040 sensor inside C4-16000 moravian camera has a special calibration procedure to correct 4 quads output difference.

Images are taken in 16 bits HDR mode
Bias, dark and flat are acquired twice, in high gain 12 bits and low gain 12 bits (converted 16 bits)
MasterDark High and Low
MasterBias High and low to calibrate flats and generate MasterFlats High and low

Then, HDR images are calibrated like this : from 0 to 3600 adu, high gain master are used, from 3600 to 65535, low gain masters are used.

This works not so bad (not 100% perfect) but it is linked to the sensor technology.

I would really like this could be included on APP option but I cannot ask Mabula to do this as GS4040 is a special sensor few people uses. There is much more powerful tools to develop for APP before this. I can deal with our Pixinsight calibration script we developed then switch to APP for all the preprocessing. 

Our images on : http://www.cielaustral.com/   are not so bad with this sensor on our new CDK24 (610mm).

Best,

philippe

Perhaps Mabula or Vincent will comment on this again. I wouldn't give up hope yet that it will work out with the camera support.

Best regards.

Hi all @ccd2048 @minusman,

Thank you Philippe for sharing your issue and minusman for helping out. I am working on this now for the next release.

In 2.0.0-beta4 a change was made in star analysis to be able to deal better with some datasets, but unfortunately, it is working worse on other datasets we realize now.

Other APP users have reported several issues with star analysis with the post 2.0.0-beta4 releases and all those problems go away with 1.083. So after careful testing, I have now decided that I will roll back this change. This means that we will go back to star analysis like in 1.083 to start with.

Now your dataset here is even not working in 1.083, so I need to see what I can do to make this work as well in the next release. I will report back later 😉

Mabula

Thank you Mabula !

I even tried a wide mosaic (SHORGB) of 12 panels I have never be able to assemble with APP (or any software but didn't try) and this is completely worse with this 2.0 beta. It even cannot detect any stars in Ha panels (where they are a lot, as it is Orion with TEC160 (1.2m focal length). I was expecting the normalization bug could correct the issue I encounted in alod version (since 1.083) as all the mosaic could assemble, but now, the stars are not detected and register gives errors. 

Cheers

Philippe

Hello CCD2048, I would suggest uploading pictures of other filters. So that Mabula can test them in detail. Not that in the end the problem is solved for the OIII filter and still exists for the Ha filter, for example.

With best regards.

One more question about the pictures. Two galaxies can be seen in the first picture, but not in the others. Is there any more information on this?

I will upload some Ha and SII files

For galaxy ??? what do you mean ? Could you show me ? (I don't have my computer here)

Please note, like CCD, some CMOS sensor can have remanence (RBI) for several seconds even minutes. We are doing auto-focusing each time the FWHM change and/or temperature change and also when we switch filter. The telescope will point a brighter star, make the focus, then go back to the field with astrometric precision (even if the L600 mount is a piece of precision). The bright star can leave a remanence trace on the next 1 or 2 images.

CCD Kodak KAF16803 had the same effect.

Ok, I look on image and dates. There was test image just after we took images of a galaxy.

Then, 1 week later, we started all images series. So yes, this is remanence of image taken just before.

Ok, I made a try with my 600 files (200 of each Ha, OIII, SII) and 250 stars.

Register only failed on 3 OIII files. I removed them and the process can continue. 

You can even lower it more as APP will then only look at the brightest stars and not the cosmic rays etc. So Mabula even tested 50 and was able to register them, he's currently working on the algorithm.

Yes, I can deal with that. It worked on 600 files 3 filters and got my 3 stacks

Anyway, the mosaic with shorter focal length was working (register 3500 stars) with 1.083 but not anymore (error on Ha ones). The bug of normalization was impossible to finish the mosaic with 1.083 nor 2.0x (and now stop by register error)

I will open a new topic on this ? or do you prefer we continue on this one ?

If it's completely a different issue, please open a new topic yes. This makes it better to find also for others.

Hi @ccd2048,

I am now able to analyse and register your CDK24 data perfectly with star count set at 500, it finds about 100 stars in each image which registration confrims are almost all stars.

Regarding the mosaic issue that you mention, please open another topic and feel free to already upload it to our server, so I can test it. Which normalization bug are you referring to?

Mabula

Posted by: @mabula-admin

Hi @ccd2048,

I am now able to analyse and register your CDK24 data perfectly with star count set at 500, it finds about 100 stars in each image which registration confrims are almost all stars.

Hi Mabula, great !

Did you use standard settings on actual beta version or did you modify your algorithm (which will be proposed in future update) ?

Regarding the mosaic issue that you mention, please open another topic and feel free to already upload it to our server, so I can test it. Which normalization bug are you referring to?

Mabula

 

This was maybe this bug you corrected in beta 7 (but as register don't work anymore I didn't be able to test)

https://www.astropixelprocessor.com/community/main-forum/mosaic-failing-to-render-correctly/

I will open a new topic and send some files (I have to test some to avoid sending you a lot of files)

Hi @ccd2048,

I modified the algorithm yesterday with some filter technique in star detection which worked very well on your data. But... it did not prove that robust on other datasets unfortunately. So I went another way now... :

In 3) analyse stars, I am adding an option that you can enable/disable : cosmic ray / noise reducer

If you enable it, you will be able to set a lower-limit area size on what you want APP to detect and analyse. On your data, if we set this to 20 pixels for example, all works beautifully, the cosmic rays are mostly structures below this threshold and so they will not be mistaken for stars and the real stars will survive this in your data easily, since the FWHM is about 4 pixels giving star areas larger than 20-30 pixels. By implementing it like this, you should be able to process data like this without further issues and other datasets without that much cosmic rays as yours will not be affected in terms of star analysis results, so I think this is the best and most robust solution for now ;-).

I will show a screenshot later today when this is finished. I intend to release 2.0.0-beta10 with this improvement later today or tomorrow as well.

Mabula

Congratulations CCD2048 on the image of the day on Astrobin.

Is the image of the Helix Nebula a mosaic?

Posted by: @minusman

Congratulations CCD2048 on the image of the day on Astrobin.

Is the image of the Helix Nebula a mosaic?

Thank you !!!

No (mosaic is ORION and is not published yet since 2 year waiting for a solution)

Unfornunately Helix had same issue than 1999 for register because of few stars on it. But 1999 is the worse case.

@mabula-admin Thnak you very much ! Hope to be able to test soon

Hi Philippe @philippe-bernhard,

I am performing some last tests, but as I promised some screenshots of the improvement for your data with cosmic rays:

The stars that are detected and analysed in 3) are allmost all actual stars as you can see in the result of 4) Register whern 90% of them is being registered in starpairs with the reference frame 😉

Hope to release the 2.0.0-beta10 later today when all testing is done !

Mabula

Hi @philippe-bernhard,

Please try 2.0.0-beta11 on your data and let me know if all is okay now 😉 !

Downloads: https://www.astropixelprocessor.com/downloads/

Mabula

@mabula-admin

Many thanks !!! I will test this evening on the full stack