Version 1.062 Errors amnd Calibration

Hello Mabula, I have now bought a new ZWO ASI294MC Pro Camera.
Wanted to create a bad pixel map now, but that does not seem to work properly. There are always two created. I suspect it's related to the Fits header. In details, the camera name is displayed correctly and in the
Framelist, but the recording date appears instead of the camera name.

Second, I have stability problems with Save Calibrated Frames or
Analysis stars sometimes crash Windows Explorer and APP will simply quit. The version 1.061 had a similar behavior, with which the difference ran the APP further.

Does your BPM issue look like mine below?

I reported this a few weeks ago and am currently using a BPM generated previously for the same camera using 1.061.  I guess that as you have anew camera, you are not able to do the same?

Dave

Have exactly the same problem, have already tried with an older version of APP to create a Bad Pixel Map but with the same result. 🤔  🤨 

Posted by: minusman

Hello Mabula, I have two issues with version 1.062. Memory will not release with time and after tasks have finished. What about Java error and crash. On the other hand, I can not properly process images taken with the Astronomik CLS Filter. The oversteer. It was wonderful in the previous version.

Hi @minusman,

Apologies for the late response on your questions.

APP 1.062 was changed internally regarding memory usage and not releasing memory while processing actually increased performance in testing. But I think I will need find a middle ground here. So in APP 1.063 I will make sure that memory is released again while processing, so APP won't consume all memory in due time.

On the other hand, I can not properly process images taken with the Astronomik CLS Filter. The oversteer. It was wonderful in the previous version.

I don't think I understand what you refer to here. What do you mean with "oversteer" ? Could the problem be related to how the data is shown in APP, like a problem in the preview filter perhaps?

Mabula

Posted by: minusman

Hello Mabula, I have now bought a new ZWO ASI294MC Pro Camera.
Wanted to create a bad pixel map now, but that does not seem to work properly. There are always two created. I suspect it's related to the Fits header. In details, the camera name is displayed correctly and in the
Framelist, but the recording date appears instead of the camera name.

Hi @minusman,

Indeed, this is a problem with the Fits Header and how APP reads this. I think this is already fixed in the current APP 1.063 beta, but to be sure, can you send me one of those fits files. I will then make sure that it's fixed ;-). After that, only 1 BPM should be created with a correct instrument name.

Second, I have stability problems with Save Calibrated Frames or
Analysis stars sometimes crash Windows Explorer and APP will simply quit. The version 1.061 had a similar behavior, with which the difference ran the APP further.

Okay, could it be related to the Operating System running out of memory ? I have never had APP just quit or the explorer crash so I need more information especially on OS level if you could share that?

( Chances are that the issue is actually on the OS level, i have never had a problem like this.)

Mabula

Posted by: Jonesdee

Does your BPM issue look like mine below?

I reported this a few weeks ago and am currently using a BPM generated previously for the same camera using 1.061.  I guess that as you have anew camera, you are not able to do the same?

Dave

Hi Dave @jonesdee and @minusman,

Indeed, there is a problem here in how APP reads the FITS metadata causing the instrument tag to have read the shot date.. I think I have fixed this for next release, but if one of you or both could send me one of the frame, I will make sure it's fixed.

Mabula

Posted by: minusman

Have exactly the same problem, have already tried with an older version of APP to create a Bad Pixel Map but with the same result. 🤔  🤨 

Hi @minusman,

I think I have found the source of the error, the fits interpreter in APP was too flexible in reading certain Fits header tags and it should be fixed in APP 1.063.

One of your frames should help me as well to guarantee it's fixed 😉

Mabula

Yes, the problem with the frames that were taken with Astronomik CLS Filter, seems to be on the preview filter. If I manually adjust settings (stretch) the preview will look good.

I sent frames and the OS configuration to support@astropixelprocessor.com.

I've tried several different ways to set bios to default. Video driver completely reinstalled. Software that does not need to be uninstalled uninstalled. Windows 10 updates up to date. What I've noticed, it's only since the Meltdown and Specter theme. Maybe it is related to it.

Hi Dave @jonesdee and @minusman, 

Thank you minusman for sending the files so I can test. The issue with the shot date being reported in the instrument column indeed is already fixed for APP 1.063.

APP 1.062:

APP 1.063:

So that should solve the issue with Bad Pixel Map creation as well.

I will now look at the preview filter problem fot the Astronomik CLS Filter.

Kind regards,

Mabula

Hi Mabula - I have just sent the 5 BPM files generated in the example I posted above, by WeTransfer 🙂

Regards

Dave

Hi @minusman,

I have located the source of the problem with previewing your Astronomik CLS Filter data.

If the data is not calibrated, all goes well right ?

As soon as we calibrate the data, APP isn't showing a correct preview now. It's over stretched... the screenshot shows the calibrated data with the l-calibrated image viewer mode:

Now the source problem is that after calibration, you are severly clipping the left side of the histogram, the blackpoint. If I adjust the preview fitler manually, we can clearly see this:

Off course, I need to apply a fix in the preview filter, so at least the data is shown better with the previewfilter, I will implement that now.

Luckily, I have an immediate nice fix for you, which should also improve your integration result. Since you are clipping the blackpoint, please enable the new calibration setting in 2) when calibrating your data:

enable Adaptive Pedestal/reduce Amp-Glow

This will adaptively add a pedestal to your light data, so no black clipping occurs in calibration:

The calibration and histogram now are better and the preview filter will work since there is no black clipping ;-). Please look at the values of the B(lack) slider. Previously this was at 0 and data was clearly clipping it. With the adaptive data pedestal this is no longer the case.

Left: calibrated data with adaptive data pedestal enabled and working preview filter

Right: B(lack) set at 0 to show no black clipping occurs after calibration.

I will correct the preview filter 😉 to cover this exception.

This issue with black clipping happens because the exposure time in combination with the CLS filter is too short. Too little light is captured, especially in the red channel. The filter blocks a lot of light in the red spectrum.

With the Adaptive Data Pedestal there is a good solution at least 😉 but I would recommend to increase the exposure time per sub if possible. The data now is a bit weak and the red channel will probably suffer when compared to the green and blue channels.

Kind regards,

Mabula

Thanks Mabula, for solving the problem, with CLS Filter. Since I have to expose longer in the future, but I wait for cooler temperatures outside.

In APP 1.061 the preview was funny displayed correctly. For calibrated frames.

Hi @minusman,

The previewfilter is now fixed for your data. The problem was the black clipping after calibration. But the previewfilter now detects this properly and will still give a good preview 😉

Left: APP 1.062 showing an overstretched image, check the histogram as well

Right: APP 1.063 beta, showing a proper preview stretch of the data.

Thank you again 😉 for reporting this issue.

Mabula

Hi Dave @jonesdee,

Thank you for your data, I have downloaded it now and will immediately check why normalization fails. This is probably the same reason why other users get a black Integration result.

Will report back as soon as I know the issue.

Mabula

Hi Dave @jonesdee,

The problem with failed normalization is severe black clipping of your data after data calibration.

Always try to locate issues with your data using the image viewer modes 😉 that really is informative:

Linear image viewer mode:

This is how the data looks after having subtracted one of the masterdarks, so l-calibrated image viewer mode:

Please look at the histogram and the B(lack) point. The B(lack) point is at 0 and the histogram clearly shows the data is being clipped.

To solve this now so you can properly process your data, in 2) Calibrate, enable the adaptive data pedestal and try again. Now it works fine and the data is not clipped on the black point.

l-calibrated image viewer mode with adaptive data pedestal enabled :

I will locate the exact point where the normalization fails, so APP will properly warn about this in the next release and will also try to find a way so normalization is still possible, despite the severe black clipping 😉

Thank you very much for sharing your data to locate the issue 😉

Do let me know if all is okay if you process with the Adaptive Data Pedestal !

Kind regards,

Mabula

Wow, many thanks for that Mabula!  Is the fundamental issue underexposure of my subs ?

I will certainly re-process the images with the Adaptive Data Pedestal set.

I am clearly lacking a detailed technical understanding of what is going on here 🙂

Best regards

Dave

Hello Mabula, where can you download the beta version of APP 1.063 for testing?

Hello Mabula, a new bug has cropped up in the Batch Rotated / Resize module. App freezes while rotating.

Hello Mabula, with the new version 1.063 the picture remains black. When loading and selecting a frame.

😥

Problem solved. On my laptop, I had to hard-fix app of Nvidia graphics card in Nvidia control.
Otherwise, OpenGL always ran on the Intel processor. 😏 

Posted by: minusman

Hello Mabula, a new bug has cropped up in the Batch Rotated / Resize module. App freezes while rotating.

Hi @minusman,

I have tested rotating/resizing your integration and it works fine over here. I did test this in APP 1.064 which will be released today.

I do notice that with a clockwise rotation of 59 degree on your integration the image dimensions grow to 11695x11448 f rom 8071x8795. A lot of dark area around the non zero data is added. I will make a note to adaptively reduce the field of view to take into account only non-zero pixels.

Is the rotation error a bug that only happened once, or can you duplicate this ?

Mabula

Posted by: minusman

Problem solved. On my laptop, I had to hard-fix app of Nvidia graphics card in Nvidia control.
Otherwise, OpenGL always ran on the Intel processor. 😏 

@minusman, Hmm, , can you show me a screenshot of what you need to adjust in Nvidia control ?

Thanks,

Mabula

Here is the screenshot of the Nvidia settings.

😉

Hello Mabula, in APP 1.064 I wanted to remove the light pollution with the save calibration. The LPC module is loaded but no image preview is displayed. Can you check that out?

Posted by: minusman

Hello Mabula, in APP 1.064 I wanted to remove the light pollution with the save calibration. The LPC module is loaded but no image preview is displayed. Can you check that out?

Thanks @minusman, I will check it out 😉

Mabula

@minusman, thank you, the progressmonitor is now corrected to NOT show in the image viewer, but in a separate window, in case of Light Pollution correction in saving calibrated frames, so it will work in APP 1.065 😉

Might release it today due to another bug that needed fixing as well a.s.a.p. ...

Mabula

Posted by: minusman

Here is the screenshot of the Nvidia settings.

😉

Thank  @minusman !