European AstroFest 2023 - 3 & 4 February 2023 - Kensington Conference and Events Centre, London.
Please come all to AstroFest in London to ask us (Mabula & Vincent) questions and to see live demos of APP!
Please note our new Downloads page here
2023-01-19: APP 2.0.0-beta13 has been released !
!!! Big performance increase due to optimizations in integration !!!
and upgraded development platform to GraalVM 22.3 based on openJDK19
We are very close now to releasing APP 2.0.0 stable with a complete printable manual...
Astro Pixel Processor Windows 64-bit
Astro Pixel Processor macOS Intel 64-bit
Astro Pixel Processor macOS Apple M Silicon 64-bit
Astro Pixel Processor Linux DEB 64-bit
Astro Pixel Processor Linux RPM 64-bit
Trying to integrate a mosaic and getting null pointer exception.
Encountered error in module:
LanczosResamplerWorker
Error message:
java.lang.NullPointerException
Cause:
null
no trace
0852 - IMAGE CONVERTER: image conversion not needed, data is already normalized...
0806 - GENERAL IMAGE LOADER: frame E:\Photos\DSS\M31_Mosaic\Light\http___www.astrobin.com_360949__nc=user-5_150sec_1x1__frame19.fit was loaded successfully
0807 - 6) INTEGRATE: integrate light frames: loaded frame 116 of 117 frames
0827 - 6) INTEGRATE: integrate light frames: wrote frame: 116 to file mapper
0838 - 6) INTEGRATE: integrate light frames: wrote weights of frame: 116 to file mapper
0838 - GENERAL IMAGE LOADER: loading frame E:\Photos\DSS\M31_Mosaic\Light\http___www.astrobin.com_360949__nc=user-5_150sec_1x1__frame20.fit
0841 - 2) CALIBRATE: Adaptive Data Pedestal: enabled
0841 - 2) CALIBRATE: Adaptive Data Pedestal set at: 1.371E-01
0841 - 2) CALIBRATE: WARNING !!! we have detected a possible sensor offset issue between the MasterBias and the Masterdark !
0841 - 2) CALIBRATE: WARNING !!! the MasterBias has a median value of: 4.172E-02 for channel 1
0841 - 2) CALIBRATE: WARNING !!! the MasterDark has a median value of: 3.858E-02 for channel 1
0841 - 2) CALIBRATE: WARNING !!! normally, the median value of the MasterDark should be the same or larger than the median value of the MasterBias
0841 - 2) CALIBRATE: WARNING !!! because a dark should always contain the entire bias signal plus the dark current and possible amp glow signals
0841 - 2) CALIBRATE: WARNING !!! some CMOS sensors however exhibit this behaviour due to CMOS technology.
0841 - 2) CALIBRATE: WARNING !!! we have detected a possible sensor offset issue between the MasterBias and the Masterdark !
0841 - 2) CALIBRATE: WARNING !!! the MasterBias has a median value of: 4.160E-02 for channel 2
0841 - 2) CALIBRATE: WARNING !!! the MasterDark has a median value of: 3.833E-02 for channel 2
0841 - 2) CALIBRATE: WARNING !!! normally, the median value of the MasterDark should be the same or larger than the median value of the MasterBias
0841 - 2) CALIBRATE: WARNING !!! because a dark should always contain the entire bias signal plus the dark current and possible amp glow signals
0841 - 2) CALIBRATE: WARNING !!! some CMOS sensors however exhibit this behaviour due to CMOS technology.
0841 - 2) CALIBRATE: WARNING !!! we have detected a possible sensor offset issue between the MasterBias and the Masterdark !
0841 - 2) CALIBRATE: WARNING !!! the MasterBias has a median value of: 4.162E-02 for channel 3
0841 - 2) CALIBRATE: WARNING !!! the MasterDark has a median value of: 3.853E-02 for channel 3
0841 - 2) CALIBRATE: WARNING !!! normally, the median value of the MasterDark should be the same or larger than the median value of the MasterBias
0841 - 2) CALIBRATE: WARNING !!! because a dark should always contain the entire bias signal plus the dark current and possible amp glow signals
0841 - 2) CALIBRATE: WARNING !!! some CMOS sensors however exhibit this behaviour due to CMOS technology.
0841 - 2) CALIBRATE: WARNING !!! we have detected a possible sensor offset issue between the MasterBias and the Masterdark !
0841 - 2) CALIBRATE: WARNING !!! the MasterBias has a median value of: 4.174E-02 for channel 4
0841 - 2) CALIBRATE: WARNING !!! the MasterDark has a median value of: 3.884E-02 for channel 4
0841 - 2) CALIBRATE: WARNING !!! normally, the median value of the MasterDark should be the same or larger than the median value of the MasterBias
0841 - 2) CALIBRATE: WARNING !!! because a dark should always contain the entire bias signal plus the dark current and possible amp glow signals
0841 - 2) CALIBRATE: WARNING !!! some CMOS sensors however exhibit this behaviour due to CMOS technology.
0841 - 2) CALIBRATE: performing calibration in 32bits normalized floats...
0841 - 2) CALIBRATE: converting data to 32bits normalized floats...
0845 - IMAGE CONVERTER: image conversion not needed, data is already normalized...
1726 - GENERAL IMAGE LOADER: cancelled loading frame E:\Photos\DSS\M31_Mosaic\Light\http___www.astrobin.com_360949__nc=user-5_150sec_1x1__frame20.fit
1726 - 6) INTEGRATE: integrate light frames: integration task was cancelled
1726 - MEMORY TO FILE MAPPER: closing memory to file mapper...E:\Photos\APP\Light.dat
1726 - MEMORY TO FILE MAPPER: removing mapped file
1728 - MEMORY TO FILE MAPPER: closing memory to file mapper...E:\Photos\APP\Light.dat
1728 - MEMORY TO FILE MAPPER: removing mapped file
1728 - MEMORY TO FILE MAPPER: closed succesfully
1728 - MEMORY TO FILE MAPPER: closing memory to file mapper...E:\Photos\APP\Light-weights.dat
1728 - MEMORY TO FILE MAPPER: removing mapped file
1728 - MEMORY TO FILE MAPPER: closed succesfully
1728 - 6) INTEGRATE: integrate light frames: integration task finished
1728 - FRAME DETAILS UPDATER: starting...
1728 - FRAME DETAILS UPDATER: no new frames to add
1728 - FRAME DETAILS UPDATER: rebuilding all frame details...
1728 - FRAME DETAILS UPDATER: checking if frames were identified earlier...
1728 - FRAME DETAILS UPDATER: adding frame marks...
1728 - FRAME DETAILS UPDATER: updated succesfully
Same thing with bilinear.
Hi @gotak,
Could you describe which steps you took before the error appears and at what stage exactly (during normalization, registration...)? Thanks.
- 116 frames from a 163C camera over 6 sections of the skies (about 20 frame for each section)
- 10% overlap in the frames
- Everything works fine up to integration. Integration fails at it looks like the part where it starts pixel interpolation?
Not sure what else I can add to this.
Is there a location for the full log files?
The error is that of java, so the log won't really help in this case. Could you maybe upload like 5 frames per section for me to try and have a look?
You can upload them to the APP server, using login and password "appuser" (without the "). Please create a folder with your name on it first, thanks!
Uploading now.
Downloading. I'll try to have a look somewhere today.
Ok, thanks for letting us know! I didn't get the error as of yet, but Mabula knows about it and is going to investigate further and hopefully fix it for the next release.
@jamesmrobins it would be great if you could share some of the data as well, Mabula is then able to have a better look to reproduce the issue.
Uploaded. The panes are each stacks of 4 images.
Settings as follows
REGISTRATION:
quadrilaterals, scale start 1, scale stop 10, flip descriptors on, use dynamic distortion correction on, same camera/optics off, mosaic mode.
NORMALIZE:
Regular, multiply scale, neutralise background off
INTEGRATE
equal weight, media, 1st LNC 1 iteration, MBB 25%, scale 0.3
Greatly appreciated James! I'll let Mabula know.
I am wondering if it is memory related. I have 12GB allocated to APP. If I build my stacks with a reduced scale then it sometimes works... my results are inconsistent on this, I can't seem to nail it down.
Yeah it would be great to have a consistent bug, way easier to find. But I think Mabula could try to avoid a null pointer situation. Which platform are you using btw?
Hi @gotak, @jamesmrobins & @vincent-mod,
I am investigating this issue right now 😉 thank you very much for uploading your data.
I will report back as soon as I have reproduced and solved the issue.
Kind regards,
Mabula
Hi @gotak, @jamesmrobins & @vincent-mod,
I have run several tests using a SSD harddrive in my laptop, the nullpointer never happens whatever setting i use from registration to integration. So I am starting to suspect an issue with using either an external drive or using a network share.
Can you both give addtional information on where the data is? External/network?
And where have you set the work directory?
Maybe that will explain where the issue is coming from. In the mean time I will continue investigating 😉 with additional test.
Kind regards,
Mabula
Hi @gotak, @jamesmrobins & @vincent-mod,
I have run several tests using a SSD harddrive in my laptop, the nullpointer never happens whatever setting i use from registration to integration. So I am starting to suspect an issue with using either an external drive or using a network share.
Can you both give addtional information on where the data is? External/network?
And where have you set the work directory?
Maybe that will explain where the issue is coming from. In the mean time I will continue investigating 😉 with additional test.
Kind regards,
Mabula
Local drive in raid mirroring array on two 3tb conventional drives.
Hi @gotak,
Okay, thanks, then I think it's safe to assume that this issue is not related to network shares/external drives.
Can you share more on the settings that you use that trigger the error? Does the error always occur? I mean each time that you try to process the data?
On which Operating System does it happen?
Thanks in advance 😉
Mabula
I am on windows 10.
I tried a few different thing and it always seems to happen.
Do you need the full set of settings? I'll have to go and try to regenerate them. Is there a log somewhere that would provide the settings?
I am on OSX, data is in a network share with local working directory.
The error occurs consistently for a given set of data - if it is going to happen it always happens with that data. But if I change the data a bit eg reduce the number of input files or scale down the input files then sometimes I can make the error go away. I can't get this workaround to be successful every time however.
Hi @gotak & @jamesmrobins & @vincent-mod,
I have run both your datasets many times now and I never get the error unfortunately, so I suspect that I am not doing things exactly the way you are doing it. To be able to solve this issue I would need to know exactly what you are doing.
@gotak, in your first post we can see that you are using Master calibration frames.. which were not part of your upload. I have only part of your light frames. Maybe the issue is related to the calibration masters? Did you verify calibration is looking okay with the L-calibrated image viewer mode? What happens if you process your lights without calibration frames? Does the error still occur?
It would really help me if you can both provide me with a step by step record of what you are doing with the data that you sent me that will give the error on your computer, so I will be able to reproduce this error and solve it 😉
This is gotak's data and it processes without problems whatever I do until now:
So no calibration frames are used since I don't have these and this is just a small subset of gotak's data..
Kind regards,
Mabula