Null Pointer exception

Same thing with bilinear. 

Hi @gotak,

Could you describe which steps you took before the error appears and at what stage exactly (during normalization, registration...)? Thanks.

• 116 frames from a 163C camera over 6 sections of the skies (about 20 frame for each section)
• 10% overlap in the frames
• Everything works fine up to integration. Integration fails at it looks like the part where it starts pixel interpolation?

Not sure what else I can add to this.

Is there a location for the full log files? 

The error is that of java, so the log won't really help in this case. Could you maybe upload like 5 frames per section for me to try and have a look?

You can upload them to the APP server, using login and password "appuser" (without the "). Please create a folder with your name on it first, thanks!

Uploading now.

Downloading. I'll try to have a look somewhere today.

I also get this error, on as few as a 2 panel mosaic. It happens during integration. Changing the pixel interpolation method has no impact even though this is where the error appears to be. Screen shots attached

Ok, thanks for letting us know! I didn't get the error as of yet, but Mabula knows about it and is going to investigate further and hopefully fix it for the next release.

@jamesmrobins it would be great if you could share some of the data as well, Mabula is then able to have a better look to reproduce the issue.

Uploaded. The panes are each stacks of 4 images.

Settings as follows

REGISTRATION:

quadrilaterals, scale start 1, scale stop 10, flip descriptors on, use dynamic distortion correction on, same camera/optics off, mosaic mode.

NORMALIZE:

Regular, multiply scale, neutralise background off

INTEGRATE

equal weight, media, 1st LNC 1 iteration, MBB 25%, scale 0.3

Greatly appreciated James! I'll let Mabula know.

I am wondering if it is memory related. I have 12GB allocated to APP. If I build my stacks with a reduced scale then it sometimes works... my results are inconsistent on this, I can't seem to nail it down.

Yeah it would be great to have a consistent bug, way easier to find. But I think Mabula could try to avoid a null pointer situation. Which platform are you using btw?

Hi @gotak, @jamesmrobins & @vincent-mod,

I am investigating this issue right now 😉 thank you very much for uploading your data.

I will report back as soon as I have reproduced and solved the issue.

Kind regards,

Mabula

Hi @gotak, @jamesmrobins & @vincent-mod,

I have run several tests using a SSD harddrive in my laptop, the nullpointer never happens whatever setting i use from registration to integration. So I am starting to suspect an issue with using either an external drive or using a network share.

Can you both give addtional information on where the data is? External/network?

And where have you set the work directory?

Maybe that will explain where the issue is coming from. In the mean time I will continue investigating 😉 with additional test.

Kind regards,

Mabula

Posted by: @mabula-admin

Hi @gotak, @jamesmrobins & @vincent-mod,

I have run several tests using a SSD harddrive in my laptop, the nullpointer never happens whatever setting i use from registration to integration. So I am starting to suspect an issue with using either an external drive or using a network share.

Can you both give addtional information on where the data is? External/network?

And where have you set the work directory?

Maybe that will explain where the issue is coming from. In the mean time I will continue investigating 😉 with additional test.

Kind regards,

Mabula

Local drive in raid mirroring array on two 3tb conventional drives.

Hi @gotak,

Okay, thanks, then I think it's safe to assume that this issue is not related to network shares/external drives.

Can you share more on the settings that you use that trigger the error? Does the error always occur? I mean each time that you try to process the data?

On which Operating System does it happen?

Thanks in advance 😉

Mabula

@mabula-admin

I am on windows 10.

I tried a few different thing and it always seems to happen.

Do you need the full set of settings? I'll have to go and try to regenerate them. Is there a log somewhere that would provide the settings?

I am on OSX, data is in a network share with local working directory. 

The error occurs consistently for a given set of data - if it is going to happen it always happens with that data. But if I change the data a bit eg reduce the number of input files or scale down the input files then sometimes I can make the error go away. I can't get this workaround to be successful every time however. 

Hi @gotak & @jamesmrobins & @vincent-mod,

I have run both your datasets many times now and I never get the error unfortunately, so I suspect that I am not doing things exactly the way you are doing it. To be able to solve this issue I would need to know exactly what you are doing.

@gotak, in your first post we can see that you are using Master calibration frames.. which were not part of your upload. I have only part of your light frames. Maybe the issue is related to the calibration masters? Did you verify calibration is looking okay with the L-calibrated image viewer mode? What happens if you process your lights without calibration frames? Does the error still occur?

It would really help me if you can both provide me with a step by step record of what you are doing with the data that you sent me that will give the error on your computer, so I will be able to reproduce this error and solve it 😉

This is gotak's data and it processes without problems whatever I do until now:

So no calibration frames are used since I don't have these and this is just a small subset of gotak's data..

Kind regards,

Mabula

I can upload the calibration frames. I did not see any issue with calibration.

The settings are the same as when I do a regular integration but just with the registration and integration setting changed to mosaic settings. So I doubt the settings other the the mosaic specific ones are the issue.

So I used the default setting with the following changes:

Normally used:

1. BGGR and force CFA
2. Adaptive pedestal checked
3. Sigma clip calibration frames (Bias, Flat and Dark)
4. LN Sigma clip for integration

For mosaic:

1. Set star num to 2500
2. 2nd degree LNC
3. MBB set to 10%
4. use dynamic distortion correction

Cal frames are being uploaded.

Thanks a lot @gotak! Hopefully they will help.

So still no idea what the cause is?

Bug hunting, especially when it's not consistent takes time.. 🙂

Hi @gotak,

I am trying to reproduce the issue right now with your calibration frames, I will let you know what I find 😉

Mabula

Posted by: @gotak

I can upload the calibration frames. I did not see any issue with calibration.

The settings are the same as when I do a regular integration but just with the registration and integration setting changed to mosaic settings. So I doubt the settings other the the mosaic specific ones are the issue.

So I used the default setting with the following changes:

Normally used:

1. BGGR and force CFA
2. Adaptive pedestal checked
3. Sigma clip calibration frames (Bias, Flat and Dark)
4. LN Sigma clip for integration

For mosaic:

1. Set star num to 2500
2. 2nd degree LNC
3. MBB set to 10%
4. use dynamic distortion correction

Hi @gotak, @jamesmrobins & @vincent-mod,

Hmm,  I have just run Gotak's data including his calibration data (masterbias, masterdark & created a masterflat from the individual flats) and no nullpointer occured. I have used the settings that Gotak indicated...

Maybe, there is another setting that is overlooked? How about the LN sigma clipping settings? Just 1 iterations with sigma 3, or more iterations and lower sigma ?

Mabula

@mabula-admin @gotak 

I think I found it! At least for me...

I have frames loaded but not selected. Registration reports that frames that are not selected "Can't be registered" (of course, there should be no attempt to register them) - see attached screen shot. If this occurs then Integration fails with the Null Pointer Exception previously reported.

BUT If I only load the frames that I intend to integrate and select all those files, then there are no errors reported at registration and integration succeeds.

Settings are exactly the same in both instances - the only difference is that I have frames loaded that I don't have checked. This explains why it was intermittent for me.

------

Settings:

I have 12Gb allocated to APP. I restarted APP after creating the panels to ensure default settings were applied then changed the following:

Analyse Stars: 2500

Register: Scale Stop 10, file descriptors on, use dynamic distortion correction on, same camera/optics off, Mosaic mode

Normalize: multiply-scale

Integrate: equal weight, medium, Not LNC, MBB 30%, Scale 0.3

@jamesmrobins

Well that should help. I did have 1 frame that failed to register in my full set. So I think you managed to find the root cause. Good job!

@Mabula 

I was going to do a stack trace if I could but since James seems to have found the problem... maybe future versions should have better logging? Null pointers sucks but usually it's shouldn't be this hard to figure out in java program.

@jamesmrobins Thanks for the workaround!

I ran into exactly the same issue. Tested using 1.083b1 on Windows.