An error happened v...
 
Share:
Notifications
Clear all

15th Feb 2024: Astro Pixel Processor 2.0.0-beta29 released - macOS native File Chooser, macOS CMD-Q fixed, read-only Fits on network fixed and other bug fixes

7th December 2023:  added payment option Alipay to purchase Astro Pixel Processor from China, Hong Kong, Macau, Taiwan, Korea, Japan and other countries where Alipay is used.

 

[Closed] An error happened version1.082

18 Posts
3 Users
2 Likes
1,081 Views
 momo
(@mo-mo)
Neutron Star
Joined: 2 years ago
Posts: 122
Topic starter  

Hello

 

After carrying out processing for the next image, an error produced me at a stage of integrate. As I captured an error screen, I attach it.

light 658 file RGB  1file about 50MB

dark 120 file RGB  1file about 50MB
It is for each 20 files according to exposure times

flat 20 file RGB  1file about 50MB

bias 20 file RGB  1file about 50MB

PC
HP Z420 Workstation
CPU Xeon E5 2650
memory 32GB
graphics NVIDIA Quadro K2000 2GB

CPU in APP, the memory setting are attached files.
The Bayer setting is an attached file.
The camera is a one-shot color type.

APP version is 1.082

I used both multichannel and multi-session, and I performed this processing.

The state that used memory 15GB during handling of ANALIZE STARS by 100% continued for a long time.

Question 1
Is it settled if APP memory setting is 25GB?

Question 2
Is there too much number of the files to handle?

Question 3
What happened in APP?

That's all.

Thank you for your cooperation.

 

java error

 

APP CPU memory

 

APP Bayer

 

This topic was modified 2 years ago 2 times by momo

   
(@vincent-mod)
Universe Admin
Joined: 7 years ago
Posts: 5707
 

I think your settings are ok, but over 600 subs does require a lot of the system. You can try to divide the data into subsets of 200 images for instance and the combine the integrations later. Please also try our latest beta (beta 4) which has a lot of improvements and I consider to be quite stable now, links can be found on top of the forum.


   
 momo
(@mo-mo)
Neutron Star
Joined: 2 years ago
Posts: 122
Topic starter  

@vincent-mod

Hello

The number of light file was reduced to 154 and processed.

I did not reduce the number of the files of Dark, flat, bias.
An error did not occur.
I did not use Beta 4.

So I will ask questions again.

What happened in APP?

What does this error mean?

This post was modified 2 years ago by momo

   
 momo
(@mo-mo)
Neutron Star
Joined: 2 years ago
Posts: 122
Topic starter  

Hello

I attach the image of the processing result.
flat may have a problem, but does not know a cause.

It did not occur to another image which I did.
bayer pattern is RGGB, algorithm is Adaptive Airly Disc.
Can I improve when change algorithm?
Or can I improve when change another setting?

Thank you for your helpful cooperation.

Rosette 010s RGB session 1 St

 


   
(@vincent-mod)
Universe Admin
Joined: 7 years ago
Posts: 5707
 

I would also advice to use APP 1.083-beta4 as it is very stable now and quite a few improvements have been made. You can also wait a bit for the 1.083 stable release, that shouldn't take very long anymore given the beta 4 performance.

The result you show above does seem to show a data problem, possibly calibration. You can always check if calibration is working, before waiting for the entire integration, by loading a light and you master calibration files... then going to the top of the APP preview window and select "L-calibrated" from the drop down menu. You then will see that single light being calibrated with your masters and if all looks good, it'll likely work on all data.

What happens when you try a small integration with no calibration data at all?


   
 momo
(@mo-mo)
Neutron Star
Joined: 2 years ago
Posts: 122
Topic starter  

@vincent-mod

Hello

I tried a small integration with no calibration data at all. Only one light file integration.

I attach that file.
From this result, I guess that calibration files, especially flat frames do not fit,

how are your opinions?

Does APP have the method of reducing the impact of flat frame that is not conforming in this way?
I want to try it.

I tried 1.083 according to your recommendation.
I feel that the processing speed is faster.
It was processed in a 2/3 time of 1.082.
However, the impact of calibration files is the same as 1.082.

Therefore, the processing result is also the same as 1.082.

Rosette 10s no caribration files St

 


   
(@vincent-mod)
Universe Admin
Joined: 7 years ago
Posts: 5707
 

Yes, so that indicates there is a problem in calibration. What you can try now is to first add the master-bias, check how that looks. If ok, add the master-dark, check again. If that still is ok, then the flats have an issue.


   
 momo
(@mo-mo)
Neutron Star
Joined: 2 years ago
Posts: 122
Topic starter  

@vincent-mod

According to Vincent's advice, I applied only Master bias to one light.
It is the first image.

Next I applied Master bias and Master dark to one light.
It is the second image.
As far as I look at these images, Master bias and Master dark seem to have no abnormality.
What do you think ?

Can you conclude that there is a cause of FLAT from this result?

Rosette 10s only Masterbias St

 

Rosette 10s Masterbias Masterdark St

 


   
(@vincent-mod)
Universe Admin
Joined: 7 years ago
Posts: 5707
 

It's also better to stretch the light well so you can check how the background looks etc. But if this looks normal to you, my guess is that you have an issue with taking proper flats. What is your process exactly in taking flats, what camera settings do you use and what do you do to the scope for taking them? Can you show me a screenshot of a master-flat?


   
 momo
(@mo-mo)
Neutron Star
Joined: 2 years ago
Posts: 122
Topic starter  

@vincent-mod

Master Flat Screenshot has been attached.
Flat, Dark, Bias and Light are all acquired by Time-lending telescope service. It is not  iTelescope.

MF screenshot
MF IG 100.0 E 1.0s ASI Camera 1  6248x4176  1 St
MD IG 0.24286 E10.0s ASI Camera 1  6248x4176 1 St
MB IG 0.24286 E 0.0s ASI Camera 1  6248x4176 2 St

   
(@vincent-mod)
Universe Admin
Joined: 7 years ago
Posts: 5707
 

Ok, that is looking strange to be honest. Maybe it's better for me to have a better look. Could you upload some (like 10) of bias, dark and flats to our server? Also include the master calibration files please.

I do have to say that my experience with a lot of remote services has been a hit or miss regarding calibration data, more often a miss. Usually it's indeed the flats that are simply not taken properly.

Go to https://upload.astropixelprocessor.com and for the username and password, use: upload

Create a directory named “mo-mo-calibrationProblem” and upload in there. Thank you!


   
 momo
(@mo-mo)
Neutron Star
Joined: 2 years ago
Posts: 122
Topic starter  

@vincent-mod

I uploaded bias, dark ,flats and light to your server.

I'm glad if you'll check the files with your cooperation.

 


   
(@vincent-mod)
Universe Admin
Joined: 7 years ago
Posts: 5707
 

Thanks for uploading! Give me a day or 1-2 to analyze, thanks!


   
(@vincent-mod)
Universe Admin
Joined: 7 years ago
Posts: 5707
 

So having a look at your data now, some things I'm noticing;

1. The lights are just 10 seconds, this is really too low to get the signal out from the background properly. Which means that quite some signal remains in the noise floor. I would advise to go for at least 3 min. I can see that the histogram is clipping on the left side.

2. For the flats you have the same gain as the lights, but this is not a requirement. For flats we tend to use lower gains and really use the maximum of the dynamic range of the sensor. Of course you then need to increase the exposure length, but that's usually a good thing as well, 2-3 seconds per flat. Aim for an exposure that gets to 60-70% of the dynamic range (make sure it doesn't clip of course). For darks and bias we use the same gain as lights, if the flats have a different gain then you take darkflats as well (basically bias-like frames but with the settings of the flats, including the same gain and offset!).

The flats do over-correct and usually point to an issue with either the flat or the calibration of the flat. The bias is shot at the fastest possible setting and I wonder if that might be an issue. Some sensors don't behave well in that range so we advice to go for 0.5 second per bias frame. Maybe that is something to try first, take new bias frames at this higher exposure length and try again. Make sure to have a bias with the same gain and offset as the flats as well.

 


   
 momo
(@mo-mo)
Neutron Star
Joined: 2 years ago
Posts: 122
Topic starter  

@vincent-mod

Hello

Thank you for your analysis.

I confirmed that gain of light, flat, dark and bias are the following value.

light gain:100, flat gain:100, dark gain:0.243, bias gain:0.243

Bias should be the same gain as flat.
Flat does not have to be the same gain as light.
By lowering the gain of flat, the bias gain is also lowered (same as flat)to make the exposure time about 0.5 seconds.

I understood the above.
As you write "for darks and bias We Use the Same Gain As Lights", do I need to make dark and bias gain the same as light gain 100?


   
(@vincent-mod)
Universe Admin
Joined: 7 years ago
Posts: 5707
 

Yes, I would use darks and bias with the same gain as the lights and another bias or darkflat (dark with the exposure lenght and gain of the flat) for the flats if you change the gain on those.

Of course you also need proper offsets for the sensor, but the remote service should know about that.


   
 momo
(@mo-mo)
Neutron Star
Joined: 2 years ago
Posts: 122
Topic starter  

@vincent-mod

When I organize your talk so far, I think that I need the following 1-5 conditions.

1.Bias should be the same gain as flat.
2.Flat does not have to be the same gain as light.
3.By lowering the gain of flat, the bias gain is also lowered (same as flat)to make the exposure time about 0.5 seconds.

4. I need to make dark and bias gain the same as light gain 100

5.if I change the gain on dark or bias, Another bias or darkflat (dark with the exposure lenght and gain of the flat) for the flats is needed.

By Conditions 1 to 3, will be the next gain, isn't it?
light gain: 100, flat gain: 50 (provisional), dark gain: 100, Bias gain: 50

If I apply Condition 4 and 5 to this, it will be the next gain to meet the condition 3, isn't it?
light gain: 100, flat gain: 50 (provisional), dark gain: 100, bias gain: 50 (for Flat) +100 (for Light), need 2bias

Is my understanding of the above correct?

 

 


   
(@wvreeven)
Quasar
Joined: 6 years ago
Posts: 2133
 

@mo-mo There are now two threads where you ask the same question. I have answered here:

https://www.astropixelprocessor.com/community/postid/20898/

and will close this thread to avoid confusion and cross-posting in both threads.


   
Share: