Problem with calibr...
 
Share:
Notifications
Clear all

Problem with calibration frames  

Page 2 / 3
  RSS

(@vincent-mod)
Universe Admin
Joined: 4 years ago
Posts: 2894
November 26, 2020 14:59  

Yes correct, that's fine as darkflats are, just like bias frames and darks.... dark. They don't depend that much on a particular session usually (unless session 1 had temperatures that were very different from session 2 for instance).


ReplyQuote
(@dv_stranger)
Main Sequence Star Customer
Joined: 1 year ago
Posts: 41
November 26, 2020 15:05  

So i can use the darkflats from one session and put ist to all sessions. 

But how i can do this with my 2 diffrent types of darks? How i can add them?

Now i unterstand why the endresult is so noisy. The darks was uses only on two sessions of 6. 

I dont unterstand why APP ist working like this? Thats a failure. It should be possible to use the same failes on other sessions like my darks. And there should be also a message thats APP is deleting assignet files. 

This post was modified 2 months ago by Dane Vetter

ReplyQuote
(@vincent-mod)
Universe Admin
Joined: 4 years ago
Posts: 2894
November 26, 2020 18:00  

Sorry but there is no failure here. Darks are not needed to be made every session, unless there are pretty big differences in temperature. So you can use all those darks for all sessions, same holds true for darkflats and bias.

When you have darks that are clearly different, then you can add them to a specific session this is asked by APP when you load them. If you do pick that option, then you need to add other darks per session as well to cover them all.


ReplyQuote
(@dv_stranger)
Main Sequence Star Customer
Joined: 1 year ago
Posts: 41
November 26, 2020 18:17  

I know that i dont need to do the darks on ervery session. I have a dark library. But i renewed my dark library because they was older than a half year. So session 1 to 3 are older and i must add the old darks to this three sessions. Sessions 4 to 6 has the newer darks. 

So i cant use the "for all sessions" function, and i have to add the darks to every session. 

Session 1 - Darks A
Session 2 - Darks A
Session 3 - Darks A
Session 4 - Darks B
Session 5 - Darks B
Session 6 - Darks B

Darks A are all the same, and darks B are all the same. Maybe i unterstood it wrong. But I have now understood that if I add the same darks, the previous assignment is deleted.

So i add the darks A to session 1, then i adds the darks to session 2 - the darks are the same, so APP delete the darks from session 1 because its the same like session 2. I add the darks for session 3 and APP delete the darks from session 3 because ist similar to session 2. So now i have only assignet darks to session 3. The same happens on session 4 to 5. In the end I have only darks in session 3 and darks in session 6. 

Did i unterstand this situation right? 

Now i learned that i can use the "for all sessions" for my darkflats, and thats good to know. Thank you. 


ReplyQuote
(@vincent-mod)
Universe Admin
Joined: 4 years ago
Posts: 2894
November 26, 2020 19:11  

Thanks for the explanation, now I understand your question better. I've asked Mabula to see if that does or doesn't work and why, I've never tried it in that way. I'll let you know.

ps. regarding his previous answer about darkflats, that indeed seems to be the case. We're discussing it to see why. 😉


ReplyQuote
(@dv_stranger)
Main Sequence Star Customer
Joined: 1 year ago
Posts: 41
November 26, 2020 19:26  

Thank you!

This also explains why the counter does not go up when I add the same darks again, even if I assign them to a different session.

I'm not satisfied with the noise, and I'm afraid that's exactly the reason why my darks were used in only 2 of 4 sessions.

NGC7023 session 1 session 2 session 3 session 4 session 5 session 6 lpc cbg St final

ReplyQuote
(@mabula-admin)
Universe Admin
Joined: 4 years ago
Posts: 2508
November 26, 2020 19:39  

Hi Dane @dv_stranger,

Always check the frame column in the frame list panel at the bottom, that will show you what is happening in terms of calibration. It will show you which masters are assigned to which lights.

Now for your case, where you have recreated new darks for an updated library, I can understand why you run into problems. Why not use the new darks on all session then ? That simply should work, does it not? Or do you have very severe fixed pattern noise on you sensor that has changed significantly between sessions 1-3 versus sessions 4-6 ?

Did you shoot the new set of darks with the same gain and exposure time as the old darks ?

I will make a note in my TODO list to make it possible that you can assing the darks and dark flats to multiple sessions as well ;-), that should prevent problems like this.

But the question remains: Why not use the new darks on all session then ? That simply should work, does it not?

Mabula


ReplyQuote
(@dv_stranger)
Main Sequence Star Customer
Joined: 1 year ago
Posts: 41
November 26, 2020 19:58  

Hi Mabula,

thats a good question. I try it at this night to stack all sessions with the newest darks and look how it works. I thought it was important that the old shots were stacked with the old darks from the same time. 

Thank you for the explanation and that you keep my problem in mind for the next update. 🙂


ReplyQuote
(@vincent-mod)
Universe Admin
Joined: 4 years ago
Posts: 2894
November 26, 2020 20:10  

That's indeed not always necessary, only when it would really be noticably different. But due to your issue, there are some changes going to be made which is nice. 😉


ReplyQuote
(@mabula-admin)
Universe Admin
Joined: 4 years ago
Posts: 2508
November 26, 2020 20:20  
Posted by: @dv_stranger

Hi Mabula,

thats a good question. I try it at this night to stack all sessions with the newest darks and look how it works. I thought it was important that the old shots were stacked with the old darks from the same time. 

Thank you for the explanation and that you keep my problem in mind for the next update. 🙂

You are most welcome Dane @dv_stranger,

Please share the result that you get with only the new darks ;-). Thank you very much for sharing your issue as well, we will address this to take away any possible issues or confusion for all APP users going forward 😉

Mabula


ReplyQuote
(@dv_stranger)
Main Sequence Star Customer
Joined: 1 year ago
Posts: 41
November 27, 2020 06:49  

Hi there!

So here the result with only the new darks. And only the dark flats from one session, assigned to all sessions. 

image
image
image

Unfortunately I can't see any difference to previous versions. The noise is identical and of equal strength. But it was worth a try.


ReplyQuote
(@wvreeven)
Black Hole Admin
Joined: 2 years ago
Posts: 484
November 27, 2020 12:00  

Hi Dane,

As Vincent corrected me here, darks actually do not help reduce noise so it is not a surprise to me that the noise hasn't gone down. Good reasons to create new darks are

  • A change in gain or temperature
  • Driver updates (ZWO is known to change the default offset in their drivers in at least one occasion)
  • An increased number of hot pixels

Darks typically have to be shot only once a year, mostly because of the third bullet.

 

Wouter


ReplyQuote
(@vincent-mod)
Universe Admin
Joined: 4 years ago
Posts: 2894
November 27, 2020 12:01  

Also be aware that noise is always part of the game. Having very good calibration data will get that to a better level, but really getting the overall noise down is only possible (if you want that to be non destructive) with more data.


ReplyQuote
(@dv_stranger)
Main Sequence Star Customer
Joined: 1 year ago
Posts: 41
November 27, 2020 12:12  

Thank you guys,

i think another problem ist, that i probably not worked backgroundlimited (I dont know its the right words in english). I mean that's almost 26 hours of exposure time at Gain100 and f/5. I've seen other shots of NGC7023, with less than half the time with identical data that contains more detail. I think there is something I have not done optimally. 


ReplyQuote
(@wvreeven)
Black Hole Admin
Joined: 2 years ago
Posts: 484
November 27, 2020 12:57  

@dv_stranger

The pictures you are posting are taken from the raw stack right? I mean, you process the subs up to and including tab 6 INTEGRATE and that's all? As I explained before, that is only part of the process. After that you will need to post-process the image, which is an entire process in itself with many subtleties, to get the most details out of the image.

 

Wouter


ReplyQuote
(@mabula-admin)
Universe Admin
Joined: 4 years ago
Posts: 2508
November 27, 2020 14:39  

Dear Dane @dv_stranger,

Thanks for sharing the result 😉

Have you checked that all light frames were calibrated as expected? By looking at the frame column in the frame list panel at the bottom ?

What kind of noise reduction do you see in the metadata/header of the stack/integration?

The noise reduction metrics will tell you and us if the integration went good or not... for instance, no dithering or too small dither steps will lead to less noise reduction as ideally possible 😉

This is an example:

Noise Reduction Metadata

HDU1 - NOISE-1 = ' 1,3622E-04' / noise level of channel 1
HDU1 - SNR-1 = ' 3,1276E+00' / Signal to Noise Ratio of channel 1
HDU1 - NOTE-2 = 'NR = Noise Reduction'
HDU1 - NOTE-3 = 'medNR = noise in median frame / noise in integration'
HDU1 - NOTE-4 = 'refNR = noise in reference frame / noise in integration'
HDU1 - NOTE-5 = 'ideal noise reduction = square root of number of frames'
HDU1 - NOTE-6 = 'the realized/ideal noise reduction ratio should approach 1 ideally'
HDU1 - NOTE-7 = 'the effective noise reduction has a correction for'
HDU1 - NOTE-8 = 'dispersion change between the frame and the integration'
HDU1 - NOTE-9 = 'because dispersion and noise are correlated'
HDU1 - medNR-1 = ' 4,6778E+00' / median noise reduction, channel 1
HDU1 - refNR-1 = ' 4,6778E+00' / reference noise reduction, channel 1
HDU1 - idNR-1 = ' 4,5826E+00' / ideal noise reduction, channel 1
HDU1 - ratNR-1 = ' 1,0208E+00' / realized/ideal noise reduction ratio, channel 1
HDU1 - medENR-1= ' 2,5241E+00' / effective median noise reduction, channel 1
HDU1 - refENR-1= ' 2,5159E+00' / effective reference noise reduction, channel 1

In this example, the ratNR-1, realized / ideal noise reduction ratio, is 1,02. Theoretically, it can't go higher than 1, practically it can. But this means that this integration is very well performed in terms of noise reduction through the integration/stacking process.

If a stack is not optimally integrated, the realized/ideal NR ratio will be clearly lower than 1. Can you check the value for your stack?

Cheers,

Mabula

This post was modified 2 months ago by Mabula-Admin

ReplyQuote
(@dv_stranger)
Main Sequence Star Customer
Joined: 1 year ago
Posts: 41
November 28, 2020 21:52  

Hi!

The final stack has this data:

image

So ratNR-1 = ' 9,2000E-01'

Is that good or bad?


ReplyQuote
(@dv_stranger)
Main Sequence Star Customer
Joined: 1 year ago
Posts: 41
December 1, 2020 19:47  

Hi! Is there any news about my noise reduction? Is there maybe something wrong wenn you see my data?


ReplyQuote
(@vincent-mod)
Universe Admin
Joined: 4 years ago
Posts: 2894
December 2, 2020 00:02  

There is nothing wrong there no, it could be a bit better maybe, but to be honest I really don't see anything wrong with the data. It looks really good, I think it's more that other people with that amount of data use quite a bit of extra post processing to make it look nicer. However, I tend to like a bit more noise, as this usually means you preserve more details.


ReplyQuote
(@mabula-admin)
Universe Admin
Joined: 4 years ago
Posts: 2508
December 2, 2020 15:19  
Posted by: @dv_stranger

Hi!

The final stack has this data:

image

So ratNR-1 = ' 9,2000E-01'

Is that good or bad?

Hi Dane @dv_stranger,

Those details look fine 😉 really !

Your noise reduction realized/ideal ratio through stacking/integrating all of your data is between 0,87 - 0,92 ( 9,2000E-01 is scientific notation for 0,92 😉 ).

Ideally it would be 1. So that means that when the data was integrated ideally/perfectly, the noise in your final integration/stack would have had 1/0,92 = 1,086 -> 8,6 % less noise.

A couple of % you will not see easily with your eyes, when it becomes 10-20% you will see the difference with your eyes in terms of how noisy the result is.

Now since, you have integrated 26 hours of exposure time divided in many ! frames I assume, there is definitely room for improvement I think.

I think the most and clear improvement will be gained in terms of noise reduction if you start to improve the dithering in the capture process. So did you dither? And if so, what kind of dither steps, how large in pixels are dither steps between the frames? If you dithered, did you do it, between all exposures?

Cheers,

Mabula


ReplyQuote
Page 2 / 3
Share: