Fix Flat-field bug ...
 
Share:

[Solved] Fix Flat-field bug !  

  RSS

(@mabula-admin)
Quasar Admin
Joined: 2 years ago
Posts: 2081
March 31, 2018 22:20  

Hi all,

As several APP users have noticed, there is currently a nasty bug in flat-field calibration.

This causes the vignetting profile in the calibrated light to look over-corrected, (instead of fully-corrected), due to a problem in the normalization of the flats before integrating the flats into the masterflat.

This bug was introduced by a change in APP's normalization engine and I have located the problem and thus fixed it now.

An example of the bug is described here extensively by Kirk @1cm69 (for which many thanks 😉 ):

https://www.astropixelprocessor.com/community/main-forum/raw-files-for-processing-practice/#post-2923

OK, I have just had another go at processing this data & I do not understand what is going wrong.

Every set of data I have tried to process in APP has shown major issues with the Flats.

I am following Mabula's @mabula-haverkamp-administrator steps here:  https://www.astropixelprocessor.com/a-dslr-data-calibration-workflow/ to the letter and nothing is working.

For this latest attempt I am using data from the folder - NIKON_D800 > 2016 > 20161106

and as specified in the LOG for this data I am using NEF files:

6111 - 6122 for BIAS ISO800 1/8000s

6281 - 6312 for FLAT ISO ISO800 1/20s

6323 - 6334 for Dark ISO800 180s

6262 - 6271 for LIGHT ISO800 180s

This is how I process:

  1. Load F & D to create BPM (this also creates MF & MD which I delete) then clear the load
  2. load F, B & BPM to create MF - then clear the load
  3. Load D to create MD - then clear load
  4. Load L, BPM, MF & MD and hit Calibrate

At this stage I took a screenshot of the Linear data & also the Calibrated data:

lin

 

cal

 

You can easily see the vignette in the Linear data but this should disappear in the Calibrated data but it doesn't, it just changes colour.

 

So what is going on??

I am currently working on a big upgrade of the entire calibration engine and I am making really good progress, so once APP 1.060 is released, this bug shall be fixed and calibration will also become a lot easier in APP.

(I have also removed the very strict calibration requirements on ISO/gain and exposure values for bias, darks and dark flats.)

APP 1.060 will be released as soon as possible.

Kind regards,

Mabula

 

Main developer of Astro Pixel Processor and owner of Aries Productions


ReplyQuote
(@lammertus)
Main Sequence Star Customer
Joined: 2 years ago
Posts: 31
May 3, 2018 20:47  

Hallo Mabula,

Is er al een time-frame voor de fix?

Ik heb ook een aantal datasets die compleet fout gaan en anderen ( veel ) die goed gaan!! Dit begrijp ik niet goed mar ik hoop dat er binnenkort de nieuwe versie aankomt! Beste groeten, Mert

Translation by Mabula :

Dear Mabula,

Is there a timeframe for the fix?

Some datasets go fine, where others are not ;-( which is odd... I am anxious for the new version !!

Kind regards,

Mert

 


ReplyQuote
(@mabula-admin)
Quasar Admin
Joined: 2 years ago
Posts: 2081
May 6, 2018 17:17  

Hi Mert @lammertus,

I am nearly ready to release APP 1.060.

This particular bug has been fixed 100%. I have tested it now on a lot of datasets so flat-fielding will be very robust.

Since I started with a complete rebuild of APP's calibration engine, it all took a bit longer than previous updates.

So APP 1.060 will be a huge update 😉 with respect to calibration and fully automatic processing of multi-channel & multi-session processing.

I will try to release APP 1.060 within the next few days !

Mabula

 

Main developer of Astro Pixel Processor and owner of Aries Productions


ReplyQuote
(@lammertus)
Main Sequence Star Customer
Joined: 2 years ago
Posts: 31
May 6, 2018 17:45  

Hi Mabula,

Thanks a lot for the info, your hard work is very much apreciated!!!

I am sure the new version will be a great success!

Regards,

Mert


ReplyQuote
 PEJU
(@peju)
Hydrogen Atom Customer
Joined: 2 years ago
Posts: 2
May 12, 2018 00:42  

Dear support,

My Master D/F  are pre-made for each iso, expo and separated for each camera model Im using. and follow the instructions how to made them.  Now I noticed they coming out with a different pixel size than the subs.

The APP told me that these M-F/D is not same size as the subs and will not be used in the process. 

Is this problem familar ?

 

Best regards, Peter

proc pixel

 

Edit by Mabula: question moved to separate topic:

https://www.astropixelprocessor.com/community/faq/dslr-calibration-master-bpm-have-different-size/


ReplyQuote
(@mabula-admin)
Quasar Admin
Joined: 2 years ago
Posts: 2081
May 12, 2018 14:58  
Posted by: Lammertus

Hi Mabula,

Thanks a lot for the info, your hard work is very much apreciated!!!

I am sure the new version will be a great success!

Regards,

Mert

Thanks @lammertus 😉

It's nearly ready now...

Main developer of Astro Pixel Processor and owner of Aries Productions


ReplyQuote
(@mabula-admin)
Quasar Admin
Joined: 2 years ago
Posts: 2081
May 17, 2018 15:58  

Hi all @1cm69 @lammertus  @xsnrg @peju @wei-hao_wang SpiritofEden

 APP 1.060, which is a major upgrade, has been released today and has the fix for the flat-field overcorrection problem 😉

Kind regards,

Mabula

 

Main developer of Astro Pixel Processor and owner of Aries Productions


1CM69 and xsnrg liked
ReplyQuote
(@1cm69)
Neutron Star Customer
Joined: 2 years ago
Posts: 121
May 17, 2018 17:24  
Posted by: Mabula Haverkamp - Admin

Hi all @1cm69 @lammertus  @xsnrg @peju @wei-hao_wang SpiritofEden

 APP 1.060, which is a major upgrade, has been released today and has the fix for the flat-field overcorrection problem 😉

Kind regards,

Mabula

 

Excellent, thanks Mabula, I'll download it right away.  😉 


ReplyQuote
(@xsnrg)
Main Sequence Star Customer
Joined: 2 years ago
Posts: 29
May 17, 2018 17:28  

Thank you very much!  Will give it a go tonight.

 

Jim


ReplyQuote
(@1cm69)
Neutron Star Customer
Joined: 2 years ago
Posts: 121
May 17, 2018 18:52  

Amazing work Mabula, APP is whole new beast, the program has definitely matured with this new release.

I have just done a very and I mean very quick process using the only data I had to hand.

Wow, what can I say, just chuck it all in like a stew and let APP do all the grunt work - so simple!!!

This image is not great but that is my limited data, NOT APP's issue!!

4 x Light 600s ISO800

4 x Dark 600s ISO800

30 x Flat 2.5s ISO100

BPM - pre-made

MasterBias ISO800 - pre-made from 100 subs

MasterBias ISO100 - pre-made from 100 subs

Put it all in the pot & APP figured out the correct MB to use for the Flats, ISO100, Yahoo!!

Looking at the final image, I cannot see any vignette, BRILLIANT!

Anyway, here's my poor excuse for an image, M64

 

St med 2396.0s NR x 1.0 LZ3 NS full StSh add sc BWMV nor AA RL noMBB cbg csc St


ReplyQuote
(@mabula-admin)
Quasar Admin
Joined: 2 years ago
Posts: 2081
May 17, 2018 18:55  

Excellent Kirk @1cm69 😉

It's my pleasure !

Cheers,

Mabula

Main developer of Astro Pixel Processor and owner of Aries Productions


1CM69 liked
ReplyQuote
(@lammertus)
Main Sequence Star Customer
Joined: 2 years ago
Posts: 31
May 17, 2018 20:11  

Great news Mabula,

Just downloaded the new release and can't wait to install and try on my

failed data sets.

Just soo many new things and updates, thanks for your great effort!!

-Mert


ReplyQuote
(@xsnrg)
Main Sequence Star Customer
Joined: 2 years ago
Posts: 29
May 18, 2018 07:41  

Running my first analysis now.  220 frames to analyze across 2 nights.  Should be a good first test.  First impression, the file loading changes and groupings are excellent.  Second, the star analysis was done in record time, even for that many frames to look at.  So far, so good.  Thanks for the great update!


ReplyQuote
Share: