Fix Flat-field bug ...
 
Share:
Notifications
Clear all

15th Feb 2024: Astro Pixel Processor 2.0.0-beta29 released - macOS native File Chooser, macOS CMD-Q fixed, read-only Fits on network fixed and other bug fixes

7th December 2023:  added payment option Alipay to purchase Astro Pixel Processor from China, Hong Kong, Macau, Taiwan, Korea, Japan and other countries where Alipay is used.

 

[Solved] Fix Flat-field bug !

13 Posts
5 Users
13 Likes
4,014 Views
(@mabula-admin)
Universe Admin
Joined: 7 years ago
Posts: 4366
Topic starter  

Hi all,

As several APP users have noticed, there is currently a nasty bug in flat-field calibration.

This causes the vignetting profile in the calibrated light to look over-corrected, (instead of fully-corrected), due to a problem in the normalization of the flats before integrating the flats into the masterflat.

This bug was introduced by a change in APP's normalization engine and I have located the problem and thus fixed it now.

An example of the bug is described here extensively by Kirk @1cm69 (for which many thanks 😉 ):

https://www.astropixelprocessor.com/community/main-forum/raw-files-for-processing-practice/#post-2923

OK, I have just had another go at processing this data & I do not understand what is going wrong.

Every set of data I have tried to process in APP has shown major issues with the Flats.

I am following Mabula's @mabula-haverkamp-administrator steps here:  https://www.astropixelprocessor.com/a-dslr-data-calibration-workflow/ to the letter and nothing is working.

For this latest attempt I am using data from the folder - NIKON_D800 > 2016 > 20161106

and as specified in the LOG for this data I am using NEF files:

6111 - 6122 for BIAS ISO800 1/8000s

6281 - 6312 for FLAT ISO ISO800 1/20s

6323 - 6334 for Dark ISO800 180s

6262 - 6271 for LIGHT ISO800 180s

This is how I process:

  1. Load F & D to create BPM (this also creates MF & MD which I delete) then clear the load
  2. load F, B & BPM to create MF - then clear the load
  3. Load D to create MD - then clear load
  4. Load L, BPM, MF & MD and hit Calibrate

At this stage I took a screenshot of the Linear data & also the Calibrated data:

lin

 

cal

 

You can easily see the vignette in the Linear data but this should disappear in the Calibrated data but it doesn't, it just changes colour.

 

So what is going on??

I am currently working on a big upgrade of the entire calibration engine and I am making really good progress, so once APP 1.060 is released, this bug shall be fixed and calibration will also become a lot easier in APP.

(I have also removed the very strict calibration requirements on ISO/gain and exposure values for bias, darks and dark flats.)

APP 1.060 will be released as soon as possible.

Kind regards,

Mabula

 


   
ReplyQuote
(@lammertus)
Neutron Star
Joined: 7 years ago
Posts: 113
 

Hallo Mabula,

Is er al een time-frame voor de fix?

Ik heb ook een aantal datasets die compleet fout gaan en anderen ( veel ) die goed gaan!! Dit begrijp ik niet goed mar ik hoop dat er binnenkort de nieuwe versie aankomt! Beste groeten, Mert

Translation by Mabula :

Dear Mabula,

Is there a timeframe for the fix?

Some datasets go fine, where others are not ;-( which is odd... I am anxious for the new version !!

Kind regards,

Mert

 


   
Mabula-Admin reacted
ReplyQuote
(@mabula-admin)
Universe Admin
Joined: 7 years ago
Posts: 4366
Topic starter  

Hi Mert @lammertus,

I am nearly ready to release APP 1.060.

This particular bug has been fixed 100%. I have tested it now on a lot of datasets so flat-fielding will be very robust.

Since I started with a complete rebuild of APP's calibration engine, it all took a bit longer than previous updates.

So APP 1.060 will be a huge update 😉 with respect to calibration and fully automatic processing of multi-channel & multi-session processing.

I will try to release APP 1.060 within the next few days !

Mabula

 


   
ReplyQuote
(@lammertus)
Neutron Star
Joined: 7 years ago
Posts: 113
 

Hi Mabula,

Thanks a lot for the info, your hard work is very much apreciated!!!

I am sure the new version will be a great success!

Regards,

Mert


   
Mabula-Admin reacted
ReplyQuote
 PEJU
(@peju)
Molecular Cloud
Joined: 7 years ago
Posts: 2
 

Dear support,

My Master D/F  are pre-made for each iso, expo and separated for each camera model Im using. and follow the instructions how to made them.  Now I noticed they coming out with a different pixel size than the subs.

The APP told me that these M-F/D is not same size as the subs and will not be used in the process. 

Is this problem familar ?

 

Best regards, Peter

proc pixel

 

Edit by Mabula: question moved to separate topic:

https://www.astropixelprocessor.com/community/faq/dslr-calibration-master-bpm-have-different-size/


   
ReplyQuote
(@mabula-admin)
Universe Admin
Joined: 7 years ago
Posts: 4366
Topic starter  
Posted by: Lammertus

Hi Mabula,

Thanks a lot for the info, your hard work is very much apreciated!!!

I am sure the new version will be a great success!

Regards,

Mert

Thanks @lammertus 😉

It's nearly ready now...


   
ReplyQuote
(@mabula-admin)
Universe Admin
Joined: 7 years ago
Posts: 4366
Topic starter  

Hi all @1cm69 @lammertus  @xsnrg @peju @wei-hao_wang SpiritofEden

 APP 1.060, which is a major upgrade, has been released today and has the fix for the flat-field overcorrection problem 😉

Kind regards,

Mabula

 


   
1CM69 and xsnrg reacted
ReplyQuote
(@1cm69)
Neutron Star
Joined: 6 years ago
Posts: 133
 
Posted by: Mabula Haverkamp - Admin

Hi all @1cm69 @lammertus  @xsnrg @peju @wei-hao_wang SpiritofEden

 APP 1.060, which is a major upgrade, has been released today and has the fix for the flat-field overcorrection problem 😉

Kind regards,

Mabula

 

Excellent, thanks Mabula, I'll download it right away.  😉 


   
Mabula-Admin reacted
ReplyQuote
(@xsnrg)
Red Giant
Joined: 7 years ago
Posts: 35
 

Thank you very much!  Will give it a go tonight.

 

Jim


   
Mabula-Admin reacted
ReplyQuote
(@1cm69)
Neutron Star
Joined: 6 years ago
Posts: 133
 

Amazing work Mabula, APP is whole new beast, the program has definitely matured with this new release.

I have just done a very and I mean very quick process using the only data I had to hand.

Wow, what can I say, just chuck it all in like a stew and let APP do all the grunt work - so simple!!!

This image is not great but that is my limited data, NOT APP's issue!!

4 x Light 600s ISO800

4 x Dark 600s ISO800

30 x Flat 2.5s ISO100

BPM - pre-made

MasterBias ISO800 - pre-made from 100 subs

MasterBias ISO100 - pre-made from 100 subs

Put it all in the pot & APP figured out the correct MB to use for the Flats, ISO100, Yahoo!!

Looking at the final image, I cannot see any vignette, BRILLIANT!

Anyway, here's my poor excuse for an image, M64

 

St med 2396.0s NR x 1.0 LZ3 NS full StSh add sc BWMV nor AA RL noMBB cbg csc St

   
Mabula-Admin reacted
ReplyQuote
(@mabula-admin)
Universe Admin
Joined: 7 years ago
Posts: 4366
Topic starter  

Excellent Kirk @1cm69 😉

It's my pleasure !

Cheers,

Mabula


   
1CM69 reacted
ReplyQuote
(@lammertus)
Neutron Star
Joined: 7 years ago
Posts: 113
 

Great news Mabula,

Just downloaded the new release and can't wait to install and try on my

failed data sets.

Just soo many new things and updates, thanks for your great effort!!

-Mert


   
Mabula-Admin reacted
ReplyQuote
(@xsnrg)
Red Giant
Joined: 7 years ago
Posts: 35
 

Running my first analysis now.  220 frames to analyze across 2 nights.  Should be a good first test.  First impression, the file loading changes and groupings are excellent.  Second, the star analysis was done in record time, even for that many frames to look at.  So far, so good.  Thanks for the great update!


   
Mabula-Admin reacted
ReplyQuote
Share: