First time processi...
 
Share:

The problem with the data upload limit for attachments has been fixed. I have restored it to 30MegaBytes. A recent forum software upgrade was responsible for the changed limit. Please accept my apologies for missing this when upgrading the forum sofware, Mabula.

First time processing RGB-picture from QHY8pro camera was perfect. Now no other is going well again. What's wrong?  

Page 1 / 2
  RSS

(@jheuser)
White Dwarf Customer
Joined: 11 months ago
Posts: 11
23/02/2019 6:17 pm  

I processed IC434 picture that I took last month with my QHY8pro cooled color camera and all went well. I was surprised by the good quality picture. But it seems to be a one time lucky processing or something, because I am not able to get it done on other pictures I made last weeks e.g. of M81-M82. I'm really getting frustrated about this. I did everything the same as I did the first time but al the results are rubbish. 

St avg 26400.0s LNMWC 1 3.0 none x 1.0 LZ3 NS full eq add sc BWMV nor AA RL noMBB St
St avg 13200.0s LNMWC 1 3.0 none x 1.0 LZ3 NS full qua add sc BWMV nor AA RL noMBB SC St NoSt

Horehead nebula went well. Processing M81-M82 went bad,  despite using exactly same parameters. I hope someone is able to give me the hint about what I'm overlooking.

 

 

 

This topic was modified 6 months ago 2 times by jheuser
This topic was modified 6 months ago by Mabula Haverkamp - Admin

ReplyQuote
(@gotak)
White Dwarf Customer
Joined: 1 year ago
Posts: 15
24/02/2019 5:56 am  

What do you actually think is wrong?

Looks like you didn't take care of the LP using the LP tool (and maybe lacking some flats?). Or are there specific things that isn't working in your image?


ReplyQuote
(@jheuser)
White Dwarf Customer
Joined: 11 months ago
Posts: 11
24/02/2019 6:17 pm  

I did try light pollution removal but that didn't work out very well either. But I expected a better result before trying anything like light pollution removal.

 

What I noticed is that the flats of M82-M81 are very blue, while the flats of the IC434 were neutral. I did same processing but different results.

This post was modified 6 months ago by jheuser

ReplyQuote
(@mabula-admin)
Quasar Admin
Joined: 2 years ago
Posts: 2120
24/02/2019 6:21 pm  
Posted by: jheuser

I processed IC434 picture that I took last month with my QHY8pro cooled color camera and all went well. I was surprised by the good quality picture. But it seems to be a one time lucky processing or something, because I am not able to get it done on other pictures I made last weeks e.g. of M81-M82. I'm really getting frustrated about this. I did everything the same as I did the first time but al the results are rubbish. 

St avg 26400.0s LNMWC 1 3.0 none x 1.0 LZ3 NS full eq add sc BWMV nor AA RL noMBB St
St avg 13200.0s LNMWC 1 3.0 none x 1.0 LZ3 NS full qua add sc BWMV nor AA RL noMBB SC St NoSt

Horehead nebula went well. Processing M81-M82 went bad,  despite using exactly same parameters. I hope someone is able to give me the hint about what I'm overlooking.

 

 

 

Hi @jheuser,

If I look at your M81,82 image, I think there is a data calibration issue here. I can clearly see that vignetting is still present which suggests that either flats are not used or the flat-field calibration was incomplete due to missing bias/dark/flatdarks..

Can you provide more information so we can solve this?

  • what kind of frames did you use and how many?
  • For each type, which iso/gain and exposure time did you use ?
  • Is it with the same camera or another?
  • If you used flats, did you create new flats for this project?

Kind regards,

Mabula

Main developer of Astro Pixel Processor and owner of Aries Productions


ReplyQuote
(@mabula-admin)
Quasar Admin
Joined: 2 years ago
Posts: 2120
24/02/2019 6:23 pm  
Posted by: gotak

What do you actually think is wrong?

Looks like you didn't take care of the LP using the LP tool (and maybe lacking some flats?). Or are there specific things that isn't working in your image?

Dear @gotak & @jheuser,

Illumination problems like this should probably not be solved with the LP correction tool, since it indicates improper flat-field calibration. First try to get good flat-field calibration of your lights. Then any remaining gradients are to be corrected with the Remove Light Pollution tool.

Kind regards,

Mabula

Main developer of Astro Pixel Processor and owner of Aries Productions


ReplyQuote
(@mabula-admin)
Quasar Admin
Joined: 2 years ago
Posts: 2120
24/02/2019 6:26 pm  
Posted by: jheuser

I did try light pollution removal but that didn't work out very well either. But I expected a better result before trying anything like light pollution removal.

 

What I noticed is that the flats of M82-M81 are very blue, while the flats of the IC434 were neutral. I did same processing but different results.

Hi @jheuser,

So that might explain the problem then 😉 If the flats are very blue, perhaps the Red and Green channels in your flats are not sufficiently illuminated.

Can you share a flat perhaps ? Or show a screenshot of a flat in APP with the dpp preset at no stretch(data as is). We want to see how that looks including the histogram 😉

Mabula

Main developer of Astro Pixel Processor and owner of Aries Productions


ReplyQuote
(@jheuser)
White Dwarf Customer
Joined: 11 months ago
Posts: 11
24/02/2019 7:28 pm  

Hi Mabula,

It might be that RGB are indeed not equally illuminated in the flats of M81-M82, but strangly the same was with the flats of IC434 and that turned out good.

Flat IC434 APP
Flat M82 M81 APP

 

As an anwser to your other questions:
I always use to make flats for each individual session, and always 21 subs.

same 50 darks used on both pictures (10 min exposures)

same 99 bias frames on both pictures. (1 ms exposure)

all my light frames are 10 minutes exposures, -15 deg. C. 22 exposures for IC434, 44 exposures for M81-M82.

No ISO values, it's a cooled RGB-CCD camera, both pictures same QHY8pro camera.

I make the pictures with Nebulosity 3 with dithering on.

Is there a way of showing the FITS-header info?

This post was modified 6 months ago 2 times by jheuser

ReplyQuote
(@jheuser)
White Dwarf Customer
Joined: 11 months ago
Posts: 11
24/02/2019 7:38 pm  

Here is the image information (from Nebulosity 3): see pictures and picture-names.

Flat info IC434
Flat info M82 M81
Light info IC434
Light info M82 M81


ReplyQuote
(@mabula-admin)
Quasar Admin
Joined: 2 years ago
Posts: 2120
24/02/2019 8:19 pm  

Dear Jan @jheuser,

I see that you are still on APP 1.067? Please update to APP 1.071, several calibration issues were resolved in the past couple of versions:

https://www.astropixelprocessor.com/community/appreleases/downloads/

And you can see all metadata in APP, simply drag the left side of the image viewer window:

LoadOkayAPP1070

Mabula

Main developer of Astro Pixel Processor and owner of Aries Productions


ReplyQuote
(@mabula-admin)
Quasar Admin
Joined: 2 years ago
Posts: 2120
24/02/2019 8:25 pm  
Posted by: jheuser

Hi Mabula,

It might be that RGB are indeed not equally illuminated in the flats of M81-M82, but strangly the same was with the flats of IC434 and that turned out good.

Flat IC434 APP
Flat M82 M81 APP

 

As an anwser to your other questions:
I always use to make flats for each individual session, and always 21 subs.

same 50 darks used on both pictures (10 min exposures)

same 99 bias frames on both pictures. (1 ms exposure)

all my light frames are 10 minutes exposures, -15 deg. C. 22 exposures for IC434, 44 exposures for M81-M82.

No ISO values, it's a cooled RGB-CCD camera, both pictures same QHY8pro camera.

I make the pictures with Nebulosity 3 with dithering on.

Is there a way of showing the FITS-header info?

Hi Jan @jheuser,

I did not say that the channels of the flats need to be equally illuminated, I said they need to be illuminated sufficiently 😉 different illumination levels should never be a problem as long as the histograms peaks are clear of both left and right side of the histogram and still in the linear domain of your camera's sensor, which it will for the more recent camera's.

Can you please show the screenshots with "Or show a screenshot of a flat in APP with the dpp preset at no stretch(data as is). " ? The masterflats are now stretched which is not interesting here.

You are also welcome to send me some light frames and the masters that you have right now. I am quite sure something is not right here judging from your integration result 😉

you can send it to support@astropixelprocessor.com using wetransfer.com

Kind regards,

Mabula

Main developer of Astro Pixel Processor and owner of Aries Productions


ReplyQuote
(@jheuser)
White Dwarf Customer
Joined: 11 months ago
Posts: 11
24/02/2019 9:36 pm  

Here are the flats + info of one original flat and master flat of both pictures.

Flat none debayered info IC434
Flat master debayered info IC434
Flat non debayered M82 M81
Flat master debayered M82 M81

 

And I just sent you the original files via Wetranfer. Hope you can discover anything.

I will also upgrade to the latest version of APP and retry.


ReplyQuote
(@mabula-admin)
Quasar Admin
Joined: 2 years ago
Posts: 2120
24/02/2019 9:55 pm  

Thanks Jan @jheuser,

I will have a look immediately 😉

Will report back later...

Mabula

Main developer of Astro Pixel Processor and owner of Aries Productions


ReplyQuote
(@mabula-admin)
Quasar Admin
Joined: 2 years ago
Posts: 2120
24/02/2019 10:14 pm  
Posted by: Mabula Haverkamp - Admin

Hi Jan @jheuser,

Your master calibration frames are not correct, they should be CFA masters and they are debayered already which should not be done. How did you create these? Did you create these from already debayered bias and dark frames?

Mabula

 

The new masterbias and masterdark are clearly different from the masters of the project IC434 where all was okay 😉

Check the lines in bold :

Fits header MasterBias of IC434:

SIMPLE = T / Java FITS: Sun Feb 10 16:50:31 CET 2019
BITPIX = -32 / bits per data value
NAXIS = 2 / number of axes
NAXIS1 = 3040 / size of the n'th axis
NAXIS2 = 2016 / size of the n'th axis
EXTEND = T / Extensions are permitted
BSCALE = 1.0 / scale factor
BZERO = 0.0 / no offset
DATE = '2019-02-10T15:56:01' / creation date of MasterBias
SOFTWARE= 'Astro Pixel Processor by Aries Productions' / software
VERSION = '1.067 ' / Astro Pixel Processor version
CALFRAME= 'MasterBias' / master bias frame
INSTRUME= 'QHY8Pro ' / instrument name
CFAIMAGE= 'RGGB ' / Color Filter Array pattern
GAIN = 1.0 / gain or ISO depending on instrument
EXPTIME = 0.001 / exposure time (s)
MEAN-R = ' 6,60E-03' / mean of channel-R
MEAN-G1 = ' 6,64E-03' / mean of channel-G1
MEAN-G2 = ' 6,64E-03' / mean of channel-G2
MEAN-B = ' 6,60E-03' / mean of channel-B
MED-R = ' 6,60E-03' / median of channel-R
MED-G1 = ' 6,63E-03' / median of channel-G1
MED-G2 = ' 6,63E-03' / median of channel-G2
MED-B = ' 6,59E-03' / median of channel-B
SIGMA-R = ' 8,68E-05' / standard deviation of channel-R
SIGMA-G1= ' 8,68E-05' / standard deviation of channel-G1
SIGMA-G2= ' 8,83E-05' / standard deviation of channel-G2
SIGMA-B = ' 8,76E-05' / standard deviation of channel-B
NOISE-R = ' 4,56E-05' / MRS gaussian noise estimate of channel-R
NOISE-G1= ' 4,54E-05' / MRS gaussian noise estimate of channel-G1
NOISE-G2= ' 4,61E-05' / MRS gaussian noise estimate of channel-G2
NOISE-B = ' 4,62E-05' / MRS gaussian noise estimate of channel-B
NUMFRAME= 99 / # number of frames used in MasterBias creation

Now see your new masterbias of M81,M82:

Image #1
SIMPLE = T / Java FITS: Fri Feb 15 16:32:40 CET 2019
BITPIX = -32 / bits per data value
NAXIS = 3 / number of axes
NAXIS1 = 3038 / size of the n'th axis
NAXIS2 = 2014 / size of the n'th axis
NAXIS3 = 3 / size of the n'th axis
EXTEND = T / Extensions are permitted
BSCALE = 1.0 / scale factor
BZERO = 0.0 / no offset
DATE = '2019-02-15T15:39:22' / creation date of MasterBias
SOFTWARE= 'Astro Pixel Processor by Aries Productions' / software
VERSION = '1.067 ' / Astro Pixel Processor version
CALFRAME= 'MasterBias' / master bias frame
INSTRUME= 'QHY8Pro ' / instrument name
CFAIMAGE= 'no ' / Color Filter Array pattern
GAIN = 1.0 / gain or ISO depending on instrument
EXPTIME = 0.001 / exposure time (s)
MEAN-R = ' 6,60E-03' / mean of channel-R
MEAN-G = ' 6,64E-03' / mean of channel-G
MEAN-B = ' 6,60E-03' / mean of channel-B
MED-R = ' 6,60E-03' / median of channel-R
MED-G = ' 6,63E-03' / median of channel-G
MED-B = ' 6,59E-03' / median of channel-B
SIGMA-R = ' 8,53E-05' / standard deviation of channel-R
SIGMA-G = ' 8,52E-05' / standard deviation of channel-G
SIGMA-B = ' 8,63E-05' / standard deviation of channel-B
NOISE-R = ' 2,63E-05' / MRS gaussian noise estimate of channel-R
NOISE-G = ' 2,51E-05' / MRS gaussian noise estimate of channel-G
NOISE-B = ' 2,63E-05' / MRS gaussian noise estimate of channel-B
NUMFRAME= 99 / # number of frames used in MasterBias creation

So the problem is that you did not create these masters from undebayered raw data in this case. The dimensions are also not correct which suggest that some of the borders were clipped, which usually happens when software debayers the data. It might be done by Nebulosity or APP. Do you recall debayering the bias and darks ?

My suspicion is that you need to use your old masterbias and masterdark and create a new masterflat from the flats for M81,M82. Before making the masters, make sure that you set the debayer pattern correcly in 0) Raw/Fits and enable Force CFA 😉

I will try as well now.

Mabula

 

This post was modified 6 months ago 2 times by Mabula Haverkamp - Admin

Main developer of Astro Pixel Processor and owner of Aries Productions


ReplyQuote
(@jheuser)
White Dwarf Customer
Joined: 11 months ago
Posts: 11
24/02/2019 10:21 pm  

Thanks so far for and all your quick replies.

B.t.w.: I upgraded to APP 1.071, but that didn't give any other result. Picture looks the exact same als posted earlier.


ReplyQuote
(@mabula-admin)
Quasar Admin
Joined: 2 years ago
Posts: 2120
24/02/2019 10:22 pm  
Posted by: jheuser

Thanks so far for and all your quick replies.

B.t.w.: I upgraded to APP 1.071, but that didn't give any other result. Picture looks the exact same als posted earlier.

Yes @jheuser, your masters are not good, they are created from already debayered data somehow... did you perhaps let Nebulosity do this?

Mabula

Main developer of Astro Pixel Processor and owner of Aries Productions


ReplyQuote
(@jheuser)
White Dwarf Customer
Joined: 11 months ago
Posts: 11
24/02/2019 10:32 pm  

Your master calibration frames are not correct, they should be CFA masters and they are debayered already which should not be done. How did you create these? Did you create these from already debayered bias and dark frames?

This is the way this QHY8pro creates pictures: debayered. 

make sure that you set the debayer pattern correcly in 0) Raw/Fits and enable Force CFA

At least I thougth I did: force CFA.

 

You might be right that Nebulosity does already some cropping. Because when I uses the capture software that was included with the camera, it gave a different pixel-frame. (bigger) I didn't realize that might be important.  I like to capture with Nebulosity because of the possibility of dithering. Maybe I should shoot everything with the software that came with the camera and retry all over again.

But it was because of the good first result in APP with IC434 Horsehead nebula, that I thought it was had noting to do with the capture software. 


ReplyQuote
(@mabula-admin)
Quasar Admin
Joined: 2 years ago
Posts: 2120
24/02/2019 10:57 pm  

Hi Jan @jheuser,

Okay, apart from the debayering issue, I can also verify that your flats are bad unfortunately. The blue channel is way too strong for your camera's sensor. Apparently the Histogram peak has entered the non-linear domain of your sensor.

To illustrate this I have created 6 screenshots,

The Masterflat from the undebayered flats, unstretched, with and withour neutralize BG in the preview filter:

Masterflat no Stretch
MasterFlat no stretch Neutralize BG

We would expect that the neutralized masterflat is gray, not colored ! We don't see a lot now, so we do this again, but with stretching of the masterflat:

MasterFlat Stretch
MasterFlat stretch neutralize BG

Now, this does not look good at all. The corners are blue in the neutralized masterflat and we can also from the histogram that blue clearly deviates from red and green 😉 A clear indication that something is wrong with the blue channel in the flats...

And this is confirmed when we apply this to a light, again stretched and with/without neutralize-BG:

calibrated light bias flat stretch
calirbated light bias flat stretch neutralize BG

The blue corners of the neutralized masterflat show as yellow in the calibrated light due to the divide operation in flat-field calibration.

So, unfortunately, the flats need to be recreated if possible. I will try what the old MasterFlat gives as welll..

Mabula

 

This post was modified 6 months ago 3 times by Mabula Haverkamp - Admin

Main developer of Astro Pixel Processor and owner of Aries Productions


ReplyQuote
(@mabula-admin)
Quasar Admin
Joined: 2 years ago
Posts: 2120
24/02/2019 11:03 pm  

Hi Jan @jheuser,

And here is the comparison of your new Masterflat to your old, we would expect them to look the same if they are both okay, and they don't...

Again, stretch and with Neutralize-BG enabled in the preview filter:

Your new and your old masterflat, please check also the histograms in the screenshots, they tell you a lot 😉

MasterFlat stretch neutralize BG
OldMasterFlat stretch neutralize BG

Your old masterflat is neutral in color when neutralized, the old one is not which to me confirms 100% that the flats are bad.

The good news is, that the dust bunnies seem at very identical places, so perhaps using the old Masterflat will still give a good result.

Kind regards,

Mabula

Main developer of Astro Pixel Processor and owner of Aries Productions


ReplyQuote
(@mabula-admin)
Quasar Admin
Joined: 2 years ago
Posts: 2120
24/02/2019 11:14 pm  
Posted by: jheuser

Your master calibration frames are not correct, they should be CFA masters and they are debayered already which should not be done. How did you create these? Did you create these from already debayered bias and dark frames?

This is the way this QHY8pro creates pictures: debayered. 

make sure that you set the debayer pattern correcly in 0) Raw/Fits and enable Force CFA

At least I thougth I did: force CFA.

 

You might be right that Nebulosity does already some cropping. Because when I uses the capture software that was included with the camera, it gave a different pixel-frame. (bigger) I didn't realize that might be important.  I like to capture with Nebulosity because of the possibility of dithering. Maybe I should shoot everything with the software that came with the camera and retry all over again.

But it was because of the good first result in APP with IC434 Horsehead nebula, that I thought it was had noting to do with the capture software. 

Hi Jan @jheuser,

Okay, that does explain things 😉

You should be able to use Nebulosity, dithering is much better than no dithering so please try 😉

Can't you choose in Nebulosity, not to debayer the frames when captured ? I would think that should be possible.

If data is cropped it gives a significant problem when you want to calibrate it with data that wasn't cropped. You can't tell how to calibrate pixel by pixel then... we can't tell for sure which pixel in the lights is which pixel in dark...

So it is always important that if the capture software gives a different image dimension, that you create all data, so all lights and all calibration frames, with that same capture software 😉  Otherwise things will go wrong for sure...

Mabula

Main developer of Astro Pixel Processor and owner of Aries Productions


ReplyQuote
(@jheuser)
White Dwarf Customer
Joined: 11 months ago
Posts: 11
24/02/2019 11:21 pm  

Mabula,

 

Thanks a lot for your investigation. I've just tried to calibrate with the old master-flat. Picture is slightly different but still far from good. I'll see how to get along with all of it. Especially how to make better flats with this camera.

M81 M82 with old masterflat from IC434

I think the first thing I'm going to try is making all pictures with he software that came with the camera and use them for processing. See how that works out.


ReplyQuote
Page 1 / 2
Share: