Combine multiple R,G & B fits files

Hi Erik,

Have you received calibration files as well, or are the light frames already calibrated?

The simplest way to combine the data is to first make seperate integrations of the R,G, B channels.

example for one of the channels if the frames are already calibrated:

load the light frames and set the options in menu 3) to 6)

To start, it's probably best to leave all settings at defaults.

In 6) click on "integrate" to integrate the 6 images of the channel.

Post the results so I can have a look 😉 and maybe give you some pointers on where we could improve more.

You can save the results to JPGs using the "save" button below the histogram. These JPGs can be easily attached to your posts on this forum.

Once you have the 3 channels integrated, you will need to register them in 4) Register and "save registered frames".

The output of save registered frames can then be loaded into the RGB combine tool for all sorts of combinations.

Let me know if this gets you started.

Sunday or Mondag I'll probably post a full LRGB workflow 😉 which was requested earlier.

Kind regards,

Mabula

Hi Mabula,

Thanks for you quick answer.

I tried what you proposed. I did not change any of the options. See attached the result.

So per color the images are aligned but not the RGB-stack.

On top of that I get that annoying vertical line that is not visible when I open the FTS-files in Fitsworks e.g.

Regards

Erik

PS: The files are already calibrate when I get them.

Hi Erik,

Okay, I probably wasn't too clear about how to get the R,G,B integrations/stacks aligned as well:

"Once you have the 3 channels integrated, you will need to register them in 4) Register and "save registered frames".

The output of save registered frames can then be loaded into the RGB combine tool for all sorts of combinations.

"

Clear the frame list panel, with "clean" in 1) LOAD.

Load your R,G & B stacks in as light frames

then 3)star analyse and 4)register them, I think it's good to turn on dynamic distortion correction in registration, disable "same camera and optics".

click on "start registration"

Then after registration, scroll all the way down in menu 4)Register

use lanczos-3 and turn on "no under/over shoot" to prevent artefacts at star borders.

click on "save"

The files that are saved, will be in your working directory with -reg.fits in the end (-reg postfix), indicating these are registered frames.

These frames will have exactly the same dimensions and you can load them into the RGB combine tool.

Let me know if this works 😉

Cheers,

Mabula

Regarding the column, I don't think I know what that is? Is this visible in the single subs? Is it a bad column?

I would be happy to look at the fits files tomorrow to help you with this?

Mabula 

Can I send you the files via Wetransfer? 

Regards 

Erik

Hi Erik,

That's perfect. You can send them to mabula@astropixelprocessor.com  😉

If the colomn is a problem in the FITS loader, i'll investigate it together with the Nasa FITS library (nom.tam.fits) developers.

Were you able to register the R,G,B channels?

Mabula

Hi Erik,

I am working on your frames.

The columns are bad columns of the camera sensor that is used.

Doesn't fitswork show this in the single light frames?

If not, then I suspect fitswork is applying some sort of bad column detection?

So these columns aren't an error of APP, and it is something that can be easily fixed by calibration normally.

For comparison crops of M51 in APP and Fits Liberator, an esa/eso/nasa fits viewer. It shows the same columns. There are several hot columns on the sensor.

And I have studied the FITS header, only Bias calibration is done apparantly. No Bad Pixel Mapping to remove the bad columns. Neither flat or dark calibration.

HDU1 - SWMODIFY= 'MaxIm DL Version 5.24 131112 219J3' / Name of software that modified
HDU1 -         the image                                                              
HDU1 - HISTORY Bias Subtraction (Bias RBI, 4096 x 4096, Bin1 x 1, Temp -30C,          
HDU1 - HISTORY Exp Time 0ms)                                                          
HDU1 - CALSTAT = 'B       '      

I would ask the provider of the FITS frames if they can provide a  couple of darks of this particular camera at least so we can remove all bad columns and hot pixels?

Hi Erik,

I have integrated the Red Channel to see of we can get rid of the bad pixels and the bad columns by using median integration instead of average in 6) Integrate: integrate, and by using rather strong outlier rejection.

First screenshot is average integration without outlier rejection, second screenshot is median with outlier rejection: sigma clip 1 iteration kappa 1

Median integration removes the hot pixels, and sigma clipping can remove some of the bad columns. But this is a very sub-optimal method.

If we can make a bad pixel map of this camera's sensor using some darks, we probably can get rid of the hot pixels and bad columns in the calibration fase before integration. Which will be way better for the end quality of the integration.

(From these integrations and the applied stretch, its'also clear that these frames badly need flat-field calibration which they clearly didn't had as is indicated by the FITS header.)

EDIT: apparantly the red channel only had bias calibration, B & G had Bias and Flat calibration according to the fits files.

Hi Mabula,

I will ask for some darks and/or a bad pixel map. I will keep you posted.

In the meanwhile I've managed to create the RGB version (still with the vertical lines). 

Thanks for the support

Erik

Hi,

The darks are coming.

In the meanwhile I receive following info from the owner of the remote telescope:

I do have some darks which should mitigate your problem, but basically what you need is to remove the bad column (with those big CCDs there are always some) with the software. A bad pixel map should do the trick, apply it before alignment of the subframes. Also, the integration of the subframes is crucial, sigma clipping or similar is supposed to remove these defects.

Regards

Erik

Posted by: Erik

Hi Mabula,

 

I will ask for some darks and/or a bad pixel map. I will keep you posted.

In the meanwhile I've managed to create the RGB version (still with the vertical lines). 

Thanks for the support

Erik

Excellent 😉 !

Posted by: Erik

Hi,

The darks are coming.

In the meanwhile I receive following info from the owner of the remote telescope:

I do have some darks which should mitigate your problem, but basically what you need is to remove the bad column (with those big CCDs there are always some) with the software. A bad pixel map should do the trick, apply it before alignment of the subframes. Also, the integration of the subframes is crucial, sigma clipping or similar is supposed to remove these defects.

Regards

Erik

Hi Erik,

Indeed, I am downloading the darks 😉

I'll create a bad pixel map and you can try as well following this topic :creating-a-bad-pixel-map

A good Bad Pixel Map is always the best way to remove bad pixels and bad columns. Because a Bad Pixel Map never introduces noise in correcting the problems and the proposed method of removing the problems with oulier rejection while integrating is a a sub optimal suggestion/solution. To remove the problems with outlier rejection, you need much more frames and outlier rejection should always be used as a last resort to solve problems, the reason being: outlier rejection when applied aggresively is very destructive for the quality of your results. It really has a bad influence on the Signal to Noise Ratio of the result if applied too aggresively.

You want to solve the problems of bad pixels and bad columns as soon as possible, that means in calibration 😉

Mabula

Hi Erik,

I have created a BPM of the 20 darks. As you can see, there are a lot of hot pixels and bad columns.

Just load the 20 darks and select create bad pixel map in 2)CALIBRATE.

I used hot kappa default of 2:

FITS HDUs: 1
HDU1 - SIMPLE  =                    T / Java FITS: Sun Jun 11 11:33:39 CEST 2017      
HDU1 - BITPIX  =                    8 / bits per data value                           
HDU1 - NAXIS   =                    2 / number of axes                                
HDU1 - NAXIS1  =                 4096 / size of the n'th axis                         
HDU1 - NAXIS2  =                 4096 / size of the n'th axis                         
HDU1 - EXTEND  =                    T / Extensions are permitted                      
HDU1 - DATE    = '2017-06-11T09:34:43' / creation date of bad pixel map               
HDU1 - SOFTWARE= 'Astro Pixel Processor by Aries Productions' / software              
HDU1 - VERSION = '1.042   '           / Astro Pixel Processor version                 
HDU1 - CALFRAME= 'bad pixel map'      / bad pixel map for instrument FLI              
HDU1 - INSTRUME= 'FLI     '           / instrument name                               
HDU1 - CFAIMAGE= 'no      '           / Color Filter Array pattern                    
HDU1 - NPIX    =             16777216 / raw number of pixels                          
HDU1 - HOTKAPPA= '2,00    '           / kappa value used for hot pixel determination  
HDU1 - COLDFRAC= '0,50    '           / percentage used for cold pixel determination  
HDU1 - NBADPIX =               623280 / number of bad pixels                          
HDU1 - PBADPIX = '3,715   '           / percentage of bad pixels                      
HDU1 - NHOTPIX =               623280 / number of hot pixels                          
HDU1 - PHOTPIX = '3,715   '           / percentage of hot pixels                      
HDU1 - NCOLDPIX=                    0 / number of cold pixels                         
HDU1 - PCOLDPIX= '0,000   '           / percentage of cold pixels                     
HDU1 - NLINPIX =             16153936 / number of linear pixels                       
HDU1 - PLINPIX = '96,285  '           / percentage of linear pixels                   
HDU1 - END                                                                            

Don't be alarmed by the amount of bad pixels, there are a lot in fact 😉

I will process your lights with this BPM and post the result.

Mabula

Hi Erik,

I am making an R,G,B composite of the data with the Bad Pixel Map used for calibration of all the light frames.

The calibration is not perfect and it's hard to correct all bad columns properly. (I think I will need to add a seperate bad column detection to improve here.)

I would suggest the next time that you order frames from a remote observatory, that you request the calibration frames for each channel as well and /or request some more subs.

6 frames per channel (12 for green) is very little to let dithering work. With a double amount of frames, the integration should prove to be much better 😉

Anyway, I also combined the channels with the RGB Combine tool. Then started work with the "remove light pollution" tool in 9. This needs some work, but as you can see in the screenshot, most of the illumination problems can be corrected to a high degree with this tool. This is the result of only applying the tool once:

Next step is the calibrate background tool:

Star Color Calibration:

Selective color to tweak the colors, removed a little bit green, added some yellow and red..

Then cropped the field of view and applied a final DDP stretch with saturation with background protection, sharpening while protecting the stars, and some Highlights protection as well. This really helps with the core of M51:

I think it's really worth it to add some more data (another session to add another 6 lights per channel perhaps ? ), this will improve the result significantly, but very nice already 😉

Hi Mabula,

Thanks for the effort. I will read your instructions carefully and try to obtain the same results as you. I will keep you posted.

Now I just need to decide on what my second object will be do be imaged on the remote telescope.

Regards

Erik

Hi Erik,

If you are going for a new object. It's probably very benificial for the end results if you were to aim to get a higher number of frames per channel possibly reduce the exposure time per sub so that you have the same integration time but more frames.. Than the drizzling of the remote telescope and the integration algorithms can better work to remove those sensor artefacts (the bad columns) 😉

Cheers,

Mabula