Nothing displaying after integration

Hi ImNewHere,

At the top if the image viewer, you will need to disable the details checkbox to see the images in the image viewer. With details enabled, you will see the metadata/header information if present and/or supported by APP of the frames.

Once disabled, the images should show. If not, then you're probably loading images that aren't supported yet by APP. I can't directly see this from your screenshot, but it looks like you are loading TIF frames into APP? If these TIF frames are created by another application from conversions from for instance a raw file type, chances are that APP isn't loading them correctly. I would gladly have a look at those TIF frames then, so I can support them in future APP versions.

I know that, currently, APP will not correctly load TIF frames  with layers created by some versions of photoshop for instance.

Let me know if you can at least view your images and/or integration result with the details selectbox disabled.

Kind regards,

Mabula

Here are examples of the files, both native .dng and the converted tif. I was able to see them with the details box unchecked, but I still keep getting too much noise that I can't get rid of when stacked. Do you have plans on a star removal for the small stars option?

Viel complex: https://www.dropbox.com/s/dy3t2dvutbvvp4p/IMGP2229.DNG?dl=0 and https://www.dropbox.com/s/9ckbog7gi7py7nl/IMGP2229.tif?dl=0

Heart Nebula:  https://www.dropbox.com/s/l0nupukbw6nyb5d/IMGP2243.DNG?dl=0 and https://www.dropbox.com/s/ga15dglwvtkgrbv/IMGP2243.tif?dl=0

Hi ImNewHere,

DNG isn't supported yet (it's on my priorities list to implement), and I would assume the tif files are no longer linear, do you know if they are linear or not?

but I still keep getting too much noise that I can't get rid of when stacked

To what kind of noise are you refering? (there are all sorts of noise) Do you refer to gaussian noise or do you refer to things like hot pixels and air plane/satellite stripes ?

In case of gaussian noise, theoretically, the noise drops with the square root of the number of images that are integrated, APP achieves optimal results in this case.

In case of hot pixels, you might want to check using a Bad Pixel Map to get rid of those immediately as part of calibration:

https://www.astropixelprocessor.com/community/tutorials-workflows/creating-a-bad-pixel-map/

Air plane/satellite stripes can be removed with either

• median integration with or without applying an outlier rejection filter or
• average integration with applying an outlier rejection filter.

Do you have plans on a star removal for the small stars option?

What is a small stars option? Do you want small stars to be removed?

Currently, you can easily protect your stars with the preview filter's HL slider, that will protect the highlights in your images and will reduces star sizes significantly if you stretch strongly.

Feautures like stretching with a star mask or deconvolution of the stars to get tighter stars are on my list to be implemented.

Does this answer your questions?

Kind regards,

Mabula

Yes, I definitely want to be able to remove small stars. When I stretch they all come out really hard.

Gaussian noise is what I am having some problems with, possibly because of the conversion to .tif. Not sure of those .tif files are linear or not, but I included a dropbox to a single .tif and single .dng from the two stacks I was working on before.

Ohh, one other question, do you plan on having APP be able to see if an image file is in landscape or portrait mode, and rotate any in portrait mode to match? The reason I ask is that some of the files show as 7360 x 4912 and others show as 4912 x 7360, and that will cause an error about them not being the same size.

On another note I did stack 60x full spectrum files of M31 last night. Got some nasty noise in there and enough light gradient that I can't kill it, but the results if I can get rid of the noise and gradient look very, very promising. Much better than what DSS can pull off.

Hi ImNewHere,

Okay, I'll have a look at your DNF and TIF files. I would suspect they aren't linear anymore, which would explain that any stretching is really upsetting the stars.

"Ohh, one other question, do you plan on having APP be able to see if an image file is in landscape or portrait mode, and rotate any in portrait mode to match? The reason I ask is that some of the files show as 7360 x 4912 and others show as 4912 x 7360, and that will cause an error about them not being the same size."

APP does this already. Even more so, APP totally ignores any orientation (TIFF) tag in the metadata in processing. So all frames are loaded unrotated. The error in this case will be the conversion from dng, to tiff. The application that does this for you, needs to be told to ignore the orientation tags in saving the TIFs. The TIFs probably are rotated from the dng files follwing the orientation tag and the TIFs themselves probably don't even have this tag anymore in the metadata? I have just checked the tif metadata, which you can do with the details selectbox above the imageviewer:

[File] File Name - IMGP2243.tif
[File] File Size - 216930526 bytes
[File] File Modified Date - Tue Sep 05 12:21:29 CEST 2017
[Exif SubIFD] Exposure Time - 119 sec
[Exif SubIFD] F-Number - f/5,6
[Exif SubIFD] Exposure Program - Manual control
[Exif SubIFD] ISO Speed Ratings - 1600
[Exif SubIFD] Sensitivity Type - Standard Output Sensitivity
[Exif SubIFD] Standard Output Sensitivity - 1600
[Exif SubIFD] Exif Version - 2.30
[Exif SubIFD] Date/Time Original - 2017:08:23 23:48:48
[Exif SubIFD] Date/Time Digitized - 2017:08:23 23:48:48
[Exif SubIFD] Shutter Speed Value - 119.0 sec
[Exif SubIFD] Aperture Value - f/5,6
[Exif SubIFD] Exposure Bias Value - 0 EV
[Exif SubIFD] Metering Mode - Multi-segment
[Exif SubIFD] Flash - Flash did not fire, auto
[Exif SubIFD] Focal Length - 310,0 mm
[Exif SubIFD] Sensing Method - One-chip color area sensor
[Exif SubIFD] Custom Rendered - Normal process
[Exif SubIFD] Exposure Mode - Manual exposure
[Exif SubIFD] White Balance Mode - Auto white balance
[Exif SubIFD] Focal Length 35 - 310mm
[Exif SubIFD] Scene Capture Type - Standard
[Exif SubIFD] Contrast - Hard
[Exif SubIFD] Saturation - High saturation
[Exif SubIFD] Sharpness - Low
[Exif SubIFD] Subject Distance Range - Distant view
[Exif IFD0] New Subfile Type - Unknown (0)
[Exif IFD0] Image Width - 7360 pixels
[Exif IFD0] Image Height - 4912 pixels
[Exif IFD0] Bits Per Sample - 16 16 16 bits/component/pixel
[Exif IFD0] Compression - Uncompressed
[Exif IFD0] Photometric Interpretation - RGB
[Exif IFD0] Make - RICOH IMAGING COMPANY, LTD.
[Exif IFD0] Model - PENTAX K-1
[Exif IFD0] Strip Offsets - 16606
[Exif IFD0] Samples Per Pixel - 3 samples/pixel
[Exif IFD0] Rows Per Strip - 4912 rows/strip
[Exif IFD0] Strip Byte Counts - 216913920 bytes
[Exif IFD0] X Resolution - 240 dots per inch
[Exif IFD0] Y Resolution - 240 dots per inch
[Exif IFD0] Planar Configuration - Chunky (contiguous for each subsampling pixel)
[Exif IFD0] Resolution Unit - Inch
[Exif IFD0] Software - Adobe Photoshop Lightroom 4.4 (Windows)
[Exif IFD0] Date/Time - 2017:08:30 21:24:27
[Exif IFD0] Artist - Scotty W. Bishop scottyspot.com
[Exif IFD0] Unknown tag (0x02bc) - [9399 shorts]
[Exif IFD0] Copyright - Scotty W. Bishop scottyspot.com
[Exif IFD0] Unknown tag (0x8649) - [5616 shorts]
[Exif IFD0] Inter Color Profile - [560 bytes]
[IPTC] Coded Character Set - UTF-8
[IPTC] Application Record Version - 4
[IPTC] Date Created - Wed Aug 23 00:00:00 CEST 2017
[IPTC] Time Created - 23:48:48
[IPTC] Digital Date Created - 20170823
[IPTC] Digital Time Created - 23:48:48
[IPTC] By-line - Scotty W. Bishop scottyspot.com
[IPTC] Copyright Notice - Scotty W. Bishop scottyspot.com

And this answers several questions:

• Lightroom converted your files and applied a ICC color profile assuming this is a normal photography image, this most likely will mean this data isn't linear unfortunately.
• No orientation tag to be found.

An auto rotating of the frames, will not work, because how should APP know how to rotate them? 90 degrees clockwise or 90 degrees counterclockwise?

Both issues can be resolved, if you can find an application that converts your Pentax frames linearly with disregard of the orientation tag. I will implement RAW support for Pentax frames, but for now, APP unfortunately can't ;-( once implemented both problems would be solved as well.

"On another note I did stack 60x full spectrum files of M31 last night. Got some nasty noise in there and enough light gradient that I can't kill it, but the results if I can get rid of the noise and gradient look very, very promising. Much better than what DSS can pull off."

The "remove light pollution" in APP can get rid of extreme light pollution with some practice, an example:

Data courtesy of Rudy Pohl

The noise is probably partly due to the non-linearity of the TIF frames I think.

Kind regards,

Mabula

I will look for a converter that will do them in a linear way and shut off auto rotation for now. The .DNG files should be linear since they are one of the two native formats Pentax uses.

One more question, will the app get rid of fixed pattern/walking noise? If so that will be a huge sell for me since most of the time I run unguided due to how much dew we have here. I lost a collective total of 9 hours due to not turning on the dew heaters on Sunday night.

Hi ImNewHere,

Yes indeed, the DNG files should be linear since they must contain the raw sensor data 😉 The metadata enables software to convert it correctly to the non-linear sRGB/Adobe 1998 or other colorspaces.

One more question, will the app get rid of fixed pattern/walking noise? If so that will be a huge sell for me since most of the time I run unguided due to how much dew we have here. I lost a collective total of 9 hours due to not turning on the dew heaters on Sunday night.

So you refer to hot pixels and possibly banding noise I think?

The hot pixels can be dealt with efficiently with a Bad Pixel Map.

Banding noise would normally need accurate dark frame calibration. I will definitely work soon on implementing dark scaling and will also investigate how to deal with Fixed Patterns in the data, but that might take more time to get it implemented correctly and robustly.

Cheers,

Mabula

I can't get any kind of converter in Windows 10 that will keep any converted files linear, so I can't get the results I know the program is capable of. I've been trying for a couple weeks now with no luck. I guess I have to check back when .dng files are supported.

Hi ImNewHere,

Yes, I fully understand.

Once the DNG are supported, I will be happy to restart your trial, just send me an e-mail then (support@astropixelprocessor.com) with reference to this thread and I will restart your trial 😉

Cheers,

Mabula

Hi, I know it's been a while, but I definitely wanted to follow up on when .DNG support might happen. There's a thread about this vs PI going on over at Cloudy Nights.

@imnewhere , thanks, I know about the thread(s) on Cloudy Nights, but thank you for mentioning this 😉 

DNG support (and other camera brands) will come, I am working on it 😉

Cheers,

Mabula

Hi Mabula,

I have a few beginning questions. Thanks for your great precision APP program! This may be too many diverse questions for this topic. You can put it anywhere you want...

I have a number of scopes such as an 11" Edge-HD, SW 190mm MN,  8" Astro-Tech newt, and 70mm Stellar-Vue Triplet but I thought for simplicity I would start learning APP with a Samyang 135mm on my Nikon D810a. For speed of imaging and because the lens rotates with the camera so only one set of Flats necessary (well at iso 800 anyway). I also have the ZWO ASI071 Pro etc, but that's for later...

In my ignorance I found myself bumping into a number of impasse walls in APP. While following your "A DSLR Data Calib Workflow" I noticed a number of discrepancies. Btw, I'm trying APP on the Mac OS side with version 10.13.6. I can do it on Windows 10 but would rather not.

1. There is only "create Masters and assign Lights" option instead of "calibrate" under BPM. What is the correct flow there?

2. Should my Flats already be calibrated with Bias BEFORE I create the BPM or the Master Flat?

3. There is no "details" button at the top middle as in your video. Is there a way to access the 'details'? I wanted to see what percentage of bad pixels there are in my Darks. Also should I use 50% for the 'Cold Pixels' instead of say 25%?

4. I can't navigate the 'Load' images to an external HD where I want to store images. My internal HDD is only 256gb...

Are some of these issues related to my new version of APP or possibly the Mac app or maybe the Nikon platform?

Best,

James

Hi James @guitarmaker,

Thank you for your kind words. Let me respond to your questions:

1) If you need to create new Master frames, simply click on Create Master and assign to Lights, APP will then automatically apply the new Masters in APP's dynamic data calibration. If you don't need to create new Masters, simply re-assign masters to lights 😉 . When the frames need to be calibrated, APP will dynamically do this, so in processes 3)-6) the frames will be automatically loaded and calibrated and you can verify calibration of your data with the L-calibrated image viewer mode.

2) It's better to calibrate your flats with bias in making a Bad Pixel Map. But in most cases, uncalibrated flats will serve well for detection of cold pixels in the Bad Pixel Map creation. It depends on the height of any Bias pedestal/offset in relation to the actual illumination of the flats.

3) the details selectbox has been removed, you can now simply use the left side of the image viewer, you can drag it to display image metadata at the same time as seeing the image. In most cases 25% is enough for detection of cold pixels. If you have little vignetting, you can safely increase the %.

4) Hmm, that should be possible. I do know that MacOS can be problematic with external drives that are partitioned with ExFAT or NTFS, which is a problem on Operating System level.

Kind regards,

Mabula