Almost at my last n...
 
Share:
Notifications
Clear all

15th Feb 2024: Astro Pixel Processor 2.0.0-beta29 released - macOS native File Chooser, macOS CMD-Q fixed, read-only Fits on network fixed and other bug fixes

7th December 2023:  added payment option Alipay to purchase Astro Pixel Processor from China, Hong Kong, Macau, Taiwan, Korea, Japan and other countries where Alipay is used.

 

Almost at my last nerve

24 Posts
7 Users
0 Likes
2,460 Views
(@1cm69)
Neutron Star
Joined: 6 years ago
Posts: 133
Topic starter  

With the still unresolved Java issues I am having with 1.074.1 I am still using 1.073 which will be apparent in the following screenshots (thought I'd put this disclaimer up first as people will spot it straight away).

Anyhow, I am nearly at the point of doing an Elvis & putting a bullet through my monitor as I am that frustrated.

Once again it's a darn Flats issue (as always) 😔 

I was happily snapping away at M3 last night, got 50 x 40s frames at ISO800

I was dithering via PHD2 through BYEOS on every second frame.

I took my Flats as usual using my Flats Panel, mid histogram as found using SGP - 0.6s at ISO100

I took Dark Flats to match the Flats, 0.6s at ISO100

I have a pre-made Master Bias at ISO800 & also a pre-made BPM

I am not using Darks because I am heavily dithering, these are not necessary and by not taking them I am saving time.

Here's a single RAW Light frame:

raw

Here's a single Flat frame:

f

Here's a single Dark Flat frame:

df

Here's my Master Bias:

mb

Now after Calibration.

A single calibrated Light frame:

rawcal

Master Flat frame:

mf

Master Dark Flat frame:

mdf

Now through to Integration:

int2

As is plainly obvious, the vignette & prism shadow of my OAG is still present but lighter.

I have read in various places that a light vignette is not a symptom of over correcting in the software but rather the opposite, under correction due to Flat exposure time being too short.

The problem is that by looking at the Linear result of my Flats, they are well placed mid histogram & also do correspond to 'Mid ADU' in SGP & APT, so should not cause a problem at all.

Many, many other places I have read that people can get very hung up on producing the perfect Flat Field Exposures but that there is a lot of wiggle room each side of mid histogram & these not so centered Flats will produced perfectly good results. Why then I am having such an issue?

'Pulling my hair out'

Kirk


   
ReplyQuote
(@steveibbotson)
Red Giant
Joined: 7 years ago
Posts: 31
 

I am by no means an expert with flats.

 

You are altering the ISO setting just for a giggle I myself would try dimming the panel and taking flats with the same ISO and with a longer exposure.


   
ReplyQuote
(@1cm69)
Neutron Star
Joined: 6 years ago
Posts: 133
Topic starter  

Hi Steve,

it is not necessary that the ISO for Flats & Lights to match as per the Calibration rules. 

There would be no way that I could dim the panel enough at ISO800 to get a purposeful exposure time. 

Thank for replying though 👍🏻

Kirk


   
ReplyQuote
(@wvreeven)
Quasar
Joined: 6 years ago
Posts: 2133
 

You can use a white t-shirt, fold that up and place it between the flat panel and the lens of the telescope. Then your flats will be dark enough at the same ISO as the lights. This is what I, and many others, do.

The one thing you need to be careful of when taking flats with a DSLR camera is that if the exposure time get too short, you will get a shadow of the moving mirror on one side. That is probably what you see in your stacked image. So, apply a white t-shirt and make sure that you use exposure times of at least 1.5 seconds for your flats.

 

HTH, Wouter


   
ReplyQuote
(@1cm69)
Neutron Star
Joined: 6 years ago
Posts: 133
Topic starter  

Thanks for replying but as I have said and Mabula has stated, it is not necessary for Flats ISO to match Lights ISO, so this cannot be the issue that I am seeing. 

I have no shadow from the mirror in my images, the shadow is that of my OAG prism which is what the Flat is supposed to remove

Kirk


   
ReplyQuote
(@kijja)
Black Hole
Joined: 6 years ago
Posts: 149
 

Hi@ICM69

Have you try flat darks without biases? 


   
ReplyQuote
(@1cm69)
Neutron Star
Joined: 6 years ago
Posts: 133
Topic starter  

I only use Bias for the Lights but I have tried with Bias for Flats and got the same result

This post was modified 5 years ago by 1CM69

   
ReplyQuote
(@rcfmitch)
Red Giant
Joined: 6 years ago
Posts: 67
 

Kirk, im glad im not the only one having DSLR flats issues.

for 1 year now all my flats have failed due to them turning white... (under correction)

so i figured after shooting 50 % ...25% ....75%... histogram with no good results,, so  i thought i will  drop the ISO to 100.

Fail Fail.. so its not the ISO that's the problem here , it definitely is Speed ( exposure time)/Light source) ..

So i said fine ~~~  i ll shoot every speed from 1/800th sec down to 2 seconds in increments....

Results  === Fail Failed Fail ~~~~   i had enough, i went out and bought  a CMOS camera  ...

end of story , i never got it solved.. this whole stick it in AV  mode and you have made is @#@@##

Joseph,,,

BTW got an Altair Astro 183MC -IMX Hypercam  and never looked back. all my flats now are 1850 ADU on a 12 bit camera regardless if its SGP or APT or SC3.2  whatever program i use,,,they are perfect...

but that  does not help you at all... i feel for you bud!


   
ReplyQuote
(@1cm69)
Neutron Star
Joined: 6 years ago
Posts: 133
Topic starter  

Interesting...

Due to the weather here & work/family commitments I am just an occasional imager so I cannot pretend to know all the mechanics but only what I have tried. 

My DSLR is the same one I have used in all the time I’ve had APP, I am by no means dissing APP before anyone gets irate, so I can rule out a DSLR issue as I have had great results in the past. 

I guess about a year ago I found someone who was sharing data to practise calibrating/processing with. Ideal I thought, that’ll give me something to do on rainy days & learn more of APP’s capabilities. 

Well, I had major issues with the Flat frames included with this data, Mabula got on the case, found a bug in APP, fixed it and all was fine. 

I did some of my own imaging, used APP to calibrate/integrate etc.... with perfect results. 

Now I cannot remember exactly if the top of my head, I’d have to check my invoices but possibly somewhere between then and now I changed from a Celestron OAG to my current SX OAG & FW setup. 

This SX OAG has a prism that is slightly smaller in surface area than the Celestron and appears to encroach more into the light cone, hence the odd shadow in my Flats. 

This shadow is more apparent in some Earthshine Moon images that I took last night, I’ll post one up later when I am at my PC. 

This prism shadow seems to be the worst area in my Flats & resulting Integrations. 

I have also found this very in-depth article about Flat Field frames: http://www.skyandtelescope.com/wp-content/uploads/documents/Flatfields+Mar11.pdf

but I need to read it through a few times to get a handle on things and see if it will help at all.

Also my light panel is the same one that I have always used & I can get excellent consistency from it, so that is ruled out as a problem.

Yes, I do have my eye on a CCD but finances will not stretch just yet.

It is just odd that I was getting hood results back along but something has changed and until Mabula is back and can take a look I’ll have to wait and see.

Kirk

 


   
ReplyQuote
(@jan-willem)
Neutron Star
Joined: 7 years ago
Posts: 105
 

Hi Kirk,

 

I did read the whole post and see why you are "pulling you're hair out of you're head".

I've got only one question, I see you have tried a lot of things but I'm missing one option.

Did you already try to make a few recordings without the OAG. Just a few lights and a few flats.

By this way you can also see if it is really you're OAG or something else.

 

Cheers Jan-Willem.


   
ReplyQuote
(@1cm69)
Neutron Star
Joined: 6 years ago
Posts: 133
Topic starter  
Posted by: Jan-Willem

Hi Kirk,

 

I did read the whole post and see why you are "pulling you're hair out of you're head".

I've got only one question, I see you have tried a lot of things but I'm missing one option.

Did you already try to make a few recordings without the OAG. Just a few lights and a few flats.

By this way you can also see if it is really you're OAG or something else.

 

Cheers Jan-Willem.

Hi,

I haven't done this yet and I may give it a shot.

So, last night I took some images of The Moon deliberately over-exposed in order to catch Earthshine & these images clearly show the Prism shadow of my OAG, see:

Prism Shadow

you can see in this image the rectangular feature of the prism shadow as well as the regular corner vignetting, especially visible on the right hand side of the image but it is in fact in all corners. The corner vignette does not seem to be much of an issue as this is pretty standard.

I have also ruled out any mirror sweep issues too because all my images have mirror lock engaged for 2 secs prior to image capture to suppress any vibration.

I have another clear night forecast tonight so I plan to try lifting my prism out of my sensor's line-of-sight as much as I possibly can and still achieve stars for guiding.

I'll post back any findings.

For the moment I have resigned myself to smaller targets, i.e. Globular Clusters and smaller Galaxies where I can centre them on my sensor so as not to bother at all with Flats because any darkening of corners or the prism shadow will not be visible due to already black sky. As I also dither, Darks are not necessary so I'll only stack lights with Bias & BPM.

If I get this issue sorted, fine, if not I'll wait until I get my CCD.

Kirk 


   
ReplyQuote
(@jan-willem)
Neutron Star
Joined: 7 years ago
Posts: 105
 

Hi,

 

I was thinking about you're problem and saw that you switched between a celestron OAG to a sw OAG and filterwheel.

Now I'm curious what kind of scope do you use?

I'm asking this because when you use a field flatter or a coma corrector it could also be an reflection problem of the prism.

Also is the OAG before or after you're filterwheel?

And what kind of filters do you use?

 

A few weeks ago I had a sort of similar problem with a DSLR lens and a filter. Long story short, I used a cheap uv/ir filter and this gave a strange reflection. Switched over to a bit expensiver kind and the reflection problems where gone...

 

Cheers Jan-Willem


   
ReplyQuote
(@1cm69)
Neutron Star
Joined: 6 years ago
Posts: 133
Topic starter  
Posted by: Jan-Willem

Hi,

 

I was thinking about you're problem and saw that you switched between a celestron OAG to a sw OAG and filterwheel.

Now I'm curious what kind of scope do you use?

I'm asking this because when you use a field flatter or a coma corrector it could also be an reflection problem of the prism.

Also is the OAG before or after you're filterwheel?

And what kind of filters do you use?

 

A few weeks ago I had a sort of similar problem with a DSLR lens and a filter. Long story short, I used a cheap uv/ir filter and this gave a strange reflection. Switched over to a bit expensiver kind and the reflection problems where gone...

 

Cheers Jan-Willem

Hi,

my scope is a Celestron CPC925 & I am imaging with a Celestron 0.63 focal reducer which is fitted before the OAG.

The OAG is before the FW

I use Baader filters

Kirk


   
ReplyQuote
(@jan-willem)
Neutron Star
Joined: 7 years ago
Posts: 105
 

Hi,

This sounds to be all right. I also use Baader filters and the have a very nice anti reflection coating. 

The only thing you can do ( what you already mentioned ) is lift the prism so it is further away of the imaging stream.

Hope you can solve it.

 

Clear skies Jan-Willem

This post was modified 5 years ago by Jan-Willem

   
ReplyQuote
(@1cm69)
Neutron Star
Joined: 6 years ago
Posts: 133
Topic starter  

Well, I tried lifting the prism but it’s a no go, anything other than all the way in and it does not pick up any stars for guiding at all. 

So, the actual opening is wide enough 48mm I believe which means that the prism isn’t over lapping the camera sensors imaging plane, it’s just the shadow. 

Interestingly I ran my lights through APP yesterday using just Bias & BPM & the resulting integration was awful. I’ll need to post up an image of it later but I assumed that not using Flats in the flow would just mean that the integration would include the vignetting as each individual Light frame shows but it looks almost completely opposite to that. 🤔

Kirk


   
ReplyQuote
(@jan-willem)
Neutron Star
Joined: 7 years ago
Posts: 105
 

Hi,

 

It is a shame that lifting the prism doesn't work.

Could it be an reflection of the prism into the reducer?

The only thing I can think of (if it is possible) is to place te OAG before the reducer. I know you will be guiding at a much higher focal length but there is not a chance of an reflection in the reducer. 

 

Did you try to make a shot without the OAG?

 

Cheers Jan-Willem

 


   
ReplyQuote
(@1cm69)
Neutron Star
Joined: 6 years ago
Posts: 133
Topic starter  
Posted by: Jan-Willem

Hi,

 

It is a shame that lifting the prism doesn't work.

Could it be an reflection of the prism into the reducer?

The only thing I can think of (if it is possible) is to place te OAG before the reducer. I know you will be guiding at a much higher focal length but there is not a chance of an reflection in the reducer. 

 

Did you try to make a shot without the OAG?

 

Cheers Jan-Willem

 

Hi,

yes I have imaged without the OAG & the shadow is not present.

Kirk


   
ReplyQuote
(@1cm69)
Neutron Star
Joined: 6 years ago
Posts: 133
Topic starter  

I have spent a lot of time last night collecting masses of data & calibration file in order to run an array of tests.

Another thing that I am noticing with my Dithered data after Integration is this:

int1 2

It is the hot pixels I guess walking the dithered path, they are White Red, Green & Blue.

So before running all the tests from last nights data, I ran my original data as used in my first post through DSS minus Bias & BPM, the result was very pleasing, much much better than the APP Integration. It had no pale vignette or walking hot pixels.

So next I made my first test of my new data in APP, this is what I loaded;

Lights ISO800 120s (undithered)

Darks ISO800 120s

Flats ISO100 0.6s

Dark Flats ISO100 0.6s

I didn't add any Bias or BPM but chose to make a BPM on the fly.

APP created the MasterDark & then the MasterDarkFlat before failing with this error:

MasterDarkFlat Error

This is extremely odd as my MasterDarkFlat does indeed match the Flats, see the log here, outlined in red:

MasterDarkFlat Error 2

So I quit the Calibration routing, removed the Dark & Dark Flat frames from the load screen, leaving just the Lights, Flats, MasterDark & MasterDarkFlat frames and ran the Calibration engine once more.

This time no error about mismatch was thrown & calibration went ahead finishing by producing the MasterFlat & BPM also.

Kirk

 


   
ReplyQuote
(@1cm69)
Neutron Star
Joined: 6 years ago
Posts: 133
Topic starter  

Right, here are the results from my first set of testing.

These are all through APP & I just saved the Integrations as .jpg without performing any post processing at all.

This first set are all Undithered Lights ISO800 120s, MasterDark ISO800 120s, BPM and varying MasterFlats as I will list against each image.

I first ran each of my 5 sets of DarkFlats ([ISO100 x ] 0.6s, 0.8s, 1.0s, 1/8s) & ISO800 1/60s, through APP to create 5 MasterDarkFlats and then used these MDF to calibrate each of my 5 matching sets of Flat frames to produce the MasterFlats before clearing the load page, then loading up Lights, MasterDark, MasterFlat & BPM 5 times to produce the 5 Integrations.

Test1: ISO100 0.6s MasterFlat

Test1

Test2: ISO100 0.8s MasterFlat

Test2

Test3: ISO100 1.0s MasterFlat

Test3

Test4: ISO100 1/8s MasterFlat via AV-Flat setting in BYEOS

Test4

Test5: ISO800 1/60s MasterFlat via AV-Flat setting in BYEOS

Test5

 

So firstly something very odd is going on with the 1s exposure length MasterFlat, there is nothing untoward on the individual Flats that make up this Master.

It is plainly obvious that there is a Flat issue in all the Integrations but what I have noticed is that there is a dust bunny just below and left of the Galaxy which is visible in the results Test1 to Test4 but is invisible in Test5.

These Flats were all captured sequentially without any adjustment of movement of the imaging train.

Everything in APP was left as default except in the Integrate tab I set 'weights' to Quality & made sure MBB was ticked at 5%

I will now run this same data through DSS.

Kirk

 


   
ReplyQuote
(@astrogee)
Neutron Star
Joined: 6 years ago
Posts: 153
 

Hi 1CM69, I know its "accepted knowledge" to use the t-shirt method or flat-panel method but honestly, I don't think you get good flats with these methods. I noticed once I had horrible flats when I put a t-shirt at the end of the dew shield, and retracting the dew shield was better but not perfect. I believe the problem is that a white flat surface at the end of the telescope is not collimated light - ie. light from the edges of the aperture is emitted from the flat surface.

I've resorted to using flats taken at dusk, I setup my scope before it gets dark and wait for the dusk light to match my required ISO, then I start snapping flats. I did this and I have great flats now.

This gets touchy the higher your ISO but if you can control your speed well, it should not be a problem 

Give it a try, you have nothing to lose! Good luck!

 


   
ReplyQuote
(@wvreeven)
Quasar
Joined: 6 years ago
Posts: 2133
 

Sorry to read that this doesn't work for you @astrogee. For me, on the contrary, it always works for my 80 mm refractor. If I don't use a t-shirt then I need to use exposure times below 1 sec and then my master flat ends up badly. However, with a t-shirt folded twice I can use exposure times of 3 seconds and then the master flat comes out perfectly fine. For the Ha filter I need to take 12 second exposures otherwise they are too faint.

I'd prefer a flat panel that I can dim, like the one I use for my 203 mm RC telescope, but I don't have one so I need to use a t-shirt.

 

Clear skies, Wouter


   
ReplyQuote
(@rcfmitch)
Red Giant
Joined: 6 years ago
Posts: 67
 

When you say (ISO) i'm assuming your using a DSLR, so where is your Histogram on those good flats?

2 years in trying and still those motes persists..

 

 


   
ReplyQuote
(@astrogee)
Neutron Star
Joined: 6 years ago
Posts: 153
 

The histogram is pretty much in the middle. Unfortunately red green and blue usually have different levels but as close to the middle overall. Since dslr histograms are stretched, the raw can live with higher levels. 


   
ReplyQuote
(@1cm69)
Neutron Star
Joined: 6 years ago
Posts: 133
Topic starter  

OK, after battling for what seems like an eternity I may possibly be getting somewhere.

Firstly, my imaging train is all 2" right through to either my CMOS, (fairly small sensor) or my DSLR, (crop sensor).

Taking measurements & even just looking through from my OAG to my attached DSLR after removing the setup from my scope, the prism doesn't look like it overlaps my DSLR sensor.

I did notice that my Flats had this shadow from the OAG prism, I know it is the prism because if I remove it the shadow is gone. However on closer inspection this shadow is not exactly central as I would expect it to be, in fact my Flats do look slightly asymmetrical.

In an attempt to rule certain things out I downloaded a trial of PixInsight & noticed that it had a Flat Contour Wizard, so I ran this wizard on one of my MasterFlats, this is the outcome:

flat BINNING 1 integration contourPlot

as you can see, the Flat is definitely not symmetrical at all, so something in my imaging train is out of whack.

Now bear in mind that when I take my Flats I have the scope pointing straight up and all the imaging train is hanging straight down with no lateral strain on it.

My thinking now is that this asymmetry is due to poor collimation.

But I first thought that if my scope is out of collimation, it also is when capturing Lights & the Flats pattern should still match up. Is it the fact that I use a light panel placed directly on top of my scope that exacerbates the issue?

Is it possible that my Focal reducer is playing a detrimental part in this too as maybe the light cone through the FR is too narrow or ever so slightly off for the sensor of my DSLR.

I really can't say that I have noticed these issues at all when using my CMOS camera where the sensor is around a quarter of the size but I will have to run test with that at the next available oppotunity.

Kirk 

 


   
ReplyQuote
Share: