Integration looks as if Darks, Dark/Flats, and Flats are not being removed.

Here is mt Master Dark 600s.

I would advice to update APP first to at least 1.079 (1.080 is around the corner), when you load in a light and your masterdark, what happens when you select "L-calibrated" on top of the screen? Do you see the dark being used (it's indicated then in the list on the bottom) and does it correct anything?

Hello Vincent!

First, thank you for your response.  It's greatly appreciated.  I have updated the revision of the application to the latest v.1.079.  Next, I loaded a single 600s light (shot with L-eNhance filter on my OSC camera) in Linear, the loaded the Master Dark (600s) in Linear.  Then with the image area of the upper-right corner enlarged to see any changes the MD would have.  When I selected "L-Calibrated" I do see some of the "Sensor Glow" streaks disappear but clearly not most of it.  Would this indicate that I have a light-leak during either taking the Darks and/or the Lights?

Many thanks,

Dale

 Linear

 L-Calibrated

Mm, I do see that your masterdark has a different ISO/gain then your lights. What happens if you tick the "scale masterdark" in tab 2? And I assume the adaptive pedestal/amp glow is also ticked?

And could you also take a screenshot of the masterdark itself? If I can't figure it out, I'll ask you for the masterdark and some lights, just to be able to check it myself. It does seem to work, maybe a bit too much even as the light streaks are now darker. What happens if you now integrate with this, the same result?

Vincent, I did another run, this time with the "scale masterdark" ticked and yes, the adaptive pedestal/amp glow is also ticked.  Same result.  The appearance of Dark & Flat artifacts. I'll attach my Master Dark ffile here.

Thank you for your efforts in helping me.  It is great having someone to turn to for advice.

Dale

This is my Master Dark

Doesn't seem to be very weird at first glance, apart from the big amp-glow signal. Could you upload the masterdark and like 10 light frames?

You can upload them to our server (<- click that link) with username/password: appuser

Please create a directory called "dalemschultz-ampglowproblem" and I'll have a look asap.

Thanks, Vincent

Hi Vincent,

Thinking that perhaps 600s subs were too long for the Sony Sensor, last night I took 26 subs at 300s and re-shot my 300s Darks as well.  This time the Darks were at the same ISO!  I only used 10 of the subs (total, 50 minutes) and I can't see where I did anything different in the tabs 0-6, but the results are good for me.  Here are the 300s Master Dark and the same Flats as before (I'm reshooting the Flats tonight), along with the satisfactory Integrated image before any additional processing.

Dale

Masterdark (300s, ISO 120)

Masterflat

Integrated file

It might also be that the ISO difference was the issue, perhaps in this case (with that amp glow) that doesn't work too well. Anyway, great to see it worked!

@vincent-mod 

Hi Vincent,

I've tried several times to log in to the Server to upload those lights and Masterdark, but I haven't been able to.  I'm using my APP login, Username, and Password, but no access.  I'd still like to understand why the Integration result of a batch of 600s lights and 600s Darks (matching iso) along with Flats, is showing most if not all of the noise from the Masterdark and the Masterflat. Honestly, I'm a bit unclear about the usage of the right-hand side variables.

These are the defaults when I start APP.  Is there a resource where I can review their usage.  I've looked for the manual, but use the hover tools that are present. 

Master Dark, 600s, iso 121.

Master Flat, 0.08s, iso 120.

Integration result!

It's hot here now and I'm only able to get my ZWO sensor down to about -14, so perhaps shooting 600s subs is too long and produces too much noise.  Maybe I should reserve the longer subs for the cooler months?  Thinking out loud...

If you can help me understand the login for the Server, I'll be happy to upload those files.

Many thanks Vincent,

Dale Schultz

Hi Vincent,

This is an outline of my procedure for taking Flats.

1. At night, I'll use an iPad placed on my objective with a blank screen showing, illuminating the sensor.

2. Use the SGP "Flats Calibration Tool" that saves the required exposure times & iso into the "Filters" area of my equipment profile in SGP.

3. Use the "Flats Wizard" in SGP to take the Flats images that targets an exposure +- 500 ADU's from the 30,000 ADU's for that sensor.

4. Take the night's run of Lights.

5. Load those Lights & Flats into APP, then create a Master Flat, during Post.

I think that's it.

Thank you,

Dale Schultz

I too have just had a scenario where my calibration frames are being ignored too. APP 1.080

Processing my tri-band data from my OSC.

I selected

0) Ha-Oiii Extract Ha and it appeared to be ok. See Integration result below:

I then went back to change this to

0) Ha-Oiii Extract Oiii, accepting the fact the data needs to be re-normalized. I didn't change anything else. The result is this mess: 

I never had this issue with the Beta release of 1.080 as I processed similar data in the same way last week with no issues.

I decided to process 0) Ha-Oiii Color and ended up with this:

I'll retry with an older version of APP as well as verify if calibration is happening. My workflow is very simple and I don't have any unusual or incorrect Gain settings etc. My Flats, DarkFlats, Lights and Darks all at same Gain (and offset where applicable).

Correction - I see the same problem in 1.080-Beta2:

I'm now not sure how I managed to process last week's data as I didn't have these issues. Why is this occurring? Why is it my HaOiii Extracted Ha process is giving good calibrated data, but HaOiii Extracted Oiii isn't?

And I hijacked this thread because of the relation to the inability to apply calibrated data, but let me know if this is a separate issue.

** UPDATE: I'm still getting this with 1.079 too so something is wrong - but I really can't work out what. My Ha-Oiii Extract Ha is calbrating fine. My Ha-Oiii Extract Oiii is not being calibrated correctly ** 

Update: The problem in my specific case [and I apologise for distracting in this thread] but it was my Flats that were the problem. The part of the workflow that had changed since last time I processed data was NINA [capture software]. I had updated this since last week. It changed some things in the Flat Wizard and I didn't verify the Flats/DarkFlats it produced. They were far too bright. Processing correct ones and my calibration issues have gone away.

Apologies for the delays, @dalemschultz. To me it looks like the flats are not correct, a nice flat should not have a rectangular spot in the frame (is that the iPad?). You should have the same focus and the sensor should be illuminated fully and I think this might be one of the issues here. I also see that the dark is not corrected, so this makes me think that the calibration frames are either not taken with the proper camera settings or have different dimensions which makes APP not be able to either correct properly or even match them. You can load a light into APP, then you Masterdark and Masterflat and on top of your screen, select “l-calibrated”, in the list APP should then add indications to the light frame that they have been added and the correction is loaded. If they are not added, the data is somehow wrong.

@itarchitectkev Thanks for finding this out on your own! Great to know these things as I can use things like this to find other people’s issues faster too. 🙂

@vincent-mod

Hi Vincent,

Thank you or your patience on this.  I wondered about the pinkish rectangle on the Flats as well. Either way, I'll reshoot them with a different light source tonight.  Then try a new run in APP to check the results.

Thank you,

Dale

No worries, if it doesn’t work still, we can look further into it together. That’s how we learn anyway. 🙂

Hi Vincent,

I've been busy scratching my head about the pinkish rectangle on the Flats images and I've discovered the source of it.  I'm shooting at full focal length (952mm) with my Refractor and I'm using the Optolong L-eNhance dual-band filter.  It has a pink look to the coatings that provide the bandwidth rejection.  The iPad was not the issue since I was pretty sure I had seen the pink rectangle previous to and subsequent to using it.  Today I used the overcast sky to capture T-shirt Flats and the pink rectangle is present when using the L-eNhance Filter, but not when I use my Infrared Filter, which is mostly clear.  SO, that issue is known, but the issue of using the Refractor without the benefit of a Reducer/Corrector shows itself as pretty bad vignetting and probably Coma.  I've included images of the Flat files directly from SGP as well as the files after they are loaded in APP, including both the L-eNhance & IR filters.  Today, I decided to clean all accessible optical surfaces.  I found a tiny spider suspended in its tiny web all the way up the inside of the 127mm Refractor!  Fun!  That means I needed new Flats as well as any Lights, which I'll attempt tonight, clouds permitting!

SGP's view of Flats, LeNh (0.16s, iso 120) & IR (0.03s, iso120).  Note, no pink rectangle on the far left one L-Enhance Filter.

APP's view of Flats, IR (0.03s, iso 120 & LeNh (0.16s, iso 120).  Note the pink rectangle on the right one with the L-eNhance Filter.

Taking Lights tonight, hopefully!

Thanks so much Vincent!

Dale

Well, that’s a lot more knowledge at least! 😉 I also see that you could increase the exposure on your flats, the histogram should be a bit more to the middle. I’m still a bit puzzled by the rectangle, but it must be some optical effect then.

The Full Well for the Sony chip is 63,700 so I shoot or around 30,000, +/-500.  Should I disregard that number and gauge by the histogram being centered?  I think the rectangle comes from the shape of the sensor.  With the scope at F952 and f7.5, there must be quite a bit of an overshoot of the light path outside of the sensor boundaries.  I hope that makes sense.  Typically, I've had a 0.70 Reducer/Corrector inline after the focuser and in front of the filter & camera.  That gave me a scope with F666 & f5.25.  Faster scope and a wider field with correction needed for a Triplet Apo-Refractor.  I wanted to try it at full focal length for some smaller objects.

Anyway, the clouds came in a few hours earlier than forecast last night, so I didn't get a chance to shoot new Lights to test with the newer Flats.  Tonight looks promising.

Best regards,

Dale

Hi Vincent,

Posted by: @dalemschultz

I think the rectangle comes from the shape of the sensor.  With the scope at F952 and f7.5, there must be quite a bit of an overshoot of the light path outside of the sensor boundaries.

Having thought more about this, it doesn't make any sense that there would be any light captured "outside of the edge of the sensor"!  So, I'm not understanding what the shape is at all.  Interestingly though, I don't see that rectangle in the images I capture with SGPro.  Looks promising for imaging tonight.  Jeez, I've said that before!

Dale

Hi Vincent,

This Master Dark is a mess!  I can't imagine that the ZWO ASI294MC Pro that I use can be used for any image over 300s.  Even with all that to be subtracted out of the lights, there's still remnants of the Glow. 

And the Master Flat!?  They were shot correctly with identical Gain values as the Lights and Darks, exposed to allow a central peak in the Histogram (40,000 ADU +/-).  I think I'll try configuring the scope back to F666 & f5.25 by adding in the 0.7x Focal reducer/Corrector to see how the Flats react to that.  Then go from there.

This is the Integrated result from that MD & MF.

This is one of the 600s Lights.

Can you help me understand how to go about logging into the Server to upload the Lights and Masters you requested?   I've tried using my login name and password, but it won't accept those.  Is there another way?

Thank you,

Dale

Posted by: @dalemschultz

The Full Well for the Sony chip is 63,700 so I shoot or around 30,000, +/-500.  Should I disregard that number and gauge by the histogram being centered?

Well that should be in the middle more or less. In Sequence generator pro, I know it always shows a 16-bit image, which is great as you can then always advice to go for that 30.000 ADU target. Maybe you can double-check if you’re actually taking it at that level.

@dalemschultz

So, are you taking the images with different capture programs? Sorry if I missed that somwhere, but it’s always to be advised to use the same capture software, just to be able to rule out differences in data processing between those. I think these problems call for an upload of you set to our server. Could you upload them to a directory named “dalemschultz-flatsproblem”?Link to our server, login with username and password: appuser

Hi Vincent,

I'm using SGPro to capture the Lights, Darks, Flats, and Bias frames.  When I was capturing new Flat frames this week, I noticed that indeed the stored exposure time (derived from the Tool; Flats Calibration) was not giving the actual 30,000 +/-500 ADU's as expected!  Curious lack of user knowledge on display probably!  So I upped the exposures until I saw a mid-range grey rendering that showed all of the artifacts known to be in the light train, and then shot 25 of those. 

I'm not able to log into the Server for some odd reason Vincent.  I'm obviously logged into the site right now and at other times, but when I try to use that link and log in, it comes back with a failure.  Ideas?

What do you make of that 600s Master Dark frame I attached!?  It's awash in Amp Glow!

Thank you,

Dale

Ok, so you did find something odd with SGP there, that’s good! The APP server has a different username and password, just use appuser as both the login and password.

A lot of amp-glow can happen with some sensors indeed. But APP should be able to correct this nicely if you use darks of the same gain and exposure length.