Help with FLAT issue (recently started)

@bscholl What camera do you use? Did the drivers get updated since you last took flats? Are all parameters, like gain, offset, temperature and exposure time the same? Do you still have the raw flats you took previously and, if yes, do you see any differences in the FITS headers? Assuming that the raw flats are FITS files of course!

Wouter

@Wouter,

Nothing has changed.  That's what is so bizzare.  

Offset, Gain, temp, exposure are the same. No change in the  drivers for my QHY183m.

No difference in Fits Header for flats or sub FITS - if I use 2019 FLATS on 2019 subs, works fine.  2020 FLATS on 2019 subs works fine.  And the integrations without flats look the same.  

APP working fine - re-processing old data again has no issues.  

Running same version of SGP for a year now.

So I guess is has to be something with the actual LIGHT subs themselves.  

This couldn't just be condensation from a bad dew heater?  Everything points to it working correctly.

@bscholl What about darks and dark flats or bias? Do you reuse the ones from 2019 or did you create new ones of those as well? Any differences there?

@wvreeven

Nope.  Everything is the exact same.  MasterDarks, Bias, BPM, imaging train, rotation, exposure, drivers, SGP, etc.

It like suddenly every sub I take for the last 30 days doesn't play well with *any* set of FLATS.  

But, *New* flats, *Old* flats, all work fine with *old* subs. 

That's what is so odd.  I can use my *New* FLATS and process 2 year old data just fine.  Just not recent data.

I'm pulling my hair out here.

So weird. I'd suggest for you to upload a few old subs, a few new subs (with this issue) and your calibration files to our server so we can check it out better.

Go to https://upload.astropixelprocessor.com and use upload2 as username and upload2 as password.

Create a directory named “bscholl-flatsissue” and upload in there. Thank you!

@vincent-mod

Thanks.  They are uploading.  

APP has worked flawlessly for me for 2 years.

And I haven't changes SGP, drivers, image train dynamics - once I get something set and working, I leave it alone.....

So it's gotta be something I'm doing.  But can't figure what it is for the life of me.  

Thanks.

Just ran a quick NB from the November.  No issues on that one. 

So I'm at a loss, unless somebody is intermittently shining a flashlight into my scope on just some NB and RGB.

And I've already ruled out the moon 😉 

Files uploaded.  Thanks for any help.

Mmm, so that does suggest, like Wouter was suggesting, something is changing somehow. Anyway, we'll have a look at the data!

@bscholl I had a look at your data and there are a few subtle differences

• The SGP profile used is different and even refers to a different camera
• The SGP version used is different. Are you sure that the divers are the same between the two versions?
• You uploaded master flats instead of raw flats. Can you upload a few of them used for each master flat please?
• Judging by the file names, the APP version used is quite old. Can you upgrade to the latest version and see if that helps?

Thanks, Wouter

@bscholl More questions and remarks 🙂 The flats from 2019 were taken at 0.008 sec integration time and the ones from 2020 at 0.002. However, the CALIB value (not sure what that is) for the 2019 master flat is set to 32618 and for 2020 to 41639.5. Any idea where that CALIB difference comes from?

I am not very familiar with QHY cameras but I'd advice you to make sure that you can take flats of around 1 to 2 seconds either by dimming the light sources that you are using or by closing off the front of the telescope with a white t-shirt, possibly folded double several times. Making GOOD flats is essential and it seems to me that the 2020 flats are not good enough. We will be able to tell you when you upload a few of them (don't forget the 2019 ones please) so we can inspect them.

@wvreeven

Went back and looked at my data.....  

So :

• I've been running SGP v3.1.0.454 and APP 1.077 since 2/22/2020.  No camera drivers updated since 2/22/2020 according to the SGP directory.  
• On or around October 2020, it was almost like my FLATS (from 12/2019) started to go *bad*, like too many photocopies of a photocopy used to do.  Although that isn't, of course, possible digitally.
• The day of my post to the forum, I upgraded to most recent 1.082? APP  - hoping it was some setting in the software and operator error - but, clearly that's not the case as 2019 FLATS and 2020 FLATS work with 2019 LIGHTS.  But anything taken say OCT 2020 til now gets the white spots.
• I don't have the old flats originals, just the master.

So now I'm figuring it has to be something with my LIGHT subs (since my new flats work across years of data, until we hit Oct 2020).

Timeline examples :

M21 RGB        4-16-2020    SGP v3.1.0.454        APP 1.077

M16   (5 subs HA)    8-13-2020     SGP v3.1.0.454      APP 1.077

Sh2-231 Ha 10-3-2020         SGP v3.1.0.454         APP 1.077

NGC1333  RGB    11-15-2020      SGP v3.1.0.454         APP 1.077

and you know the rest..

At the end of the day, it is just darn perplexing that nothing has changed in my system since 2/20/2020 and Flats from December 2019 and December 2020 work perfectly on *any* data prior to October 2020.  But not after October 2020 (except for that one Ha of the California Nebula)

I hope my camera isn't going bad.

Thanks.

But what about Wouter seeing different SGP profiles, with a different camera setting? That alone might already change things I think.

The 2019 data was under a different profile "name", but same settings.  Turns out, that example was a different version of SGP... but I used that in my original post just to illustrate the time differential - and that *all* my flats seem to work with any data prior to Oct 2020.  Just not moving forward.  Even that new set of flats taken 4 days ago.

CREATOR = 'Sequence Generator Pro v3.0.1.0' / Capture software
PROFILE = 'QHY 163 M' / SGPro profile name

However, from Feb 2020 til present, no change in Camera profile in FITS headers.  All of mine look like the following example images I posted....

The Good:

M21 :

SIMPLE = T / file does conform to FITS standard
BITPIX = 16 / number of bits per data pixel
NAXIS = 2 / number of data axes
NAXIS1 = 4656 / length of data axis 1
NAXIS2 = 3522 / length of data axis 2
BZERO = 32768 / offset and data range to that of unsigned short
BSCALE = 1 / default scaling factor
CRPIX1 = 2328 / reference spectrum pixel coordinate for axis 1
CRPIX2 = 1761 / reference spectrum pixel coordinate for axis 2
CTYPE1 = 'RA---TAN' / standard system and projection
CTYPE2 = 'DEC--TAN' / standard system and projection
OBJECT = 'M21 ' / Object name
DATE-LOC= '2020-04-16T22:41:32' / Local observation date
DATE-OBS= '2020-04-17T03:41:32' / UTC observation date
IMAGETYP= 'LIGHT ' / Type of frame
CREATOR = 'Sequence Generator Pro v3.1.0.454' / Capture software
PROFILE = 'Observatory 183m' / SGPro profile name
INSTRUME= 'QHYCCD-Cameras-Capture' / Instrument name
FOCUSER = 'NiteCrawler Focuser' / Focuser name
FOCPOS = 8799 / Absolute focuser position
FOCTEMP = 10.5666666666667 / Focuser temperature
FWHEEL = 'Starlight Xpress Filter Wheels' / Filter Wheel name
FILTER = 'G ' / Filter name
EXPOSURE= 120 / Exposure time in seconds
CCD-TEMP= -10 / Camera cooler temperature
SET-TEMP= -10 / Camera cooler target temperature
XBINNING= 1 / Camera X Bin
CCDXBIN = 1 / Camera X Bin
YBINNING= 1 / Camera Y Bin
CCDYBIN = 1 / Camera Y Bin
XPIXSZ = 3.8 / Pixel Width in microns (with binning)
YPIXSZ = 3.8 / Pixel Height in microns (with binning)
TELESCOP= 'EQMOD HEQ5/6' / Telescope name
RA = 270.842320981087 / Object Right Ascension in degrees
DEC = -22.6991015625 / Object Declination in degrees
CRVAL1 = 270.842320981087 / RA at image center in degrees
CRVAL2 = -22.6991015625 / DEC at image center in degrees
OBJCTRA = '18 03 22.157' / Object Right Ascension in hms
OBJCTDEC= '-22 41 56.766' / Object Declination in degrees
AIRMASS = 2.0481605844973 / Average airmass
OBJCTALT= 29.3222164557419 / Altitude of the object
CENTALT = 29.3222164557419 / Altitude of the object
FOCALLEN= 840 / The focal length of the telescope in mm
FLIPPED = F / Is image flipped
ANGLE = 270.36 / Image angle
SCALE = 0.900805 / Image scale (arcsec / pixel)
PIXSCALE= 0.900805 / Image scale (arcsec / pixel)
POSANGLE= 181.653762817383 / Camera rotator postion angle (degrees)
GAIN = 174 / Camera gain
EGAIN = 1 / Electrons Per ADU

M16 :

SIMPLE = T / file does conform to FITS standard
BITPIX = 16 / number of bits per data pixel
NAXIS = 2 / number of data axes
NAXIS1 = 4656 / length of data axis 1
NAXIS2 = 3522 / length of data axis 2
BZERO = 32768 / offset and data range to that of unsigned short
BSCALE = 1 / default scaling factor
CRPIX1 = 2328 / reference spectrum pixel coordinate for axis 1
CRPIX2 = 1761 / reference spectrum pixel coordinate for axis 2
CTYPE1 = 'RA---TAN' / standard system and projection
CTYPE2 = 'DEC--TAN' / standard system and projection
OBJECT = 'M16 ' / Object name
DATE-LOC= '2020-08-08T15:57:03' / Local observation date
DATE-OBS= '2020-08-08T20:57:03' / UTC observation date
IMAGETYP= 'LIGHT ' / Type of frame
CREATOR = 'Sequence Generator Pro v3.1.0.454' / Capture software
PROFILE = 'Observatory 183m' / SGPro profile name
INSTRUME= 'QHYCCD-Cameras-Capture' / Instrument name
FOCUSER = 'NiteCrawler Focuser' / Focuser name
FOCPOS = 9580 / Absolute focuser position
FOCTEMP = 24.82 / Focuser temperature
FWHEEL = 'Starlight Xpress Filter Wheels' / Filter Wheel name
FILTER = 'Ha ' / Filter name
EXPOSURE= 300 / Exposure time in seconds
CCD-TEMP= -10 / Camera cooler temperature
SET-TEMP= -10 / Camera cooler target temperature
XBINNING= 1 / Camera X Bin
CCDXBIN = 1 / Camera X Bin
YBINNING= 1 / Camera Y Bin
CCDYBIN = 1 / Camera Y Bin
XPIXSZ = 3.8 / Pixel Width in microns (with binning)
YPIXSZ = 3.8 / Pixel Height in microns (with binning)
TELESCOP= 'EQMOD HEQ5/6' / Telescope name
RA = 274.712749890913 / Object Right Ascension in degrees
DEC = -13.9102734375 / Object Declination in degrees
CRVAL1 = 274.712749890913 / RA at image center in degrees
CRVAL2 = -13.9102734375 / DEC at image center in degrees
OBJCTRA = '18 18 51.060' / Object Right Ascension in hms
OBJCTDEC= '-13 54 36.984' / Object Declination in degrees
AIRMASS = 1.56529458828298 / Average airmass
OBJCTALT= 39.8647913501987 / Altitude of the object
CENTALT = 39.8647913501987 / Altitude of the object
FOCALLEN= 840 / The focal length of the telescope in mm
FLIPPED = F / Is image flipped
ANGLE = 271.05 / Image angle
SCALE = 0.899225 / Image scale (arcsec / pixel)
PIXSCALE= 0.899225 / Image scale (arcsec / pixel)
POSANGLE= 181.653762817383 / Camera rotator postion angle (degrees)
GAIN = 174 / Camera gain
EGAIN = 1 / Electrons Per ADU
END

The Bad : 

SH2-231 : Threw away the lights, only have the APP integration...

NGC 1333 :

SIMPLE = T / file does conform to FITS standard
BITPIX = 16 / number of bits per data pixel
NAXIS = 2 / number of data axes
NAXIS1 = 4656 / length of data axis 1
NAXIS2 = 3522 / length of data axis 2
BZERO = 32768 / offset and data range to that of unsigned short
BSCALE = 1 / default scaling factor
CRPIX1 = 2328 / reference spectrum pixel coordinate for axis 1
CRPIX2 = 1761 / reference spectrum pixel coordinate for axis 2
CTYPE1 = 'RA---TAN' / standard system and projection
CTYPE2 = 'DEC--TAN' / standard system and projection
OBJECT = 'ngc1333 ' / Object name
DATE-LOC= '2020-11-15T14:30:55' / Local observation date
DATE-OBS= '2020-11-15T20:30:55' / UTC observation date
IMAGETYP= 'LIGHT ' / Type of frame
CREATOR = 'Sequence Generator Pro v3.1.0.454' / Capture software
PROFILE = 'Observatory 183m' / SGPro profile name
INSTRUME= 'QHYCCD-Cameras-Capture' / Instrument name
FOCUSER = 'NiteCrawler Focuser' / Focuser name
FOCPOS = 3314 / Absolute focuser position
FOCTEMP = 6.78666666666667 / Focuser temperature
FWHEEL = 'Starlight Xpress Filter Wheels' / Filter Wheel name
FILTER = 'B ' / Filter name
EXPOSURE= 120 / Exposure time in seconds
CCD-TEMP= -10 / Camera cooler temperature
SET-TEMP= -10 / Camera cooler target temperature
XBINNING= 1 / Camera X Bin
CCDXBIN = 1 / Camera X Bin
YBINNING= 1 / Camera Y Bin
CCDYBIN = 1 / Camera Y Bin
XPIXSZ = 3.8 / Pixel Width in microns (with binning)
YPIXSZ = 3.8 / Pixel Height in microns (with binning)
TELESCOP= 'EQMOD HEQ5/6' / Telescope name
RA = 52.1122124866324 / Object Right Ascension in degrees
DEC = 31.33359375 / Object Declination in degrees
CRVAL1 = 52.1122124866324 / RA at image center in degrees
CRVAL2 = 31.33359375 / DEC at image center in degrees
OBJCTRA = '03 28 26.931' / Object Right Ascension in hms
OBJCTDEC= '+31 20 00.937' / Object Declination in degrees
AIRMASS = 1.26362209825794 / Average airmass
OBJCTALT= 52.5588791531984 / Altitude of the object
CENTALT = 52.5588791531984 / Altitude of the object
FOCALLEN= 840 / The focal length of the telescope in mm
FLIPPED = F / Is image flipped
ANGLE = 270.15 / Image angle
SCALE = 0.900235 / Image scale (arcsec / pixel)
PIXSCALE= 0.900235 / Image scale (arcsec / pixel)
POSANGLE= 181.653762817383 / Camera rotator postion angle (degrees)
GAIN = 174 / Camera gain
EGAIN = 1 / Electrons Per ADU
END

@bscholl OK thanks. We are just trying to make absolutely sure that nothing has changed  in that sense. The FITS headers you posted are for lights judging by the exposure times. But my request remains: can you make available flats so we can look at those?

@wvreeven

Uploaded into the 2020 FLATS folder.  I don't have any left from 2019, just masters.

Somebody else mentioned maybe an offset problem?  I don't have it set in SGP, and in ASCOM it is set at 77.  Its not specified in FITS header.   Since I don't let SGP specify offset, I'm assuming it always takes at 77?

Could that be an issue?  SGP logs are un-revealing for differences (offset is always listed as 100 going back to 2019 - don't know where that value comes from - but it is consistent)

Thanks, I’ll have a look as soon as possible. Yes, offset can certainly be an issue. All lights and all calibration frames need to have the same offset. Pity that SGP doesn’t store the value in the FITS headers. Is there any way you can find out what value was used for all the frames?

@wvreeven

I'm guessing SGP doesn't record offset because I haven't selected any.  It should default to whatever the ASCOM setting is.  Which is 77.

I'm going to run some new darks, flats and lights next clear night and specify offset in SGP and see if that's the problem.

But still strange that this problem randomly appeared October despite nothing changing.

@bscholl I agree that it is strange. Perhaps your question belongs in the SGP forum and not here?

@bschollI have had a look at your flats and they, indeed, look OK. This strengthens your and my suspicion that the OFFSET is different.

So odd.  I'll re-aquire everything and see if it helps.

Thanks.