Diagonal pattern in...
 
Share:
Notifications
Clear all

2022-05-29: APP 2.0.0-beta2 has been released !

Release notes

Download links per platform:

windows 2.0.0-beta2

macOS x86_64 2.0.0-beta2

macOS arm64 M1 2.0.0-beta2

Linux DEB 2.0.0-beta2

Linux RPM 2.0.0-beta2

Diagonal pattern in images

Page 1 / 2

(@dtc1999)
White Dwarf Customer
Joined: 3 years ago
Posts: 29
Topic starter  

Hello everyone,

For the last several months I have been troubleshooting an issue without success. I have been getting a diagonal pattern in my images. I am not sure if the issue is within APP, but I have troubleshot the hardware, replaced USB, tried different USB ports, updated camera drivers, etc. I wanted to share my issue here to the hope that someone maybe able to see where the issue is coming from. I will give all info that I can:

The equipment I use is two Redcat51 scopes with ZWO 2600MM and 2600MC cameras. The MM uses a ZWO IR-Cut filter. My sub lengths are 300 seconds at unity gain. Flats and DarkFlats are 2 seconds, with the Flats created with a white t-shirt and an ipad with flashlight app for the light source. Darks are 300 seconds. I dither between every frame. I have recently only imaged with one Redcat and the 2600MM to make troubleshooting easier. 

All settings in APP are at the default other than Tab6, Integrate is set to Average (over 20 subs) and Filter set to adaptive rejection. If I need to create a new MasterFlat, I will shoot 50 and select Average and Adaptive Rejection in the Calibrate Tab2.  

The attached image is to show an example. This is from my last imaging session, which was during a new moon. The image is stretched to the max of 30% to better show the diagonal pattern. The pattern is still visible at lower stretches.

Please let me know if you would like me to upload a number of subs and my master calibration files. I appreciate any help that you can give me as I have been working hard for months to figure this out without much luck..

M101 MONO session 1 crop lpc St

ReplyQuote
(@wvreeven)
Galaxy Admin
Joined: 4 years ago
Posts: 1724
 

@dtc1999 Hi Damian,

I am tempted to say that this is walking noise but I am not entirely sure. Walking noise usually isn't as broad as the lines in your image. A few questions for you:

  • Good that you're dithering! How much do you dither? In order for walking noise to be effectively eliminated, you need to dither at least 15 or 20 pixels in the imaging camera.
  • Is the pattern in your images always in the same direction? If yes, what happens to the pattern if you rotate the camera over 90º?
  • Do you have a fixed set up or do you set it up every time you go imaging? In both cases, are you sure that the polar alignment is OK? How do you do polar alignement?
  • Does the pattern show in the raw lights as well or only after integrating? You may need to stretch a raw light to the extreme in order to be able to see it. If only after integrating, do you see it as well if you integrate without calibration frames?
  • What do you mean with "unity gain"? ASI2600 cameras work best at gain 0 or gain 100.

Depending on your answers I may have more questions 🙂

Thanks, Wouter


ReplyQuote
(@dtc1999)
White Dwarf Customer
Joined: 3 years ago
Posts: 29
Topic starter  

@Wouter-Moderator Thank you for the reply Wouter.

I too thought this was walking noise when I first noticed the issue, but many people in another forum did not believe that it was walking noise, as walking noise will give a much thinner "rain-like" appearance. I am not sure how many pixels I am dithering over the imaging camera. I have entered all the correct info in PHD2 for my guide scope and camera, other than that all settings are default.

I have not yet tried to rotate the camera to see if the pattern follows the change or not. However I am setup to automatically perform meridian flips. I know one occurred in my example image. The pattern stayed the same on both sides of the flip. My thinking this could eliminate an outside stray light source as the cause, correct?

I keep my setup fully built, but I do wheel it in and out of the garage for my sessions. I perform a new polar alignment every time I bring it out. I polar align manually by looking through the pole scope. I am very through when I do my alignment and have been doing it this way for over 5 years. I am very confident that my polar alignments are good.

It is very hard to tell when I look at individual light frames that are uncalibrated. I would be more than happy to upload them along with my calibration masters if you or anyone else wanted to take a look at them.

I do image my 2600s in gain 100. The example image is 100. I have also imaged at gain 0 and got the same results.


ReplyQuote
(@vincent-mod)
Quasar Admin
Joined: 5 years ago
Posts: 4854
 

Could you show us your flat? In linear and stretched form? Linear you get by loading the flat and then switching to "no stretch" in the stretch presets on the right of APP.

image

ReplyQuote
(@dtc1999)
White Dwarf Customer
Joined: 3 years ago
Posts: 29
Topic starter  

Hello @Vincent-Moderator 

Here is an unstretched masterflat file:

Flat no stretch

 

This is the masterflat in the default stretch of 15%BG 3sigma 2,5%base:

Flat 15 stretch

 

These files were uploaded in jpg, let me know if you want to see the fits


ReplyQuote
(@vincent-mod)
Quasar Admin
Joined: 5 years ago
Posts: 4854
 

Thanks, I was wondering if it had to do with the flats maybe. The diagonal lines are very subtle and I think bigger than anything related to walking noise. What happens when you make an integration without any calibration data?


ReplyQuote
(@dtc1999)
White Dwarf Customer
Joined: 3 years ago
Posts: 29
Topic starter  

Here is a stack of 20 of the subs with just a crop and LP removal. Stretched to the max 30%

20 subs only MONO session 1 crop St

ReplyQuote
(@vincent-mod)
Quasar Admin
Joined: 5 years ago
Posts: 4854
 

Ah, there it's super clear indeed, so it's definitely in the data itself. I am wondering if bigger dithering steps will help, normally people tend to do dithering with just a few pixels, but it can be very good to dither with 10-15 pixels even. I would try that first. It does look a little like walking noise here so that may work.


ReplyQuote
(@dtc1999)
White Dwarf Customer
Joined: 3 years ago
Posts: 29
Topic starter  

What is strange is that with only the subs, the pattern is vertical. With calibration files, the pattern is diagonal and looks much wider. 


ReplyQuote
(@dtc1999)
White Dwarf Customer
Joined: 3 years ago
Posts: 29
Topic starter  

Here is one sub that is calibrated, LP removal, and stretched 30%. Although faint, it looks if the diagonal pattern is still there. Would walking noise appear in one sub?

one sub cal

ReplyQuote
(@vincent-mod)
Quasar Admin
Joined: 5 years ago
Posts: 4854
 

@dtc1999 I find it hard to see in the single sub, but I think it's right that walking noise you see only after integration. I think this is where we should ask for your data to have a closer look. 🙂 I suspect multiple noise patterns in your data.


ReplyQuote
(@dtc1999)
White Dwarf Customer
Joined: 3 years ago
Posts: 29
Topic starter  

Thanks Vincent. I have uploaded under the folder name you suggested and included my master calibration files and 24 subs.. 12 subs are before the meridian flip, and 12 after 


ReplyQuote
(@vincent-mod)
Quasar Admin
Joined: 5 years ago
Posts: 4854
 

Thanks for uploading! Please allow for a day or 2 as we're always a bit busy in general. 😉


ReplyQuote
(@vincent-mod)
Quasar Admin
Joined: 5 years ago
Posts: 4854
 

Ok having a first look at the data;

What I notice is that the signal is very faint with a single sub, I'm wondering if that may cause a bit of an issue as it's more difficult to get out of the noise floor. Another thing is that the unstretched histogram is quite far from the left, is your offset very high by any chance?


ReplyQuote
(@dtc1999)
White Dwarf Customer
Joined: 3 years ago
Posts: 29
Topic starter  

My subs are 300 seconds, gain 100, offset 25

How far is it showing you that they are off the left of the histogram? It's showing me that they are roughly 5-10% from the left.

 


ReplyQuote
(@dtc1999)
White Dwarf Customer
Joined: 3 years ago
Posts: 29
Topic starter  

actually I was looking at the FIT viewer histogram. The histogram in APP is showing between 20-30% to the left


ReplyQuote
(@dtc1999)
White Dwarf Customer
Joined: 3 years ago
Posts: 29
Topic starter  

I know you guys are busy, and I appreciate you looking at my data. I noticed something after looking over my imaging software. Between late last year and early this year, which is roughly around the time I noticed the issue happening, I switched from SharpCap to NINA for my imaging software. Around the same time period, I also switched from an OAG back to a regular guide scope. I felt the guide scope was working better than the OAG with my Redcat51. 

What I have noticed was with SharpCap, my dither pixels was set to 10. NINA still had the default setting of 5 pixels. I can't say for certain that this was the cause of the issue. Even though when I changed my guiding system, I did update PHD2 with the new focal length for guiding, the amount of actual pixels I was dithering did change since NINA used a smaller number by default. 

The more I look at my data, while it doesn't resemble walking noise when there is no guiding at all, it could be faint remaining artifacts from not enough dithering. Possibly even Fixed Pattern Noise. For troubleshooting, I have changed the dither pixel value in NINA to 20. Starting on Sunday, I have a stretch of clear skies for 3 or 4 days. I will get a chance to test this changed setting. I am also going to shoot a variety of exposure lengths: 120, 180, 240, and 300 seconds to compare the results. In addition, I will try NINA's Flat Wizard for the first time, as all other calibration frames are made with NINA, but I was still using Sharpcap to create flats as I felt Sharpcap has a much better detailed histogram. Using NINA's Flat Wizard, I will not need to visually inspect a histogram.

I will report back with the results.


ReplyQuote
(@vincent-mod)
Quasar Admin
Joined: 5 years ago
Posts: 4854
 

Great! Yes, I think the effect isn't very big, but indeed is there. My guess is that this might get sorted with a small change in data collection, like the dither increase. Very curious if that solves it!


ReplyQuote
(@dtc1999)
White Dwarf Customer
Joined: 3 years ago
Posts: 29
Topic starter  

After a few nights of clear skies this week, I had a chance to change some dither settings. The first night I set my dither pixels to 20. I timed the dithers from start to finish and they were completing in about 30 seconds. After processing the data, I noticed the pattern was improved a lot, but I was still able to see faint remains of the pattern in aggressive stretches. 

Last night, I increased the dither pixels to 40. I also increased the dither settle timeout to 60 seconds to be safe. Again I timed the dithers and they were now completing in 45-50 seconds. I just finished processing the data, trying to get the pattern to appear in the image. Here are two results from last night with a 15% and 30% stretch. There is 6 hours of 300 second subs, gain 100, and using an IR Cut filter. 

M101 MONO session 1 crop lpc St 15
M101 MONO session 1 crop lpc St 30

The data looks good as far as I can see. Dithering 40 pixels with my setup is actually moving the imaging camera 83 pixels. I am sure that trial and error will show the minimum number needed to dither will fall somewhere between 20 and 40. At first, that seemed very high, but realizing my APS-C imaging camera has a pixel size of 3.76 and the Redcat51 is only a 250mm focal length, a larger number of pixels needed to dither does start to make sense. What do you guys think? 


ReplyQuote
(@dtc1999)
White Dwarf Customer
Joined: 3 years ago
Posts: 29
Topic starter  

Going through all the data I collected while troubleshooting my issue, including the sessions that had dithering set to 5 pixels, I had 36 hours of mono data. I stayed on M101 so it was easier to compare results after making changes. 

Today I decided to go ahead and stack it all to give APP ver2.0 a workout... 433 subs at 5 minutes each.. 6 sessions total. I always keep my OTA built, so I only needed 2 sets of flats and I was able to stack everything in 2 sessions The subs with the diagonal patterns seemed to average out pretty well and completely disappeared when I selected 10% stretch for this image.

M101 MONO session 1 session 2 crop lpc St DeNoiseAI low light

I must say that Ver2.0 handled the data quite well, although I cant give a total completion time because I got a hard drive space warning in the middle of the integration and had to delete a few files to continue. A good suggestion for an upcoming version would be for a sound alert when you get a warning. Just something to get your attention like the Gong sound..

Tomorrow I will start working on my color data.. 


ReplyQuote
Page 1 / 2
Share: