Share:
Notifications
Clear all

Removing Bad Rows


(@ckwadrat)
Brown Dwarf Customer
Joined: 3 years ago
Posts: 12
Topic starter  

I am new APP user that is using the iTelescope system to image with its calibrated filter frames.  I really like APP's ability to remove hot pixels and bad columns.  I followed iTelescope's APP CCD LRGB tutorial by Dr.Sasse and was easily able to reproduce the tutorial results with the supplied downloadable data.  I then used APP to preprocess some of my own CCD LRGB data of M101 from another iTelescope astrograph. 

The problem that I immediately ran into was the presence of a bad row in addition to bad columns.  I was able to remove the bad columns easily, but the bad row remained.  I finally had to abandon using APP for another program to eliminate the bad rows and columns.  My question is: can I get APP to eliminate bad rows as easily as it removes bad columns?   I have not located any information that allows me to easily remove bad rows with APP.  Any information the you can supply me with will be greatly appreciated.

Thanks,

Carl


ReplyQuote
(@vincent-mod)
Universe Admin
Joined: 6 years ago
Posts: 5268
 

Mmm, bad rows in addition to bad columns sounds a little like a sensor that needs to be replaced. But, dithering might help with that as well, did you select dithering for your session?


ReplyQuote
(@ckwadrat)
Brown Dwarf Customer
Joined: 3 years ago
Posts: 12
Topic starter  

Hello Vincent,

Encountering bad rows are not as frequent as bad columns.  I've only encountered a bad row on one iTelescope CCD Camera sensor so far.  I finally used PixInsight cosmetic correction to take out the bad rows and columns.  CCDStack has the same cosmetic correction functionality as PixInsght.  The iTelescope sensor is not bad,  the data just needs to be cosmetically corrected in both bad rows and columns.  So far, I have not been able to find any information about removing bad rows in APP. 

I just started using APP, so I am not familiar enough with the APP software to know if there is a way to handle the removal of bad rows in APP.  Having to use PixInsight to cover a possibel short coming in APP is not that convenient but it is doable.  The cosmetic correction in APP on lists bad pixel and bad column removal options.  This is the only shortcoming of APP that I have found so far in my first attempts at using APP.  Perhaps there is a building defect map option in APP that I am not familiar with like that which exists in PixInsight and CCDStack that would take care of bad rows in addition to bad columns? 

Thanks,

Carl

 

 


ReplyQuote
(@vincent-mod)
Universe Admin
Joined: 6 years ago
Posts: 5268
 
Posted by: @ckwadrat

Hello Vincent,

Encountering bad rows are not as frequent as bad columns.  I've only encountered a bad row on one iTelescope CCD Camera sensor so far.

Exactly, which is why I think they should be replacing that one.

 I finally used PixInsight cosmetic correction to take out the bad rows and columns.  CCDStack has the same cosmetic correction functionality as PixInsght.  The iTelescope sensor is not bad,  the data just needs to be cosmetically corrected in both bad rows and columns.  So far, I have not been able to find any information about removing bad rows in APP. 

It doesn't have a specific "bad row" option, but dithering might help a lot with that and maybe a BPM (Bad Pixel Map). Not 100% sure as I've never seen a bad row in my workflow. A BPM can be made by using a long dark and (when cold pixels are there) a flat.

I just started using APP, so I am not familiar enough with the APP software to know if there is a way to handle the removal of bad rows in APP.  Having to use PixInsight to cover a possibel short coming in APP is not that convenient but it is doable.  The cosmetic correction in APP on lists bad pixel and bad column removal options.  This is the only shortcoming of APP that I have found so far in my first attempts at using APP.  Perhaps there is a building defect map option in APP that I am not familiar with like that which exists in PixInsight and CCDStack that would take care of bad rows in addition to bad columns? 

If the above options don't fix it I'll ask Mabula if this is interesting as an additional option. Thanks for the suggestion!

 


ReplyQuote
(@ckwadrat)
Brown Dwarf Customer
Joined: 3 years ago
Posts: 12
Topic starter  

Hi Vincent,

The iTelescope data was dithered. That didn't remove the predominant bad row.  It only reduced its intensity to some degree and introduced multiple visible bad row artifacts after registering the frames.  The fact that other processing programs have both bad columns and bad row removal as part of their cosmetic corrections processes indicates that it is useful and that it is acknowledged as a possible problem that can be encountered and taken care of without the need to replace the sensor. 

It seems to me that covering both bad columns and rows is reasonable given the fact that I eventually used PixInsight's cosmetic correction process to take care of both the bad rows and columns by constructing a bad row and column defect map.  The easy bad column removal feature of APP is one of several reasons that I started to use APP.  I was disappointed to run into the missing bad row feature in APP and had to resort to PixInsight to solve the problem. 

I was lucky since I already had PixInsight and knew how to use its cosmetic correction process to solve the problem. What has attracted me to APP was easy of use.  I do not feel that it should be necessary for APP to rely on PixInsight to solve a useful APP shortcoming even if it is a low frequency occurrence to encounter data with bad rows.  Fortunately, my other data sets from other iTelescope astrograph sensors only have bad columns and they are easily removed by via APP.  

Thanks,

Carl


ReplyQuote
(@vincent-mod)
Universe Admin
Joined: 6 years ago
Posts: 5268
 

Yes I understand, so that might be interesting although it's usually not a thing in sensors (more likely some dark current or something like that). Btw, this should be present in the masterdark as well, doesn't that work?

If not, I'd love to see the data myself to check.


ReplyQuote
(@ckwadrat)
Brown Dwarf Customer
Joined: 3 years ago
Posts: 12
Topic starter  

The data that I acquired was automatically calibrated by iTelescope as one of their data preprocessing options using their Maxim D/L.  Looking at the data is not worth the effort since I dealt with the problem using PixInsight.  Online Maxim D/L documentation indicates manual removal of bad rows is available just like PixInsight does in its cosmetic correction features. If I have anymore bad row data in the future, I'll use PixInsight to handle the job since it works quite well. 

My original question of handling bad rows was based on the possibility that I am a new user of APP and that I may have been overlooking another way to deal with removing bad rows in APP.  If APP has no functionality for removing bad rows I'll keep using PixInsight to solve the problem.


ReplyQuote
(@vincent-mod)
Universe Admin
Joined: 6 years ago
Posts: 5268
 

Ah, so I think that might be the actual issue here. I know that some calibration data at iTelescope hasn't been the best. They have a new manager now so that should be taken care of, but I would advice to download the uncalibrated data and their master-calibration files. This will tell us a lot, besides being able to use APP's calibartion process, which is better anyway (and the reason iTelescope is switching to APP as well). A proper dark should be able to take care of this and this is likely the reason APP doesn't have such an option, it shouldn't be needed.


ReplyQuote
(@ckwadrat)
Brown Dwarf Customer
Joined: 3 years ago
Posts: 12
Topic starter  

Hi Vincent,

I acquired the data with the bad rows a few years ago using the T16 astrograph at iTelescope in Spain.  I took it from my image archives a week or so ago and then and was in the process of using it with APP when I experienced the bad row problem.  I don't have access to the old camera calibration data since I used iTelescope provided Calibrated files.  I believe that the camera used on T16 at the acquisition time was an SBIG STL-1000M. 

I just looked at the iTelescope site and the current camera on T16 is the Moravian G4-16000.  I have not used T16 with the new camera so I don't have any data from it. 

I regularly use iTelescope astrographs in New Mexico and at the Siding Springs Observatory in Australia.  All the data from these astrographs is automatically precalibrated by iTelescope using Maxim D/L.  I never download the Master Darks, Flats, and Bias files since I am currently satisfied with using the precalibrated data download option.  So far, the only problem with the bad rows has been with my archived Calibrated data from T16.  The data from all the other astrographs in New Mexico and Australia only have bad columns which APP handles easily (so far).  

Whether or not the problem of a bad row is caused by an irregularity with the sensor electronics is irrelevant to me when I already have bad rows in 4 hours of archived data and want to correct the problem at hand via processing.  PixInsight processing solved the problem at hand under the circumstances. 

According to the PixInsight documentation, bad row along with columns occur and its software has a way to deal with it using it cosmetic correction process.  The PixInsight process worked well on my T16 data and I now have cosmetically corrected data without bad rows.  My original inquiry was in effect whether or not APP had this bad row removal capability and that maybe I had overlooked the functionality since I just started using a licensed copy of APP a few weeks ago.

Thanks,

Carl


ReplyQuote
(@vincent-mod)
Universe Admin
Joined: 6 years ago
Posts: 5268
 

Ok, yes I understand Carl, so that might still be interesting to add as a capability, I'll mention this to Mabula.

ps. Would you like to share the data still? Like to try if it still can be done maybe.


ReplyQuote
Share: