[Sticky] Very strange integration result - check if you're using a FAT32 filesystem

Oh dear, that is not right indeed. What did you exactly load into APP, the raw data frames of the dslr? And what APP version are you using?

@vincent-mod

Hello Vincent,

thank you for your reply!
I'm using the DSLR Canon Lights. The data files have the ending "CR2" and I'm using the trial version of APP (wanted to test it before purchasing)... the version ist 1.079. Additional information: I stacked a series of Lights of M13 and here I had no Problems. All attempts with galaxies (M51, M101) failed like the Orion Nebula in the picture I posted.

Hi Erich,

I was just browsing the forum and am intrigued by your problem.  Looking at your screen-shot it seems to me that there was perhaps a failure during star registration as I see irregular lateral movement/repeating ghost patterns in the integrated image.

I suggest you

a)  Look at your subs after initial loading with the viewer set to 'Image' mode:  If the individual subs look OK here then I think APP must have correctly recognised the CR2 Bayer CFA pattern

b)  After Star Registration, look at your subs with the viewer set to l-c-registered mode:  Focus attention on an important part of the image - eg the center of M42. This should not change position in the viewer window as you move through the subs. You will certainly see small movement at the frame edges owing to dither movements and/or polar alignment issues at capture.

c)  IAs a result of failed registration, I think that a few of your frames will show significant lateral movement as you page through them. - giving rise to the ghosting when integrated.    I would inspect these subs closely and decide if they are reasonable or poor quality.  If quality is poor I suggest you omit these subs, but if reasonable quality I suggest experimenting with the parameters on Tab 4) Registration to see if you can get correct alignment.

I do not know why you have such severe banding in the integration.  Do you see any similar banding in the individual subs - look with a high stretch eg 30% bg, 2 sigma, 0.0% base on the right hand panel ?

You do say whether you  used any calibration frames but maybe there is a problem here calibrating out read noise.   For what its worth, I once had a problem somewhat like yours that I eventually traced to low voltage at my camera that exhibited as banding like yours in my subs.

Hope you get a decent result soon.

regards

Mike

I've had this very same problem recently. It went away when I made sure I had enough disk space for the integration. APP warns if there is too little disk space, but in my case (old macbook pro with external SSD) happily continues with the integration even when there is not enough space, with the following results. Identical setting in APP, just more disk space (500 GB!) for the picture that looks good:

I am actually having the same issue. I've checked to make sure the star registration was good. I also checked my HDD space and it says I have 747gb. Any other ideas to try and figure this out? 1st photo is without Master Dark Frames; the last photo was calibrated with dark frames. 

Thanks in advance!

Very, very weird. Ok I'll inform Mabula for this one, we probably need some of the data to investigate, thank you all for notifying!

Hi Erich @e-nowaczinski @mestutters @ralph @codeandlight & @vincent-mod,

I think I know why this is happening, I have seen it before. Please check what kind of filesystem you have on the partition where your work directory is/was when you were processing the data.

If it is FAT32, that will be the cause of the problem. That is a very old filesystem only capable of processing files with a file size limit of 4GB. Chances are that the amount of data that APP needs to process is larger than this and then odd results like this could happen.

If the filesystem is however NTFS or ExFAT or a modern linux/macOS filesystem, then it should work properly. In that case, please share the whole dataset so I can solve this problem as soon as possible.

You can upload it here:

https://ariesprodstor.astropixelprocessor.com:7001/

use login and password: appuser

and please make a folder with your name 😉 so we can find it and then let us know when you have finished the upload.

I will make a note and try to prevent this problem on Fat32 filesystems though.

EDIT: if you use fat32, try to format the drive first with ExFAT to solve this issue 🙂 !

Kind regards,

Mabula

@mabula-admin

Hi Mabula, thank you a lot for the detailed response! ... I looked it up and the file-system is FAT32. I will try later to run the dataset again and post the result.
Thanks again!!

Erich

If it's on an external drive, ExFat will be a lot better to have as a filesystem.

Posted by: @vincent-mod

If it's on an external drive, ExFat will be a lot better to have as a filesystem.

Thanks Vincent @e-nowaczinski @mestutters @ralph @codeandlight & @vincent-mod

Indeed, ExFAT is not a lot better only, in fact it is the solution to the problem 🙂 ! I forgot to mention that in my initial response.

Mabula

@mabula-admin thank you! I do have it in an external drive. I will need to check when I get home from work. Thanks for the response and hopefully, that is the solution! 🙂

@codeandlight

@mabula-admin

Here is the result after copying the M42 files to my SSD (NTFS). Looks like that the problem was solved! 😉
Thank you a lot for the great support!

Clear Skies

Erich

@e-nowaczinski @mabula-admin @vincent-mod

This is great news! I just got home and checked my external HD and it is formatted to FAT32!

I will reformat to exFAT and try to process again. 

@e-nowaczinski 

That did it! I reformated my extHD to exFAT. Then, reprocessed my data from NGC4169. This is a screenshot right after integration! 

Thank you @mabula-admin @vincent-mod for all the help! 

@codeandlight @mabula-admin @vincent-mod

supplementary information:

I tried to stack the files which are saved at my HDD (FAT32-formatted) but this time chose the APP working directroy at my SSD (NFTS-formatted). This also worked properly. Therefore it seems that the format (FAT32 or NTFC etc.) of the working directory where APP is "working on" is crucial if the error appears or not.

Thank you once again for the help!

CS

Erich

Superb! Ok, now I also know to remember this as a possibility when the result makes no sense whatsoever. 🙂

I am also trying to put APP through it's paces before purchasing, and have had similar issues with horizontal banding, but my external hard drive is using  NTFS.

Should I reformat to exFAT?

Side note, that i have not been getting the horizontal banding with DSS

No, I think that should be fine. Can you post a screenshot of what you're seeing?

@vincent-mod

My file is too large to send, I will send via DropBox and Zip it for you.

https://www.dropbox.com/t/bJ9oIJmp3ynPMYuq

Duh,  you said Screen shot

Mm, that looks more like a data issue. I can have a look at the data if you want.

You can upload some to our server (<- click that link) with username/password: appuser

Please create a directory called "tmyers-stripes" and I'll have a look asap.

Thanks, Vincent

To be honest, I did have some reservations about some of my data. This image from the stack above was from three separate nights and I did have some issues the first couple nights. Rather than upload a mountain of bad data I decided to re-process everything using only the last nights imaging. I believe it is much better and during processing APP rejected 2 frames in Star Analysis and 5 more during normalization, so I don't think 7 out of 200 is that bad. I will let you know how that turns out.

I do have a question though, when using the Load tab I get a window with lots and lots of possible channels in which to load my frames. Since I have a one shot color camera, ASI1600 MC Cool, I chose RGB, IS that correct or should I have chosen something different?

If you only shot RGB lights, then that is the correct choice yes. I'll await your new results! 🙂

Well, it did it again, how many frames do you want to see of my data?

And for what it is worth, I don't get this banding in DSS

i seem to have lost my help!!

No, you didn't loose me. 🙂 I can't be here every day, although I try to. I'm a bit busy with family for 2 days, so please bear with me, I'll be back to action on Thursday. Sorry for that.

In the mean time you can upload like 10 lights and 10 of the calibration files you use. You can use the link a few comments above.

No need to apologize, I was really just kidding, I figure you have a life outside of this forum . I tried to log into the file access, but did not notice you had provided a special password. Tried to use mine for the forum and am now locked out. i am assuming I will need a reset because I am now blocked.