15th Feb 2024: Astro Pixel Processor 2.0.0-beta29 released - macOS native File Chooser, macOS CMD-Q fixed, read-only Fits on network fixed and other bug fixes
7th December 2023: added payment option Alipay to purchase Astro Pixel Processor from China, Hong Kong, Macau, Taiwan, Korea, Japan and other countries where Alipay is used.
HI!
my name is Riccardo, and I am a new (trial) user of APP.
I am trying to calibrate images created with a ZWO, but every time I try to preview the FITS file I get this error:
"This fits file seems to be incomplete and / or corrupt Error message: IO error: java.io.IOException: Invalid FITS Header "
With other programs I can load and align files.
Can you help me?
That sounds like the file itself does have a problem somehow. In fact the FITS header is the issue, so while other programs may load it, they must be ignoring that while APP is using it to judge what image it is. What program did you use for shooting and saving the images?
Hi!
Voyager on Zwo Asi last driver.
this is my FITS file created with Voyager.
Mm, I'm not familiar with Voyager, do you still have the raw data before using Voyager? If so, please try to load those, I suspect Voyager isn't writing the Fits header properly. I'll check the fits as well here.
Having said that, I also can load it in another package, so I'll forward this one to Mabula to see what the issue is if at all. Thanks for sharing!
Hi!
Ufortunately I have no raw data, voyager writes the file directly in the updated format. I am waiting for any developments.
Riccardo.
Mm, that's something you can't change in preferences? I would advice to always keep the original data when possible. There should be no need to process the raw data in between as APP is able to do this together with the raw calibration data.
HI!
my opinion APP expects to find a value in the header that is not written by vayager in the FITS file. But all the other programs read without problem the FITS files generated by this program APP seems to be the only one to have this type of problem (pixinshight, maximDL, DeepStaker, FITS Viewer read without problems)
Yes this could very well be, I've forwarded the file to Mabula so I'm waiting for his analysis. The original raw data however will very likely work anyway.
Hi, any news on the problem? If the trial license expires I will not be able to test this software.
Best Regards, Spectre
Not yet no, Mabula is working hard on the next version, when he doesn't respond in a few days I'll notify him again (please ask here again as well then, so we're sure we can get you the help needed).
Hi Riccardo @spectre_68,
I am investigating your voyager fits file right now. It does seem that the FITS is not created properly according to FITS NASA specification. We use a NASA library to READ/WRITE FITS files, so a properly created FITS would not give this error I think... but let me investigate a bit more for now. I will report back later 😉
Mabula
Hi Riccardo @spectre_68,
After more investigation, it is becoming more clear now that this FITS file has a serious problem in terms of the FITS specification. I can only conclude at the moment that something went wrong in Voyager when the file was written to your harddisk. Do the other Fits files from voyager work properly?
Mabula
Hi,
my name is Paolo,
same problem encountered with images from a Starlight Xpress Trius-SX26C Color - ATEO2A Insight Observatory -
, see FIT attached. (change type file from .txt to .zip and dezip the file)
Hi,
I don't know, but other files always downloaded from insight observatory's Starbase platform - from mono sensors like FLI Proline 16803 or SBIG STL11000 - I work very very well with A.P.P.
Hi,
problem solved, I opened and saved all files with M DL. Now it would seem that A P P works them.
Hi all @mpemme, @wvreeven, @spectre_68, @vincent-mod
Together with Leonardo Orazi, developer of Voyager, we have found the problem. The FITS writer code that Voyager uses has a bug in terms of FITS specification.
The FITS headers are not closed properly according to FITS specification.
https://fits.gsfc.nasa.gov/fits_primer.html
"Every HDU consists of an ASCII formatted `Header Unit' followed by an optional `Data Unit'. Each header or data unit is a multiple of 2880 bytes long. If necessary, the header or data unit is padded out to the required length with ASCII blanks or NULLs depending on the type of unit.
The header is padded with additional blank records if necessary so that it is a multiple of 2880 bytes (equivalent to 36 80-byte keywords) long. Note that the header unit may only contain ASCII text characters ranging from hexadecimal 20 to 7E"
The problematic voyager files have the HDU bytes filled up with 00 which is not allowed in the FITS specification. Those 00 should be 20 in hexadecimals... which is 32 in decimals... which is the ASCII blank character.
I understand from Leonardo that the FITS writer code in Voyager is a port of the same NASA library that we use and so this port was not done correctly then... Apparently, even the ASCOM driver of ASI has this bug it seems then.
I will make sure that APP will catch this bug and deals with it so the files can still be read in APP's next release 😉 allthough the problem is caused by the capture software or driver.
Cheers,
Mabula
I have actually just run in to this problem with a few .fit files shot using Voyager and a QSI6162 camera. I tried using the current beta but that still threw an invalid fits header error. Is the workaround that was based on the ASI cameras fully implemented in the current beta 1? Can I send a file via wetransfer or similar? The output of the 6162 is 30.9 meg so I can't attach a sample file to examine here.
Incidentally, I have always used an ASI camera with Voyager (First an ASI294MC Pro and now an ASI2600MC Pro) without ever seeing this error, it must be specific to particular cameras or drivers.
I just realised that I think I have misread release notes for 1.83 beta 2 as applying to beta 1. If that is the case I will wait with bated breath for the beta 2 release, hopefully in the coming days.
@the_bluester Yes there will be better handling of some types of corrupt FITS files in 1.083 beta 2. If you want to know if your files can be opened with that version then please upload a few files to
https://upload.astropixelprocessor.com/
using upload5 for both username and password. Please create a directory called the_bluester_corrupt_fits and place the files there. As soon as the files are there I will ask Mabula to have a look at them.
Thanks, I have uploaded two, even one would probably do. My friend is considering a move to Voyager so it would be good to know up front that 1.083 will open files produced by it from his camera.
Hi @the_bluester & @wvreeven,
Yes, APP 1.083-beta2 has the fix for this problem:
WORK-AROUND IN FITS READER, FOR APPLICATIONS THAT USE A FAULTY FITS WRITER: some users reported problems with some FITS files where APP could not read these, but other applications could. The topic is : https://www.astropixelprocessor.com/community/camerasupport/invalid-fits-header-on-zwo-asi-178mc/ Some of the problem FITS files come from the Voyager capture software developed by Leonardo Orazi. Together with Leonardo, we have found the root cause of the problem. It turned out that the FITS writer code that is used in Voyager is a badly ported version of the nasa java fits library that we use in Astro Pixel Processor. The ported version is not following the FITS specification for how the FITS Header bytes should be filled with byte values once all metadata has been stored. So the root cause is not in APP, the root cause of the problem is in the capture software or the ascom driver that such a capture package uses. Leonardo has fixed this issue in Voyager and in APP, I have implemented a work-around so that this particular FITS issue is detected and the files are still read properly.
So the error actually is in older Voyager versions and other applications that use a faulty fits writer. Leonardo Orazi from Voyager has solved this issue form his end as well 😉 And APP 1.083 will be able to read these problem files as well.
To be sure, I have just checked your uploaded files in APP 1.083-beta2 and yes they work fine now 😉
Mabula
Thank you for checking. I will have to follow up with Leo, the version I was using is pretty up to date, it might be a separate problem if the fix has been implemented already. It might be a while before I can test again, we were supposed to have another go tonight but we both live in Victoria (Australia) and are under a COVID lockdown at the moment so that is not going to happen.
Regardless of that, it will be no problem in the future given 1.083 will open the files anyway, looking forward to the beta 2 release.
