19 June 2021: Our upload server https://upload.astropixelprocessor.com/ has been migrated successfully to our new office with higher upload and download speeds (nearly 10MByte/sec up/down ) ! We now have 1 general upload user called: upload with password: upload. The users upload1 - upload5 have been disabled.
31 May 2021: APP 1.083-beta2 has been released ! APP 1.083 stable will follow soon afterwards. It includes a completely new Star Reducer Tool, New File Saver Module, Improved Comet registration and much more, check the release notes here!
IO error encountered while saving fits file
In the final stages of integration APP v1.082 running in iMac with latest version of Catalina OS I get the following error:
"IO error encountered while saving fits file" in this case M33-session_1_session_2.fits.
I have 604Gb of space available on disk. 4Gb of RAM allocated to APP and 1477 used.
129 20Mb lights from ASI294mc, 1 master dark, no bias or flats. All APP settings default. Working folder is /users/Tom/Astro/Lights
This is the fourth aggravating crash after several hours of processing each time. I did have one successful run using just 30 of the 129 subframes.
An IO exception is usually only triggered if, for some reason, APP is not able to write to that location. This can be because of a number of reasons, like an actual disk issue to having no permissions. Could you try to write to a different location?
Apology for long delay. I thought I'd found the problem, but apparently not. Now, after 4 days of frustrating failures, I again think I see a new, different cause.
You're right, APP can't write to my main HDD because it thinks it's full. It's a 1T drive, managed by Catalina. As such, the drive has over 629Gb of purgeable/relocatable space including images and Time Machine backups. But APP doesn't seem to know that space can be used.
APP reports that I only have 100Gb of free space. And that space can easily be eaten up by APP during processing. So when the program finishes integration, it simply reports an IO error (or disk full) and #$%^&* quits. Why doesn't it at least ask for another drive to write on?
Is there a setting that will make APP understand that I have lots and lots of free space?
@tailspin45 Unfortunately it is not that simple. MacOS doesn't provide a Java API to check the Volumes with, nor to check whether or not consumed disk space is purgeable/relocatable. MacOS is supposed to take care of that automagically and Java doesn't know about it. The disk space check has been built in APP on purpose and during the processing steps you should get warnings that, according to APP/Java, you are running out of disk space. This should give you ample time to free up some. I know it is frustrating because I am a Mac user myself but Apple unfortunately doesn't expose that kind of functionality to Java.
@wvreeven I received oen warning in a half dozen runs, deleted a bunch of files, but it still failed. Apparently, as soon as I make more space, OS X fills it back up with Time Machine data or whatever.
So APP is useless as a Mac app, as much as I like it. How do I go about getting a refund? I purchased a renter license just a few days ago.
@tailspin45 I am sorry to read that you are unhappy but it APP certainly is not useless on MacOS. Perhaps it doesn’t work as you expected but there really is no need to word it like that.
I’d advise you to contact firstname.lastname@example.org and discuss the details further there.
@wvreeven Sorry, worded like what? You say APP can't tell if there is space available on my drive. If that's the case, I can't use it because OS X fills my drive with relocatable files whenever possible.
As I said, I like APP. I tried it on an old, very slow Windows machine with just a few light subframes, and I liked the workflow, I liked the results.
But I use a Mac, and if APP can't reliably run to completion or provide a way to provide a checkpoint, or way to save results in the event of a problem, after running for hours and hours then I don't need the pain.
If you're a Mac user and use the program reliably, can you explain to me how?
@tailspin45 Like I said, APP is not useless on Mac.
My MacBook has a 1 Tb disk. Also, I occasionally use a 2 Tb external SSD. And I don’t have local time machine backups.
@wvreeven Yes, I have a 1Tb drive too. How did you permanently remove the local TimeMachine snapshots? My understanding they are not under user control.
I can temporarily remove the current snapshots by turning TimeMachine off. But when I turn it back on it starts to create them again.
APP is very slow already, but I'll try an external USB 3.0 SSD. It that works, then at least I won't lose hours and hours of processing.
In any case, is there a reason that a user can't be given an option to pick another drive if the current drive is full after, in my worst case, a day and half of chugging along?
Hi Tom, @tailspin45
Please accept my apologies for any inconvenience caused.
I have read the entire topic on our forum.
- If you process a big dataset on an external disc, please access the extern folder using a permanent mount in the file system. The IO issue is caused by MacOS not being very clever unfortunately… if you use the automatic mount, MacOS can suddenly unmount it although clearly it is being used by you in APP, causing the IO error. If you simply use a permanent mount for any external disc, the issue should never occur. You do need to be aware of using a 100% compatibile filesystem for both READ/WRITE support on the external drive.
- If you start an integration, APP will check the available amount of space, if there does not seem to be enough due to the temporary storage that MacOS claims, you can simply overrule this and let APP do it’s thing. It should work without issues If you have enough real space left on your Apple computer.
Many people use APP on MacOS and they are able to process very big datasets using the internal drive and with external storage, so from our perspective there is no real issue here, except that MacOS is not transparent at all about how much free space there actually is unfortunately.
I have checked our records, and I can see that you had a 30-day Trial period in advance of your purchase. I that case, we normally can’t offer a refund. Then we have a strict no-refund policy because the Trial license works 100% so you can test it completely before buying.
Now I do understand that you used the trial on windows and only started using APP on a Mac with the Renter’s license, right?
Is there no way whatsoever for you to make APP work nicely on your Mac?
I will be happy to help you in any way I can.
@mabula-admin. Thank you for your prompt and thoughtful reply.
I was not using an external drive, the problem was with the built in 1Tb drive. But I bought a new external 1Tb SSD (US$130) drive and will try using APP with that.
But I don't understand what you mean by a permanent or automatic mount for an external disk. I'm not aware that I have any such choice. I plug in the USB cable, it mounts, and stays there until I unplug it.
Anyway, yes, you're right. I used the demo on an old PC with just a few subframes and it seemed to work albeit slowly. So I bought the package for my Mac knowing it was a much newer and faster machine but not knowing that your program couldn't tell that I had 619Gb of space available or that the Mac couldn;t tell that you're a legitimate process and it should make room for whatever you want to write.
I appreciate your offer to do whatever is necessary to make your product work nicely on my machine. Let me try a couple more sessions with the SSD and see where we are. I'll let you know if I have problems.
I really do like the program, but I really don't like arriving at the end of a more than day-long session and have no option but to quit when your program thinks there no space available. Wouldn't a choice of drives be a better solution if the one assigned appears to be full?