Linux 64bit ARM version planned?

Not sure, maybe as it is cross-platform capable. However, running APP on a Raspberry will be quite painful as it's very CPU and RAM intensive so I don't think that would be worth the effort of porting it.

Hmm, my plan is not a Raspberry, this was on an example of an ARM system. I thought more about ARM based machines in the cloud. Like in AWS, there you can have a machine with up to 64 cores and 256GB of memory. Sure, you can also have a x86 virtual machine, but the ARM machines are much cheaper. The crazy idea I have in my mind is uploading the images already during the imaging session in the night to an cloud storage (S3, Drop-Box, Google-Cloud) and then launch a powerfull virtual machine running APP and crunch the images. It is just an idea but it all starts with an APP version running on an ARM system :-).

Ah ok, sorry I assumed it as you mentioned the Pi, but you meant Pi-like systems. 🙂 Those cloud systems do sound interesting indeed, I can kind of ask carefully what he thinks of that and if it would be even possible. Chances are not high I think as we're still very busy on the code-base we already have.

edit: Ok, just had a little chat. We have to look a bit into it. But he didn't immediately say no. 😉 I think if we would be able to compile it, maybe... Can't give any more than that though, needs some thorough looking into.

Testing out on a cloud linux machine (32 vCPU / 128GB of memory) with my last dataset from M13 to compare it to the time my Mac Pro has needed to compute the image. 
For what I can see already is that analyzing stars is much faster as it seem to rely mostly on cpu power and the number of threads.

So finally I can call this test a success. It took under one hour with this 32 vCPU virtual machine to compute 30 dark frames, 30 bias frames, 100 light frames all saved as really big TIFF files, because I have a Fuji X-Trans sensor that is currently not supported directly. But it will be soon, so I can repeat the test. 
I made this test in Amazon AWS, because I have the most experience with this cloud environment, but you can do the same in any other cloud environment in a similar way. It was easy to create a Linux Template called AMI (Amazon Machine Image) with an VNC Server installed running the Mate desktop and also APP installed. 
Using this prepared Image I could simply start any virtual machine size up to 96 vCPUs that is ready to use after a few seconds. You login via SSH with the VNC-Port tunneled into the remote machine. Then you can access the Desktop on the remote machine and start APP. 

Before all that I created a so called S3 bucket what is basically a cloud storage container where I uploaded all my dark, bias and light images. Uploading the images was the longest part, so it should be done already during the imaging session to save time. Otherwise uploading (at least with my 10 MBit upstream) takes longer then the time you safe with faster computing. 
Once the images are located in the S3 bucket I can spawn my virtual machine (in my test case a 32 vCPU / 128GB memory machine that costs 1,24€ per hour) and copied the 30GB of my test data from the S3 bucket to the virtual machine. This took only a few minutes with over 100MB/s from S3 to the local SSD. 
If you start APP for the first time then you have to enter your license data what is cool, because so one could provide an general / public image for this purpose that anyone can use without the risk of your license getting compromised. 

The rest is processing the data as you would do also on your machine at home. For me the speedup was 3 times compared to my 2 x 3,46 GHz 6-Core Intel Xeon MacPro at home. I cannot provide the total cost of this cloud experiment, because I still have some credits / free tier values left in AWS, so I will not be charged for anything. But I think it is under 2,-€. Assuming that I would have paid also additional costs for power consumption of my big Mac at home having it running full throttle for over 3 hours, this is not the worst thing. 

If there's any interest I could provide a more details tutorial / show case for using cloud computing for APP. 

Cheers, Mario.

P.S. Important note - Don't forget to delete all cloud resources you have created after you are done. This is the most common case where additional costs are created.

This is really awesome Mario, yes please, if you have the time and like to do it you're welcome to make a nice workflow. I will put this up as a sticky for all to see.

So we did some looking into ARM support and at least there is Java for ARM. So, if we have some time, we could try to do a compilation to that platform. I'll make a note to remember myself to do that, or at least to ask Mabula to try it. 😉

Posted by: @mkeller0815

For what I can see already is that analyzing stars is much faster as it seem to rely mostly on cpu power and the number of threads.

That is true, APP is using as many threads as there are on your system for most tasks. Not all yet, but most. In the future we want to add the GPU to do processing, which will likely speed things up even more.

Hi Mario @mkeller0815 & @vincent-mod,

Thanks for all the info ! This is really cool indeed, running APP in the cloud like this, I like the idea a lot 🙂 ! I know how powerfull and usefull this can be.

How do you get the screen output from the ARM machine? with SSH tunneling I guess?

Anyway, about running APP on ARM hardware:

I have not tried it yet, but I think it should work actually.

APP has Oracle Java 8 64bits as it's platform at the moment (soon we will go to Java 11 64bits) and

https://www.oracle.com/java/technologies/javase/javase-jdk8-downloads.html

as you can see, there is Oracle Java 8 for ARM 64bits.

Even OpenGL hardware display acceleration might work, because the OpenGL library that we use also supports ARMv6 I think. Otherwise we can fall back on software rendering of the image data, that will work anyway.

Now.... I think it will be beneficial to try to make it possible for you and others that might want to run APP on ARM hardware.

To make it possible, I would need to make a build machine for an ARM APP installer, maybe I can do this easily with a virtual machine in VMWare, I will check that first. Otherwise, I would need to buy some ARM hardware...in that case, what would you recommend to buy? 

Cheers,

Mabula

Good Morning everyone,

so screen output is done via VNC. running a vnc server on the virtual machine and connecting via ssh with a forwarded local port. like "ssh -L127.0.0.1:5901:127.0.0.1:5901 user@machine". It looks strange, because you are using "localhost" two times, but the first is your local machine and the second is the localhost on the remote machine. Then you can easily connect your local VNC viewer to 127.0.0.1:5901 and it will connect to the VNC server on the remote machine. This is an easy and secure way to connect to the remote machine, because only the ssh port is exposed to the internet. 
If your connections is slow, you can also add the parameter "-C" (ssh -C -L127.0.0.1....) to the ssh command to enable compression for the connection. 

To start out with an ARM system I personally would go for the current Raspberry Pi 4. The new model is available up to 8GB of memory and runs a quad-core cpu that is quite powerful. 
But to start out you can just use a virtual ARM server in AWS, Azure, Google Cloud, Packet.com, or were ever. 
For AWS the prices are about 23 Cents/hour for an 8 core 16gb memory machine and about 46 Cents/hour for 16 cores and 32gb memory. This is quite good because you will run it only when you need it.

Mario.

Much appreciated Mario, thank you!

Ok, lets pull this up again, because with Apple announcing to move to ARM architecture soon with the new models I think there is a much higher demand on running APP on an ARM machine. Is there any chance to have at least a "proof of concept" version of APP for ARM linux? 

Mario.

Jep well we think it's possible but we're not coding to the speed of new Apple hardware. 😉 The chance of having a fast enough ARM Mac with all bells and whistles is not super high I think. It's very early days for that so I think we have enough time to get that working. Priority now is the upgrade to Java 11 and other important changes. When that has been done we can do trials for ARM.

So with Apple selling ARM Macs now, with APP working at least on the X86 emulation mode of the M1 chip and with the announcement of porting APP to run on M1 natively, I want to raise this thread again. 
According to this site: https://www.graalvm.org/docs/introduction/
and also this post: https://stackoverflow.com/questions/54705341/graalvm-as-jit-compiler-for-arm-jre
GraalVM and OpenJDK 11 are reported to work also on ARM. 
So would it be possible to give this also a shot? 

We are actually gearing towards using GraalVM and JDK11, so yes, that will become possible. And by then we might also look at specific ARM compilations.