Running a CRaC Java application on Raspberry Pi - UPDATE
On July 15th of 2023, I published a post here about my initial experiments with CRaC on the Raspberry Pi. At that time, I found out that both the Linux kernel in Raspberry Pi OS and the Zulu Build of OpenJDK still needed some changes to work on the Raspberry Pi. I created a ticket in the Linux kernel project, which was solved by Phil Elwell. Last week, a new version of the Raspberry Pi OS, based on Debian Bookworm, was released. And in september, version 21 of OpenJDK was released and the Zulu Build of it, includes CRaC. So let’s see if we can use CRaC without issues, if we bring all this togheter.
TL;DR; Yes, we have a winning combination! ;-)
What is CRaC?
As described on Azul Docs:
Coordinated Restore at Checkpoint (CRaC) is a JDK project that can start projects - on Linux - with shorter time to first transaction and less time and resources to achieve full code speed. CRaC effectively takes a snapshot of the Java process when it is fully warmed up, then uses that snapshot to launch any number of JVMs from this captured state. Not all existing Java programs can run without modification as all resources need to be explicitly checkpointed and restored using the CRaC API. Popular frameworks like Micronaut and Quarkus support CRaC checkpointing out of the box.
New Raspberry Pi OS
Using the Raspberry Pi Imager, I created a new SD card. Select “Raspberry Pi OS (other)” and “Raspberry Pi OS (64-bit), a port of Debian Bookworm”. This new version of the OS uses Wayland as the display system, and there seems to be a problem with VNC so you’ll need to use TigerVNC instead of RealVNC. And in my case, Wayland also didn’t show a task bar, which I could solve by switching back to X11 via the “Advanced Options” in sudo raspi-config. But these issues are not important while testing CRaC, as I could do all this via SSH from another computer.
Check OS and kernel version
In the GitHub Linux project of Raspberry Pi, I found that a tag stable_20231004 was created and I was wondering if I could find back this reference in the kernel version.
$ uname -srm
Linux 6.1.0-rpi4-rpi-v8 aarch64
$ hostnamectl
Static hostname: crac
Icon name: computer
Machine ID: ccce8a0c23d146b9a17c17116862b5b4
Boot ID: 6ac6015a728a41ba94be2dfe7facc958
Operating System: Debian GNU/Linux 12 (bookworm)
Kernel: Linux 6.1.0-rpi4-rpi-v8
Architecture: arm64
$ cat /proc/version
Linux version 6.1.0-rpi4-rpi-v8 (debian-kernel@lists.debian.org) (gcc-12 (Debian 12.2.0-14) 12.2.0, GNU ld (GNU Binutils for Debian) 2.40) #1 SMP PREEMPT Debian 1:6.1.54-1+rpt2 (2023-10-05)
As you can see there is no direct reference to this tag, but the version date is 2023-10-05, so the day after the kernel tag was created.
When comparing that tag with the previous one 1.20230405, you can find that the setting to enable CRIU, is part of the included changes. See arch/arm64/configs/bcm2711_defconfig > CONFIG_CHECKPOINT_RESTORE=y.
Raspberry Pi 5
Starting from Octoboer 23th, the new Raspberry Pi will be available. That board is based on the BCM2712. It seems CRaC should also be available on it from the start, as the same comparison of tags reveals the file arch/arm64/configs/bcm2712_defconfig that also has the same setting for CONFIG_CHECKPOINT_RESTORE=y.
I have my first RPi5 in backorder, so I hope I’ll be able to test this very soon…
Testing CRaC on Raspberry Pi
Using this new Raspberry Pi OS release, it’s time to run the CRaC experiment again!
Install SDKMAN, Java and Maven
Let’s first use SDKMAN to install Zulu 21 with CRaC and Maven.
$ curl -s "https://get.sdkman.io" | bash
$ source "/home/crac/.sdkman/bin/sdkman-init.sh"
$ sdk list java | grep crac
| | 21.crac | zulu | | 21.crac-zulu
| | 17.0.8.crac | zulu | | 17.0.8.crac-zulu
| | 17.0.8.1.crac | zulu | | 17.0.8.1.crac-zulu
| | 17.0.7.crac | zulu | | 17.0.7.crac-zulu
$ sdk install java 21.crac-zulu
$ java -version
openjdk version "21" 2023-09-19 LTS
OpenJDK Runtime Environment Zulu21.28+89-CRaC-CA (build 21+35-LTS)
OpenJDK 64-Bit Server VM Zulu21.28+89-CRaC-CA (build 21+35-LTS, mixed mode, sharing)
$ sdk install maven
$ mvn -version
Apache Maven 3.9.5 (57804ffe001d7215b5e7bcb531cf83df38f93546)
Maven home: /home/crac/.sdkman/candidates/maven/current
Java version: 21, vendor: Azul Systems, Inc., runtime: /home/crac/.sdkman/candidates/java/21.crac-zulu
Default locale: en_US, platform encoding: UTF-8
OS name: "linux", version: "6.1.0-rpi4-rpi-v8", arch: "aarch64", family: "unix"
Jetty Example Application
Now we can get the Jetty CRaC example application, compile it, run it, and try to create a checkpoint.
$ git clone https://github.com/CRaC/example-jetty.git
$ cd example-jetty
# Run the application
$ java -XX:CRaCCheckpointTo=cr -jar target/example-jetty-1.0-SNAPSHOT.jar
# In a second terminal, warm up the application
$ curl localhost:8080
Hello World
# Still in the second terminal, create the checkpoint
$ jcmd target/example-jetty-1.0-SNAPSHOT.jar JDK.checkpoint
6754:
CR: Checkpoint ...
# Check the first terminal for the output of the application, it should show that the checkpoint has been created
[0.004s][warning][os] CRaC closing file descriptor 63: pipe:[18355]
2023-10-16 13:21:15.574:INFO::main: Logging initialized @464ms to org.eclipse.jetty.util.log.StdErrLog
2023-10-16 13:21:15.809:INFO:oejs.Server:main: jetty-9.4.48.v20220622; built: 2022-06-21T20:42:25.880Z; git: 6b67c5719d1f4371b33655ff2d047d24e171e49a; jvm 21+35-LTS
2023-10-16 13:21:15.934:INFO:oejs.AbstractConnector:main: Started ServerConnector@67f89fa3{HTTP/1.1, (http/1.1)}{0.0.0.0:8080}
2023-10-16 13:21:15.940:INFO:oejs.Server:main: Started @885ms
Oct 16, 2023 1:21:49 PM jdk.internal.crac.LoggerContainer info
INFO: Starting checkpoint
2023-10-16 13:21:49.141:INFO:oejs.AbstractConnector:Attach Listener: Stopped ServerConnector@67f89fa3{HTTP/1.1, (http/1.1)}{0.0.0.0:8080}
Oct 16, 2023 1:21:49 PM jdk.internal.crac.LoggerContainer info
INFO: /home/crac/example-jetty/target/dependency/crac-1.3.0.jar is recorded as always available on restore
Oct 16, 2023 1:21:49 PM jdk.internal.crac.LoggerContainer info
INFO: /home/crac/example-jetty/target/dependency/jetty-io-9.4.48.v20220622.jar is recorded as always available on restore
Oct 16, 2023 1:21:49 PM jdk.internal.crac.LoggerContainer info
INFO: /home/crac/example-jetty/target/dependency/jetty-util-9.4.48.v20220622.jar is recorded as always available on restore
Oct 16, 2023 1:21:49 PM jdk.internal.crac.LoggerContainer info
INFO: /home/crac/example-jetty/target/dependency/jetty-http-9.4.48.v20220622.jar is recorded as always available on restore
Oct 16, 2023 1:21:49 PM jdk.internal.crac.LoggerContainer info
INFO: /home/crac/example-jetty/target/dependency/javax.servlet-api-3.1.0.jar is recorded as always available on restore
Oct 16, 2023 1:21:49 PM jdk.internal.crac.LoggerContainer info
INFO: /home/crac/example-jetty/target/dependency/jetty-server-9.4.48.v20220622.jar is recorded as always available on restore
Oct 16, 2023 1:21:49 PM jdk.internal.crac.LoggerContainer info
INFO: /home/crac/example-jetty/target/example-jetty-1.0-SNAPSHOT.jar is recorded as always available on restore
Killed
That’s nice, exactly the output we expected - without an error - showing that a checkpoint could be created.
Run the Application from the Checkpoint
Now we can restart the application using the checkpoint, but first must give additional permissions to criu, as described here:
$ whereis java
java: /usr/share/java /home/crac/.sdkman/candidates/java/21.crac-zulu/bin/java
$ sudo chown root:root /home/crac/.sdkman/candidates/java/21.crac-zulu/lib/criu
$ sudo chmod u+s /home/crac/.sdkman/candidates/java/21.crac-zulu/lib/criu
# Restart application from checkpoint
$ java -XX:CRaCRestoreFrom=cr
2023-10-16 14:07:38.333:INFO:oejs.Server:Attach Listener: jetty-9.4.48.v20220622; built: 2022-06-21T20:42:25.880Z; git: 6b67c5719d1f4371b33655ff2d047d24e171e49a; jvm 21+35-LTS
2023-10-16 14:07:38.344:INFO:oejs.AbstractConnector:Attach Listener: Started ServerConnector@67f89fa3{HTTP/1.1, (http/1.1)}{0.0.0.0:8080}
2023-10-16 14:07:38.345:INFO:oejs.Server:Attach Listener: Started @2783290ms
And What About Java 17?
Not only Zulu 21 has the improvements in CRaC which are needed on the Raspberry Pi. In the meantime, also a newer version of Zulu 17 is available, 17.0.8.crac-zulu. And with that version, compiling, running, creating the checkpoint, and restarting from it, works flawlessly!
$ sdk install java 17.0.8.crac-zulu
$ whereis java
java: /usr/share/java /home/crac/.sdkman/candidates/java/17.0.8.crac-zulu/bin/java
$ sudo chown root:root /home/crac/.sdkman/candidates/java/17.0.8.crac-zulu/lib/criu
$ sudo chmod u+s /home/crac/.sdkman/candidates/java/17.0.8.crac-zulu/lib/criu
$ mvn -version
Apache Maven 3.9.5 (57804ffe001d7215b5e7bcb531cf83df38f93546)
Maven home: /home/crac/.sdkman/candidates/maven/current
Java version: 17.0.8, vendor: Azul Systems, Inc., runtime: /home/crac/.sdkman/candidates/java/17.0.8.crac-zulu
Default locale: en_US, platform encoding: ANSI_X3.4-1968
OS name: "linux", version: "6.1.0-rpi4-rpi-v8", arch: "aarch64", family: "unix"
$ mvn package
$ java -XX:CRaCCheckpointTo=cr -jar target/example-jetty-1.0-SNAPSHOT.jar
# In a second terminal, warm up the application, and create the checkpoint
$ curl localhost:8080
Hello World
$ jcmd target/example-jetty-1.0-SNAPSHOT.jar JDK.checkpoint
# Back in the first terminal, check the output, it should show errors
# Restart the application from the checkpoint
$ java -XX:CRaCRestoreFrom=cr
2023-10-16 14:35:51.676:INFO:oejs.Server:Attach Listener: jetty-9.4.48.v20220622; built: 2022-06-21T20:42:25.880Z; git: 6b67c5719d1f4371b33655ff2d047d24e171e49a; jvm 17.0.8+7-LTS
2023-10-16 14:35:51.695:INFO:oejs.AbstractConnector:Attach Listener: Started ServerConnector@59f95c5d{HTTP/1.1, (http/1.1)}{0.0.0.0:8080}
2023-10-16 14:35:51.697:INFO:oejs.Server:Attach Listener: Started @71492ms
Conclusion
Boom! CRaC out-of-the-box on Raspberry Pi OS with both Zulu 17 and 21! This is a perfect way to try out some applications using CRaC and test the startup improvements, using very inexpensive hardware!
And I’m now even more curious to see what the new Raspberry Pi 5 will bring regarding Java application performance improvements…