The Next Level
FreeBSD on arm64 in the Cloud
FreeBSD/arm64 is the FreeBSD port to the 64-bit ARM architecture, also known as AArch64 or ARMv8. All supported FreeBSD releases include support for ARMv8 and there are many packages and ports (3rd party applications) available to support the software you normally deploy with FreeBSD.
arm64 extends and modernizes the ARM architecture, adding specialized instructions and accelerators, as well as modern virtualization capabilities. The arm64 hardware, on which FreeBSD runs, ranges from small single board computers to mammoth multi-processor server machines with more than 100 cores on a single socket.
arm64 supports two models for describing hardware. Most single board computers (such as the raspberry pi or ROCK64) use flattened device tree (FDT) to describe the hardware, while other devices use the same ACPI (Advanced Configuration and Power Interface) as x86 CPUs. arm64 scales up to server platforms and the ARM specifications intend to make it possible to have images that ‘run anywhere’ through the SBSA (Server Base System Architecture). This standardizes the interface the machine uses to boot and interface with hardware. With SBSA and SBBR (Server Base Boot Requirement) arm64 hosts are able to boot using UEFI and discover hardware through ACPI. This makes it easier to deploy single images that work on many different hardware and virtual platforms, and brings arm64 deployment in line with amd64.
FreeBSD/arm64 has had support for server class platforms for a long time. The FreeBSD project uses Ampere servers to build packages for ARMv7 and ARMv8. These machines can be expensive and difficult to acquire, hence for experimentation it has been easier to pick up single board computers that are available for less than $50.
If you aren’t an embedded device vendor, then single board computers are not a proof-of-concept platform for more than the smallest experimentation. Thankfully FreeBSD/arm64 runs really well in the cloud and we can use cloud services to show that FreeBSD/arm64 is just as capable a server platform as any other of the supported architectures.
Cloud Services
arm64 is still breaking into the world of cloud hosting for virtual machines. AWS is the major host of virtual machines that will run arm64 images. At the time of writing Azure is experimenting with offering deployment pipelines on arm64, and other providers have experimented with arm64 virtual machines and full arm64 machine provisioning.
FreeBSD runs very well on the Amazon Web Services (AWS) with their Graviton 2 hosts. Graviton 2 is the second generation of general access arm64 servers that amazon has offered in the cloud.
AWS introduced their Graviton platform in November 2018 with their first generation of hardware using Annapurna Labs processors. This platform featured 16 Cortex-A72 cores organized as four quad-core clusters, all operating at 2.3 GHz.
At ReInvent, in December 2019, AWS introduced the Graviton2 platform built on top of ARM Neoverse N1processors. These processors quadruple the number of cores compared to Graviton 1 from 16 to 64 and claim to offer up to 7x the performance of the first generation with the newer 7nm process.
Graviton2 processors power EC2 T4g, M6g, C6g and R6g instances, and their variants offer high performance NVMe-based storage and high-performance networking with Elastic Network Adapters (ENA).
This is the perfect time to try running FreeBSD/arm64 in EC2.
Running FreeBSD on EC2
EC2 t4g instances offer great general workload performance with the ability to burst performance of CPU, storage and network. CPU burst credits are determined by instance size with all sizes of instance being able to burst network performance up to 5 Gigabit/s.
The current release of FreeBSD (12.2) is available as an official image produced by the FreeBSD Release Engineering team and offered on the Amazon Market Place by the FreeBSD Foundation.
Launching a t4g.micro instance is a matter of hitting the launch button from the EC2 web interface, searching for FreeBSD AMIs, selecting FreeBSD/ARM 12 AMI type and launching an instance.
After a few minutes the machine will be available in the instance dashboard and you will be able to ssh to the virtual machine with your ssh key. FreeBSD AMIs are configured with a user named ec2-user which is available for use with ssh. Once you connect you will be shown the motd as on any other FreeBSD host:
Code:
[user@computer] $ ssh ec2-user@3.249.84.154
Last login: Mon Mar 15 10:21:09 2021 from tmux(2897).%0
FreeBSD 12.2-RELEASE r366954 GENERIC
Welcome to FreeBSD!
Release Notes, Errata: https://www.FreeBSD.org/releases/
Security Advisories: https://www.FreeBSD.org/security/
FreeBSD Handbook: https://www.FreeBSD.org/handbook/
FreeBSD FAQ: https://www.FreeBSD.org/faq/
Questions List: https://lists.FreeBSD.org/mailman/listinfo/freebsd-questions/
FreeBSD Forums: https://forums.FreeBSD.org/
Documents installed with the system are in the /usr/local/share/doc/freebsd/
directory, or can be installed later with: pkg install en-freebsd-doc
For other languages, replace "en" with a language code like de or fr.
Show the version of FreeBSD installed: freebsd-version ; uname -a
Please include that output and any error messages when posting questions.
Introduction to manual pages: man man
FreeBSD directory layout: man hier
Edit /etc/motd to change this login announcement.
To easily configure your installed FreeBSD use bsdconfig(8).
-- Lars Engels <lme@FreeBSD.org>
ec2-user@freebsd:~ $ uname -a
FreeBSD freebsd 12.2-RELEASE FreeBSD 12.2-RELEASE r366954 GENERIC arm64The ec2-user is in the wheel group and you can elevate to root by running the su command without a password.
Did you know?
You can maximize the power of your FreeBSD infrastructure with our Support Subscription!
Check it out!Software Side
A lot of work has gone into FreeBSD/arm64 from both src and ports committers to make the platform a high-quality experience. Because of all of this hard work, using FreeBSD/arm64 in the cloud should not be very different than using amd64 based cloud images in any provider.
I ran some tests services with nginx web workloads and some iperf3 network performance tests without any issue on the FreeBSD/arm64 EC2 instances. The places you may encounter architecture related issues will be when software has not been written with portability or where FreeBSD/arm64 is not well supported upstream.
The second area of where you may encounter issues is when ports have been marked as amd64 only because in the past amd64 was the main platform these tools would be run. With the SBSA, arm64 is much more similar to amd64 server hardware, and most of these tools will run without any problem once the port metadata has been updated to include arm64.
t4g Burstable Instances
EC2 t4g instances are burstable, when the CPU, storage (EBS) or network (ENA) is not running above a baseline capacity the instance accrues credits which can later be used when there is an increase in workload. When the instance runs above the baseline capacity it will spend accrued credit.
The free trial t4g.micro instance we deployed in this example will accrue CPU credits when the CPU load is below the baseline of 10% utilization per vCPU. When below the baseline the instance will accrue 12 credits per hour up to a maximum of 288 credits. Credits are spent at the millisecond interval by EC2 and information about credits is made available at 5-minute intervals through cloudwatch.
Starting with ENA version 2.3.0 t4g-network, FreeBSD supports exposing ENA network metrics via sysctl. You first have to start collecting metrics by setting the sysctl that governs the metrics interval:
Code:
# sysctl dev.ena.1.eni_metrics.sample_interval=10
# sysctl dev.ena.network_interface.en1_metricsI was unable to test this feature; all of the instances I started came up with ENA version 2.2.0. The latest release of the ENA driver from Semihalf suggests that support should be in future images.
Disk and network are also burstable and are throttled to a base line when their credit has been exhausted. The t4g burstable instances are a great way to experiment with FreeBSD/arm64 in the cloud, but don’t expect them to be able to handle sustained load as well as non-burstable instance types.
Conclusion
FreeBSD/arm64 is a growing platform, and I would expect to see more cloud providers offering services on arm64 and more and more powerful server hardware to come on to the market. arm64 is already able to provide significant power savings and new server offerings from companies such as Ampere offer very high core counts as well.
There is ongoing discussion in the FreeBSD project to promote FreeBSD/arm64 to a Tier 1 platform in FreeBSD 13. There is clear enthusiasm from FreeBSD developers for the arm64 architecture and it looks to have a great future in the project.
Meet the Author: Tom Jones
Tom Jones is an Internet Researcher and FreeBSD developer that works on improving the core protocols that drive the Internet. He is a
contributor to open standards in the IETF and is enthusiastic about using FreeBSD as a platform to experiment with new networking ideas as they progress towards standardisation.
Read more
Like this article? Share it!
- Share on Twitter
- Share on Linkedin
- Share via Email
- Share on Reddit
You might also want be interested in
Get more out of your FreeBSD development
Kernel development is crucial to many companies. If you have a FreeBSD implementation or you’re looking at scoping out work for the future, our team can help you further enable your efforts.
FreeBSD Support
FreeBSD Development
Continue reading...