Running Google Chrome in a dedicated Linux-Jail

Holger · Jun 12, 2022

Introduction and motivation

There are great articles ([1] and [2]) by patovm04 here on the forum explaining how to run Chrome and Brave in a Linux chroot environment (usually /compat/linux or /compat/ubuntu).

These approaches work great. However, I am a big fan of FreeBSD's jails and it has always bothered me that these Linux compatibility layers were some kind of “sort-of” jail, but not real ones. By “real one” I mean a lightweight virtualization environment that makes use of FreeBSD's jail infrastructure (jexec, jls, /etc/jail.conf, etc.).

I am hoping that the approach I am presenting here will illustrate the following benefits:

Leverage standard jail infrastructure to uniform handling of Linux and FreeBSD jails.
Create as many Linux jails as you want (not just one in /compat/linux, respectively /compat/ubuntu).

There is at least one negative side to putting the Linux emulation into a separate jail: It is less convenient, because you won't be able to execute Linux programs as if they were part of the host system. However, you will get a clean separation between the FreeBSD world and the Linux world.

This tutorial was tested on FreeBSD-13.1-RELEASE.

Basic setup

The setup I am starting with is the following:

I am using ZFS, but do not rely on it in this tutorial. I have added remarks when appropriate on how to do the same thing on UFS.
I am using “bare metal” jails, no management utility like sysutils/bastille or sysutils/iocage. The use of them could possibly simplify some aspects of this tutorial, but I like the down-to-earth approach when using jails directly.
My jails are attached to a loopback device lo1 and I use pf to NAT to forward network traffic to my physical network. This way I can assign static IP addresses to my jails and keep using DHCP on my physical network. If you are interested in the details of this setup, just give me a call and I post them here.
I use Xorg, not Wayland, which I am unfamiliar with. If you know Wayland and a simple way to modify this tutorial to support both, please let me know!

Ok, let's go.

Enabling Linux support

Why we probably don't need /etc/rc.d/linux or similar

Normally, the service /etc/rc.d/linux takes care of everything that is needed to run Linux binaries. The service basically does the following:

Load the required kernel module(s).
Configure how to handle unbranded ELF executables.
Mount Linux system directories like /dev, /proc, etc., under the path specified by compat.linux.emul_path.

Loading the kernel modules can be done in rc.conf. Mounting the Linux system directories will be dealt with by FreeBSD's Jail infractructure.

So, besides the issue with the unbranded ELF executables, there is not much need for starting the service. Therefore, I omit it in this tutorial.

Loading the modules

Ensure that Linux support is activated:

Code:

# kldload linux64

To make it permanent, add the module in /etc/rc.conf to the kld_list variable, but ensure that your other modules remain there:

Code:

kld_list="... linux64 ..."

E.g., my kld_list now looks like this:

Code:

kld_list="i915kms linux64"

Creating the jail

I like to put each of my jails into a separate ZFS dataset. This, for example, makes it easier to create snapshots or to transfer the jail to another computer. The zpool I am using for this is called scratch. You'll have to adjust that name to your particular configuration (e.g. zroot or similar):

Code:

# zfs create scratch/jail/ubuntu
# zfs set mountpoint=/jail/ubuntu scratch/jail/ubuntu

If you are using UFS instead of ZFS, a simple mkdir -p /jail/ubuntu will do.

Next, we'll need to prepare the system directories:

Code:

# for DIR in /dev/fd /dev/shm /tmp /proc /sys; do mkdir -p /jail/ubuntu/${DIR}; done

Now we can populate the jail with an Ubuntu root filesystem. For this, we'll use sysutils/debootstrap:

Code:

# pkg install debootstrap
# debootstrap --arch=amd64 --no-check-gpg focal /jail/ubuntu

Before we can actually start the jail, we need to specify what has to be mounted inside the jail. FreeBSD's jail mechanism can parse an fstab-like file that does exactly that. The location of this file is irrelevant, but for simplicity, we put it right into the jail itself. So, create it using an editor:

Code:

# vi /jail/ubuntu/etc/fstab

and put the following content into it:

Code:

devfs           /jail/ubuntu/dev      devfs           rw                      0       0
tmpfs           /jail/ubuntu/dev/shm  tmpfs           rw,size=1g,mode=1777    0       0
fdescfs         /jail/ubuntu/dev/fd   fdescfs         rw,linrdlnk             0       0
linprocfs       /jail/ubuntu/proc     linprocfs       rw                      0       0
linsysfs        /jail/ubuntu/sys      linsysfs        rw                      0       0
/tmp            /jail/ubuntu/tmp      nullfs          rw                      0       0

Note that, with this configuration, the jail will share the host's /tmp directory. This makes it easier to share audio and X11 sockets with the jail, but, for security reasons, you might want a different configuration sometime later when everything is up and running. –

Next, we need to configure the jail. Open the file /etc/jail.conf and append the following content:

Code:

ubuntu {
    host.hostname="ubuntu.schattenwelt.org";
    ip4.addr="lo1|10.10.0.5/24";
    path="/jail/ubuntu";
    allow.raw_sockets=1;
    exec.start='/bin/true';
    exec.stop='/bin/true';
    persist;
    mount.fstab="/jail/ubuntu/etc/fstab";
}

You will probably need to adjust the following:

The domain name (mine is schattenwelt.org, but your's will be different).
The IP address. In my configuration I use a loopback device (lo1) and assign static IP address for each jail manually. If you wonder how I get internet access inside the jail: I use the packet filter pf for that, redirecting network traffic from and to the jail to my physical network.

We are now ready to start the jail:

Code:

# service jail onestart ubuntu

To start the jail at each boot, put the following into your /etc/rc.conf:

Code:

jail_enable="YES"

You can check if the jail is properly running by doing

Code:

# jls

which should give the following output:

Code:

   JID  IP Address      Hostname                      Path
     ...
     4  10.10.0.5       ubuntu.schattenwelt.org       /jail/ubuntu

Setting up the Ubuntu jail

Now that the Ubuntu jail is up and running, we can set up the Linux system. First, enter the jail:

Code:

# jexec ubuntu /bin/bash

Configure locales and timezone:

Code:

# dpkg-reconfigure locales
# dpkg-reconfigure tzdata

Install necessary packages (for running Google Chrome later on):

Code:

# printf "deb http://archive.ubuntu.com/ubuntu/ focal main restricted universe multiverse" > /etc/apt/sources.list
# apt update
# apt install curl gnupg pulseaudio fonts-symbola ttf-mscorefonts-installer

Install GPG key enabling the Google Chrome DEB repository:

Code:

# curl -s https://dl.google.com/linux/linux_signing_key.pub | apt-key --keyring /etc/apt/trusted.gpg.d/google-chrome-stable.gpg add -

Install Google Chrome:

Code:

# printf "deb [arch=amd64] http://dl.google.com/linux/chrome/deb/ stable main" > /etc/apt/sources.list.d/google-chrome.list
# apt update
# apt install google-chrome-stable

Create a wrapper script for launching Chrome: Create a file /opt/google/chrome/chrome-wrapper and add the following content:

Code:

#!/bin/bash
#
# Copyright (c) 2011 The Chromium Authors. All rights reserved.
# Use of this source code is governed by a BSD-style license that can be
# found in the LICENSE file.

HERE="/opt/google/chrome"

export CHROME_VERSION_EXTRA="stable"

# We don't want bug-buddy intercepting our crashes. http://crbug.com/24120
export GNOME_DISABLE_CRASH_DIALOG=SET_BY_GOOGLE_CHROME

# Sanitize std{in,out,err} because they'll be shared with untrusted child
# processes (http://crbug.com/376567).
exec < /dev/null
exec > >(exec cat)
exec 2> >(exec cat >&2)

export DISPLAY=:0
export LIBGL_DRI3_DISABLE=1

"$HERE/chrome" --in-process-gpu --no-sandbox --no-zygote --test-type --enable-features=UseOzonePlatform --ozone-platform=x11 --v=0 "$@" || true

Make it executable:

Code:

# chmod +x /opt/google/chrome/chrome-wrapper

Finally, add a non-privileged user that will be able to run Google Chrome (in my case this is hsebert, myself):

Code:

# adduser hsebert

Add this point, you should be able to run Chrome. There will be no sound yet, though (see a later chapter to fix that). To run it, first change to our non-privileged user created above:

Code:

# su - hsebert

Then run Chrome:

Code:

$ /opt/google/chrome/chrome-wrapper

When everything is allright, quit Chrome, leave the su-environment and ultimatlely the jail:

Code:

$ exit
# exit

Troubleshooting

You may need to allow the Ubuntu jail to connect to your Xorg server on the host. Do this by using the following command (on the host!):

Code:

$ xhost +

To make this change permanent, place it, e.g. in ~/.xinitrc.

Allow non-root execution

You can now start Chrome from outside the jail using jexec as root:

Code:

# jexec ubuntu /bin/bash -c "su -c /opt/google/chrome/chrome-wrapper - hsebert"

Note that we use su inside the jail to drop privileges and use our non-privileged user inside the jail (hsebert).

Having to be root in order to launch Chrome is a little bit inconvenient. FreeBSD doesn't allow non-privileged jail-execution for good reasons, though. So we will use a different approach using security/sudo:

Code:

# pkg install sudo

First, we'll create a launcher script on our host and call it /usr/local/bin/linux-chrome:

Code:

#!/bin/sh

if [ $# -lt 1 ] ; then
    echo "usage: ${0} <user>"
    exit 1
fi

if [ "${1}" = "root" ] ; then
    echo "Cowardly refusing running Chrome as root"
    exit 1
fi

jexec ubuntu /bin/bash -c "su -c /opt/google/chrome/chrome-wrapper - ${1}"

Make it executable:

Code:

# chmod +x /usr/local/bin/linux-chrome

Next, we allow a particular user on the host system to execute linux-chrome using sudo, but without requiring a passwort for this particular command (this is important!). For this, create a dedicated sudoers file as follows:

Code:

# visudo -f /usr/local/etc/sudoers.d/linux-chrome

and put a line into it having the following format:

Code:

<user> <host> = (root) NOPASSWD: <cmd>

The user (<user>) on my host system who is to be allowed to run linux-chrome is hsebert (same user name as inside the Linux jail, but this is not a necessity). My host name (<host>) is thinkle. The command to be run (<cmd>)is /usr/local/bin/linux-chrome (full path!). So, the resulting line looks like this:

Code:

hsebert thinkle = (root) NOPASSWD: /usr/local/bin/linux-chrome

After saving and closing vi, your regular user should be able to run Chrome as follows:

Code:

$ sudo linux-chrome $(whoami)

You can put this command inside a Desktop launcher, for example.

Enabling sound

Google Chrome on Linux relies on Pulseaudio, so we have to deal with it. In this guide I chose to run Pulseaudio as a system-wide service, which has pros and cons:

Pro: It can be used by multiple jails at the same time, and even by Bhyve virtual machines, if remote access is configured (see below).
Con: Even the Pulseaudio developers discourage the use as a system-wide service because of security issues, see here. Pick your poison. –

First, we need to install the package:

Code:

# pkg install pulseaudio

Pulseaudio aware applications can communicate with a Pulseaudio server either by TCP or a Unix-Domain-Socket (similar to Xorg). We configure this feature explicitly by editing /usr/local/etc/pulse/system.pa and add the following two lines:

Code:

load-module module-native-protocol-tcp auth-anonymous=1 auth-ip-acl=127.0.0.1;192.168.178.0/24
load-module module-native-protocol-unix auth-anonymous=1 socket=/tmp/pulse-native

The second line is most important for us: It tells us how the Pulseaudio socket will be named (/tmp/pulse-native). We'll need that information later when we configure Pulseaudio in the Linux jail.

Important: If you do not want remote connections over the network, delete the first line containing module-native-protocol-tcp. If remote connections are o.k. for you (e.g. because you would like to play audio in your Bhyve virtual machines), be sure to set the correct subnet mask (mine is in this case: 192.168.178.0/24).

Now that we have configured the Pulseaudio server, we need to enable it. The package audio/pulseaudio does not come with an rc script to start it up at boot time. Therefore, we have to create one: Create the file /usr/local/etc/rc.d/pulseaudio and let it have the following content:

Code:

#!/bin/sh

# PROVIDE: pulseaudio
# REQUIRE: DAEMON FILESYSTEMS
# KEYWORD: nojail shutdown

. /etc/rc.subr

name="pulseaudio"
desc="Start the Pulseaudio server"
rcvar="pulseaudio_enable"
pulseaudio_bin="/usr/local/bin/${name}"
pulseaudio_pidfile="/var/run/pulse/pid"
start_cmd="${name}_start"
stop_cmd="${name}_stop"
load_rc_config "${name}"

pulseaudio_start()
{
    ${pulseaudio_bin} --system --disallow-module-loading &
}

pulseaudio_stop()
{
    if [ -f "${pulseaudio_pidfile}" ]
    then
        kill $(cat "${pulseaudio_pidfile}")
    fi
}

run_rc_command "$1"

Next, enable the service in /etc/rc.conf:

Code:

pulseaudio_enable="YES"

Finally, start the service so the we do not have to reboot the machine:

Code:

# service pulseaudio start

We are almost there. The one thing left is to let Chrome in the Linux jail know which socket to use when talking to our Pulseaudio server. We could put this information directly into the chrome-wrapper script introduced further up, but I think it's better to make it a system-wide default for our Linux jail. Therefore, create the file /jail/ubuntu/etc/profile.d/05-pulseaudio.sh and add the following line:

Code:

export PULSE_SERVER=unix:/tmp/pulse-native

Note: If you would like to configure remote access, simply replace to line above with

Code:

export PULSE_SERVER="<Host-IP-address>"

where "Host-IP-address" is the addres of the machine running the actual Pulseaudio server.

Debugging sound

If there is no sound, ensure that the Pulseaudio connection is working: Enter the Linux jail and run

Code:

$ pactl list

If everything works, you should see a list of available sources and sinks. If you get something like this:

Code:

Connection failure: Connection refused
pa_context_connect() failed: Connection refused

then ensure the following:

Is the PULSE_SERVER variable set correctly?
Is the socket /tmp/pulse-native available, i.e. is /tmp properly mounted?
Is the Pulseaudio server running on the host: ps -a | grep pulseaudio?
Have you tried turning it off and on again?

References

snakedoctr · Aug 17, 2022

Hi Holger. Great write-up! I'm close to getting it to work, but ran into an issue with running Chrome. I'm getting this when running the wrapper script:

Code:

./chrome-wrapper: line 17: /dev/fd/62: Operation not supported
./chrome-wrapper: line 18: /dev/fd/62: Operation not supported
[15782:110082:0817/142226.801489:ERROR:file_path_watcher_inotify.cc(329)] inotify_init() failed: Function not implemented (38)
[15782:110088:0817/142226.857930:ERROR:bus.cc(398)] Failed to connect to the bus: Failed to connect to socket /var/run/dbus/system_bus_socket: No such file or directory
[15782:110088:0817/142226.858034:ERROR:bus.cc(398)] Failed to connect to the bus: Failed to connect to socket /var/run/dbus/system_bus_socket: No such file or directory
[15782:110085:0817/142226.860094:ERROR:address_tracker_linux.cc(190)] Could not create NETLINK socket: Address family not supported by protocol (97)
[15782:110087:0817/142226.860371:ERROR:bus.cc(398)] Failed to connect to the bus: Could not parse server address: Unknown address type (examples of valid types are "tcp" and on UNIX "unix")
[15782:110087:0817/142226.860434:ERROR:bus.cc(398)] Failed to connect to the bus: Could not parse server address: Unknown address type (examples of valid types are "tcp" and on UNIX "unix")
[15782:15782:0817/142226.861194:ERROR:platform_shared_memory_region_posix.cc(217)] Creating shared memory in /dev/shm/.com.google.Chrome.yZEuZA failed: No such file or directory (2)
[15782:15782:0817/142226.861215:ERROR:platform_shared_memory_region_posix.cc(220)] Unable to access(W_OK|X_OK) /dev/shm: No such file or directory (2)
[15782:15782:0817/142226.861222:FATAL:platform_shared_memory_region_posix.cc(222)] This is frequently caused by incorrect permissions on /dev/shm.  Try 'sudo chmod 1777 /dev/shm' to fix.
[0817/142226.906687:ERROR:ptracer.cc(43)] ptrace: Invalid argument (22)
[0817/142226.906767:WARNING:process_reader_linux.cc(379)] Couldn't initialize main thread.
[0817/142226.906794:ERROR:proc_task_reader.cc(46)] format error
[0817/142226.906807:WARNING:exception_snapshot_linux.cc(349)] thread ID 15782 not found in process
[0817/142226.906851:ERROR:process_snapshot_linux.cc(129)] thread not found 15782
[0817/142226.907065:ERROR:proc_task_reader.cc(46)] format error
./chrome-wrapper: line 23: 15782 Trace/breakpoint trap   (core dumped) "$HERE/chrome" --in-process-gpu --no-sandbox --no-zygote --test-type --enable-features=UseOzonePlatform --ozone-platform=x11 --v=0 "$@"

My mounts look like this:

Code:

zroot/usr/jails/ubuntu on / type zfs (rw,noatime)
devfs on /dev type devfs (rw)
tmpfs on /dev/shm type tmpfs (rw)
fdescfs on /dev/fd type fdescfs (rw)
proc on /proc type proc (rw)
/sys on /sys type sysfs (rw)
/tmp on /tmp type nullfs (rw,nosuid,noatime)
devfs on /dev type devfs (rw)

Odd that /dev is mounted twice. It looks like /dev/shm is mounted as well (one of the errors). The other errors during Chrome startup are related to dbus. Did you have to run dbus inside the Ubuntu jail to get it to work? Or what am I missing?

Thanks again!

Holger · Aug 17, 2022

Hi! I'm glad you find this tutorial useful! –

Are the mounts inside the jail properly configured?

Could you please post the output of mount on the FreeBSD host while your Linux jail is running?

snakedoctr · Aug 17, 2022

It's fixed!

The issue was in fact because /dev was being mounted twice. It was being mounted once via mount.devfs in jail.conf, and again in the jail's /etc/fstab (via mount.fstab). Once I removed mount.devfs it only mounted /dev once and now there are a LOT more devices listed now (apparently the ones that make Chrome work

). Launching Chrome via the wrapper script brings it right up!

Now to continue with your guide and getting sound working and launching it outside the jail on the FreeBSD host.

Thanks again!

Holger · Aug 18, 2022

Great, you fixed it!

Could the problem have come from the fact that use are using the linux service? I.e. do you have in your rc.conf something like:

Code:

linux_enable="YES"

If yes, this might have cased the double-mounts. In such a case, I should update the tutorial to explicitly mention that the linux should not be enabled. I.e. that the above line should be removed from rc.conf.

snakedoctr · Aug 19, 2022

I don't think linux_enable="YES" caused it. It had /dev mounted in /compat/linux/dev, and /dev for my jail was in /usr/jails/ubuntu/dev, so they weren't colliding.

snakedoctr · Sep 2, 2022

Hey Holger. I finally got the time to implement the rest of your guide (running Chrome from the FreeBSD host as a regular user, setting up audio, etc.). It's all working great now -- amazing! One thing I did have to fix was the pulseaudio startup script you have mentioned. You need to add load_rc_config "${name}" before the pulseaudio_start() function, otherwise I was getting an error that pulseaudio_enable="YES" wasn't set in /etc/rc.conf when it actually was (because the startup script wasn't actually importing the variable from rc.conf).

Holger · Sep 10, 2022

hp550c said:
Hey Holger. I finally got the time to implement the rest of your guide (running Chrome from the FreeBSD host as a regular user, setting up audio, etc.). It's all working great now -- amazing! One thing I did have to fix was the pulseaudio startup script you have mentioned. You need to add load_rc_config "${name}" before the pulseaudio_start() function, otherwise I was getting an error that pulseaudio_enable="YES" wasn't set in /etc/rc.conf when it actually was (because the startup script wasn't actually importing the variable from rc.conf).

Thanks for the fix! I will update the tutorial ASAP.

Edit: Tutorial has been updated.

Jerald · Jan 29, 2023

Holger said:
Introduction and motivation

There are great articles ([1] and [2]) by patovm04 here on the forum explaining how to run Chrome and Brave in a Linux chroot environment (usually /compat/linux or /compat/ubuntu).

These approaches work great. However, I am a big fan of FreeBSD's jails and it has always bothered me that these Linux compatibility layers were some kind of “sort-of” jail, but not real ones. By “real one” I mean a lightweight virtualization environment that makes use of FreeBSD's jail infrastructure (jexec, jls, /etc/jail.conf, etc.).

I am hoping that the approach I am presenting here will illustrate the following benefits:

Leverage standard jail infrastructure to uniform handling of Linux and FreeBSD jails.

Create as many Linux jails as you want (not just one in /compat/linux, respectively /compat/ubuntu).

There is at least one negative side to putting the Linux emulation into a separate jail: It is less convenient, because you won't be able to execute Linux programs as if they were part of the host system. However, you will get a clean separation between the FreeBSD world and the Linux world.

This tutorial was tested on FreeBSD-13.1-RELEASE.

Basic setup

The setup I am starting with is the following:

I am using ZFS, but do not rely on it in this tutorial. I have added remarks when appropriate on how to do the same thing on UFS.

I am using “bare metal” jails, no management utility like sysutils/bastille or sysutils/iocage. The use of them could possibly simplify some aspects of this tutorial, but I like the down-to-earth approach when using jails directly.

My jails are attached to a loopback device lo1 and I use pf to NAT to forward network traffic to my physical network. This way I can assign static IP addresses to my jails and keep using DHCP on my physical network. If you are interested in the details of this setup, just give me a call and I post them here.

I use Xorg, not Wayland, which I am unfamiliar with. If you know Wayland and a simple way to modify this tutorial to support both, please let me know!

Ok, let's go.

Enabling Linux support

Why we probably don't need /etc/rc.d/linux or similar

Normally, the service /etc/rc.d/linux takes care of everything that is needed to run Linux binaries. The service basically does the following:

Load the required kernel module(s).

Configure how to handle unbranded ELF executables.

Mount Linux system directories like /dev, /proc, etc., under the path specified by compat.linux.emul_path.

Loading the kernel modules can be done in rc.conf. Mounting the Linux system directories will be dealt with by FreeBSD's Jail infractructure.

So, besides the issue with the unbranded ELF executables, there is not much need for starting the service. Therefore, I omit it in this tutorial.

Loading the modules

Ensure that Linux support is activated:

Code:

# kldload linux64

To make it permanent, add the module in /etc/rc.conf to the kld_list variable, but ensure that your other modules remain there:

Code:

kld_list="... linux64 ..."

E.g., my kld_list now looks like this:

Code:

kld_list="i915kms linux64"

Creating the jail

I like to put each of my jails into a separate ZFS dataset. This, for example, makes it easier to create snapshots or to transfer the jail to another computer. The zpool I am using for this is called scratch. You'll have to adjust that name to your particular configuration (e.g. zroot or similar):

Code:

# zfs create scratch/jail/ubuntu # zfs set mountpoint=/jail/ubuntu scratch/jail/ubuntu

If you are using UFS instead of ZFS, a simple mkdir -p /jail/ubuntu will do.

Next, we'll need to prepare the system directories:

Code:

# for DIR in /dev/fd /dev/shm /tmp /proc /sys; do mkdir -p /jail/ubuntu/${DIR}; done

Now we can populate the jail with an Ubuntu root filesystem. For this, we'll use sysutils/debootstrap:

Code:

# pkg install debootstrap # debootstrap --arch=amd64 --no-check-gpg focal /jail/ubuntu

Before we can actually start the jail, we need to specify what has to be mounted inside the jail. FreeBSD's jail mechanism can parse an fstab-like file that does exactly that. The location of this file is irrelevant, but for simplicity, we put it right into the jail itself. So, create it using an editor:

Code:

# vi /jail/ubuntu/etc/fstab

and put the following content into it:

Code:

devfs /jail/ubuntu/dev devfs rw 0 0 tmpfs /jail/ubuntu/dev/shm tmpfs rw,size=1g,mode=1777 0 0 fdescfs /jail/ubuntu/dev/fd fdescfs rw,linrdlnk 0 0 linprocfs /jail/ubuntu/proc linprocfs rw 0 0 linsysfs /jail/ubuntu/sys linsysfs rw 0 0 /tmp /jail/ubuntu/tmp nullfs rw 0 0

Note that, with this configuration, the jail will share the host's /tmp directory. This makes it easier to share audio and X11 sockets with the jail, but, for security reasons, you might want a different configuration sometime later when everything is up and running. –

Next, we need to configure the jail. Open the file /etc/jail.conf and append the following content:

Code:

ubuntu { host.hostname="ubuntu.schattenwelt.org"; ip4.addr="lo1|10.10.0.5/24"; path="/jail/ubuntu"; allow.raw_sockets=1; exec.start='/bin/true'; exec.stop='/bin/true'; persist; mount.fstab="/jail/ubuntu/etc/fstab"; }

You will probably need to adjust the following:

The domain name (mine is schattenwelt.org, but your's will be different).

The IP address. In my configuration I use a loopback device (lo1) and assign static IP address for each jail manually. If you wonder how I get internet access inside the jail: I use the packet filter pf for that, redirecting network traffic from and to the jail to my physical network.

We are now ready to start the jail:

Code:

# service jail onestart ubuntu

To start the jail at each boot, put the following into your /etc/rc.conf:

Code:

jail_enable="YES"

You can check if the jail is properly running by doing

Code:

# jls

which should give the following output:

Code:

JID IP Address Hostname Path ... 4 10.10.0.5 ubuntu.schattenwelt.org /jail/ubuntu

Setting up the Ubuntu jail

Now that the Ubuntu jail is up and running, we can set up the Linux system. First, enter the jail:

Code:

# jexec ubuntu /bin/bash

Configure locales and timezone:

Code:

# dpkg-reconfigure locales # dpkg-reconfigure tzdata

Install necessary packages (for running Google Chrome later on):

Code:

# printf "deb http://archive.ubuntu.com/ubuntu/ focal main restricted universe multiverse" > /etc/apt/sources.list # apt update # apt install curl gnupg pulseaudio fonts-symbola ttf-mscorefonts-installer

Install GPG key enabling the Google Chrome DEB repository:

Code:

# curl -s https://dl.google.com/linux/linux_signing_key.pub | apt-key --keyring /etc/apt/trusted.gpg.d/google-chrome-stable.gpg add -

Install Google Chrome:

Code:

# printf "deb [arch=amd64] http://dl.google.com/linux/chrome/deb/ stable main" > /etc/apt/sources.list.d/google-chrome.list # apt update # apt install google-chrome-stable

Create a wrapper script for launching Chrome: Create a file /opt/google/chrome/chrome-wrapper and add the following content:

Code:

#!/bin/bash # # Copyright (c) 2011 The Chromium Authors. All rights reserved. # Use of this source code is governed by a BSD-style license that can be # found in the LICENSE file. HERE="/opt/google/chrome" export CHROME_VERSION_EXTRA="stable" # We don't want bug-buddy intercepting our crashes. http://crbug.com/24120 export GNOME_DISABLE_CRASH_DIALOG=SET_BY_GOOGLE_CHROME # Sanitize std{in,out,err} because they'll be shared with untrusted child # processes (http://crbug.com/376567). exec < /dev/null exec > >(exec cat) exec 2> >(exec cat >&2) export DISPLAY=:0 export LIBGL_DRI3_DISABLE=1 "$HERE/chrome" --in-process-gpu --no-sandbox --no-zygote --test-type --enable-features=UseOzonePlatform --ozone-platform=x11 --v=0 "$@" || true

Make it executable:

Code:

# chmod +x /opt/google/chrome/chrome-wrapper

Finally, add a non-privileged user that will be able to run Google Chrome (in my case this is hsebert, myself):

Code:

# adduser hsebert

Add this point, you should be able to run Chrome. There will be no sound yet, though (see a later chapter to fix that). To run it, first change to our non-privileged user created above:

Code:

# su - hsebert

Then run Chrome:

Code:

$ /opt/google/chrome/chrome-wrapper

When everything is allright, quit Chrome, leave the su-environment and ultimatlely the jail:

Code:

$ exit # exit

Troubleshooting

You may need to allow the Ubuntu jail to connect to your Xorg server on the host. Do this by using the following command (on the host!):

Code:

$ xhost +

To make this change permanent, place it, e.g. in ~/.xinitrc.

Allow non-root execution

You can now start Chrome from outside the jail using jexec as root:

Code:

# jexec ubuntu /bin/bash -c "su -c /opt/google/chrome/chrome-wrapper - hsebert"

Note that we use su inside the jail to drop privileges and use our non-privileged user inside the jail (hsebert).

Having to be root in order to launch Chrome is a little bit inconvenient. FreeBSD doesn't allow non-privileged jail-execution for good reasons, though. So we will use a different approach using security/sudo:

Code:

# pkg install sudo

First, we'll create a launcher script on our host and call it /usr/local/bin/linux-chrome:

Code:

#!/bin/sh if [ $# -lt 1 ] ; then echo "usage: ${0} <user>" exit 1 fi if [ "${1}" = "root" ] ; then echo "Cowardly refusing running Chrome as root" exit 1 fi jexec ubuntu /bin/bash -c "su -c /opt/google/chrome/chrome-wrapper - ${1}"

Make it executable:

Code:

# chmod +x /usr/local/bin/linux-chrome

Next, we allow a particular user on the host system to execute linux-chrome using sudo, but without requiring a passwort for this particular command (this is important!). For this, create a dedicated sudoers file as follows:

Code:

# visudo -f /usr/local/etc/sudoers.d/linux-chrome

and put a line into it having the following format:

Code:

<user> <host> = (root) NOPASSWD: <cmd>

The user (<user>) on my host system who is to be allowed to run linux-chrome is hsebert (same user name as inside the Linux jail, but this is not a necessity). My host name (<host>) is thinkle. The command to be run (<cmd>)is /usr/local/bin/linux-chrome (full path!). So, the resulting line looks like this:

Code:

hsebert thinkle = (root) NOPASSWD: /usr/local/bin/linux-chrome

After saving and closing vi, your regular user should be able to run Chrome as follows:

Code:

$ sudo linux-chrome $(whoami)

You can put this command inside a Desktop launcher, for example.

Enabling sound

Google Chrome on Linux relies on Pulseaudio, so we have to deal with it. In this guide I chose to run Pulseaudio as a system-wide service, which has pros and cons:

Pro: It can be used by multiple jails at the same time, and even by Bhyve virtual machines, if remote access is configured (see below).

Con: Even the Pulseaudio developers discourage the use as a system-wide service because of security issues, see here. Pick your poison. –

First, we need to install the package:

Code:

# pkg install pulseaudio

Pulseaudio aware applications can communicate with a Pulseaudio server either by TCP or a Unix-Domain-Socket (similar to Xorg). We configure this feature explicitly by editing /usr/local/etc/pulse/system.pa and add the following two lines:

Code:

load-module module-native-protocol-tcp auth-anonymous=1 auth-ip-acl=127.0.0.1;192.168.178.0/24 load-module module-native-protocol-unix auth-anonymous=1 socket=/tmp/pulse-native

The second line is most important for us: It tells us how the Pulseaudio socket will be named (/tmp/pulse-native). We'll need that information later when we configure Pulseaudio in the Linux jail.

Important: If you do not want remote connections over the network, delete the first line containing module-native-protocol-tcp. If remote connections are o.k. for you (e.g. because you would like to play audio in your Bhyve virtual machines), be sure to set the correct subnet mask (mine is in this case: 192.168.178.0/24).

Now that we have configured the Pulseaudio server, we need to enable it. The package audio/pulseaudio does not come with an rc script to start it up at boot time. Therefore, we have to create one: Create the file /usr/local/etc/rc.d/pulseaudio and let it have the following content:

Code:

#!/bin/sh # PROVIDE: pulseaudio # REQUIRE: DAEMON FILESYSTEMS # KEYWORD: nojail shutdown . /etc/rc.subr name="pulseaudio" desc="Start the Pulseaudio server" rcvar="pulseaudio_enable" pulseaudio_bin="/usr/local/bin/${name}" pulseaudio_pidfile="/var/run/pulse/pid" start_cmd="${name}_start" stop_cmd="${name}_stop" load_rc_config "${name}" pulseaudio_start() { ${pulseaudio_bin} --system --disallow-module-loading & } pulseaudio_stop() { if [ -f "${pulseaudio_pidfile}" ] then kill $(cat "${pulseaudio_pidfile}") fi } run_rc_command "$1"

Next, enable the service in /etc/rc.conf:

Code:

pulseaudio_enable="YES"

Finally, start the service so the we do not have to reboot the machine:

Code:

# service pulseaudio start

We are almost there. The one thing left is to let Chrome in the Linux jail know which socket to use when talking to our Pulseaudio server. We could put this information directly into the chrome-wrapper script introduced further up, but I think it's better to make it a system-wide default for our Linux jail. Therefore, create the file /jail/ubuntu/etc/profile.d/05-pulseaudio.sh and add the following line:

Code:

export PULSE_SERVER=unix:/tmp/pulse-native

Note: If you would like to configure remote access, simply replace to line above with

Code:

export PULSE_SERVER="<Host-IP-address>"

where "Host-IP-address" is the addres of the machine running the actual Pulseaudio server.

Debugging sound

If there is no sound, ensure that the Pulseaudio connection is working: Enter the Linux jail and run

Code:

$ pactl list

If everything works, you should see a list of available sources and sinks. If you get something like this:

Code:

Connection failure: Connection refused pa_context_connect() failed: Connection refused

then ensure the following:

Is the PULSE_SERVER variable set correctly?

Is the socket /tmp/pulse-native available, i.e. is /tmp properly mounted?

Is the Pulseaudio server running on the host: ps -a | grep pulseaudio?

Have you tried turning it off and on again?

References

[Linuxulator] How to install Brave (Linux app) on FreeBSD 13.0+

[Linuxulator] How to run Google Chrome (linux-binary) on FreeBSD

Create an Ubuntu Linux jail on FreeBSD 12.2

fstab in FreeBSD jails

Write up has been immensely useful in making me believe I did the right choice adopting FreeBSD as an operating system as opposed to my former distribution. The type of methodical documentation on FreeBSD makes me believe that it is best suited for people who truly want to understand the operating system they are using- all aspects of it.
One great trouble I had after installing FreeBSD was playing Netflix. It was difficult, as a fresh user, to use Linux emulation on FreeBSD to get Chrome or Brave working. There was just too much connectivity issues etc. Nevertheless, the search began thereafter and I started looking deeper. I mean- FreeBSD folks must watch Netflix right? Afterall, Netflix is run on FreeBSD. I'm glad I found this tutorial. For the moment, I've gone back to Linux. But I have a dedicated computer in which I am trying toFreeBSDs Jails/ and Linux emulation. There is so much to learn form this proper UNIX operating system. I wish this was taught in university instead of anything else. The documentation of FreeBSD is from the future! Thanks for eveyone making it possible.

bsduser2112 · Apr 13, 2023

if you wonder how I get internet access inside the jail: I use the packet filter pf for that, redirecting network traffic from and to the jail to my physical network.

Could the op or someone elaborate how the redirection is performed to get internet access within the jail using pf? In particular what rules are used to configure pf to accomplish this?.

Holger · Apr 13, 2023

bsduser2112 said:
Could the op or someone elaborate how the redirection is performed to get internet access within the jail using pf? In particular what rules are used to configure pf to accomplish this?.

Here is how I do it:
1. Create a loopback device that has enough IP-addresses for your jails: Put the following into /etc/rc.conf:

Code:

cloned_interfaces="lo1"
ifconfig_lo1_aliases="inet 10.10.0.1-12/24"

This will create a loopback device named lo1 with 10.10.0.x IP-addresses

2. Pick an 10.10.0.x address for each of your jail. E.g. in /etc/jail.conf:

Code:

ubuntu {
    ...
    ip4.addr="lo1|10.10.0.5/24";
    ...
}

Observe how the lo1-loopback device is mentioned in the ip4.addr statement.

3. Create the relevant rules in /etc/pf.conf:

Code:

nat on em0 from {lo1:network} to any -> (em0)

Here em0 is my physical network. Yours may be different!

Hope that helps!