jails Using Makefiles to start applications in a Podman container

NapoleonWils0n

NapoleonWils0n

I have Ollama using Cuda running in a Podman container using this project

github.com

GitHub - NapoleonWils0n/freebsd-cuda: Run Linux command line and gui applications on FreeBSD with full Nvidia Cuda hardware acceleration.

Run Linux command line and gui applications on FreeBSD with full Nvidia Cuda hardware acceleration. - NapoleonWils0n/freebsd-cuda
github.com github.com

However the issue was starting Ollama in the container required the following commands

1) change into the ollama directory

Code: 
cd ~/podman/freebsd-cuda/containers/ai/ollama

2) Start the container

Code: 
doas podman-compose up -d

3) Enter the container

Code: 
doas podman exec -it -u ${USER} ollama /bin/bash

4) Start Ollama

Note because im running a turing gpu
i have to prefix the ollama serve command with OLLAMA_LLM_LIBRARY=cuda_v12

Code: 
OLLAMA_LLM_LIBRARY=cuda_v12 ollama serve

So i wanted to simplify starting Ollama

The solution is using using a Makefile with a couple of scripts

The new way i start Ollama is

1) change into the directory with the Makefile

Code: 
cd ~/make/ollama

2) Run make which starts the container and runs the ollama serve command using a script

Code: 
make

And to stop Ollama and the container i run

Code: 
make clean

The Makefile

Code: 
#===============================================================================
# FILE: Makefile
# Project: Ollama
#===============================================================================


#===============================================================================
# PHONY
#===============================================================================

.PHONY: all clean


#===============================================================================
# commands
#===============================================================================

# compose.yaml
COMPOSE_FILE      := ${HOME}/podman/freebsd-cuda/containers/ai/ollama/compose.yaml
COMPOSE_CMD    := doas podman-compose -f
ENV_FILE      := ${HOME}/podman/freebsd-cuda/containers/ai/ollama/.env
JAIL_NAME    := ollama
STARTUP_SCRIPT    := start-ollama
COMPOSE_DOWN    := $(COMPOSE_CMD) $(COMPOSE_FILE) down


#===============================================================================
# default target
#===============================================================================

all: container


#===============================================================================
# compose up
#===============================================================================

container:
    wrapper-freebsd -j $(JAIL_NAME) -a $(STARTUP_SCRIPT) -c $(COMPOSE_FILE) -e $(ENV_FILE)


#===============================================================================
# clean
#===============================================================================

clean:
    $(COMPOSE_DOWN)


The Makefile then calls the freebsd-wrapper script

The freebsd-wrapper script gets the Podman container id string
and because Podman containers are actually Jails on Freebsd

I then use jexec with the jail id which is the same as the podman id
to call a script called wrapper-podman inside the Podman container and the name of the application to start

Code: 
#!/bin/sh

#===============================================================================
# file: wrapper-freebsd
# project: wrapper-freebsd script that run wrapper-podman
#===============================================================================

#===============================================================================
# script usage
#===============================================================================

usage () {
# if argument passed to function echo it
[ -z "${1}" ] || echo "! ${1}"
# display help
echo "\
# script usage
$(basename "$0") -j jail -a application -c compose.yaml -e .env -p on|off
-p on = enable audio on the freebsd host for the jail
"
exit 2
}


#===============================================================================
# check the number of arguments passed to the script
#===============================================================================

[ $# -gt 0 ] || usage "${WRONG_ARGS_ERR}"


#===============================================================================
# getopts check the options passed to the script
#===============================================================================

while getopts ':j:a:c:e:p:h' opt
do
  case ${opt} in
     j) jail="${OPTARG}";;
     a) app="${OPTARG}";;
     c) compose="${OPTARG}";;
     e) env="${OPTARG}";;
     p) audio="${OPTARG}";;
     h) usage;;
     \?) usage "${INVALID_OPT_ERR} ${OPTARG}" 1>&2;;
     :) usage "${INVALID_OPT_ERR} ${OPTARG} ${REQ_ARG_ERR}" 1>&2;;
  esac
done
shift $((OPTIND-1))


#===============================================================================
# start audio if -p on option is specified
#===============================================================================

# default audio on
audio_default="on"

# start audio
audio () {
pgrep pulseaudio || pulseaudio --start --daemonize 2>/dev/null
}

# check if pulseaudio should be started
case "${audio:=${audio_default}}" in
    on) audio;;
    off) ;;
    *) usage;;
esac


#===============================================================================
#  Check if the container is already running. If not, spin it up backgrounded
#===============================================================================

if [ -z "$(doas podman ps -q -f name="${jail}")" ]; then
    # Change directory to ensure podman-compose finds your compose.yaml path cleanly
    doas podman-compose -f "${compose}" --env-file "${env}" up -d
  
    # Strictly wait until podman inspect confirms the jail is fully running
    while [ "$(doas podman inspect --format '{{.State.Running}}' "${jail}" 2>/dev/null)" != "true" ]; do
        :
    done
fi


#===============================================================================
# Resolve jail IDs and execute command
#===============================================================================

# find the podman id
full_id=$(doas podman ps -q --no-trunc -f name="${jail}")

# Ensure jls has registered the long ID string before moving forward
jail_id=""
while [ -z "${jail_id}" ]; do
    jail_id=$(doas jls jid name | awk -v id="${full_id}" '$2 == id {print $1}')
done

# Use jexec to run the wrapper-podman script in the podman container
doas jexec -u "${USER}" "${jail_id}" /bin/bash -ilc "${HOME}/bin/wrapper-podman -a ${app}" -- "${@}"

The wrapper-podman script

Code: 
#!/bin/sh

#===============================================================================
# wrapper-podman
#===============================================================================

#===============================================================================
# script usage
#===============================================================================

usage () {
# if argument passed to function echo it
[ -z "${1}" ] || echo "! ${1}"
# display help
echo "\
# script usage
$(basename "$0") -a application"
exit 2
}

#===============================================================================
# check the number of arguments passed to the script
#===============================================================================

[ $# -gt 0 ] || usage "${WRONG_ARGS_ERR}"


#===============================================================================
# getopts check the options passed to the script
#===============================================================================

while getopts ':a:h' opt
do
  case ${opt} in
     a) app="${OPTARG}";;
     h) usage;;
     \?) usage "${INVALID_OPT_ERR} ${OPTARG}" 1>&2;;
     :) usage "${INVALID_OPT_ERR} ${OPTARG} ${REQ_ARG_ERR}" 1>&2;;
  esac
done
shift $((OPTIND-1))


#===============================================================================
# start the application and exit cleanly
#===============================================================================

"${app}" "${@}" >/dev/null 2>&1 &
exit 0

The wrapper-podman script inside the Podman container
then runs a script called start-ollama

The start-ollama script detects the gpu
and calls the correct command to start ollama

Code: 
#!/bin/sh

#===============================================================================
# start-ollama
#===============================================================================

# Query nvidia-smi for the GPU name
GPU_NAME=$(nvidia-smi --query-gpu=gpu_name --format=csv,noheader)

# Check if the gpu is (GTX 16xx, RTX 20xx, or explicit Turing)
if echo "$GPU_NAME" | grep -qiE "turing|GTX 16|RTX 20"; then
    OLLAMA_LLM_LIBRARY=cuda_v12 ollama serve
else
    ollama serve
fi

You have to start applications from inside the Podman container
instead of from the Freebsd side because Podman is rootfull on Freebsd

And if you try and start application from Freebsd they will be run as root in the container
Using this method the application is started as the same user in the Podman container as on Freebsd

Because i map the user id and gid from Freebsd to the Podman container

I also use a Makefile to start Invidious running in a Bhyve vm as well

The Makefile starts the vm uses netcat to check if port 22 is open
then use ssh to login, changes into the Podman container and runs a script to start Invidious

So i just run make to start the vm and the Invidious Podman conatiner
and make clean to ssh into the container stop Invidious and then stop the VM

github.com

freebsd-dotfiles-xps/make/invidious/Makefile at master · NapoleonWils0n/freebsd-dotfiles-xps

freebsd dotfiles dell xps 15 2019. Contribute to NapoleonWils0n/freebsd-dotfiles-xps development by creating an account on GitHub.
github.com github.com
 
You must log in or register to reply here.
Back
Top