Useful scripts

mjollnir

Daemon

Reaction score: 831
Messages: 1,267

Hi people!
Please, post here useful scripts which can help to do some useful or difficult operation on FreeBSD OS.
Found a script on one of my old laptops to attach/insert a geom(4) I/O scheduler - RTFM gsched(8).
  • An I/O scheduler is at least useful on a UFS (or alike) system; I don't know and didn't try with ZFS. Other use cases might be a DB on a dedicated geom(4) device.
  • The I/O scheduler can significantly increase throughput and interactivity with concurrent tasks, as described by the authors of gsched(8).
    I can could verify that - the system runs much "smoother" when running background tasks with heavy disk I/O. The script served me well for years.
  • The script might violate some guidelines for rc scripting that I was not aware of.
  • Very Likely it has bugs, since I was the only user.
    Drop me a note and I'll fix that.
  • If a few others could prove it's safe, it could go to /usr/share/examples/etc/rc.d or even in the base.
  • Suggested directories:{/usr/local}/etc/rc.d/geom_sched, {/usr/local}/libexec/fs_summarize.awk
    or the respective directories in /usr/share/examples
    and rc.geom_sched.conf is for insertion in rc.conf{.local}
The other script fs_summarize.awk is useful to determine the parameters for newfs(8) or just if you want some info about a FS or directory.
Just in case this is needed: the standard FreeBSD BSD license applies to the scripts, the snippet rc_geom_sched.conf obviously does not need any license.
 

Attachments

kpedersen

Daemon

Reaction score: 1,108
Messages: 2,101

Moving UNIX domain sockets
Not really a script as such but more an example that UNIX domain sockets can be moved (not copied). This is really handy for allowing X11 access into a Jail.

Code:
UNIQUE=$$
Xephyr :$UNIQUE &

# Wait until the socket exists

mv /tmp/.X11-unix/X$UNIQUE /jails/myjail/tmp/.X11-unix/X0
jexec myjail xterm -display :0
Finding absolute path of script or program
Also, a handy way to get the absolute path to your script (or more usefully, the prefix) without faffing with procfs and all that other stuff. This is useful because sometimes relative paths are not supported or not desired.

Code:
PREFIX="$(cd "$(dirname "$(which "$0")")" && cd .. && pwd)"

# which "$0" ensures that a path is returned if the program was run from $PATH
# dirname ensures that the directory containing the program (i.e bin) is returned
# cd, cd .., pwd ensures that an absolute path is given by going into the directory and using pwd.
It seems a bit "dirty" but I also use this (via popen) in C++ and perl. It is the only guaranteed way that I have found over the years. And I have tried quite a few!
 

olli@

Aspiring Daemon
Developer

Reaction score: 848
Messages: 801

Not really a script as such but more an example that UNIX domain sockets can be moved (not copied). This is really handy for allowing X11 access into a Jail.

Code:
[...]
mv /tmp/.X11-unix/X$UNIQUE /jails/myjail/tmp/.X11-unix/X0
Does that also work if /tmp and /jails are on different file systems? Because in that case a you’re technically copying, not moving.
Also, a handy way to get the absolute path to your script (or more usefully, the prefix) without faffing with procfs and all that other stuff. This is useful because sometimes relative paths are not supported or not desired.

Code:
PREFIX="$(cd "$(dirname "$(which "$0")")" && cd .. && pwd)"
The realpath(3) function (and the corresponding realpath(1) command) is very useful in this context. It converts relative paths to absolute paths and resolves symbolic links. In shell scripts I'm using the following to find out the script’s directory:
Code:
SCRIPTDIR=$(realpath "${0%/*}")
So, to get the prefix, that would be rather simple:
Code:
PREFIX=$(realpath "${0%/*}/..")
Sometimes I resort to writing a script in zsh, because FreeBSD's sh is missing quite a few features (it’s not a POSIX-compatible shell). In that case, the prefix can be found without calling any external programs, so this is most efficient:
Code:
PREFIX=${0:A:h:h}
Explanation: Basically, the “:A” modifier is equivalent to realpath(3), and the “:h” modifier is equivalent to dirname(3).

By the way, in Python you would do this:
Code:
from sys import argv
from os.path import realpath, dirname

prefix = dirname(dirname(realpath(argv[0])))
Other languages like Perl or Ruby also have bindings to the standard library functions realpath(3) and dirname(3), so it basically works the same there, too (modulo different syntax, of course). C and C++ can use those functions directly anyway, of course – the former is in <stdlib.h> and the latter is in <libgen.h>. Both are POSIX / SUS standards.
 

kpedersen

Daemon

Reaction score: 1,108
Messages: 2,101

Does that also work if /tmp and /jails are on different file systems? Because in that case a you’re technically copying, not moving.
Unfortunately no, I don't think that is possible. Perhaps with something like nullfs? You can trick it into opening the socket on the other FS?

Perhaps it is time for Zirias's project to come in handy? :): https://forums.freebsd.org/threads/new-tool-remusock-remote-unix-socket-access.76026/


The realpath(3) function (and the corresponding realpath(1) command) is very useful in this context. It converts relative paths to absolute paths and resolves symbolic links.
From what I could see, this doesn't work for things in PATH? I.e realpath ls returns /home/kpedersen/ls
 

olli@

Aspiring Daemon
Developer

Reaction score: 848
Messages: 801

From what I could see, this doesn't work for things in PATH? I.e realpath ls returns /home/kpedersen/ls
Yes, it does work for things in PATH. When you call a shell script via PATH, $0 always includes the directory in which it was found.
 

olli@

Aspiring Daemon
Developer

Reaction score: 848
Messages: 801

By the way, some generic advice for writing shell scripts, for those wo are not already experts … ;)

Try to write shell scripts using /bin/sh only. This will make the scripts most portable and doesn't require people to install a specific shell if you share them with others. Note that /bin/sh scripts will usually work without change with POSIX-compatible shells, too (zsh, ksh, bash). If you absolutely have to use POSIX features that are not supported by FreeBSD's /bin/sh, I suggest using ksh because it most closely resembles the POSIX standard, and also works with bash and zsh. Avoid using bashisms or zshisms that don't work anywhere else.

Begin your script with set -Cefu.
  • -C (“noclobber”) prevents the script from overwriting files with > accidentally. If you need to overwrite a file on purpose and you're absolutely sure that it’s the correct thing to do, you can use >| instead to override the -C option, or enclose the respective code between set +C and set -C, or simply rm -f the file beforehand.
  • -e (“errexit”) causes the script to terminate immediately if a command returns an error (i.e. a non-zero return code). See the sh(1) manual page for details. If you know that a command may return a non-zero exit code, but you want to continue anyway, append || true, or (better) put it into an if clause and print a message to the user if appropriate.
  • -f (“noglob”) disables globbing, also known as pathname expansion or filename generation using wildcards like *. This is often a cause of subtle bugs and malfunctions, so it’s best to disable it altogether. To create a list of files, it’s better to use things like ls | grep ... or find(1) which is much more powerful.
  • -u (“nounset”) causes the script to print an error message and exit immediately if you try to use a variable that has not been set before. unset FOO; echo $FOO will trigger this, for example. This is useful to quickly detect typos in variable names. If you need to expand a variable that might be unset (and you known what you’re doing), ${FOO-} will prevent the error (if FOO is unset, it will expand to an empty string). Again, see the sh(1) manual page for details.
These options are very helpful to write scripts that are more robust and have a lower risk of malfunctions.

The next line in most of my scripts is: ME=${0##*/}
That sets the variable ME to the basename of the script, so it can conveniently be used in error messages and similar, like in this little helper function:
Code:
Err ()
{
        echo "${ME}: $*" >&2
        exit 1
}
So you can write things like this:
Code:
test -f "$INFILE" || Err "File not found: $INFILE"
If in doubt, always write variable expansions in double quotes.
If a variable might begin with a dash, use -- to prevent it from being confused with an option if appropriate, e.g.:
ls -l -- "$MYDIR"

By the way, do not use backticks ( `foo`) for command substitution. Nesting them is difficult, they can be confused with single quote characters ( 'foo' looks very similar), and the quoting rules involving backticks are complicated. Better use the alternative $(foo) syntax. This can be nested arbitrarily deep without having to use heaps of backslashes, and the quoting rules are more intuitive.

If you need to debug a shell script, the -v and -x options are very useful: sh -vx ./myscript
  • -v causes the script code to be printed as it is read from the file during execution.
  • -x causes the resulting commands (after variable expansion etc.) to be printed right before they're executed. This is especially useful to see what your script is actually doing.

Happy hacking!
 

kpedersen

Daemon

Reaction score: 1,108
Messages: 2,101

  • -u (“nounset”) causes the script to print an error message and exit immediately if you try to use a variable that has
That is so very useful for constructs like:

rm -rf "${LOGDIR}/${DATE}"

Where you just happen to leave those variables unassigned in an incorrect code-path. XD
 

decuser

Active Member

Reaction score: 50
Messages: 173

Sleep daemon for monitoring battery life and putting laptop to sleep when needed

Here's a useful set of scripts for creating a daemon to monitor battery life and sleep the system when it's critically low that works without a desktop (it also works with TWM, but it's not required). The original discussion on this topic is here, the workhorse is the monitor, which is a modified version of the script DutchDaemon originally posted here. The script basically checks the battery level every 5 minutes and when it drops to 10%, it sets a 2 minute timer and checks again, if it's getting lower at that time, it sends an email to root warning of imminent shutdown and 2 minutes after that sleeps the system.

I use the daemon because I have not been using a DE lately and my laptop kept powering down when the battery got low. This keeps that from happening.

The zipfile - sleepd.zip contains these three files:

1. rc-sleepd - the rc script to control the daemon, gets installed in /etc/rc.d
2. sbin-sleepd - the script that does the actual monitoring, gets installed in /usr/sbin
3. install.sh - a simple install script to install the daemon and remind you how to enable the service

To install, just download the zipfile as a normal user with wheel privileges and:

unzip sleepd.zip
cd sleepd
sudo sh ./install.sh


and

sudo sysrc -f /etc/rc.conf sleepd_enable="YES"
sudo service sleepd start


20200722-1418 wds - updated the scripts to use /usr/local prefixes in line with hier()
 

Attachments

kpedersen

Daemon

Reaction score: 1,108
Messages: 2,101

OpenSSH on Windows is kinda bearable compared to remote powershell. The biggest issue is no native tmux / screen support.
I was not happy with the existing practice of using a tabbed console or tmux on the local host and opening multiple ssh connections so I made my own tool.


Basically run bin/vimux and then you can use tmux-like shortcuts like ctrl-c to create new terminals within the same session.

(the underlying platform enabling this is vim and its virtual-pty support ;)
 

olli@

Aspiring Daemon
Developer

Reaction score: 848
Messages: 801

In another thread I mentioned this shell function, and mjollnir asked me to put it here, so here it is.
It uses zsh syntax and is meant to be put in your ~/.zshrc, but I guess it could be adapted to bash without much effort.

Alternatively you can turn it into a standalone script: Remove the first line “inplace ()” and the outermost level of braces, and prepend the line “#!/usr/bin/env zsh”. Save the script with the name inplace in some bin directory (so it can be found via your $PATH setting), and make it executable: chmod 755 inplace. Make sure you have shells/zsh installed.
Code:
inplace ()
{
        #   Execute a command that reads from a file and writes to stdout,
        #   and move the output back to the input file.  The file name must
        #   be the last argument.  Timestamps (atime, mtime) and permissions
        #   are preserved.  A backup of the original file is retained with the
        #   extension ".BAK".  If an error occurs, the orignal file is not
        #   changed, and a temporary file is kept in the same directory
        #   (unless it is empty).

        emulate -L zsh
        local LASTARG="${@[-1]}"
        local BAKNAME="$LASTARG".BAK
        local TMPNAME="$LASTARG".inplace_tmp

        echo "Note: Keeping backup in $BAKNAME" >&2
        rm -f -- "$TMPNAME" \
        && "$@" > "$TMPNAME" \
        && touch -r "$LASTARG" -- "$TMPNAME" \
        && chmod 0$(stat -f"%OLp" -- "$LASTARG") "$TMPNAME" \
        && mv -f -- "$LASTARG" "$BAKNAME" \
        && mv -f -- "$TMPNAME" "$LASTARG" \
        || {
                if [[ ! -s "$TMPNAME" ]]; then
                        rm -f -- "$TMPNAME"
                fi
                return 1
        }
}
Usage example:
inplace iconv -f ISO8859-16 -t UTF-8 somefile.txt
 

Neubert

Member

Reaction score: 11
Messages: 23

Here's my current backup script in case it helps other new users with similar needs.

All of my data from multiple computers fits on a single hard drive, so I back it all up to a central FreeBSD server with a removable drive tray and store the drives offsite.

The morning protocol takes about 15 seconds; if the programmable LED is flashing green, I open the tray and swap the drive, then take it with me when I leave. At night I bring home an older backup drive and then repeat the swap procedure the next morning.

The script runs zfs scrub on insertion every Saturday, so all of the backup drives get scrubbed for errors once every few weeks as they rotate through.

I'm currently backing up a Windows desktop, an Ubuntu laptop, a FreeBSD server, and the backup server itself to the external drive.

The system detects drive insertion using this devd config:

Bash:
# cat /usr/local/etc/devd/backup.conf
notify 100 {
    # insert backup drive
    match "system" "GEOM";
    match "subsystem" "DEV";
    match "type" "CREATE";
    match "cdev" "gpt/backup";
    action "su -l backup -c 'doas /usr/local/sbin/backup.sh --on-insert' &";
};

notify 100 {
    # eject backup drive
    match "system" "GEOM";
    match "subsystem" "DEV";
    match "type" "DESTROY";
    match "cdev" "gpt/backup";
    action "su -l backup -c 'doas /usr/local/sbin/backup.sh --on-eject' &";
};
The script runs using doas for a dedicated user with this config:

Bash:
# cat /usr/local/etc/doas.conf
permit nopass backup as root cmd /usr/local/sbin/backup.sh
The backup script itself is attached below.
 

Attachments

Last edited:
Top