Solved bash oddity on FreeBSD with version 5.1 and beyond

patpro

patpro

Hello,

I'm currently trying to test borgwarehouse.com to make sure it can get installed and ran properly on FreeBSD. It's very early stage software but sounds promising. Anyway, the software relies on a handful of bash shell scripts that could be ported to FreeBSD in no time.
One of those shell scripts tries to validate an SSH key handled by the user, relevant code is:

Bash: 
# Check if the SSH public key is a valid format
# This pattern validates SSH public keys for : rsa, ed25519, ed25519-sk
pattern='(ssh-ed25519 AAAAC3NzaC1lZDI1NTE5|sk-ssh-ed25519@openssh.com AAAAGnNrLXNzaC1lZDI1NTE5QG9wZW5zc2guY29t|ssh-rsa AAAAB3NzaC1yc2)[0-9A-Za-z+/]+[=]{0,3}(\s.*)?'
if [[ ! "$2" =~ $pattern ]]
then
    echo "Invalid public SSH KEY format. Provide a key in OpenSSH format (rsa, ed25519, ed25519-sk)"
    exit 2
fi

and full script is available at https://github.com/Ravinou/borgwarehouse/blob/main/helpers/shells/createRepo.sh

Problem: that code block will work OK on most machines I've tested it on
  • macOS bash 3.x
  • Linux bash 4.x, 5.0.x, 5.1.4
  • FreeBSD 12 bash 5.0.x
But it will always fail on:
  • FreeBSD 13 bash 5.1.16
  • FreeBSD 13 bash 5.2.12
I can't find a proper change log for Bash to see if something happened between 5.1.4 and 5.1.16 that could explain this behavior.
I don't have access to Linux with bash 5.1.16 in order to validate it's a Bash problem and not a FreeBSD problem.

Any help appreciated :)
 
Thanks, it's working. Do you have any idea if it comes from the bash version it self, or from a FreeBSD only bash patch?
 
patpro said:
Hello,

I'm currently trying to test borgwarehouse.com to make sure it can get installed and ran properly on FreeBSD. It's very early stage software but sounds promising. Anyway, the software relies on a handful of bash shell scripts that could be ported to FreeBSD in no time.
One of those shell scripts tries to validate an SSH key handled by the user, relevant code is:

Bash: 
# Check if the SSH public key is a valid format
# This pattern validates SSH public keys for : rsa, ed25519, ed25519-sk
pattern='(ssh-ed25519 AAAAC3NzaC1lZDI1NTE5|sk-ssh-ed25519@openssh.com AAAAGnNrLXNzaC1lZDI1NTE5QG9wZW5zc2guY29t|ssh-rsa AAAAB3NzaC1yc2)[0-9A-Za-z+/]+[=]{0,3}(\s.*)?'
if [[ ! "$2" =~ $pattern ]]
then
    echo "Invalid public SSH KEY format. Provide a key in OpenSSH format (rsa, ed25519, ed25519-sk)"
    exit 2
fi

and full script is available at https://github.com/Ravinou/borgwarehouse/blob/main/helpers/shells/createRepo.sh

Problem: that code block will work OK on most machines I've tested it on
  • macOS bash 3.x
  • Linux bash 4.x, 5.0.x, 5.1.4
  • FreeBSD 12 bash 5.0.x
But it will always fail on:
  • FreeBSD 13 bash 5.1.16
  • FreeBSD 13 bash 5.2.12
I can't find a proper change log for Bash to see if something happened between 5.1.4 and 5.1.16 that could explain this behavior.
I don't have access to Linux with bash 5.1.16 in order to validate it's a Bash problem and not a FreeBSD problem.

Any help appreciated :)
Click to expand...

FreeBSD comes with a version of Thomson's SHell installed on the base system, not BASH (which is ironic because this forum only has BASH syntax highlighting for the code blocks below). From my experience, this means that some of the typical syntax shortcuts we would expect to use won't work and should be replaced with the long-handed versions. (e.g. use single brackets for test, not two, use GREP and/or case for pattern matching)

Bash: 
if [[ ! "$@" =~ $pattern ]]
then
    ...
fi

Bash: 
M=$( /bin/echo "$@" | /usr/bin/grep -sE "${pattern}\$" );
if [ -z "$M" ]; then
    ...something to do when pattern isn't matched...
fi
 
Well. My question is about the behavior of bash, the default shell on FreeBSD is irrelevant :)
 
Is bash handling that regexp on its own (i.e. using its own, built in regexp-engine) or is a system-specific library used? This might be the culprit here - I remember grep vs gnu grep being a quite annoying source of errors when porting scripts...
 
sko said:
Is bash handling that regexp on its own (i.e. using its own, built in regexp-engine) or is a system-specific library used? This might be the culprit here - I remember grep vs gnu grep being a quite annoying source of errors when porting scripts...
Click to expand...
That's most likely the issue here. \s and friends are perl regex syntax. The GNU tooling supports this, FreeBSD strictly sticks to POSIX (extended) regex.

Portable scripts should never rely on such extensions, but this here is an interesting problem, as with requiring bash, you already left the path of portability :rolleyes:
 
I can only say that I found bash to be one big PITA with how it changes. One VM I updated bash on refused to configure a project it had done in the past. Turns out the bash version 2 digits behind the point make all the difference.
 
To expand on what Crivens said: On any system other than Linux, don't use bash if you need stability and reliability. In the last 15 or 20 years of using bash as my default shell on 3-4 operating systems (only one of which is Linux), I have had bash break several times; most recently so badly that logging in to my FreeBSD server with bash become impossible (thread here).

That having been said: If you are porting software that heavily relies on bash to FreeBSD, there is just no other option than to use it. And most of the time, it works just fine, and some people (me!) find it comfortable to use as a login shell. Just be aware of the potential downside.
 
I agree with all of the above. There are major compatibility issues with the various versions of bash that just break things.

So if you choose to script with it, you have to watch the version numbers carefully!

Of all the shells, bash has the largest memory footprint, and the worst execution speed. This matters little for an interactive shell, but it's really not well suited to any form of "heavy lifting".
 
I wouldn't find any of this annoying (because I don't use bash and avoid it like the plague), if it weren't for the multitude of "shell"-scripts out there that define '#/bin/sh' but are actually bash scripts...
 
ralphbsz said:
To expand on what Crivens said: On any system other than Linux, don't use bash if you need stability and reliability. In the last 15 or 20 years of using bash as my default shell on 3-4 operating systems (only one of which is Linux), I have had bash break several times; most recently so badly that logging in to my FreeBSD server with bash become impossible (thread here).

That having been said: If you are porting software that heavily relies on bash to FreeBSD, there is just no other option than to use it. And most of the time, it works just fine, and some people (me!) find it comfortable to use as a login shell. Just be aware of the potential downside.
Click to expand...

Well, they COULD just create a/the port(s) with the appropriate patch(es) so it can use Thomson's SHell. :p
 
You must log in or register to reply here.
Back
Top