Useful scripts

graudeejs

Son of Beastie

Reaction score: 698
Messages: 4,617

graudeejs said:
$ echo $PATH | tr ':' ' ' | xargs -n 1 echo
You can probably write this even shorter
Here's better one
$ echo $PATH | awk 'BEGIN {RS=":"}; {print $0}'

this one will respect spaces in directories
 

graudeejs

Son of Beastie

Reaction score: 698
Messages: 4,617

wblock@ said:
How about
% echo $PATH | tr ':' '\n'
Totally forgot, that tr can translate to \n
I tried with sed, but failed, so I just wrote with awk
 

aa

Member

Reaction score: 17
Messages: 67

I frequently use this script when preparing FreeBSD-box since REL-4

Code:
.type catch
catch is a function
catch ()
{
    [ "$1" ] && $PAGER `which $*`
}
.
.catch mona
#!/bin/sh
ROOT=/tmp/mounts
showbanner(){
cat <<::
---------
my banner
---------

Synopsys:
  mount/unmount any ufs and/or dos filesystems

Changelog:
  version 1.0.4d, last update: 2008.11.5
    (small fix add freebsd release 8 & 9)
  version 1.0.5, last update: 2009.01.03
    (allow more than 1 filter, remaining args
     will be treated as filters. examples added)
  version 1.0.5a, last update: 2010.01.01
    (update: FreeBSD 8+ allows more slices, upto silce-z)

Usage:
  [sh] ${0##*/} mount_arg filesystem_arg filter_args...

Where:
  mount_arg is either mount (read-only, read-write) or unmount
    valid choices are: -m, -mr, -mw and -u
    - preceding dash (-) IS required

  valid choices for filesystem_arg are:
    u, ufs, f, fat, n, nt, ntfs, d, dos, and all
    - could be in any (upper/lower/mixed) case,
    - preceding dash is allowed but not required
    - if not supplied, default to UFS

Examples:
  [sh] ${0##*/} -m -a
  mount all partitions, ufs and dos (fat/ntfs)

  [sh] ${0##*/} -u
  unmount ufs only (dos partition will NOT be unmounted)

  [sh] ${0##*/} -mw ufs ad4s2[a-e] ad8s?[a-e]
  mount read-write ufs disk4 slice 2 and disk8 (any slices)
  partition a to e (usually root, var and usr/local)

Notes:
  when filter_args is given, then the preceding 2 arguments
  (mount and filesystem) must also be supplied

  default mount (-m) is read-only
  root dir will be under $ROOT/BSD or $ROOT/DOS

::
#ROOT=""
#ROOT=/tmp/mounts
}; case "$1" in -[?hH]|--help|[?]) showbanner; exit 0;; esac
printerr() { showbanner; echo -e "Error - $*\nProgram aborted.\n"; exit 1; }

getmountedslices() { local IFS='
'; for dev in `df`; { test "$dev" -a ! "${dev%%/*}" && echo "${dev%% *}" "${dev##* }"; }
}

getmountpoint() {
  test "${1#/}" || { echo "$1"; return 1; }
  test "$MOUNTPOINTS" || MOUNTPOINTS=`getmountedslices`
  local IFS='
'; for pair in $MOUNTPOINTS
  { test -z "${pair##$1 *}" && { echo "${pair##* }"; return 0; } }
}

getMSDOSFS() {
  REL=`sysctl -b kern.osrelease`
  case ${REL%%-*} in
    4.*|7.47*) echo msdos;;
    [5-9].*) echo msdosfs;;
    *) echo msdosfs;;
  esac
}

case "$1" in
  -a) unset mod; set -- "" -a;;
  -m|-mr) mod=ro,;;
  -mw) mod=rw,;;
  -u) unset mod;;
  "") unset mod; showbanner;;
   *) printerr unknown option $1;;
esac

enum_fs() {
  set -f
  local c1="[0-9]" c2="[0-9][0-9]" c3="[1-4][a-z]"
  local sc1="s$c1" sc2="s$c2" sc3="s$c3"
  local FS="" r=""
  test "$UFS" = 1 && r="$r $c1$sc3 $c2$sc3"
  test "$NT4" = 1 -o "$FAT" = 1 && r="$r $c1$sc1 $c1$sc2 $c2$sc1 $c2$sc2"
  for n in $r; { FS="$FS /dev/ad$n /dev/da$n"; }
  set +f; echo $FS
}

getfilters() { test -n "$3" || return; shift 2; echo -n "$@"; }
filters="`getfilters "$@"`"
#filter="$3";
#echo filters="$filters"

UFS=""; NT4=""; DOS=""; FAT="";
case "${2#-}" in
  [aA]|[aA][lL][lL]) UFS=1; NT4=1; FAT=1;;
    #nodes="/dev/ad*s*[0-9ad-z] /dev/da*s*[0-9ad-z]";;
  [dD]|[dD][oO][sS]|[fF]|[fF][aA][tT]|[nN]|[nN][tT]|[nN][tT][fF][sS])
    #nodes="/dev/ad*s*[0-9] /dev/da*s*[0-9]";;
    case "${2#-}" in
      [dD]|[dD][oO][sS]) NT4=1; FAT=1;;
      [fF]|[fF][aA][tT]) FAT=1;;
      [nN]|[nN][tT]|[nN][tT][fF][sS]) NT4=1;;
    esac;;
  ""|[uU]|[uU][fF][sS]) UFS=1;;
    #nodes="/dev/ad*s[1-4][ad-z] /dev/da*s[1-4][ad-z]";;
  *) echo unknown filesystem identifier: $2; exit 1;;
esac
nodes="`enum_fs`"

#Constants:
BSD=BSD
DOS=DOS

AA=999:999
OWNER="-uaa -gaa"
OWNER=""

opt_UFS="noatime -t ufs"
opt_DOS="noatime,noexec,nosuid,nodev"
opt_DOS="noatime,noexec,nosuid"         #FreeBSD7 no longer accepts nodev
opt_NT4="-m775 -t ntfs" #could never be written anyway
                        #limited (if any at all) write only for small system
opt_NT4="-m775"
opt_FAT="-m664,-M775 -t `getMSDOSFS`"

MOUNTPOINTS=`getmountedslices`

for node in $nodes; do test -e "$node" || continue
  #[ "$filter" ] && [ "${node##*$filter*}" ] && continue

  SKIP=""
  if [ "$filters" ]; then SKIP=1
    for f in $filters; do
      test -z "${node##*$f*}" && { SKIP=""; break; }
    done
  fi

  [ "$SKIP" ] && continue

  test "$1" != -u && mountpt="`getmountpoint $node`" && {
    echo -n $node already mounted in: $mountpt\ ;
    test -w $mountpt && echo "(rw)" || echo "(ro)"
    test -z ""; continue;
  }

  slice=s"${node#/dev/*s}"      # = s2a
  chunk="${node%$slice}"        # = /dev/ad2
  dev="${chunk#/dev/}"          # = ad2

  disk=""       #disk="$root/$dev"
  mountpoint="" #mountpoint="$disk/$dev$slice"

  __setmountvars() { # got vars: disk, mountpoint
    test "$1" -a ! "$2" || printerr "invalid construct: [$@]"
    #echo disk=$disk slice=$slice mountpoint=$mountpoint
    case "$1" in
      UFS) test -z "${slice%%*[1-4][ad-z]}" || return 1
        root=$ROOT/$BSD; OPTS="$mod$opt_UFS";;
      DOS|FAT|NT4) test -z "${slice%%*[0-9]}" || return 1
        root=$ROOT/$DOS; OPTS="$mod$opt_DOS"
        test $1 = FAT && OPTS="$OPTS,$opt_FAT" || OPTS="$OPTS $opt_NT4"
        OPTS="$OPTS $OWNER";;
      *) printerr unknown filesystem: $*;;
    esac
    disk=$root/$dev             # /mnt/UFS/ad2
    mountpoint=$disk/$dev$slice # /mnt/UFS/ad2/ad2s2a
    #echo now disk=$disk slice=$slice mountpoint=$mountpoint
  }

  __delif_empty() { test -d "$1" -a "$1/*" = "`echo $1/*`" && rm -R "$1"; }
  __delife_all() { __delif_empty $mountpoint; __delif_empty $disk; }
  __mountcheck() { __setmountvars $1 && echo "  $1: $node will be mounted on $mountpoint"; }

  # updated for freebsd7's minor bug: does not return error on failed mount_ntfs
  __mount() { __setmountvars $1 || return
    [ "$1" = "NT4" ] && FBSD7_BUG=_ntfs || unset FBSD7_BUG
    test -d $mountpoint || { mkdir -p $mountpoint && \
      chown $AA $mountpoint || printerr $mountpoint creation failed; }
      (mount$FBSD7_BUG -o $OPTS $node $mountpoint > /dev/null 2>&1) && \
      echo "mounted $1 in $mountpoint (${mod%,})" && return
    __delife_all; return 1 #must be returns fail
  }
  __unmount() { __setmountvars $1 || return
    test -d $mountpoint && umount $mountpoint && \
      echo "  unmounted $1: $node on $mountpoint"
    __delife_all
  }

  case "$1" in
    -m|-m[wr])
      test "$UFS" = 1 && __mount UFS && continue
      test "$NT4" = 1 && __mount NT4 && continue
      test "$FAT" = 1 && __mount FAT && continue
      #echo "*** failed to mount $node";;
      ;;
    -u)
      test "$UFS" = 1 && __unmount UFS
      test "$FAT" = 1 -o "$NT4" = 1 && __unmount DOS
      #echo "*** failed to unmount $node";;
      ;;
    "")
      test "$UFS" = 1 && __mountcheck UFS && continue
      test "$FAT" = 1 -o "$NT4" = 1 && __mountcheck DOS  && continue
      #echo "*** invalid $node";;
      ;;
  esac

done

#test "$1" || { echo; showbanner; }

.
 

debguy

Active Member

Reaction score: 11
Messages: 139

Code:
# these can make some script list processing problems
# easier to make and understand - and are efficient as sort.
#


#!/bin/sh
set -e
if [ c"$1" = c"-h" ] ; then
cat << EOF >> /dev/stderr
USAGE: [intersection union complement complement-intersection] [-n] file1 file2
DESCR: files are flat sets, lines in are elements (newline list)
       these small shell scripts simplify list work [in scripts]
       show all not in both (c-i) : Comp [ Union[s1,s2] , Int[s1,s2] ]
limit:
       uses sort(1), use -n for numberic for numberic sets, uses uniq(1)
EOF
exit ; fi
[ c"$1" = c"-n" ] && p="-n" && shift
[ -f "$1" ] && [ -f "$2" ] || exit 0
cat "$1" "$2" | sort $p | uniq -d
Code:
#!/bin/sh
set -e
if [ c"$1" = c"-h" ] ; then
cat << EOF >> /dev/stderr
USAGE: [intersection union complement complement-intersection] [-n] file1 file2
DESCR: files are flat sets, lines in are elements (newline list)
       these small shell scripts simplify list work [in scripts]
       show all not in both (c-i) : Comp [ Union[s1,s2] , Int[s1,s2] ]
limit:
       uses sort(1), use -n for numberic for numberic sets, uses uniq(1)
EOF
exit ; fi
[ c"$1" = c"-n" ] && p="-n" && shift
[ ! -f "$1" ] && exit 0
cat "$1" "$2" | sort $p | uniq
Code:
#!/bin/sh
set -e
if [ c"$1" = c"-h" ] ; then
cat << EOF >> /dev/stderr
USAGE: [intersection union complement complement-intersection] [-n] file1 file2
DESCR: files are flat sets, lines in are elements (newline list)
       these small shell scripts simplify list work [in scripts]
       show all not in both (c-i) : Comp [ Union[s1,s2] , Int[s1,s2] ]
limit:
       uses sort(1), use -n for numberic for numberic sets, uses uniq(1)
EOF
exit ; fi
[ c"$1" = c"-n" ] && p="-n" && shift
[ ! -f "$1" ] && exit 0
cat "$1" | sort $p | uniq > s1.$$
[ ! -f "$2" ] && cat s1.$$ && rm s1.$$ && exit
intersection $p "$1" "$2" > s2.$$
cat s1.$$ s2.$$ | sort $p | uniq -u
rm s1.$$ s2.$$

Code:
#!/bin/sh
set -e
if [ c"$1" = c"-h" ] ; then
cat << EOF >> /dev/stderr
USAGE: [intersection union complement complement-intersection] [-n] file1 file2
DESCR: files are flat sets, lines in are elements (newline list)
       these small shell scripts simplify list work [in scripts]
       show all not in both (c-i) : Comp [ Union[s1,s2] , Int[s1,s2] ]
limit:
       uses sort(1), use -n for numberic for numberic sets, uses uniq(1)
EOF
exit ; fi
[ c"$1" = c"-n" ] && p="-n" && shift
[ ! -f "$1" ] && exit 0
cat "$1" "$2" | sort $p | uniq -u
intersection union complement complement-intersection

OOPS ^^^ names for the above :)
 

debguy

Active Member

Reaction score: 11
Messages: 139

# This little sed(1) line below reads in an "/etc/config" that is an easy to edit and free floating Left Right file for a user to work with, for a script that needs to read in LR settings (assuming the script is not allowed to just source'ing the config to assign variables, assuming script may need one L to many R; list support).

Code:
scr=':lbl;/^[[:space:]+]/{H;n;x;s/[[:cntrl:]+]/ /g;x;b lbl};x;/^./p'

IFS=$'\n'
for ea_line in `cat $file | sed -e 's/#.*$//' | sed -ne $scr` ; do

IFS=$' \t'
for ea_str in $ea_line ; do
   foo
   # first str is left label, all other are rights
done

# That reads in an "easy edit" slipshod file like the below :)  ...

# COMMAND       DEPENDANCY to meet before command
# (label of)    (filename that must exist, but see Detours)

goal            end
terminal        %bogus        # some comment
request         begin  terminal
end             begin terminal request records-begin remove-list
                install-list command-list response pre-check
                post-check records-end show-result clean
goal            end

# # (see [url]www.souceforge.net/projects/dep-trace[/url] for shmake(1)
Rights can be split across lines anywhere until a left is visited - making it a "readable" and "editable" file.
 

graudeejs

Son of Beastie

Reaction score: 698
Messages: 4,617

debguy said:
OOPS - website striped the carriage returns and tabs.
Rights can be split across lines anywhere until a left is visited - making it a "readable" and "editable" file.
Ups... you didn't use the
Code:
 tags
 

graudeejs

Son of Beastie

Reaction score: 698
Messages: 4,617

graudeejs said:
I had enough of updating vim plugins and help tags manually, so I wrote 2 scripts:

This script updates my plugins by downloading tar.gz from github:
update_plugins.sh modified and renamed: https://github.com/graudeejs/dot.vim/blob/master/github_plugin_update.sh

And this script updates all help tags for all doc directories in ~/.vim/
https://github.com/graudeejs/dot.vim/blob/master/update_tags.sh


EDIT, I updated scripts.... so now there's also Makefile:
https://github.com/graudeejs/dot.vim/blob/master/Makefile

enjoy!
Updated and renamed script to update_plugin
Now it also supports gitorious.
 

abefar

Member

Reaction score: 7
Messages: 29

System updating script

I wrote a small script for my own use for making the daily update checking process (using portmaster) on my boxes easier. It can also check jails for updates using ezjail.
The script has two modes of operation: ports (portsnap -> portmaster) and source (csup).

I am pretty much a bash newbie, but I hope that someone will find it useful anyway. :)

Code:
#!/usr/local/bin/bash
#

################
### SETTINGS ###
################

# FreeBSD branch to check out - change to match
# installed version (or newer if upgrading)
SUPBRANCH="RELENG_9_0"

# Which CVSup mirror to use
SUPHOST="cvsup.dk.freebsd.org"

# Use compression when checking out sources?
SUPCOMP=1

# Check jails for updates, too?
CHECK_JAILS=1

# If yes, list of jails to check
JAILS=('jail1' 'jail2' 'jail3')

# Editor when PAGER variable is not set
DEFAULTPAGER="less"

# Name of the script (auto-detected)
SCRIPTNAME=$(basename $0)

# Path for system updating supfile (recreated every time)
SUPFILE=/tmp/$SCRIPTNAME-supfile

# Miscellaneous paths
# (leave at defaults)
EZJAIL_BIN="/usr/local/bin/ezjail-admin"
PORTSNAP_BIN="/usr/sbin/portsnap"
PORTMASTER_BIN="/usr/local/sbin/portmaster"
PORTAUDIT_BIN="/usr/local/sbin/portaudit"
JAILS_DIR="/usr/jails"
PORTS_DIR="/usr/ports"

############
### CODE ###
############

if [ `id -u` -ne 0 ]; then
	echo "Please run this script as root. Exiting."
	exit 1
fi

if [ ! -x $PORTSNAP_BIN ]; then
	echo "Portsnap not installed. Exiting."
	exit 1
fi

if [ ! -x $PORTMASTER_BIN ]; then
	echo "Portmaster not installed. Exiting."
	exit 1
fi

if [ ! -x $EZJAIL_BIN ]; then
	echo "Ezjail not installed. Skipping jails."
	echo
	CHECK_JAILS=0
fi

case $1 in
	ports)
		if [ -x $PORTAUDIT_BIN ]; then
			read -p "Check for vulnerabilities? [N/y] " yn
			if [ "$yn" = "Y" -o "$yn" = "y" ]; then
				echo -n "Fetching latest vulnerabilities..."
				$PORTAUDIT_BIN -Fq
				echo "done"
				echo
				echo -n "Checking host system: "
				/usr/sbin/pkg_info | awk '{ print $1 }' | xargs $PORTAUDIT_BIN
				if [ $CHECK_JAILS == 1 -a ${JAILS[0]} ]; then
					for jail in ${JAILS[@]}; do
						echo -n "Checking '$jail' jail: "
						$EZJAIL_BIN console -e '/usr/sbin/pkg_info' $jail | awk '{ print $1 }' | xargs $PORTAUDIT_BIN
					done
				fi
			fi
		else
			echo
			echo "Portaudit not installed. Skipping vulnerability checks."
		fi
		echo

		read -p "Update ports tree? [Y/n] " yn
		if [ "$yn" != "N" -a "$yn" != "n" ]; then
			$PORTSNAP_BIN fetch update
		fi
		echo

		read -p "Check host system for updates? [Y/n] " yn
		if [ "$yn" != "N" -a "$yn" != "n" ]; then
			$PORTMASTER_BIN -L
		fi

		if [ $CHECK_JAILS == 1 -a ${JAILS[0]} ]; then
		for jail in ${JAILS[@]}; do
			echo
			if [ -x $JAILS_DIR/$jail$PORTMASTER_BIN ]; then
				read -p "Check '$jail' for updates? [Y/n] " yn
				if [ "$yn" != "N" -a "$yn" != "n" ]; then
					$EZJAIL_BIN console -e '$PORTMASTER_BIN -L' $jail
				fi
			else
				echo "Portmaster not installed on jail '$jail'. Skipping."
			fi
		done
		echo
		fi

		read -p "Read /usr/ports/UPDATING? [Y/n] " yn
		if [ "$yn" != "N" -a "$yn" != "n" ]; then
			${PAGER:=$DEFAULTPAGER} /usr/ports/UPDATING
		fi
		echo
	;;

	sys)
		read -p "Update system sources? [Y/n] " yn
		if [ "$yn" != "N" -a "$yn" != "n" ]; then
			echo -n "Creating a supfile for $SUPBRANCH at $SUPFILE..."
			echo "*default host=$SUPHOST" > $SUPFILE
			echo "*default base=/var/db" >> $SUPFILE
			echo "*default prefix=/usr" >> $SUPFILE
			echo "*default release=cvs tag=$SUPBRANCH" >> $SUPFILE
			echo "*default delete use-rel-suffix" >> $SUPFILE
			if [ $SUPCOMP -eq 1 ]; then
				echo "*default compress" >> $SUPFILE
			fi
			echo "src-all" >> $SUPFILE
			echo "done"
			/usr/bin/csup -L 1 $SUPFILE
		fi
		echo

		read -p "Read /usr/src/UPDATING? [Y/n] " yn
		if [ "$yn" != "N" -a "$yn" != "n" ]; then
			${PAGER:=$DEFAULTPAGER} /usr/src/UPDATING
		fi
		echo
	;;

	*)
		echo "USAGE: $SCRIPTNAME [ports | sys]"
	;;
esac
 

UNIXgod

Daemon

Reaction score: 200
Messages: 1,088

abefar said:
I wrote a small script for my own use for making the daily update checking process (using Portmaster) on my boxes easier. It can also check jails for updates using ezjail.
The script has two modes of operation: ports (portsnap -> portmaster) and src (csup).

I am pretty much a Bash newbie, but I hope that someone will find it useful anyway. :)

Code:
#!/usr/local/bin/bash
Your script looks good. In fact it looks like posix so you could use #!/bin/sh. As a pet peeve of mine I prefer any external applications to use env() for portability (i.e. #!/usr/bin/env bash) This would be the same for ruby, perl etc. Though your script is FreeBSD specific I guess it really doesn't matter.

If you'd like to transverse the jails running you can use jls() and create a list from there to run your commands with jexec(). Here is an example to get the current running jails for use with jexec:

Code:
jls | tail -n +2 | awk '{print $1}'
If you need the names you can change the awk print statement to $3 and ip addresses are $2.

Here is an old script I used for jails to generate a report on out of date packages with pkg_version() and/or portversion if installed. Maybe you might find it useful for your jails supervision scripts as it has no actual dependencies on ports.

https://forums.freebsd.org/showpost.php?p=139538&postcount=123
 

graudeejs

Son of Beastie

Reaction score: 698
Messages: 4,617

I'm using x11/slim as my login manager.

Today I had this idea:
Why not to save last used WM and use that as default next time I log in
Slim doesn't support this. But it doesn't have to. Everything you need is to adjust you ~/.xinitrc

The magic code snippet
Code:
DEFAULT_WM="fvwm"
if [ "$1" ]; then
    export WM="$1"
else
    export WM="$DEFAULT_WM"
    [ -f "$HOME/.last_wm" ] && WM="`cat "$HOME/.last_wm"`"
fi

echo "$WM" > "$HOME/.last_wm"

# now exec your WM
Here's my full ~/.xinitrc
https://github.com/graudeejs/dot.files/blob/master/dot.xinitrc
 

vermaden

Son of Beastie

Reaction score: 1,332
Messages: 2,907

change national characters to ASCII equivalents

If You move files from one filesystem to another, especially to FAT32/NTFS and then back to UNIX filesystems, then You probably had encoding issues because of 'national' characters, which means unable to locate the file while You know its there or unable to remove it.

The solution to this is to rename/keep all files in ASCII names, so wherever they go, they are 'safe'.

Its 'configurable' by changing the ASCII='...' line, the default is:
ASCII="echo \"${FILE}\" ${TR} ${Tr} ${Or} ${OdQ}"

Which also translates various variants of '-' to ASCII one, ™ to TM, ® to R and removes ! character.

It also can be 'configured' to remove brackets, both (/) and [/] and {/}.

Below is the script that I use to make such ASCII 'normalizations', and an example.

Code:
% [B][color="Blue"]ls [/color][/B]
ÅÆ?ĄÇĆČĎÐÉÈÊËĘĚƐƏŒĒÍÎÏÌĸĹĽŁÑŇŃŊÔÖŐÒØÓÕŔŘ?ΒßŜŞŠŚŤÙÚÛÜŮŰŸÝŽŻŹ,—–™®![](){}
àáâäăãåæɐąçćčďðéèêëęěɛəœēíîïìĸĺľłñňńŋôöőòøóõŕřßβſŝşšśťùúûüůűÿýžżźÀÁÂÄĂÃ
% [B][color="blue"]toascii.sh *[/color][/B][color="Gray"] (without brackets removal)[/color]
% [B][color="blue"]ls[/color][/B]
AAAACCCDDEEEEEEEEEEIIIIKLLLNNNNOOOOOOORRSSSSSSSTUUUUUUYYZZZ.--TMR[](){}
aaaaaaaaaacccddeeeeeeeeeeiiiiKlllnnnnooooooorrSsssssstuuuuuuyyzzzAAAAAA
% [B][color="blue"]toascii.sh *[/color][/B] [color="gray"](with brackets removal)[/color]
% [B][color="blue"]ls[/color][/B]      
AAAACCCDDEEEEEEEEEEIIIIKLLLNNNNOOOOOOORRSSSSSSSTUUUUUUYYZZZ.--TMR
aaaaaaaaaacccddeeeeeeeeeeiiiiKlllnnnnooooooorrSsssssstuuuuuuyyzzzAAAAAA
toascii.sh
Code:
#! /bin/sh

[ ${#} -eq 0 ] && {
  echo "usage: ${0##*/} FILE|DIRECTORY"
  exit 0
}

a0=" | tr 'à' 'a' "; A0=" | tr 'À' 'A' ";
a1=" | tr 'á' 'a' "; A1=" | tr 'Á' 'A' ";
a2=" | tr 'â' 'a' "; A2=" | tr 'Â' 'A' ";
a3=" | tr 'ä' 'a' "; A3=" | tr 'Ä' 'A' ";
a4=" | tr 'ă' 'a' "; A4=" | tr 'Ă' 'A' ";
a5=" | tr 'ã' 'a' "; A5=" | tr 'Ã' 'A' ";
a6=" | tr 'Ã¥' 'a' "; A6=" | tr 'Ã…' 'A' ";
a7=" | tr 'æ' 'a' "; A7=" | tr 'Æ' 'A' ";
a8=" | tr 'ɐ' 'a' "; A8=" | tr 'Ɐ' 'A' ";
a9=" | tr 'Ä…' 'a' "; A9=" | tr 'Ä„' 'A' ";
c0=" | tr 'ç' 'c' "; C0=" | tr 'Ç' 'C' ";
c1=" | tr 'ć' 'c' "; C1=" | tr 'Ć' 'C' ";
c2=" | tr 'č' 'c' "; C2=" | tr 'Č' 'C' ";
d0=" | tr 'ď' 'd' "; D0=" | tr 'Ď' 'D' ";
d1=" | tr 'ð' 'd' "; D1=" | tr 'Ð' 'D' ";
e0=" | tr 'é' 'e' "; E0=" | tr 'É' 'E' ";
e1=" | tr 'è' 'e' "; E1=" | tr 'È' 'E' ";
e2=" | tr 'ê' 'e' "; E2=" | tr 'Ê' 'E' ";
e3=" | tr 'ë' 'e' "; E3=" | tr 'Ë' 'E' ";
e4=" | tr 'ę' 'e' "; E4=" | tr 'Ę' 'E' ";
e5=" | tr 'Ä›' 'e' "; E5=" | tr 'Äš' 'E' ";
e6=" | tr 'ɛ' 'e' "; E6=" | tr 'Ɛ' 'E' ";
e7=" | tr 'ə' 'e' "; E7=" | tr 'Ə' 'E' ";
e8=" | tr 'Å“' 'e' "; E8=" | tr 'Å’' 'E' ";
e9=" | tr 'Ä“' 'e' "; E9=" | tr 'Ä’' 'E' ";
i0=" | tr 'í' 'i' "; I0=" | tr 'Í' 'I' ";
i1=" | tr 'î' 'i' "; I1=" | tr 'Î' 'I' ";
i2=" | tr 'ï' 'i' "; I2=" | tr 'Ï' 'I' ";
i3=" | tr 'ì' 'i' "; I3=" | tr 'Ì' 'I' ";
k0=" | tr 'ĸ' 'k' "; K0=" | tr 'ĸ' 'K' ";
l0=" | tr 'ĺ' 'l' "; L0=" | tr 'Ĺ' 'L' ";
l1=" | tr 'ľ' 'l' "; L1=" | tr 'Ľ' 'L' ";
l2=" | tr 'ł' 'l' "; L2=" | tr 'Ł' 'L' ";
n0=" | tr 'ñ' 'n' "; N0=" | tr 'Ñ' 'N' ";
n1=" | tr 'ň' 'n' "; N1=" | tr 'Ň' 'N' ";
n2=" | tr 'ń' 'n' "; N2=" | tr 'Ń' 'N' ";
n3=" | tr 'Å‹' 'n' "; N3=" | tr 'ÅŠ' 'N' ";
o0=" | tr 'ô' 'o' "; O0=" | tr 'Ô' 'O' ";
o1=" | tr 'ö' 'o' "; O1=" | tr 'Ö' 'O' ";
o2=" | tr 'ő' 'o' "; O2=" | tr 'Ő' 'O' ";
o3=" | tr 'ò' 'o' "; O3=" | tr 'Ò' 'O' ";
o4=" | tr 'ø' 'o' "; O4=" | tr 'Ø' 'O' ";
o5=" | tr 'ó' 'o' "; O5=" | tr 'Ó' 'O' ";
o6=" | tr 'õ' 'o' "; O6=" | tr 'Õ' 'O' ";
r0=" | tr 'Å•' 'r' "; R0=" | tr 'Å”' 'R' ";
r1=" | tr 'ř' 'r' "; R1=" | tr 'Ř' 'R' ";
s0=" | tr 'ß' 's' "; S0=" | tr 'ẞ' 'S' ";
s1=" | tr 'β' 's' "; S1=" | tr 'Β' 'S' ";
s2=" | tr 'ſ' 's' "; S2=" | tr 'ß' 'S' ";
s3=" | tr 'ŝ' 's' "; S3=" | tr 'Ŝ' 'S' ";
s4=" | tr 'ÅŸ' 's' "; S4=" | tr 'Åž' 'S' ";
s5=" | tr 'Å¡' 's' "; S5=" | tr 'Å ' 'S' ";
s6=" | tr 'Å›' 's' "; S6=" | tr 'Åš' 'S' ";
t0=" | tr 'ť' 't' "; T0=" | tr 'Ť' 'T' ";
u0=" | tr 'ù' 'u' "; U0=" | tr 'Ù' 'U' ";
u1=" | tr 'ú' 'u' "; U1=" | tr 'Ú' 'U' ";
u2=" | tr 'û' 'u' "; U2=" | tr 'Û' 'U' ";
u3=" | tr 'ü' 'u' "; U3=" | tr 'Ü' 'U' ";
u4=" | tr 'ů' 'u' "; U4=" | tr 'Ů' 'U' ";
u5=" | tr 'ű' 'u' "; U5=" | tr 'Ű' 'U' ";
y0=" | tr 'ÿ' 'y' "; Y0=" | tr 'Ÿ' 'Y' ";
y1=" | tr 'ý' 'y' "; Y1=" | tr 'Ý' 'Y' ";
z0=" | tr 'ž' 'z' "; Z0=" | tr 'Ž' 'Z' ";
z1=" | tr 'ż' 'z' "; Z1=" | tr 'Ż' 'Z' ";
z2=" | tr 'ź' 'z' "; Z2=" | tr 'Ź' 'Z' ";

Oco=" | sed s/,/./g";
Oh0=" | tr '—' '-' ";
Oh1=" | tr '–' '-' ";
Otm=" | sed s/'â„¢'/'TM'/g ";
Ore=" | tr '®' 'R' ";
OdQ=" | tr -d '!' ";
Ob0=" | tr -d '[' "; OB0=" | tr -d ']' ";
Ob1=" | tr -d '(' "; OB1=" | tr -d ')' ";
Ob2=" | tr -d '{' "; OB2=" | tr -d '}' ";

Tr="${Tr} ${a0} ${a1} ${a2} ${a3} ${a4} ${a5} ${a6} ${a7} ${a8} ${a9} ${c0}"
Tr="${Tr} ${c1} ${c2} ${d0} ${d1} ${e0} ${e1} ${e2} ${e3} ${e4} ${e5} ${e6}"
Tr="${Tr} ${e7} ${e8} ${e9} ${i0} ${i1} ${i2} ${i3} ${k0} ${l0} ${l1} ${l2}"
Tr="${Tr} ${n0} ${n1} ${n2} ${n3} ${o0} ${o1} ${o2} ${o3} ${o4} ${o5} ${o6}"
Tr="${Tr} ${r0} ${r1} ${s0} ${s1} ${s2} ${s3} ${s4} ${s5} ${s6} ${t0} ${u0}"
Tr="${Tr} ${u1} ${u2} ${u3} ${u4} ${u5} ${y0} ${y1} ${z0} ${z1} ${z2}"

TR="${TR} ${A0} ${A1} ${A2} ${A3} ${A4} ${A5} ${A6} ${A7} ${A8} ${A9} ${C0}"
TR="${TR} ${C1} ${C2} ${D0} ${D1} ${E0} ${E1} ${E2} ${E3} ${E4} ${E5} ${E6}"
TR="${TR} ${E7} ${E8} ${E9} ${I0} ${I1} ${I2} ${I3} ${K0} ${L0} ${L1} ${L2}"
TR="${TR} ${N0} ${N1} ${N2} ${N3} ${O0} ${O1} ${O2} ${O3} ${O4} ${O5} ${O6}"
TR="${TR} ${R0} ${R1} ${S0} ${S1} ${S2} ${S3} ${S4} ${S5} ${S6} ${T0} ${U0}"
TR="${TR} ${U1} ${U2} ${U3} ${U4} ${U5} ${Y0} ${Y1} ${Z0} ${Z1} ${Z2}"

Os="${Oh0} ${Oh1} ${Otm} ${Ore} ${Oco}"

Od="${Ob0} ${OB0} ${Ob1} ${OB1} ${Ob2} ${OB2} ${OdQ}"

for FILE in "${@}"
do
  ASCII="echo \"${FILE}\" ${TR} ${Tr} ${Os} ${Od}"
  mv "${FILE}" "$( eval ${ASCII} 2> /dev/null )"
done
Any suggestions welcome ;)
 

Dereckson

Member

Reaction score: 16
Messages: 65

Good morning,

I don't like your solution, as it creates more problems than it solves.

Here some thoughts about the way you handle this issue:
  • The solution doesn't solve the problem you announce it solved (FAT32/NTFS -> UNIX filesystem file moves).
  • This method shows disdain to your international users: their glyphs are't allowed on your system.
  • The script doesn't care about proper transliteration. For example, ß should be replaced by ss (like s0=" | tr 'ß' 'ss' ").
  • There is a vocabulary issue, letting know you've some progress to do (or you don't really care) about l10n. For example, a language doesn't match a country, so “national characters” doesn't mean anything. You even need non ASCII characters in English too. Here two filenames I've found Wikimedia Commons' pictures: Production drawing Pink Panther In “Olym-Pinks”, 1980.jpg and Everybody Loves Eric Raymond 2006-01-11, The GPL3 according to….jpg.
  • And the killer argument: there is an encoding well supported by your UNIX filesystems and your OS: this is called UTF-8.
  • If you use UTF-8, you can mount a NTFS filesystem (which internally stores filenames in UTF-16) in UTF-8, and so have directly compatible filenames.

Now, your method would be useful in this case to convert FAT32 filenames.

Code:
#!/bin/sh
ENCODING_SOURCE=iso-8859-1
#To import Windows FAT32, use instead: ENCODING_SOURCE=cp1252
ENCODING_TARGET=utf-8

[ ${#} -eq 0 ] && {
  echo "usage: ${0##*/} FILE|DIRECTORY"
  exit 0
}
for FILE in "${@}"
do
  NEWNAME = `iconv "${FILE} -s -c -f ${ENCODING_SOURCE} -t ${ENCODING_TARGET}//TRANSLIT`
  if [ -z "${NEWNAME}" ]
  then
    echo "Can't rename ${FILE}"
  else
    mv "${FILE}" "${NEWNAME}"
  fi
done
It could also be used to keep characters your encoding should support, like é or ß with ENCODING_SOURCE=cp1252 and ENCODING_TARGET=cp437 (if you use cons25 encoding).

iconv with use transliteration to convert any character not existing in target encoding according state of art admitted rules:
Code:
A origem da povoação de Pereira parece estar ligada às vic….pdf (UTF-8, original)
A origem da povoaâ–’~ao de Pereira parece estar ligada â–’s vic....pdf (cp437, for cons25)
A origem da povoaç~ao de Pereira parece estar ligada às vic....pdf (converted back in UTF-8, this bulletin board encoding)
Note: There are two ports, converters/libiconv and converters/iconv. You probably already have libiconv installed, and probably with this option enabled ([X] EXTRA_ENCODINGS). If you have to install one and choose converter/iconv instead converter/libiconv, don't forget to install also converter/iconv-extra port and to note your binary will be "biconv" to avoid conflicts.
 

vermaden

Son of Beastie

Reaction score: 1,332
Messages: 2,907

@Dereckson

Thank You for Your arguments, I will try to address them below.

I don't like your solution, as it creates more problems than it solves.
It solves problems for me, but I do not agree that this is the best way to solve others' people encoding problems.

[*] The solution doesn't solve the problem you announce it solved (FAT32/NTFS -> UNIX filesystem file moves).
I use it to rename files that are still 'valid', ofter when copying from FAT32/NTFS with different encoding then neede mounted, files will copy with 'bad' characters. I was thinking generally about issues with transferring files between FAT/NTFS and UNIX filesystems, not directly about FAT/NTFS -> UNIX migration.

This method shows disdain to your international users: their glyphs are't allowed on your system.
I personally disdain MY national language/characters and think that they are harm/dangerous in filenames and also change them. I haven't thought about changing OTHER people filenames with that, only mine.

The script doesn't care about proper transliteration. For example, ß should be replaced by ss (like s0=" | tr 'ß' 'ss' ").
I have read about 'ß' in Wikipedia, but, for simplicity I used only one 's', the script is editable and You can make it to change 'ß' to 'ss' instead of 's'. By definition/language rules, 'ß' is a 'ss' representation in one character, but I do not care (yes, its my ignorance) how *strong* is this 's', its still an 's'.

By the way, to make it work, You will have to change it like that (tr will not work):
Code:
[color="Red"]-s0=" | tr 'ß' 'ss' "[/color]
[color="Lime"]+s0=" | sed s/'ß'/'ss'/g "[/color]
[*] There is a vocabulary issue, letting know you've some progress to do (or you don't really care) about l10n. For example, a language doesn't match a country, so “national characters” doesn't mean anything. You even need non ASCII characters in English too.
My knowledge about l10n (and i18n) is limited, I did not dig that topic, I just wanted to solve the problem, and as tiem has passed I first created that script only with characters from my native language, then added all other possible characters, that will make my life easier.

And the killer argument: there is an encoding well supported by your UNIX filesystems and your OS: this is called UTF-8.
Great, but does FreeBSD console is fully UTF-8? Nope. Is everything in computing UTF-8 compatible? Nope. Until then I will translate 'national' [*] characters to english harmless ones.

[*] yes I remember that You mentioned that this naming convention is not valid, but its what best describes that issue for me

Now, your method would be useful in this case to convert FAT32 filenames.

Code:
#!/bin/sh
ENCODING_SOURCE=iso-8859-1
#To import Windows FAT32, use instead: ENCODING_SOURCE=cp1252
ENCODING_TARGET=utf-8

[ ${#} -eq 0 ] && {
  echo "usage: ${0##*/} FILE|DIRECTORY"
  exit 0
}
for FILE in "${@}"
do
  NEWNAME = `iconv "${FILE} -s -c -f ${ENCODING_SOURCE} -t ${ENCODING_TARGET}//TRANSLIT`
  if [ -z "${NEWNAME}" ]
  then
    echo "Can't rename ${FILE}"
  else
    mv "${FILE}" "${NEWNAME}"
  fi
done
I will look into that, thanks.

It could also be used to keep characters your encoding should support.
Maybe it will for encodings that I use, but the I would take that drive to the system that does not support these characters and the problem begins again.

Will it rename the filenames as my script does (removing all national characters?), if yes, then maybe my scripts is useless and there is far better tool to do that?

Regards,
vermaden
 

vermaden

Son of Beastie

Reaction score: 1,332
Messages: 2,907

@Dereckson

I just tried using iconv, it translates with 'additional characters, which is unacceptable:

Code:
% echo ẃŵẅÿýỳŷžżź | iconv -c -f utf-8 -t iso-8859-1//TRANSLIT 
�w^w"w��`y^yzz�z
While my toascii.sh script does the job and translates 'ẃŵẅÿýỳŷžżź' into 'wwwyyyyzzz'.

Here is the latest version, with more characters:

toascii.sh
Code:
#! /bin/sh

[ ${#} -eq 0 ] && {
  echo "usage: ${0##*/} FILE|DIRECTORY"
  exit 0
}

a0=" | tr 'à' 'a' "; A0=" | tr 'À' 'A' ";
a1=" | tr 'á' 'a' "; A1=" | tr 'Á' 'A' ";
a2=" | tr 'â' 'a' "; A2=" | tr 'Â' 'A' ";
a3=" | tr 'ä' 'a' "; A3=" | tr 'Ä' 'A' ";
a4=" | tr 'ă' 'a' "; A4=" | tr 'Ă' 'A' ";
a5=" | tr 'ã' 'a' "; A5=" | tr 'Ã' 'A' ";
a6=" | tr 'Ã¥' 'a' "; A6=" | tr 'Ã…' 'A' ";
a7=" | sed s/'æ'/'ae'/g "; A7=" | sed s/'Æ'/'AE'/g ";
a8=" | tr 'ɐ' 'a' "; A8=" | tr 'Ɐ' 'A' ";
a9=" | tr 'Ä…' 'a' "; A9=" | tr 'Ä„' 'A' ";
aA=" | tr 'ā' 'a' "; AA=" | tr 'Ā' 'A' ";
aB=" | tr 'ă' 'a' "; AB=" | tr 'Ă' 'A' ";
aC=" | tr 'ÇŸ' 'a' "; AC=" | tr 'Çž' 'A' ";
aD=" | sed s/'ǽ'/'ae'/g "; AD=" | sed s/'Ǽ'/'AE'/g ";
b0=" | tr 'ḃ' 'b' "; B0=" | tr 'Ḃ' 'B' ";
c0=" | tr 'ç' 'c' "; C0=" | tr 'Ç' 'C' ";
c1=" | tr 'ć' 'c' "; C1=" | tr 'Ć' 'C' ";
c2=" | tr 'č' 'c' "; C2=" | tr 'Č' 'C' ";
c3=" | tr 'Ä‹' 'c' "; C3=" | tr 'ÄŠ' 'C' ";
d0=" | tr 'ď' 'd' "; D0=" | tr 'Ď' 'D' ";
d1=" | tr 'ð' 'd' "; D1=" | tr 'Ð' 'D' ";
d2=" | tr 'ḑ' 'd' "; D2=" | tr 'Ḑ' 'D' ";
d3=" | tr 'ď' 'd' "; D3=" | tr 'Ď' 'D' ";
d4=" | tr 'ḋ' 'd' "; D4=" | tr 'Ḋ' 'D' ";
d5=" | tr 'đ' 'd' "; D5=" | tr 'Đ' 'D' ";
d6=" | tr 'ð' 'd' "; D6=" | tr 'Ð' 'D' ";
d7=" | sed s/'dž'/'dz'/g "; D5=" | sed s/'DŽ'/'DZ'/g ";
d8=" | sed s/'dz'/'dz'/g "; D6=" | sed s/'DZ'/'DZ'/g ";
e0=" | tr 'é' 'e' "; E0=" | tr 'É' 'E' ";
e1=" | tr 'è' 'e' "; E1=" | tr 'È' 'E' ";
e2=" | tr 'ê' 'e' "; E2=" | tr 'Ê' 'E' ";
e3=" | tr 'ë' 'e' "; E3=" | tr 'Ë' 'E' ";
e4=" | tr 'ę' 'e' "; E4=" | tr 'Ę' 'E' ";
e5=" | tr 'Ä›' 'e' "; E5=" | tr 'Äš' 'E' ";
e6=" | tr 'ɛ' 'e' "; E6=" | tr 'Ɛ' 'E' ";
e7=" | tr 'ə' 'e' "; E7=" | tr 'Ə' 'E' ";
e8=" | sed s/'Å“'/'oe'/g "; E8=" | sed s/'Å’'/'OE'/g ";
e9=" | tr 'Ä“' 'e' "; E9=" | tr 'Ä’' 'E' ";
eA=" | tr 'Ä•' 'e' "; E9=" | tr 'Ä”' 'E' ";
eB=" | tr 'Ä—' 'e' "; E9=" | tr 'Ä–' 'E' ";
e9=" | tr 'Ê’' 'e' "; E9=" | tr 'Æ·' 'E' ";
e9=" | tr 'ǯ' 'e' "; E9=" | tr 'Ǯ' 'E' ";
f0=" | tr 'ḟ' 'f' "; F0=" | tr 'Ḟ' 'F' ";
f1=" | tr 'Æ’' 'f' "; F1=" | tr 'Æ‘' 'F' ";
g0=" | tr 'ǵ' 'g' "; G0=" | tr 'Ǵ' 'G' ";
g1=" | tr 'Ä£' 'g' "; G1=" | tr 'Ä¢' 'G' ";
g2=" | tr 'ǧ' 'g' "; G2=" | tr 'Ǧ' 'G' ";
g3=" | tr 'ĝ' 'g' "; G3=" | tr 'Ĝ' 'G' ";
g4=" | tr 'ÄŸ' 'g' "; G4=" | tr 'Äž' 'G' ";
g5=" | tr 'Ä¡' 'g' "; G5=" | tr 'Ä ' 'G' ";
g6=" | tr 'ǥ' 'g' "; G6=" | tr 'Ǥ' 'G' ";
h0=" | tr 'ĥ' 'h' "; H0=" | tr 'Ĥ' 'H' ";
h1=" | tr 'ħ' 'h' "; H1=" | tr 'Ħ' 'H' ";
i0=" | tr 'í' 'i' "; I0=" | tr 'Í' 'I' ";
i1=" | tr 'î' 'i' "; I1=" | tr 'Î' 'I' ";
i2=" | tr 'ï' 'i' "; I2=" | tr 'Ï' 'I' ";
i3=" | tr 'ì' 'i' "; I3=" | tr 'Ì' 'I' ";
i4=" | tr 'ĩ' 'i' "; I4=" | tr 'Ĩ' 'I' ";
i5=" | tr 'ī' 'i' "; I5=" | tr 'Ī' 'I' ";
i6=" | tr 'ĭ' 'i' "; I6=" | tr 'Ĭ' 'I' ";
i7=" | tr 'į' 'i' "; I7=" | tr 'Į' 'I' ";
i8=" | tr 'ı' 'i' "; I8=" | tr 'I' 'I' ";
j0=" | tr 'ĵ' 'j' "; J0=" | tr 'Ĵ' 'J' ";
k0=" | tr 'ĸ' 'k' ";
k1=" | tr 'ḱ' 'k' "; K1=" | tr 'Ḱ' 'K' ";
k2=" | tr 'ķ' 'k' "; K2=" | tr 'Ķ' 'K' ";
k3=" | tr 'ǩ' 'k' "; K3=" | tr 'Ǩ' 'K' ";
l0=" | tr 'ĺ' 'l' "; L0=" | tr 'Ĺ' 'L' ";
l1=" | tr 'ľ' 'l' "; L1=" | tr 'Ľ' 'L' ";
l2=" | tr 'ł' 'l' "; L2=" | tr 'Ł' 'L' ";
l3=" | tr 'Å€' 'l' "; L3=" | tr 'Ä¿' 'L' ";
l4=" | tr 'ļ' 'l' "; L4=" | tr 'Ļ' 'L' ";
m0=" | tr 'ṁ' 'm' "; M0=" | tr 'Ṁ' 'M' ";
n0=" | tr 'ñ' 'n' "; N0=" | tr 'Ñ' 'N' ";
n1=" | tr 'ň' 'n' "; N1=" | tr 'Ň' 'N' ";
n2=" | tr 'ń' 'n' "; N2=" | tr 'Ń' 'N' ";
n3=" | tr 'Å‹' 'n' "; N3=" | tr 'ÅŠ' 'N' ";
n4=" | tr 'ņ' 'n' "; N4=" | tr 'Ņ' 'N' ";
n5=" | tr 'ʼn' 'n' ";
o0=" | tr 'ô' 'o' "; O0=" | tr 'Ô' 'O' ";
o1=" | tr 'ö' 'o' "; O1=" | tr 'Ö' 'O' ";
o2=" | tr 'ő' 'o' "; O2=" | tr 'Ő' 'O' ";
o3=" | tr 'ò' 'o' "; O3=" | tr 'Ò' 'O' ";
o4=" | tr 'ø' 'o' "; O4=" | tr 'Ø' 'O' ";
o5=" | tr 'ó' 'o' "; O5=" | tr 'Ó' 'O' ";
o6=" | tr 'õ' 'o' "; O6=" | tr 'Õ' 'O' ";
o7=" | tr 'ō' 'o' "; O7=" | tr 'Ō' 'O' ";
o8=" | tr 'ŏ' 'o' "; O8=" | tr 'Ŏ' 'O' ";
o9=" | tr 'ø' 'o' "; O9=" | tr 'Ø' 'O' ";
oA=" | tr 'ǿ' 'o' "; OA=" | tr 'Ǿ' 'O' ";
p0=" | tr 'á¹—' 'p' "; P0=" | tr 'á¹–' 'P' ";
r0=" | tr 'Å•' 'r' "; R0=" | tr 'Å”' 'R' ";
r1=" | tr 'ř' 'r' "; R1=" | tr 'Ř' 'R' ";
r2=" | tr 'Å—' 'r' "; R2=" | tr 'Å–' 'R' ";
s0=" | sed s/'ß'/'ss'/g ";
s1=" | sed s/'β'/'ss'/g "; S1=" | sed s/'Β'/'SS'/g ";
s2=" | sed s/'ſ'/'ss'/g "; S2=" | sed s/'ß'/'SS'/g ";
s3=" | tr 'ŝ' 's' "; S3=" | tr 'Ŝ' 'S' ";
s4=" | tr 'ÅŸ' 's' "; S4=" | tr 'Åž' 'S' ";
s5=" | tr 'Å¡' 's' "; S5=" | tr 'Å ' 'S' ";
s6=" | tr 'Å›' 's' "; S6=" | tr 'Åš' 'S' ";
s7=" | tr 'ṡ' 's' "; S7=" | tr 'Ṡ' 'S' ";
t0=" | tr 'ť' 't' "; T0=" | tr 'Ť' 'T' ";
t1=" | tr 'Å£' 't' "; T1=" | tr 'Å¢' 'T' ";
t2=" | tr 'ŧ' 't' "; T2=" | tr 'Ŧ' 'T' ";
t3=" | tr 'þ' 't' "; T3=" | tr 'Þ' 'T' ";
u0=" | tr 'ù' 'u' "; U0=" | tr 'Ù' 'U' ";
u1=" | tr 'ú' 'u' "; U1=" | tr 'Ú' 'U' ";
u2=" | tr 'û' 'u' "; U2=" | tr 'Û' 'U' ";
u3=" | tr 'ü' 'u' "; U3=" | tr 'Ü' 'U' ";
u4=" | tr 'ů' 'u' "; U4=" | tr 'Ů' 'U' ";
u5=" | tr 'ű' 'u' "; U5=" | tr 'Ű' 'U' ";
u6=" | tr 'ũ' 'u' "; U6=" | tr 'Ũ' 'U' ";
u7=" | tr 'ū' 'u' "; U7=" | tr 'Ū' 'U' ";
u8=" | tr 'ŭ' 'u' "; U8=" | tr 'Ŭ' 'U' ";
u9=" | tr 'ų' 'u' "; U9=" | tr 'Ų' 'U' ";
w0=" | tr 'ẁ' 'w' "; W0=" | tr 'Ẁ' 'W' ";
w1=" | tr 'ẃ' 'w' "; W1=" | tr 'Ẃ' 'W' ";
w2=" | tr 'ŵ' 'w' "; W2=" | tr 'Ŵ' 'W' ";
w3=" | tr 'ẅ' 'w' "; W3=" | tr 'Ẅ' 'W' ";
y0=" | tr 'ÿ' 'y' "; Y0=" | tr 'Ÿ' 'Y' ";
y1=" | tr 'ý' 'y' "; Y1=" | tr 'Ý' 'Y' ";
y2=" | tr 'ỳ' 'y' "; Y2=" | tr 'Ỳ' 'Y' ";
y3=" | tr 'ŷ' 'y' "; Y3=" | tr 'Ŷ' 'Y' ";
z0=" | tr 'ž' 'z' "; Z0=" | tr 'Ž' 'Z' ";
z1=" | tr 'ż' 'z' "; Z1=" | tr 'Ż' 'Z' ";
z2=" | tr 'ź' 'z' "; Z2=" | tr 'Ź' 'Z' ";

Oco=" | sed s/,/./g";
Oh0=" | tr '—' '-' ";
Oh1=" | tr '–' '-' ";
Otm=" | sed s/'â„¢'/'TM'/g ";
Ore=" | tr '®' 'R' ";
OdQ=" | tr -d '!' ";
Ob0=" | tr -d '[' "; OB0=" | tr -d ']' ";
Ob1=" | tr -d '(' "; OB1=" | tr -d ')' ";
Ob2=" | tr -d '{' "; OB2=" | tr -d '}' ";

Tr="${Tr} ${a0} ${a1} ${a2} ${a3} ${a4} ${a5} ${a6} ${a7} ${a8} ${a9} ${Aa}"
Tr="${Tr} ${Ab} ${Ac} ${Ad} ${b0} ${c0} ${c1} ${c2} ${c3} ${d0} ${d1} ${d2}"
Tr="${Tr} ${d3} ${d4} ${d5} ${d6} ${e0} ${e1} ${e2} ${e3} ${e4} ${e5} ${e6}"
Tr="${Tr} ${e7} ${e8} ${e9} ${eA} ${eB} ${eC} ${eD} ${f0} ${f1} ${g0} ${g1}"
Tr="${Tr} ${g2} ${g3} ${g4} ${g5} ${g6} ${h0} ${h1} ${i0} ${i1} ${i2} ${i3}"
Tr="${Tr} ${i4} ${i5} ${i6} ${i7} ${i8} ${j0} ${k0} ${k1} ${k2} ${k3} ${l0}"
Tr="${Tr} ${l1} ${l2} ${l3} ${l4} ${m0} ${n0} ${n1} ${n2} ${n3} ${n4} ${n5}"
Tr="${Tr} ${o0} ${o1} ${o2} ${o3} ${o4} ${o5} ${o6} ${o7} ${o8} ${o9} ${oA}"
Tr="${Tr} ${p0} ${r0} ${r1} ${r2} ${s0} ${s1} ${s2} ${s3} ${s4} ${s5} ${s6}"
Tr="${Tr} ${s7} ${t0} ${T1} ${T2} ${T3} ${u0} ${u1} ${u2} ${u3} ${u4} ${u5}"
Tr="${Tr} ${u6} ${u7} ${u8} ${u9} ${w0} ${w1} ${w2} ${w3} ${y0} ${y1} ${y2}"
Tr="${Tr} ${y3} ${z0} ${z1} ${z2}"

TR="${TR} ${A0} ${A1} ${A2} ${A3} ${A4} ${A5} ${A6} ${A7} ${A8} ${A9} ${AA}"
TR="${TR} ${AB} ${AC} ${AD} ${B0} ${C0} ${C1} ${C2} ${C3} ${D0} ${D1} ${D2}"
TR="${TR} ${D3} ${D4} ${D5} ${D6} ${E0} ${E1} ${E2} ${E3} ${E4} ${E5} ${E6}"
TR="${TR} ${E7} ${E8} ${E9} ${EA} ${EB} ${EC} ${ED} ${F0} ${F1} ${G0} ${G1}"
TR="${TR} ${G2} ${G3} ${G4} ${G5} ${G6} ${H0} ${H1} ${I0} ${I1} ${I2} ${I3}"
TR="${TR} ${I4} ${I5} ${I6} ${I7} ${I8} ${J0} ${K1} ${K2} ${K3} ${L0} ${L1}"
TR="${TR} ${L2} ${L3} ${L4} ${M0} ${N0} ${N1} ${N2} ${N3} ${N4} ${O0} ${O1}"
TR="${TR} ${O2} ${O3} ${O4} ${O5} ${O6} ${O7} ${O8} ${O9} ${OA} ${P0} ${R0}"
TR="${TR} ${R1} ${R2} ${S1} ${S2} ${S3} ${S4} ${S5} ${S6} ${S7} ${T0} ${T1}"
TR="${TR} ${T2} ${T3} ${U0} ${U1} ${U2} ${U3} ${U4} ${U5} ${U6} ${U7} ${U8}"
TR="${TR} ${U9} ${W0} ${W1} ${W2} ${W3} ${Y0} ${Y1} ${Y2} ${Y3} ${Z0} ${Z1}"
TR="${TR} ${Z2}"

Os="${Oh0} ${Oh1} ${Otm} ${Ore} ${Oco}"

Od="${Ob0} ${OB0} ${Ob1} ${OB1} ${Ob2} ${OB2} ${OdQ}"

for FILE in "${@}"
do
  ASCII="echo \"${FILE}\" ${TR} ${Tr} ${Os} ${Od}"
  mv "${FILE}" "$( eval ${ASCII} 2> /dev/null )"
done
 

mwatkins

Active Member

Reaction score: 14
Messages: 120

Perhaps consider a script in Python? Hooking in the file renaming is easy enough, here is the character normalization using the "normal form KD" decomposition/composition:

Code:
import unicodedata

def normalize(string):
    return unicodedata.normalize('NFKD', input_string).encode('ascii', 'ignore')

print(normalize("ẃŵẅÿýỳŷžżź"))
Produces:
Code:
wwwyyyyzzz
There is a little more to it than this; UTF-8 encoded strings read from the console or file system must be decoded as Unicode bytestrings, so the code will grow by a few more characters here and there. I'm posting this to show the unicodedata module in Python has done a lot of legwork needed for tasks like this. I guess I'll need to round it out and provide a complete script to be in keeping with this thread. Stay tuned, will be back.
 

vermaden

Son of Beastie

Reaction score: 1,332
Messages: 2,907

@mwatkins

Thanks for that great tip, I am gonna dig into that python normalization right away! ;)

EDIT:

I tried something like that:

Code:
#! /usr/bin/env python
# -*- coding: utf-8 -*

import unicodedata

def normalize(input_string):
    return unicodedata.normalize('NFKD', input_string).encode('ascii', 'ignore')

print normalize("àÀáÁâÂäÄăĂãÃåÅɐⱯąĄāĀăĂǟǞḃḂçÇćĆčČċĊďĎðÐḑḐďĎḋḊđĐðÐéÉèÈêÊëËęĘěĚɛƐəƏēĒĕĔėĖʒƷǯǮḟḞƒƑǵǴģĢǧǦĝĜğĞġĠǥǤĥĤħĦíÍîÎïÏìÌĩĨīĪĭĬįĮıIĵĴĸḱḰķĶǩǨĺĹľĽłŁŀĿļĻṁṀñÑňŇńŃŋŊņŅʼnôÔöÖőŐòÒøØóÓõÕōŌŏŎøØǿǾṗṖŕŔřŘŗŖŝŜşŞšŠśŚṡṠťŤţŢŧŦþÞùÙúÚûÛüÜůŮűŰũŨūŪŭŬųŲẁẀẃẂŵŴẅẄÿŸýÝỳỲŷŶžŽżŻźŹæÆǽǼdžDŽdzDZœŒßβΒſß™®![](){}")
... but:

Code:
% [B][color="Blue"]python python-toascii.sh[/color][/B]
Traceback (most recent call last):
  File "python-toascii.sh", line 9, in <module>
    print normalize("àÀáÁâÂäÄăĂãÃåÅɐⱯąĄāĀăĂǟǞḃḂçÇćĆčČċĊďĎðÐḑḐďĎḋḊđĐðÐéÉèÈêÊëËęĘěĚɛƐəƏēĒĕĔėĖʒƷǯǮḟḞƒƑǵǴģĢǧǦĝĜğĞġĠǥǤĥĤħĦíÍîÎïÏìÌĩĨīĪĭĮıIĵĴĸḱḰķĶǩǨĺĹľĽłŁŀĿļĻṁṀñÑňŇńŃŋŊņŅʼnôÔöÖőŐòÒøØóÓõÕōŌŏŎøØǿǾṗṖŕŔřŘŗŖŝŜşŞšŠśŚṡṠťŤţŢŧŦþÞùÙúÚûÛüÜůŮűŰũŨūŪŭŬųŲẁẀẃẂŵŴẅẄÿŸýÝỳỲŷŶžŽżŻźŹæÆǽǼdžDŽdzDZœŒßβΒſß™®![](){}")
  File "python-toascii.sh", line 7, in normalize
    return unicodedata.normalize('NFKD', input_string).encode('ascii', 'ignore')
TypeError: must be unicode, not str
 

kpa

Beastie's Twin

Reaction score: 1,822
Messages: 6,318

You have to tell python that the string literal is unicode:
Code:
    print normalize(u"àÀáÁâÂäÄă...
On python3 there is no such problem because all strings are unicode. Another reason to prefer python 3 over 2 :)
 

kpa

Beastie's Twin

Reaction score: 1,822
Messages: 6,318

Simple one-liner for PKGNG. This prints out installed packages sorted in order of their installation times. Output is installation time formatted with current locale and the package name.

$ pkg query "%t %n-%v" | sort -n -k1 | while read timestamp pkgname; do echo "$(date -r $timestamp) $pkgname"; done


This works only in bourne type shells.
 

SNK

Active Member

Reaction score: 31
Messages: 128

Code:
#!/bin/sh

if [ $# -eq 1 ] && [ $1 = "cron" ]
    then /bin/sleep $(/usr/bin/jot -r 1 0 3600)
fi

echo "  ~  1  ~   Updating the ports tree..."
/usr/sbin/portsnap fetch update || exit 1

echo "  ~  2  ~   Checking for missing dependencies..."
/usr/sbin/pkg check -d || exit 1

echo "  ~  3  ~   Checking for vulnerabilities..."
/usr/sbin/pkg audit -F

echo "  ~  4  ~   Cleaning old packages..."
/usr/sbin/pkg clean || exit 1

echo "  ~  5  ~   What to upgrade..."
/usr/sbin/pkg version -IvL \= || exit 1

echo "  ~  6  ~   What is available in the repositories..."
/usr/sbin/pkg update || exit 1
/usr/sbin/pkg upgrade -Lqn || exit 1

echo "  ~  7  ~   Messages from UPDATING..."
weekago=$( /bin/date -v-1w "+%Y%m%d" )
lastpkg=$( /bin/date -r $( /usr/sbin/pkg query "%t" | /usr/bin/sort -n | /usr/bin/tail -n1 ) "+%Y%m%d" )

if [ $weekago -lt $lastpkg ]
    then usedate=$weekago
    else usedate=$lastpkg
fi

/usr/sbin/pkg updating -d $usedate
Hereby a small update of DutchDaemon's script. This one uses the new pkg(8) tools.
 

johnd

Member

Reaction score: 14
Messages: 53

When it comes to programming/scripting I´m a pretty newbie. But I try ;)
Here is a script that I´m using occasionally:

Code:
#!/bin/sh -e

# mfetch - a wrapper to fetch(1)
# it mimics wget --input-file behaviour
# argument $1: inputlist
# optional argument $2: directory

unset PATH

# Handling traps
trap 'echo "Aborting on signal." >&2; exit 1;' INT TERM

# Checking arguments
[ $# -lt 1 ] && { echo "Usage: ${0##*/} inputlist [directory]" >&2; exit 1; }

# Checking / Creating directories as needed
[ ! -d $2 ] && /bin/mkdir -p $2

# Feed fetch with URLs from inputlist
while read i; do
    /usr/bin/fetch -q -o ${2:-./} $i
done <$1

exit 0
 

lockdoc

Active Member

Reaction score: 6
Messages: 124

Hi,
here is a nice script to show the temperature of all attached hard disks


[1]

It currently works with ata disks.
I have tried to include disks connected to a ciss raid controller, but I am not sure if it works as I do not have one. Could anyone with a ciss controller please check this out and give some feedback.

The source of the script is here
https://github.com/lockdoc/hddtemp/blob/master/hddtemp.sh

It requires root privileges and smartmontools to be installed.

[1] My SSD does not report the proper temperature, that's why it always displays 128
 

kpedersen

Daemon

Reaction score: 850
Messages: 1,849

The following script disables the fan if the temp is lower than 45 and then enables it again at 55.

I know this isn't a brilliant idea but for some reason my Thinkpad x61 keeps its fan running trying to reach a very low temperature. It becomes quite cold for my wrist and quite uncomfortable for my lap ;)

This will only work when using the acpi_ibm module.

Code:
#!/bin/sh

disable_fan()
{
  echo "disable"
  sysctl dev.acpi_ibm.0.fan=0
}

enable_fan()
{
  echo "enable"
  sysctl dev.acpi_ibm.0.fan=1
}

try_disable_fan()
{
  if [ `sysctl -n dev.acpi_ibm.0.fan` = 1 ]; then
    disable_fan
  fi
}

try_enable_fan()
{
  if [ `sysctl -n dev.acpi_ibm.0.fan` = 0 ]; then
    enable_fan
  fi
}

if [ `id -u` != 0 ]; then
  echo "Error: Must be root"
  exit 1
fi

while [ true ]; do

  HIGHEST_TEMP=0
  TEMPERATURES=`sysctl -n dev.acpi_ibm.0.thermal`

  for TEMP in $TEMPERATURES; do
    if [ $TEMP -gt $HIGHEST_TEMP ]; then
      HIGHEST_TEMP=$TEMP
    fi
  done

  if [ $HIGHEST_TEMP -gt 55 ]; then
    try_enable_fan
  elif [ $HIGHEST_TEMP -lt 45 ]; then
    try_disable_fan
  fi

  sleep 5

done
 
Top