Useful scripts

I see an AI post on a forums as being the same etiquette as posting a google link. I.e:

https://www.google.com/search?q=scripts

But sometimes (admittedly fairly rarely) that is valid. If its a complex or non-intuitive search, likewise if it is a complex query and it took a while to stage the issue within the LLM context.

People here (myself included) got frustrated by one guy spamming a bunch of reddit links. As always everything is fine in moderation.
 
balanga said:
I disagree, AI has helped me out numerous times. It doesn't always give me the correcr solution but often points me in the correct direction.
Click to expand...
You misunderstood.
I did not say, "Don't use AI!"
I said, "Please, stop posting AI generated stuff here in the forums!"
That's quite a difference.
 
Code: 
#!/usr/local/bin/bash
# 0-------------------------------------------------------------------------0
# | make_All_ports_dir                                                      |
# | fill directory /usr/ports/All with symlinks to all portdirs             |
# | (so you can find the nvidia-drivers in category x11 and not x11-drivers |
# 0-------------------------------------------------------------------------0
cd /usr/ports
mkdir -p All
ls | while read cat_name
do
  if [ -d "$cat_name" ]
  then
    cd "$cat_name"
    ls | while read port_name
    do
      if [ -d "$port_name" ]
      then
        echo "port: $cat_name/$port_name"
        ln -fs "/usr/ports/$cat_name/$port_name" "/usr/ports/All/$port_name"
      fi
    done
    cd - > /dev/null
  fi
done
 
Script that transparently wraps www/yt-dlp.

Features:
- Console/terminal compatible.
- Downloads subtitles in all available languages.
- Downloads and embeds thumbnail image.
- Creates an "mkv" file, and separate subtitle file(s) if subtitles are available.
- Prioritises best video quality.
- Can use a configuration file to set defaults, some of which can be overridden from the command line.
- Can specify proxy string for authenticating proxy traversal.
- Can specify individual item URL or playlist (useful for seasons) URL.
- Can specify a file with a list of URLs to download (which can be all playlists or all individual items).

Sample configuration file (~/.config/yt-dlp_wrapper/yt-dlp_wrapper.conf):
Code: 
    format='bv*+ba/b'
    proxyString='http://user:password@proxy:3128'
    urlList='false'
    playList='--no-playlist'
The above configuration file would set format to prioritise highest video quality, set an authenticating proxy and credentials, the source would not be a file of URLs, and the URL would be a single item and not a playlist.

The script itself (for above configuration file example: yt-dlp_wrapper.sh):
Code: 
#!/bin/sh

#-------------------------------------------------------------------------#
# References
#-------------------------------------------------------------------------#
# man sh
# man sysexits
# man test
# read --help
#
# https://stackoverflow.com/questions/592620/how-can-i-check-if-a-program-exists-from-a-bash-script
# - Test if command/programme exists/is installed.
#
# https://www.grymoire.com/Unix/Sh.html#uh-78
# - Iterate lines in file.
# - Which leads to warning: https://www.shellcheck.net/wiki/SC2002 .
#
# https://www.shellcheck.net/wiki/SC2002
# - Finally get "correct" syntax for reading file line by line.
#
# https://unix.stackexchange.com/questions/1763/how-do-i-split-the-0-variable-to-find-directory-and-relative-paths-in-bash
# - Get full path in script invocation ($0).
#
# https://www.grymoire.com/Unix/Sh.html#uh-96
# - Checking for optional arguments.
#
# https://www.grymoire.com/Unix/Sh.html#uh-71
# - $@ versus ${1+$@} .
# **!! First incorrect option terminates processing of command line,
#   what is left becomes parameters.
#-------------------------------------------------------------------------#

#-------------------------------------------------------------------------#
# Contributors
#-------------------------------------------------------------------------#
#-------------------------------------------------------------------------#

#-------------------------------------------------------------------------#
# Dependencies
#-------------------------------------------------------------------------#
# Bourne/POSIX shell, or Bourne/POSIX compatible shell.
# 'yt-dlp' port/package.
#-------------------------------------------------------------------------#

#-------------------------------------------------------------------------#
# Known issues
#-------------------------------------------------------------------------#
# 'yt-dlp' always generates "ERROR: [generic] '' is not a valid URL", but
# continues the download anyway.
#
# Mixing playlist and non playlist URLs in a URL list file may yield
# unpredictable results.
#-------------------------------------------------------------------------#



#-------------------------------------------------------------------------#
# Version history
#-------------------------------------------------------------------------#
version="1.0.0"
# Version 1.0.0:
# - Initial release.
#-------------------------------------------------------------------------#

#-------------------------------------------------------------------------#
# Signal Traps
#-------------------------------------------------------------------------#
cleanup ( )
{
    # cleanup code goes here
    echo "INT or QUIT trap received..."
}

# [Ctrl][c]=SIGINT [Ctrl][\]=SIGQUIT
trap "cleanup" INT QUIT
#-------------------------------------------------------------------------#

#-------------------------------------------------------------------------#
# Functions
#-------------------------------------------------------------------------#
version ( )
{
    printf "%s\nVersion: %s\n" "${scriptNameExt}" "${version}"
    exit 0
}

show_usage ( )
{
    printf "%s\n" "Usage:"
    printf "%s\n" "${scriptNameExt} -v | --version"
    printf "%s\n" "${scriptNameExt} -h | --help"
    printf "%s\n" "${scriptNameExt} [-d|--debug] [--proxy proxy_string | --proxy=proxy_string] [--url_playlist] [--url_list_file] source"
    printf "%s\n" ''
    printf "%s\n" "-v|--version - shows the version and exits"
    printf "%s\n" "-h|--help - shows usage and exits"
    printf "%s\n" "-d|--debug - shows verbose yt-dlp output"
    printf "%s\n" "--url_playlist - indicates source URL (or source URL list file) is a playlist (or file containing playlist URLs)"
    printf "%s\n" "--url_list_file - indicates source URL is a file containing a list of URLs to download, which are playlists if '--url_playlist' is defined"
    printf "\n"
    printf "%s\n" "Where:"
    printf "%s\n" "- proxy_string is the value to set http_proxy to."
    printf "%s\n" "  E.g. http://user:passwd@proxy:3128"
    printf "%s\n" "- source is either a single item URL, a playlist URL, a file of URLs, or a file of playlist URLs depending on whether --url_list_file and/or --url_playlist are specified."
    printf "\n"
    printf "%s\n" "Note:"
    printf "%s\n" "- mixing URLs that are playlists and not playlists in the same URL list file is not defined, behaviour is unpredictable."
    exit 0
}

command_check ( )
{
    # $1 = command to be checked for existence
    # $2 = Action: [ exit (default) | continue ]
    # $3 = Message
    # $4 = Exit code

    commandCheckAction="${2:-exit}"
    commandCheckMessage="${3:-${1} not available.}"
    commandCheckExitCode="${4:-69}" # man sysexits: 69 = service unavailable: seems closest

    if [ "$( command -v "${1}" > /dev/null 2>&1; echo "$?" )" != '0' ]
    then
        if ! [ "${commandCheckAction}" = 'continue' ]
        then
            printf "%s\n" "Error: ${commandCheckMessage}" >&2
            exit "${commandCheckExitCode}"
        else
            printf "%s\n" "Warning: ${commandCheckMessage}"
        fi
    fi
}

get_script_configuration ( )
{
    if [ -f "${scriptName}/${scriptName}.conf" ] # Settings in script/product subdirectory - presupposes run from bin directory
    then
        ⚠         . "${scriptName}/${scriptName}.conf"
    fi
    if [ -f "/etc/${scriptName}/${scriptName}.conf" ] # Linux system wide settings
    then
        ⚠         . "/etc/${scriptName}/${scriptName}.conf"
    fi
    if [ -f "/usr/local/etc/${scriptName}/${scriptName}.conf" ] # FreeBSD system wide settings
    then
        ⚠         . "/usr/local/etc/${scriptName}/${scriptName}.conf"
    fi
    if [ -f "$HOME/${scriptName}/${scriptName}.conf" ] # User settings
    then
        ⚠         . "$HOME/${scriptName}/${scriptName}.conf"
    fi
    if [ -f "$HOME/.config/${scriptName}/${scriptName}.conf" ] # User settings
    then
        ⚠         . "$HOME/.config/${scriptName}/${scriptName}.conf"
    fi
}

set_defaults ()
{
    format="${format:-bv*+ba/b}"
    proxyString=''
    urlList="${urlList:-false}"
    playList="${playList:---no-playlist}"
}

process_cmdline ( )
{
    while :
    do
        case "$1" in
            -d|--debug)
                debugLevel="--verbose"
                ;;
            -h|--help)
                show_usage
                ;;
            -v|--version)
                version
                ;;
            --proxy=*)
                proxyString="${1##--proxy=}";
                ;;
            --proxy)
                processedTokenCount=$((processedTokenCount + 1));
                shift;
                proxyString="${1}";
                ;;
            --url_list_file)
                urlList="true";
                ;;
            --url_playlist)
                playList='';
                ;;
            *)
                # removing '*)' case will require '--' for "pass through" options/parameters to work
                #   BUT: not providing the '--' will never exit option parsing - so DO NOT disable '*)' case!
                break;
                ;;
        esac
        processedTokenCount=$((processedTokenCount + 1));
        shift
    done
}

url_download ()
{
    # $1 = URL to download

    printf "%s\n" "/------------------------------"
    printf "Downloading media link: %s\n" "${1}"
    ${runEnv} yt-dlp "${debugLevel}" --no-overwrites "${playList}" --format "${format}" --remux mkv --write-subs --sub-langs all --embed-thumbnail --embed-metadata --embed-chapters "${1}"
    printf "%s\n" "------------------------------/"
}
#-------------------------------------------------------------------------#


#-------------------------------------------------------------------------#
# Initialisation
#-------------------------------------------------------------------------#
requiredDependencies='yt-dlp' # list of required programmes (exit script if missing)
optionalDependencies='' # list of optional programmes (continue script if missing)
debugLevel=''
runEnv=''
#-------------------------------------------------------------------------#


#-------------------------------------------------------------------------#
# main
#-------------------------------------------------------------------------#
# script name:
scriptNameExt="${0##*/}"

# script name minus extension:
scriptName="${scriptNameExt%.sh}"

# get script/product default options (stored as global variables)
get_script_configuration
set_defaults

# override script/product default configuration with options if specified
processedTokenCount=0
process_cmdline "$@"
shift $((processedTokenCount)) # must remove processed tokens from "$@" outside function

# store download source (file or item)
download_source="${1}"

# Check for required commands/programmes
if [ -n "${optionalDependencies}" ]
then
    for dependency in ${optionalDependencies}
    do
        command_check "${dependency}" 'continue'
    done
fi
if [ -n "${requiredDependencies}" ]
then
    for dependency in ${requiredDependencies}
    do
        command_check "${dependency}" 'exit'
    done
fi

# set proxy URL if specified:
if [ -n "${proxyString}" ]
then
    runEnv="env http_proxy=${proxyString}"
fi

if [ "${urlList}" != "true" ]
then
    # URL is single download (which may be a playlist)
    url_download "${download_source}"
else
    # URL is filename of file containing multiple download URLs (all of which may be a playlist)
    while read -r line;
    do
        url_download "${line}"
    done < "${download_source}"
fi

exit 0
#-------------------------------------------------------------------------
 
MG said:
Code: 
[…] so you can find the nvidia-drivers in category x11 and not x11-drivers […]
Click to expand...
You know, if the potential port name can be expressed by a basic regular expression, re_format(7), you can write:​
Bash: 
ls -d /usr/ports/*/*nvidia*
Do you still need a directory full of symbolic links?​
MG said:
Code: 
#!/usr/local/bin/bash
Click to expand...
You do not appear to use any bash(1)isms, your script does not consider that things may go wrong (⇒ at least use the ‑errexit flag), and the bulk of it – the looping – can be solved with the standardized find(1) utility (albeit -mindepth/‑maxdepth are extensions):​
Bash: 
#!/bin/sh -e
mkdir /usr/ports/All
cd    /usr/ports/All

# If the symbolic link name is omitted to `ln(1)`,
# it just uses the target’s pathname’s last component:
# ln -s ../category/name name  =  ln -s ../category/name
find    .. -type d -mindepth 2 -maxdepth 2 -exec ln -fs '{}' ';'

# Remove broken links. Taken from find(1) § Examples.
#
# -L : If symbolic link target exists, return target’s information.
#      This is usually a directory.
#      If the target has been deleted, the link file’s information is used.
#      That means the `-type` predicate is `l`.
#      This assumes that no symbolic link points to another symbolic link.
find -L .  -type l -exec rm -- '{}' '+'
 
Kai Burghardt said:
You know, if the potential port name can be expressed by a basic regular expression, re_format(7), you can write:​
Bash: 
ls -d /usr/ports/*/*nvidia*
Do you still need a directory full of symbolic links?
You do not appear to use any bash(1)isms, your script does not consider that things may go wrong (⇒ at least use the ‑errexit flag), and the bulk of it – the looping – can be solved with the standardized find(1) utility (albeit -mindepth/‑maxdepth are extensions):​
Bash: 
#!/bin/sh -e
mkdir /usr/ports/All
cd    /usr/ports/All

# If the symbolic link name is omitted to `ln(1)`,
# it just uses the target’s pathname’s last component:
# ln -s ../category/name name  =  ln -s ../category/name
find    .. -type d -mindepth 2 -maxdepth 2 -exec ln -fs '{}' ';'

# Remove broken links. Taken from find(1) § Examples.
#
# -L : If symbolic link target exists, return target’s information.
#      This is usually a directory.
#      If the target has been deleted, the link file’s information is used.
#      That means the `-type` predicate is `l`.
#      This assumes that no symbolic link points to another symbolic link.
find -L .  -type l -exec rm -- '{}' '+'
Click to expand...
Never had any problem with it. This script is very old and it's only goal is to not waste time on determining inconsistent category placement. It's bash for the tab- completion. The only bashism that I notice now and rhen is an error because I changed a running script with an editor. (Only happens if you insert or delete a line) Never found the reason for that. It wants to save memory by not making a runtime copy of itself or do they want a running script to be able to change itself?

There may be many problems with the script in a different situation but I used to spend more time on finetuning scripts than actually using them. Got rid of that habit. This 1 exists and I use it a lot, so it saves time. It has to be run 1 time on a system with a new portstree. This is also a storage speed indication for me.
 
You must log in or register to reply here.
Back
Top