Out of curiosity: can you post your previous (non-SQL) sh script? (termbin if you like/applicable)
Out of curiosity: can you post your previous (non-SQL) sh script? (termbin if you like/applicable)
What a wonderful, thoroughly useful script! Thank you.
awk -F":" '/authentication failed/ {print $5}' maillog | sort -n | uniq -c | sort -nr | head -10pkg-shell(8) issues a stark warning:
DESCRIPTION
pkg shell is used to interact with the local or remote database through
a sqlite3 console. Extreme care should be taken when using this com-
mand.
Do not rely on this command. The underlying schema is subject to
change on any release. Use pkg-query(8), pkg-rquery(8), pkg-search(8)
and pkg-set(8) instead for querying and modifying the database. These
commands are expected to have a stable API, unlike the database schema.[1-0] % poudlist-org-pkg.sh | grep gpu-firmware
graphics/gpu-firmware-intel-kmod@alderlake broxton cannonlake dg1 dg2 elkhartlake geminilake icelake kabylake rocketlake skylake tigerlake
[2-0] %#!/bin/sh
# Based on installed packages, create a list of ports (flavored and non-flavored) for a poudriere bulk build
pkg query -x -e '%a = 0 && %?A = 1' '%At="%Av"\t%o' '.*' \
| sed -E '/^flavor=/! s/.*\t(.*)/\1 \1/
/^flavor=/ s/^flavor="([^"]*)"\t(.*)/\2@\1 \2/ ' \
| sort -r -k 1 \
| uniq -f 1 \
| cut -w -f 1#!/bin/sh
# Based on installed packages, create a list of ports (flavored and non-flavored) for a poudriere bulk build
pkg query -x -e '%a = 0 && %?A = 1' '%At %Av %o' '.*' \
| awk '
FNR == 1 { origin = $3
flavoredPort = ( $1 == "flavor" ? origin "@" $2 : "" )
}
FNR != 1 { if ( $3 != origin ) {
print ( flavoredPort ? flavoredPort : origin )
origin = $3
flavoredPort = ( $1 == "flavor" ? origin "@" $2 : "" )
} else if ( $1 == "flavor" ) flavoredPort = origin "@" $2
}
END { print ( flavoredPort ? flavoredPort : origin ) } 'Annotation entry. Current FreeBSD remote package repositories suggest that this is the case, most notably they all seem to have repository="FreeBSD". However, it may be that locally generated packages of an administered local package server may have 0 annotations if set up with that intent. To include installed packages that have no annotations at all, an additional pkg command has to be used: pkg query -x -e '%a = 0 && %?A = 0' '%o '.*'graphics/gpu-firmware-intel-kmod@kabylake in my setup:[11-0] % poudlist-sql-pkg.sh | grep firmware
graphics/gpu-firmware-intel-kmod@kabylake
net/wifi-firmware-iwlwifi-kmod@22000
[12-0] %CONFLICTS_INSTALL), like for example with vim(1), /editors/vim:Package flavors (<flavor>: <package>)
console: vim
gtk2: vim-gtk2
gtk3: vim-gtk3
motif: vim-motif
x11: vim-x11
tiny: vim-tinyPackage flavors (<flavor>: <package>)
skylake: gpu-firmware-intel-kmod-skylake
broxton: gpu-firmware-intel-kmod-broxton
kabylake: gpu-firmware-intel-kmod-kabylake
[...][1-0] % cat poudlist-org-pkg.sh | nl
1 #!/bin/sh
2 list=
3 for origin in $(pkg prime-origins)
4 do
5 flavor=$(pkg annotate -qS ${origin} flavor)
6 [ -n "${flavor}" ] && origin=${origin}@${flavor}
7 echo ${origin}
8 done
[2-0] % poudlist-org-pkg.sh | grep gpu-firmware
graphics/gpu-firmware-intel-kmod@alderlake broxton cannonlake dg1 dg2 elkhartlake geminilake icelake kabylake rocketlake skylake tigerlake$(pkg prime-origins) in line 3 lists all packages that are constraint by '%a = 0', as its definition shows:% pkg alias prime-origins
prime-origins 'query -e '%a = 0' '%o''${origin}@${flavor} in line 6 is used without the restriction of %a = 0'. Thus, the original sh script generates a list of all flavors for the specified ${origin}, irrespective of a particular flavored package was installed automatically or not. In my set up there happened to one flavored packages (gpu-firmware-intel-kmod-kabylake-20230625.1402000) that was reported not to be automatically installed; all other flavors for that origin however were automatically installed, for example:[10-0] % pkg query -x '(automatically_installed=%a) %o %n' 'kabylake|alderlake'
(automatically_installed=1) graphics/gpu-firmware-intel-kmod gpu-firmware-intel-kmod-alderlake
(automatically_installed=0) graphics/gpu-firmware-intel-kmod gpu-firmware-intel-kmod-kabylake
[11-0] %%?A = 1 isn't strictly necessary, however it may serve to emphasize that it is necessary to produce any output given the '%At %Av %o' as query format. For a specified <pattern> when dealing with packages that have no annotations at all this strangely blocks any output. For testing, I used pkg-annotate(8) to delete all annotations from sysutils/lsof.[0-0] % pkg inf lsof
lsof-4.99.4_2,8
Name : lsof
Version : 4.99.4_2,8
Installed on : Mon Mar 31 20:27:35 2025 CEST
Origin : sysutils/lsof
Architecture : FreeBSD:14:amd64
Prefix : /usr/local
Categories : sysutils
Licenses : lsof
Maintainer : ler@FreeBSD.org
WWW : https://github.com/lsof-org/lsof/
Comment : Lists information about open files (similar to fstat(1))
Shared Libs required:
libc.so.7
libkvm.so.7
libutil.so.9
Annotations :
Flat size : 236KiB
Description :
Lsof (LiSt Open Files) lists information about files that are open by the
running processes. An open file may be a regular file, a directory, a block
special file, a character special file, an executing text reference, a library,
a stream or a network file (Internet socket, NFS file or Unix domain socket).
See also fstat(1) in the base system.
[1-0] % pkg query -x -e '%a = 0 && %?A = 0' '%At %Av %o' 'lsof'
[2-0] % pkg query -x -e '%a = 0 && %?A = 0' '%o' 'lsof'
sysutils/lsoflsof) Annotations as compared to the above provided awk(1) solution:[0-0] % poudlist-sql-pkg.sh | egrep '(lsof|gsed|gpu-firmware-intel-kmod)'
graphics/gpu-firmware-intel-kmod@kabylake
sysutils/lsof
textproc/gsed
[1-0] % poudlist-awk.sh | egrep '(lsof|gsed|gpu-firmware-intel-kmod)'
graphics/gpu-firmware-intel-kmod@kabylake
textproc/gsed
[2-0] %That's impressive, but I'm curious how that would be re-usable? I don't know what 2/3rds of
pkg query -x -e '%a = 0 && %?A = 1' '%At="%Av"\t%o' '.*' is 
curl .sh. That line looks cool, but what if you wanted to use it in another script just slightly differently? Or would you re-create the whole command? -a 'a' was something weird like allow all cookies but good luck with the rest + different casing (iirc the flags also differed Linux/FreeBSD)GDK_BACKEND='x11' surf -a 'a' -b -D -f -g -I -k -m -n -S -t -x 'http://localhost:8888' 1 #!/bin/sh
2 # Based on installed packages, create a list of ports (flavored and non-flavored) for a poudriere bulk build
3 pkg query -x -e '%a = 0 && %?A = 1' '%At %Av %o' '.*' \
4 | awk '
5 FNR == 1 { origin = $3
6 flavoredPort = ( $1 == "flavor" ? origin "@" $2 : "" )
7 }
8 FNR != 1 { if ( $3 != origin ) {
9 print ( flavoredPort ? flavoredPort : origin )
10 origin = $3
11 flavoredPort = ( $1 == "flavor" ? origin "@" $2 : "" )
12 } else if ( $1 == "flavor" ) flavoredPort = origin "@" $2
13 }
14 END { print ( flavoredPort ? flavoredPort : origin ) } 'FNR == 1 (line 5) and FNR != 1 (line 8), or also to the code between { ... } in lines 5-13?function select_flavor(f) {
if (match(f, "py|qt"))
return ""
return f
}
{
if ($3 == origin) {
if ($1 == "flavor")
flavor = select_flavor($2)
} else {
if (origin)
print (flavor ? origin"@"flavor : origin)
origin = $3
flavor = ""
if ($1 == "flavor")
flavor = select_flavor($2)
}
}
END { print (flavor ? origin"@"flavor : origin) }select_flavor() can be expanded to take the origin as an argument and fit someone's need for more complex testing.Well, given "I don't know what 2/3rds [...] is" I'd say, you are well on your way and have travelled the first 1/3rd of the wayI'm curious how that would be re-usable? I don't know what 2/3rds ofpkg query -x -e '%a = 0 && %?A = 1' '%At="%Av"\t%o' '.*'is
I see random stuff like that in scripts online, and there's no way everyone is understanding what that stuff is before passing it behind acurl .sh. That line looks cool, but what if you wanted to use it in another script just slightly differently? Or would you re-create the whole command?
poudlist-sql-pkg.sh below) and of the original sh script (see the 'Spoiler: Explanation' part in message #405 ) the use of pkg prime-origins and use its non-aliased format, then extending that to the regex format; then limiting the packages selected and then noticing the possibility of 'dumping' all annotations:[0-0] % poudlist-sql-pkg.sh | grep -E '(vim|gsed)'
editors/vim@console
textproc/gsed
[1-0] % pkg alias prime-origins
prime-origins 'query -e '%a = 0' '%o''
[2-0] % pkg query -e '%a = 0' '%o'
[ very long list ]
[3-0] % pkg query -x -e '%a = 0' '%o' '.*'
[ very long list ]
[4-0] % pkg query -x -e '%a = 0' '%o' '(gsed|vim)'
textproc/gsed
editors/vim
[5-0] % pkg query -x -e '%a = 0' '%At=%Av %o' '(gsed|vim)'
FreeBSD_version=1401000 textproc/gsed
build_timestamp=2025-02-27T01:11:15+0000 textproc/gsed
built_by=poudriere-git-3.4.2 textproc/gsed
cpe=cpe:2.3:a:gnu:sed:4.9:::::freebsd14:x64 textproc/gsed
port_checkout_unclean=no textproc/gsed
port_git_hash=1bb147345 textproc/gsed
ports_top_checkout_unclean=no textproc/gsed
ports_top_git_hash=6ec2dc26d textproc/gsed
repo_type=binary textproc/gsed
repository=FreeBSD textproc/gsed
FreeBSD_version=1402000 editors/vim
build_timestamp=2025-04-24T07:30:03+0000 editors/vim
built_by=poudriere-git-3.4.2-11-gfa886a3d editors/vim
cpe=cpe:2.3:a:vim:vim:9.1:::::freebsd14:x64 editors/vim
flavor=console editors/vim
port_checkout_unclean=no editors/vim
port_git_hash=9d7c8ff2cb editors/vim
ports_top_checkout_unclean=no editors/vim
ports_top_git_hash=4de34b9860 editors/vim
repo_type=binary editors/vim
repository=FreeBSD editors/vim
[6-0] %
. For me it involves reading trying stuff, thinking about stuff, trying more stuff etc.; I imagine that this process wasn't altogether that different for W.hâ/t while developing his sql and sh script.That looks like you have achieved the integration of the two patterns and does not need the use of
FNR anymore; personally I find the control flow harder to follow, but that may well be personal preference.next statement*, or use the getline function in the BEGIN pattern, the latter results in not needing any line number tests (like with FNR) and eliminates the test for an 'empty' origin:#!/bin/sh
# Based on installed packages, create a list of ports (flavored and non-flavored) for a poudriere bulk build
pkg query -x -e '%a = 0 && %?A = 1' '%At %Av %o' '.*' \
| awk '
BEGIN { # BEGIN pattern gets & processes first line as initialisation; next pattern starts at line/record #2
getline
origin = $3
flavor = ( $1 == "flavor" ? $2 : "" )
}
{ if ( $3 != origin ) {
print ( flavor ? origin "@" flavor : origin )
origin = $3
flavor = ( $1 == "flavor" ? $2 : "" )
} else if ( $1 == "flavor" ) flavor = $2
}
END { print ( flavor ? origin "@" flavor : origin ) } ' **#!/bin/sh
# Based on installed packages, create a list of ports (flavored and non-flavored) for a poudriere bulk build
pkg query -x -e '%a = 0 && %?A = 1' '%At %Av %o' '.*' \
| awk ' \
FNR == 1 { origin = $3
flavor = ( $1 == "flavor" ? $2 : "" )
next
}
{ if ( $3 != origin ) {
print ( flavor ? origin "@" flavor : origin )
origin = $3
flavor = ( $1 == "flavor" ? $2 : "" )
} else if ( $1 == "flavor" ) flavor = $2
}
END { print ( flavor ? origin "@" flavor : origin ) } '-: END { print ( flavor ? origin "@" $flavor : origin ) } '
+: END { print ( flavor ? origin "@" flavor : origin ) } 'FNR didn't belong there. I like the BEGIN version better though. The initialisation is explicit.FNR == 1 { origin = $3
flavor = ( $1 == "flavor" ? $2 : "" )
next
}
if ( $3 != origin )built_by which comes before flavor and seems to be annotated by poudriere to every package it builds.#!/bin/sh
match_flavor='
function select_flavor(f) {
if (match(f, "^py|^qt"))
return ""
return "@"f
}
{
if ($1 == "built_by") {
print origin
origin = $3
} else if ($1 == "flavor" )
origin = origin select_flavor($2)
}
END { print origin }
'
${@:-eval} "pkg query -e '%a = 0' '%At %Av %o'" | awk "${match_flavor}" | sort$ sh mkplist.sh
$ sh mkplist.sh ssh admin@server PKG_DBDIR=/jail/var/db/pkgYes, of the first pattern containing
next the test is executed for every record/line. When reading the first line this test is executed and, as it evaluates to true, the action is performed; for all other lines the test evaluates to false and as a consequence the action is not performed. However, the second test: FNR != 1 is eliminated; so one out of two, for that version. I don't think that will result in a relevant difference in execution. Along the same lines about optimization as you noted.hold space; #5. You can observe some of the differences between awk(1) and sed(1) in their respective scripts for this problemGood catch. This made me realise a fundamental shortcoming in my thinking about the problem so far. Addressing this I changed my script for the generation of a poudriere list where, as far as annotations are concerned, the decision regarding the flavor is based on the presence or absence of the flavor annotation, and not on any other annotation; more on that below. There's also a typo in my previous script above (amended).
pkg query -x -e '%a == 0 && %At == flavor' '%At=%Av %o' '.*'green-sardine of graphics/gpu-firmware-amd-kmodtahiti of graphics/gpu-firmware-amd-kmodverde of graphics/gpu-firmware-radeon-kmod[1-0] % pkg query -x -e '%a = 0 && %?A = 1' '%At %Av %o' '(green-sardine|verde)' | column -t
FreeBSD_version 1402000 graphics/gpu-firmware-amd-kmod
build_timestamp 2025-04-24T07:46:24+0000 graphics/gpu-firmware-amd-kmod
built_by poudriere-git-3.4.2-11-gfa886a3d graphics/gpu-firmware-amd-kmod
flavor green_sardine graphics/gpu-firmware-amd-kmod
port_checkout_unclean no graphics/gpu-firmware-amd-kmod
port_git_hash 1bb147345d graphics/gpu-firmware-amd-kmod
ports_top_checkout_unclean no graphics/gpu-firmware-amd-kmod
ports_top_git_hash 4de34b9860 graphics/gpu-firmware-amd-kmod
repo_type binary graphics/gpu-firmware-amd-kmod
repository FreeBSD graphics/gpu-firmware-amd-kmod
FreeBSD_version 1402000 graphics/gpu-firmware-radeon-kmod
build_timestamp 2025-04-24T07:52:26+0000 graphics/gpu-firmware-radeon-kmod
built_by poudriere-git-3.4.2-11-gfa886a3d graphics/gpu-firmware-radeon-kmod
flavor verde graphics/gpu-firmware-radeon-kmod
port_checkout_unclean no graphics/gpu-firmware-radeon-kmod
port_git_hash 1bb147345d graphics/gpu-firmware-radeon-kmod
ports_top_checkout_unclean no graphics/gpu-firmware-radeon-kmod
ports_top_git_hash 4de34b9860 graphics/gpu-firmware-radeon-kmod
repo_type binary graphics/gpu-firmware-radeon-kmod
repository FreeBSD graphics/gpu-firmware-radeon-kmod[1-0] % pkg query -x -e '%a = 0 && %?A = 1' '%At %Av %o' '(tahiti|verde)' | column -t
FreeBSD_version 1402000 graphics/gpu-firmware-radeon-kmod
build_timestamp 2025-04-24T07:46:49+0000 graphics/gpu-firmware-radeon-kmod
built_by poudriere-git-3.4.2-11-gfa886a3d graphics/gpu-firmware-radeon-kmod
flavor tahiti graphics/gpu-firmware-radeon-kmod
port_checkout_unclean no graphics/gpu-firmware-radeon-kmod
port_git_hash 1bb147345d graphics/gpu-firmware-radeon-kmod
ports_top_checkout_unclean no graphics/gpu-firmware-radeon-kmod
ports_top_git_hash 4de34b9860 graphics/gpu-firmware-radeon-kmod
repo_type binary graphics/gpu-firmware-radeon-kmod
repository FreeBSD graphics/gpu-firmware-radeon-kmod
FreeBSD_version 1402000 graphics/gpu-firmware-radeon-kmod
build_timestamp 2025-04-24T07:52:26+0000 graphics/gpu-firmware-radeon-kmod
built_by poudriere-git-3.4.2-11-gfa886a3d graphics/gpu-firmware-radeon-kmod
flavor verde graphics/gpu-firmware-radeon-kmod
port_checkout_unclean no graphics/gpu-firmware-radeon-kmod
port_git_hash 1bb147345d graphics/gpu-firmware-radeon-kmod
ports_top_checkout_unclean no graphics/gpu-firmware-radeon-kmod
ports_top_git_hash 4de34b9860 graphics/gpu-firmware-radeon-kmod
repo_type binary graphics/gpu-firmware-radeon-kmod
repository FreeBSD graphics/gpu-firmware-radeon-kmodbuilt_by tag. Given the pkg-query(8) output ordening it correctly identifies each transition, even with the same origin. However, there are a few limitations/problems. Your script does not initialise the origin variable, as a consequence it (used with pkg query -x -e '%a = 0' '%At %Av %o' '(yellow|green)') outputs an empty first line:[1-0] % mkplist.sh | nl -b a
1
2 graphics/gpu-firmware-amd-kmod@green_sardine
3 graphics/gpu-firmware-amd-kmod@yellow_carp
[2-0] %-f file
Build ports listed in the file.
The path to the file has to be absolute. Ports must be specified in the form of “category/port”
and sh(1)-style comments are allowed. Multiple -f file arguments may be specified at once.BEGIN pattern as indicated, or by changing the print statement (line 12) into the version on line 11 as shown below: 1 #!/bin/sh
2 match_flavor='
3 function select_flavor(f) {
4 if (match(f, "^py|^qt"))
5 return ""
6 return "@"f
7 }
8 #BEGIN { origin = "#" }
9 {
10 if ($1 == "built_by") {
11 #if ( origin ) print origin
12 print origin
13 origin = $3
14 } else if ($1 == "flavor" )
15 origin = origin select_flavor($2)
16 }
17 END { print origin }
18 '
19 ${@:-eval} "pkg query -e '%a = 0' '%At %Av %o'" | awk "${match_flavor}" | sortbuilt_by annotation is output before the flavor annotation: this is essential. If the built_by annotation was output after the flavor annotation the script would fail in producing a correct result. The current output is probably sorted by annotation tag by default or by an explicit SQL sort. I would feel more secure if at least pkg-query(8) would have mentioned at its Multiline patterns sub heading something like that these particular lines are sorted but that is not mentioned. More importantly the built_by annotation must always be present, otherwise the transition would not be noticed correctly. However, when building a port locally the built_by annotation is not generated, sysutils/lsof build locally results in:[1-0] % pkg info -A lsof
lsof-4.99.4_2,8:
FreeBSD_version: 1402000
cpe : cpe:2.3:a:lsof_project:lsof:4.99.4:::::freebsd14:x64:2
[2-0] %#!/bin/sh
# Based on installed packages, create a list of ports (flavored and non-flavored) for a poudriere bulk build
pkg query -x -e '%a = 0 && %?A = 1' '%At %Av %o %n' '.*' \
| awk '
BEGIN { #BEGIN pattern gets & processes first line as initialisation; next pattern starts at line 2
getline
origin = $3
pkgname = $4
flavor = ( $1 == "flavor" ? $2 : "" )
}
{ if ( $4 != pkgname ) {
print ( flavor ? origin "@" flavor : origin )
origin = $3
pkgname = $4
flavor = ( $1 == "flavor" ? $2 : "" )
} else if ( $1 == "flavor" ) flavor = $2
}
END { print ( flavor ? origin "@" flavor : origin ) } '%n-%o and randomly 'shuffle' the entire pkg-query(8) output using shuf(1) from sysutils/shuf *#!/bin/sh
# Based on installed packages, create a list of ports (flavored and non-flavored) for a poudriere bulk build
pkg query -x -e '%a = 0 && %?A = 1' '%At %Av %o %n-%o' '.*' \
| shuf \
| sort -k 4 \
| awk '
BEGIN { #BEGIN pattern gets & processes first line as initialisation; next pattern starts at line 2
getline
origin = $3
pkgname = $4
flavor = ( $1 == "flavor" ? $2 : "" )
}
{ if ( $4 != pkgname ) {
print ( flavor ? origin "@" flavor : origin )
origin = $3
pkgname = $4
flavor = ( $1 == "flavor" ? $2 : "" )
} else if ( $1 == "flavor" ) flavor = $2
}
END { print ( flavor ? origin "@" flavor : origin ) } '[1-0] % mkpl-4f.sh | sort > plist.txt
[2-0] % mkpl-4f-shuf.sh | sort > plist-shuf.txt
[3-0] % diff plist.txt plist-shuf.txt
[4-0] %
- and don't know how to convert the octets of SERIAL_NUMBER into decimal. The octets does not seem to follow the rule of 8^(digit-position). For example, one of the QuoVadis certificates has serial 1289 in decimal, parenthesized as (0x509), which is quite funny, and then a series of octet that should hold the same value: 002 002 005 011. That is 2*(8^3) + 2*(8^2) + 5*(8^1) + 11*(8^0), which is 1201?#!/bin/sh -e
_tb=' '
_nl='
'
LC_ALL=C
export LC_ALL
_lst=$(printf "\002\020\120\224\154\354\030\352\325\234\115\325\227\357\165\217\240\255" | vis -h)
_lst="${_lst#?}"
IFS="%"
for i in $_lst
do
if test ${#i} -eq 2
then _str="$_str $i"
else
#convert ascii to hex - #1
# NB hexdump(1) reverse char sequence
_str="$_str ${i%?} x"
_hxstr="${i#??}$_hxstr"
fi
done
_str="${_str# }"
#convert ascii to hex - #2
_hxstr=$(printf "$_hxstr"|hexdump -e '"%x"')
_lst=
IFS=" "$_tb$_nl
for i in $_str
do
if test "$i" = "x"
then
#convert ascii to hex - #3
_tmp="${_hxstr#??}"
_lst="$_lst ${_hxstr%$_tmp}"
_hxstr="${_hxstr#??}"
else _lst="$_lst $i"
fi
done
echo "Serial for \"XRamp Global CA Root\" in nss format \"\002\020\120\224\154\354\030\352\325\234\115\325\227\357\165\217\240\255\""
echo
echo "valid: 50:94:6c:ec:18:ea:d5:9c:4d:d5:97:ef:75:8f:a0:ad"
echo "script convert:"
echo "$_lst"printf "\002\020\120\224\154\354\030\352\325\234\115\325\227\357\165\217\240\255" \
| hexdump -e '/1 "%02x" " "'
# takesnap
Pool to take snapshot from:
storage
Comment to add to the snapshot name:
backup after zfs copy
Will operate on storage
Date is: 2025-05-30-15:45
Comment is: backup-after-zfs-copy
Snapshot is taken: done!
storage@2025-05-30-15:45:backup-after-zfs-copy 0B - 165G -
# takesnap zroot
Comment to add to the snapshot name:
Will operate on zroot
Date is: 2025-05-30-15:41
Snapshot is taken: done!
zroot@2025-05-30-15:41 0B - 96K -
# takesnap zroot "test comment"
Will operate on zroot
Date is: 2025-05-30-15:41
Comment is: test-comment
Snapshot is taken: done!
zroot@2025-05-30-15:41:test-comment 0B - 96K -#!/bin/csh
set date = `date +%F-%H:%M`
if ( "$#argv" > 0 ) then
set pool = "$argv[1]"
else
echo "Pool to take snapshot from: "
set pool = "$<"
endif
if ( "$#argv" >= 2 ) then
set comment = "$argv[2]"
set isComment = 1
set fixedComment = `echo "$comment" | sed 's/ /-/g'`
set c = ":$fixedComment"
else
echo "Comment to add to the snapshot name: "
set comment = "$<"
if ( "$comment" == "" ) then
set isComment = 0
else
set isComment = 1
set fixedComment = `echo "$comment" | sed 's/ /-/g'`
set c = ":$fixedComment"
endif
endif
if ( ! "$?isComment" ) then
set isComment = 0
endif
if ( "$pool" == "" ) then
echo "No pools given."
exit 1
endif
echo "Will operate on $pool"
echo "Date is: $date"
if ( "$isComment" != 0 ) then
echo "Comment is: $fixedComment"
set takesnap = "zfs snapshot -r $pool@$date$c"
else
set takesnap = "zfs snapshot -r $pool@$date"
endif
eval $takesnap
if ( "$status" != 0 ) then
echo "Error taking snapshot of $pool"
exit 1
else
echo "Snapshot is taken: done!"
if ( "$isComment" != 0 ) then
set showsnap = "zfs list -H -t snap $pool@$date$c"
else
set showsnap = "zfs list -H -t snap $pool@$date"
endif
eval $showsnap
endif
exit 0# s.parse_nss_bundle -f /tmp/certdata.txt -m
Certificates in bundle: 176
Certificates onhold: 41# s.parse_nss_bundle -f /tmp/certdata.txt -l
Certificates in bundle: 176
Certificates onhold: 12# time s.parse_nss_bundle -f /tmp/certdata.txt -s
Certificates in bundle: 176
Certificates onhold: 41
9.75 real 9.84 user 0.56 sys# mkdir /tmp/store
# time /usr/src/secure/caroot/MAca-bundle.pl -i /tmp/certdata.txt -o store
## Untrusted certificates omitted from this bundle: 41
## Number of certificates: 135
0.89 real 0.70 user 0.19 sys#!/bin/sh -e
# Interpreting escaped octal values relies on sh(1) builtin
# 'printf'. Transposing name of issuer to a kind of plain ascii is
# implemented by setting LC_ALL and with help from sed(1).
# Date format strings can be looked up at strftime(3)
#/usr/src/usr.sbin/certctl/certctl.sh:
# -> certctl.sh rehash
#/usr/src/secure/caroot/Makefile:
#|fetchcerts: .PHONY
#| fetch --no-sslv3 --no-tlsv1 -o certdata.txt 'https://hg.mozilla.org/projects/nss/raw-file/tip/lib/ckfw/builtins/certdata.txt'
_script=s.parse_nss_bundle
_script_v="tst 01"
HELP="
A shell script to parse the nss bundle:
$_script -f <file> [-l|-m] [-d <dirname>] [-s|-a] [-v [<str>]]
Options:
-l - (logical) evaluate expiry plus trust tokens
-m - (mimick) evaluate trust token for 'server auth'
This is default
-f <file> - read from file
-d <dirname> - topdir to the two sort orders \"ok\" & \"onhold\".
Defaults to /tmp/nss_local
-a - (annotations) serialize both \"ok\" and \"onhold\"
certificates
Default is to only write \"onhold\" certificates
-s - serialize the bundle via calls to
\"openssl x509 -text -inform DER -fingerprint\"
Default is to only write the annotations
-v <str> - (version) use <str> to mark the certificates.
Leave empty to use timestamp of inputfile
The script sort certificates to one of two categories, \"ok\" & \"onhold\". The \"onhold\" certificates are annotated to document their lesser priority. Use 'head -n5 onhold/*|less' to see through the annotations.
BUGS
Version string cannot begin with hyphen.
"
test "${0##*/}" = "$_script" ||
{
echo "Expecting script to have name:$_script. If you have sourced this file, then pls don't. Too many variables$HELP"
return 1
}
# 3 causes for onhold + 1 not annotated
_ca_msg_nxplicit="# no explicit trust token"
_ca_msg_wdraw="# explicit trust token gainsaid:"
_ca_msg_expi="# Expired "
_ca_cnt=0
_ca_onhold_cnt=0
# nss octal encoded date format according to MAca-bundle.pl
#YYMMDDhhmmss plus a "Z"
_ca_sftm3="+%y%m%d%H%M%S"
_ca_now=$(date $_ca_sftm3)
# Date string written in cleartxt
# Tue Sep 01 12:00:00 1998
alias caroot_ctm='date -j -f "%a %b %d %H:%M:%S %Y"'
#/usr/src/secure/caroot/MAca-bundle.pl, the cli call
alias caroot_ossl='openssl x509 -text -inform DER -fingerprint'
# Pattern @[^[:alnum:]\-]@_@ also from MAca-bundle.pl
# work with sed(1) from locale C
alias caroot_sed="LC_ALL=C sed 's@[^[:alnum:]\-]@_@g'"
# use in pipeline - similar at MAca-bundle.pl
print_header_()
{
cat - << EoStr
# idx: $_ca_idx ; mode: ${_logic:-mimick} - $_script version $_script_v${1:+${_nl}# CA: $@}
${_tjek#?}
EoStr
cat -
}
_logic=
_if=
_all=
_seria=
_ca_dir=/tmp/nss_local
_ca_idx=
while test $# -gt 0
do
case $1 in
"-l") _logic=logical ;;
"-m") : ;;
"-f")
# _if=$(readlink -f $2) ||
_ca_idx=$(stat -f "%Sm" -t "${_ca_sftm3#?}" $2) ||
{
echo "File not recognized, ..exiting$HELP"
exit 1
}
_if=$2
shift
;;
"-d")
_ca_dir=$2
shift
;;
"-s") _seria=1 ;;
"-a") _all=1 ;;
"-v")
test "$2" = "${2#-}" &&
{
_ca_idx="$2"
shift
}
;;
*)
echo "Args jumble, ..exiting$HELP"
exit 1
;;
esac
shift
done
#return 0
test "$_if" ||
{
echo "Missing input file, ..exiting$HELP"
exit 1
}
test "$_all" -a "$_seria" &&
{
echo "Args mixup - choose either of annotations or certificate formatting, ..exiting$HELP"
exit 1
}
_nl='
'
_tb=" "
_ca_item_ok_dir=$_ca_dir/ok
_ca_item_onhold_dir=$_ca_dir/onhold
_ca_todo_label=
_ca_todo_msg=
_ca_todo_ln=
_ca_todo_calst=
IFS=
_s_flag=1
_e_flag=1
#_tjek
#_ln
#_label
# expiry
#_onhold
# conditionally set _s/e_onhold
#_dt_e
#_dt_s
# confounds 'distrust after' & explicit distrust
#_s_onhold
#_e_onhold
echo > /tmp/dbg.parsenss
while read -r tgt
do
test "$tgt" &&
{
_tjek="${tgt#* }"
#{ condit - ln no blanks - begin
if test "$_tjek" = "$tgt"
then
if test "$_ln"
then
if test "$tgt" = "END"
then
_ca_todo_label="$_ca_todo_label$_nl${_label%?}"
test "$_seria" &&
_ca_todo_ln="$_ca_todo_ln$_nl${_ln#?}"
_ln=
_label=
else _ln="$_ln$tgt"
fi
elif test "$_dt_s"
then
if test "$tgt" = "END"
then
_dt_s=$(printf "$_dt_s")
test $_dt_s -lt $_ca_now &&
_s_onhold="# CKA_NSS_SERVER_DISTRUST_AFTER $_dt_s"
_dt_s=
# cut trailing Z
else _dt_s=${tgt%?132}
fi
elif test "$_logic"
then
if test "$_dt_e"
then
if test "$tgt" = "END"
then
_dt_e=$(printf "$_dt_e")
test $_dt_e -lt $_ca_now &&
_e_onhold="# CKA_NSS_EMAIL_DISTRUST_AFTER $_dt_e"
_dt_e=
# cut trailing Z
else _dt_e=${tgt%?132}
fi
fi
fi
# condit - ln no blanks
else
if test "${_tjek%% *}" = "Certificate"
then _label="${_tjek#*\"}"
elif test "$tgt" = "CKA_NSS_SERVER_DISTRUST_AFTER MULTILINE_OCTAL"
then _dt_s=.
elif test "$tgt" = "CKA_VALUE MULTILINE_OCTAL"
then _ln=.
elif test "${tgt%% *}" = "CKA_TRUST_SERVER_AUTH"
then
_tjek="${_tjek##* }"
if test "$_tjek" = "CKT_NSS_TRUSTED_DELEGATOR"
then _s_flag=
elif test "$_tjek" = "CKT_NSS_NOT_TRUSTED"
then _s_onhold="# $tgt"
fi
#placeholder end-of-record
elif test "${tgt%% *}" = "CKA_TRUST_STEP_UP_APPROVED"
then
_ca_cnt=$(($_ca_cnt+1))
if test "$_logic"
then
# any non empty line will trigger onhold
#LN1 / LN2 / LN3 / LN4
#LN1
_ca_todo_msg="$_ca_todo_msg$_nl$_onhold"
#LN2 / LN3
if test "$_s_onhold" -a "$_e_onhold"
then
_ca_todo_msg="$_ca_todo_msg$_nl$_s_onhold$_nl$_e_onhold"
else
# TODO - ? too many condits - ? _s_onhold & _s_flag mutually exclusive
# evaluate explicit trust against 'prognostic' withdrawing that trust
# nota bene, when dateline is transgressed
# negates both evaluations
# if test ! "$_s_flag" -a "$_s_onhold"
if test "$_s_onhold" -a ! "$_s_flag"
then
_ca_todo_msg="$_ca_todo_msg${_nl}$_ca_msg_wdraw$_nl$_s_onhold"
elif test "$_e_onhold" -a ! "$_e_flag"
then
_ca_todo_msg="$_ca_todo_msg${_nl}$_ca_msg_wdraw$_nl$_e_onhold"
else _ca_todo_msg="$_ca_todo_msg$_nl$_nl"
fi
fi
#LN4
if test "$_s_flag" -a "$_e_flag"
then _ca_todo_msg="$_ca_todo_msg$_nl$_ca_msg_nxplicit"
else _ca_todo_msg="$_ca_todo_msg$_nl"
fi
_onhold=
_dt_s=
_s_onhold=
_dt_e=
_e_onhold=
_s_flag=1
_e_flag=1
else
#TODO - begin
#LN1
if test "$_s_onhold"
then _ca_todo_msg="$_ca_todo_msg$_nl$_s_onhold"
else _ca_todo_msg="$_ca_todo_msg$_nl"
fi
#LN2
if test "$_s_flag"
then
_ca_todo_msg="$_ca_todo_msg${_nl}$_ca_msg_nxplicit"
else
_ca_todo_msg="$_ca_todo_msg${_nl}"
fi
_dt_s=
_s_onhold=
_s_flag=1
#TODO - fini
fi
elif test "$_logic"
then
if test "${_tjek#Not Valid After}" != "$_tjek"
then
test $(caroot_ctm "${_tjek#*: }" "$_ca_sftm3") \
-lt $_ca_now && _onhold="$_ca_msg_expi${_tjek#*: }"
elif test "$tgt" = "CKA_NSS_EMAIL_DISTRUST_AFTER MULTILINE_OCTAL"
then _dt_e=.
elif test "${tgt%% *}" = "CKA_TRUST_EMAIL_PROTECTION"
then
_tjek="${_tjek##* }"
if test "$_tjek" = "CKT_NSS_TRUSTED_DELEGATOR"
then _e_flag=
elif test "$_tjek" = "CKT_NSS_NOT_TRUSTED"
then _e_onhold="# $tgt"
fi
fi
fi
fi
#} condit - ln no blanks - end
}
done < $_if
_ca_todo_ln="${_ca_todo_ln#$_nl}"
_ca_todo_msg="${_ca_todo_msg#$_nl}"
_ca_todo_calst="${_ca_todo_label#$_nl}"
_ca_todo_label=$(printf "${_ca_todo_label#$_nl}"\
| caroot_sed)
#tee << EoStr >> /tmp/dbg.parsenss
#LN
#$_ca_todo_ln
#MSG
#$_ca_todo_msg
#LABEL
#$_ca_todo_label
#EoStr
#exit
test -d "$_ca_item_ok_dir" || mkdir -p "$_ca_item_ok_dir"
test -d "$_ca_item_onhold_dir" || mkdir -p "$_ca_item_onhold_dir"
if test "$_seria"
then
#_flag
#_tjek
#_res
IFS=" ""$_tb"$_nl
for i in $_ca_todo_label
do
_tjek=
_res=$_ca_item_ok_dir
for ii in 1 2${_logic:+ 3 4}
do
_flag="${_ca_todo_msg%%$_nl*}"
test "$_flag" && _res=$_ca_item_onhold_dir
_tjek="$_tjek$_nl${_flag:-#}"
_ca_todo_msg="${_ca_todo_msg#*$_nl}"
done
test "$_res" = "$_ca_item_ok_dir" ||
_ca_onhold_cnt=$(($_ca_onhold_cnt+1))
#{
printf "${_ca_todo_ln%%$_nl*}" \
| caroot_ossl \
| print_header_ \
> $_res/$i.pem
#} &
_ca_todo_ln="${_ca_todo_ln#*$_nl}"
done
else
#_flag
#_tjek
#_res
IFS=" ""$_tb"$_nl
for i in $_ca_todo_label
do
_tjek=
if test "$_all"
then _res=$_ca_item_ok_dir
else _res=
fi
for ii in 1 2${_logic:+ 3 4}
do
_flag="${_ca_todo_msg%%$_nl*}"
test "$_flag" && _res=$_ca_item_onhold_dir
_tjek="$_tjek$_nl${_flag:-#}"
_ca_todo_msg="${_ca_todo_msg#*$_nl}"
done
test "$_res" != "${_all:+$_ca_item_ok_dir}" &&
_ca_onhold_cnt=$(($_ca_onhold_cnt+1))
test "$_res" &&
{
echo | print_header_ "${_ca_todo_calst%%$_nl*}"\
> $_res/$i.pem
}
_ca_todo_ln="${_ca_todo_ln#*$_nl}"
_ca_todo_calst="${_ca_todo_calst#*$_nl}"
done
fi
printf " Certificates in bundle: $_ca_cnt
Certificates onhold: $_ca_onhold_cnt
"# Expiry:
# Seemingly harmless, not technical, just an arbitrary rulesetting
# Distrust:
# Inexplicable worthing&context, especially naming a date after which
# distrust occur - is it technical? is it a blacklist notion? WITHOUT_CAROOT) It is sort of an simple uncertainty, concretely installing 'untrusted' certificates to the base system. ($maytrust and not $distrust) since the script does not parse for the trust token relating to 'email protection'.