Shell Getting a handle on sh syntax

astyle

Daemon

Reaction score: 626
Messages: 1,421

I'd like for some help with getting a handle on sh syntax... The goal of my rather simple script is to generate a list of all installed ports whose category is kde or kde-applications... To this end, I have the following in list_kde_ports.sh:
Bash:
for port in `cat port_list.txt`
do
if [ -x pkg query %C $port | grep kde ] ; then echo $port >> kde_ports.txt
done
port_list.txt exists, and can be fed to Poudriere or Portsnap.
Thing is, I get the following error:
Bash:
grep: ]: No such file or directory
[: missing ]

Where in my syntax did I go wrong?
 

Vull

Aspiring Daemon

Reaction score: 518
Messages: 820

"-x" tests for the existence of an executable filename. You want to test if there is any output from pkg-query. Try something like:

if [ "$(pkg query %C $port | grep kde)" ]; then echo $port >> kde_ports.txt

">>" will create a file if it doesn't already exist, so, to get consistent results, you might also want to initialize (i.e., empty) the output file, with rm kde_ports.txt before starting the for loop.
 
OP
astyle

astyle

Daemon

Reaction score: 626
Messages: 1,421

What do you think about something in the trend off,
Code:
pkg query %o | grep "kde"
%o is for origin, and I want to catch stuff like graphics/spectacle, which is a KDE app for taking screenshots. I did play with that, before realizing I need Category, rather than Origin. My big idea is to eventually compare the output of using # pkg query and find(1) (for resolving an installed package to a port). But still, I appreciate all the help, Vull and Alain De Vos !
 

memreflect

Well-Known Member

Reaction score: 221
Messages: 257

Bash:
[ -x pkg query %C $port | grep kde ]
It may help you to know that [ -x path/to/file ] is not shell syntax; it's another way to write test -x path/to/file, except the ']' is required if you use the '[' form. Here's what you're really executing:
Bash:
test -x pkg query %C $port | grep kde ]
You might consider switching to test(1) instead of '[' until you feel more comfortable with shell scripting and the 'test' command. It's easier to understand the behavior of a script written with an explicit 'test' command, and that also makes things easier to debug:
Bash:
$ port=misc/hicolor-icon-theme
$ test -x pkg query %C $port | grep gnome
test: pkg: unexpected operator
$ test -x pkg query
test: pkg: unexpected operator
$ test -x pkg
$ echo $?
1
$ ls ./pkg
ls: ./pkg: No such file or directory
 

Neubert

Member

Reaction score: 35
Messages: 56

You might also try using ShellCheck to see if it helps:

Code:
if [ -x pkg query %C $port | grep kde ] ; then echo $port >> kde_ports.txt
^-- SC1009: The mentioned syntax error was in this if expression.
   ^-- SC1073: Couldn't parse this test expression. Fix to allow more checks.
            ^-- SC1072: Expected test to end here (don't wrap commands in []/[[]]). Fix any mentioned problems and try again.
 
OP
astyle

astyle

Daemon

Reaction score: 626
Messages: 1,421

It may help you to know that [ -x path/to/file ] is not shell syntax; it's another way to write test -x path/to/file, except the ']' is required if you use the '[' form. Here's what you're really executing:
Bash:
test -x pkg query %C $port | grep kde ]
You might consider switching to test(1) instead of '[' until you feel more comfortable with shell scripting and the 'test' command. It's easier to understand the behavior of a script written with an explicit 'test' command, and that also makes things easier to debug:
Bash:
$ port=misc/hicolor-icon-theme
$ test -x pkg query %C $port | grep gnome
test: pkg: unexpected operator
$ test -x pkg query
test: pkg: unexpected operator
$ test -x pkg
$ echo $?
1
$ ls ./pkg
ls: ./pkg: No such file or directory

To be honest, Vull got it right:

You want to test if there is any output from pkg-query.

But I'm still picking up some good ideas here. :)
 

vigole

Daemon

Reaction score: 1,546
Messages: 1,382

An error handling is missing! This a generic error handling. It's portable (POSIX shell, everywhere). You can use in any script, to separate output and error messages.
Bash:
1: #!/bin/sh
2: error=${TMPDIR:-/tmp}/error.$$
3: if true; then
4:        BAD!
5: fi 2> "$error"

Line 2:

:-
IF TMPDIR is set THEN use it.
IF TMPDIR is unset or null THEN set it to /tmp

$$ the process ID of the shell.

Lines 3,5:
Am example of a block of code.

Line 4:
An error!

Line 5:
2> Redirect the entire error to the /tmp/error.PID

2
is the descriptor 2, aka standard error.
It's unbufered and by default shows up in standard output.
Standard output, i,e, descriptor 1 is buffered. As a result, error may come before the actual output.
By redirecting standard error to a file, you can separate the output and error messages.

Reference:
mktemp(1), sh(1), hier(7)
 
OP
astyle

astyle

Daemon

Reaction score: 626
Messages: 1,421

Got a different problem, but along the same lines...

I have a script, mydiff.sh:
Code:
#!/bin/sh
for port in `cat portlist_b_01.txt`;
do
if [ "$(cat kde_qt_ports.txt | grep $port)" ] 
    then echo "YES"
    else echo $port 
fi
done
That works, but, I'd like to pass abitrary filenames to it like this in mydiff2.sh:
Code:
#!/bin/sh
for port in $1;
do
if [ "$(cat $2 | grep $port)" ] 
    then echo "YES"
    else echo $port 
fi
done
I may be doing something wrong with my sh code with those $1 and $2 variables, but I'm not sure what. The script mydiff2.sh spits out this:
Code:
root@beastie:/var/db/portsnap/help/lists # ./mydiff2.sh all_qt_ports.txt installed_b_01.txt | less
all_qt_ports.txt
root@beastie:/var/db/portsnap/help/lists #
 

Vull

Aspiring Daemon

Reaction score: 518
Messages: 820

Three things:

1. You have an unnecessary pipe. grep $port kde_qt_ports.txt is easier-to-read, more terse, straight-forward, and efficient, than cat kde_qt_ports.txt | grep $port

2. You really only need one script, with argument substitution, but only when command-line arguments are missing.

3. You could do with a little more error checking.

Try this:
Code:
#!/bin/sh
# yourdiff - check port names in file1: are they included in file2?

#   l o c a l   f u n c t i o n s

Badname() {
  echo $sfn': File not accessible: "'$1'"' >&2
  exit 1
}
Syntax() {
  echo $sfn': Bad syntax:' $1 >&2
  exit 1
}

#   m a i n   l o g i c

sfn=${0##*/} #--- this script file's name

#--- check command line argument(s)

if [ "$1" ]&&[ ! "$2" ]; then
  #--- bad command syntax: need 0 or 2 filename arguments
  Syntax "Only one filename specified"

elif [ "$1" ]; then
  #--- substitute command line arguments for hard-coded filenames
  file1=$1
  file2=$2

else #--- supply hard-coded filenames
  file1=portlist_b_01.txt
  file2=kde_qt_ports.txt
fi

#--- check for missing filenames
if [ ! -f $file1 ]; then Badname $file1; fi
if [ ! -f $file2 ]; then Badname $file2; fi

#--- check names in file1: if in file2, echo "YES", else echo problem port name
for port in $(cat $file1); do
  if [ "$(grep $port $file2)" ]; then
    echo "YES"
  else
    echo $port
  fi
done
 

Vull

Aspiring Daemon

Reaction score: 518
Messages: 820

astyle, A correction: where I previously wrote redirects as 2>&1, it should have been >&2 -- this is intended to redirect error messages to standard error. I have now included those corrections in 2 places in my previous post. Sorry! My mistake.
 

gpw928

Aspiring Daemon

Reaction score: 266
Messages: 593

The test
Code:
if [ "$(grep $port $file2)" ]
is catching the output of grep into a temporary shell variable for the sole purpose of determining if the variable is an empty string.
That's awkward, and, depending on the data presented, could potentially blow up the shell script on memory usage.
A much more satisfactory mechanism is to run grep silently, and test the exit status of grep:
Code:
if grep -q "$port" "$file2"
Note that I have quoted all shell variables to prevent confusion when expanding arguments (typically from the presence of null strings, and Input Field Separators).
 
OP
astyle

astyle

Daemon

Reaction score: 626
Messages: 1,421

Good thing I named the thread well! :p

'Cuz I have yet another sh-related question: I have the official listing of KDE's Plasma 5 tarballs. I want to go down this list, and to check if a port's distinfo file mentions the tarball. If it does, I want to spit out a line that references the port itself, like "x11/plasma5-plasma-desktop". To this end, I came up with some code that borrows on ideas earlier in this thread, but I'm stumped... Any help would be appreciated!
Code:
#!/bin/sh
for tarball in `cat plasma_tar-1.txt`;
do
if [ "$(find /usr/ports/ -name distinfo -exec grep $tarball {} ";" )" ]  
  then echo port_name #  like "x11/plasma5-plasma-desktop"
  else # continue the loop.  
fi
done
 
OP
astyle

astyle

Daemon

Reaction score: 626
Messages: 1,421

Good thing I named the thread well! :p

'Cuz I have yet another sh-related question: I have the official listing of KDE's Plasma 5 tarballs. I want to go down this list, and to check if a port's distinfo file mentions the tarball. If it does, I want to spit out a line that references the port itself, like "x11/plasma5-plasma-desktop". To this end, I came up with some code that borrows on ideas earlier in this thread, but I'm stumped... Any help would be appreciated!
Code:
#!/bin/sh
for tarball in `cat plasma_tar-1.txt`;
do
if [ "$(find /usr/ports/ -name distinfo -exec grep $tarball {} ";" )" ] 
  then echo port_name #  like "x11/plasma5-plasma-desktop"
  else # continue the loop. 
fi
done
Solved it myself... silly me. 😅 I had the solution in my notes all along, just needed to re-frame the problem!
Code:
#!/bin/sh
for tarball in `cat plasma_tar-1.txt`;
do
 find /usr/ports/clean_backup -name distinfo -exec grep $tarball {} "+" | grep SIZE | cut -d/ -f5-6 >> all_plasma5_ports_2.txt; 
done

Sometimes, I gotta take a break, and allow myself to get distracted.
 
Top