Shell The /bin/sh $LINENO value just after a nested function.

jambock.84

jambock.84

Considering the following excerpt:
Code: 
 1  #!/bin/sh
 2
 3  check()
 4  {
 5      echo $1 - \
 6      $(
 7          [ $2 -eq $3 ] \
 8              && echo OK \
 9              || echo FAIL "($2 != $3)"
10      )
11  }
12
13  main()
14  {
15      check 2 $LINENO 3
16
17      inner()
18      {
19          check 4 $LINENO 3
20      }
21
22      check 3 $LINENO 10
23      inner
24  }
25
26  check 1 $LINENO 26
27  main
On FreeBSD 13.0-RELEASE-p11 I'm getting the following output:
Rich (BB code): 
# ./lineno 
1 - OK
2 - OK
3 - FAIL (22 != 10)
4 - OK
Is this a (known) bug or am I missing something?
Thanks in advance.
 
It looks like a bug to me. I also tried your test with ksh, bash, and zsh. Ksh and bash gave the same answers (but different to /bin/sh). Zsh was different again. All three were "correct" depending on how you read the manpage on LINENO.
 
POSIX says that LINENO counts, starting at 1, within the script or function. Which implies that you just found a bug, assuming that the shell tries to be POSIX compliant. Have you opened a PR?
 
Think of it as a C's __LINE__ macro.

According to the IEEE Std 1003.1-2008 (“POSIX.1”):

LINENO:
"Set by the shell to a decimal number representing the current sequential line number (numbered starting with 1) within a script or function before it executes each command. If the user unsets or resets LINENO, the variable may lose its special meaning for the life of the shell. If the shell is not currently executing a script or function, the value of LINENO is unspecified. This volume of POSIX.1-2017 specifies the effects of the variable only for systems supporting the User Portability Utilities option."

* which states "within _a_ script or function",
* which implies _one_ script or _one_ function,
* hence, in my opinion there's no contradiction.

I think that's why infamous bash provides a set of auxiliary variables,
e.g. FUNCNAME, BASH_SOURCE, BASH_LINENO
which by the way, I'm not familiar with.

P.S.

sh(1) in
* FreeBSD is supposed to be close to IEEE Std 1003.1 ("POSIX.1"), and
* OpenBSD is compliant with the IEEE Std 1003.1-2008 (“POSIX.1”), and
* and there's also MSYS2/CYGwin shebang
....
anyway, to sum up: in all three cases the result is the same.

See: https://pubs.opengroup.org/onlinepubs/9699919799/utilities/V3_chap02.html
 
On the face value it looks like a bug, in that it would appear the LINENO is losing context when there's a fork within a function to run another function.

On the other hand, I would argue such a construct as the demonstrated example is needlessly convoluted and you get what you deserve. ;)
 
Your replies are very much appreciated, thank you!

IMHO I think it's a bug although of that kind that bothers only if you put the thing through its paces. And with respect of "putting the thing through its paces" it doesn't necessarily means being "convoluted". ;) Again IMHO I think that the capability of nesting functions is great and scoping seems to work fine! The only flaw I've hit is this $LINENO issue.

As I have noted it seems to only happen after a nested function, most probably because the internal logic around $LINENO (by mistake or deliberately, who knows?) "forgets" to note "where exactly it is" upon reaching the end of a function body (which it always consider as never nested).

I also think this may be that kind of bug which is unlikely to be fixed on a short term, thus I have worked on a personal "work-around" which seems to be "good enough" (due to the obvious constrains) for those like me interested on having the right/expected behavior. This "work-around" relieves me from resorting to some other shell or language; I've learned to appreciate the power of /bin/sh specially the implementation provided by FreeBSD.

I believe my work-around involves minimal overhead. It only requires "kind of decorating" the function definitions of interest and instead of using the plain $LINENO elsewhere within the function body, wrap it with another fixing helper function. The additional machinery I've crafted for the work-around is:

Code: 
# Just for machinery.
LINE_BEFORE_script=0

# To be understood as AT LINE $2 OF SCOPE $1
AT() echo $(( $2 < LINE_BEFORE_${1} ? $2 : $2 - LINE_BEFORE_${1} ))

# To be understood as FUNCTION $1 AT LINE $3 OF PARENT_SCOPE $2.
# MUST BE declared on the immediately preceding line of the function being defined.
FUNCTION() setvar LINE_BEFORE_${1} $(( LINE_BEFORE_${2} + $( AT $2 $3 ) ))

For the sake of simplicity, to keep focused on my point, in the following sample script I've kept everything on a single source-file:

Code: 
  1 #!/bin/sh
  2
  3 # Dummy comment.
  4
  5 print() printf "%*s : %03d\n" $1 $2 $3
  6
  7 LINE_BEFORE_script=0
  8
  9 AT() echo $(( $2 < LINE_BEFORE_${1} ? $2 : $2 - LINE_BEFORE_${1} ))
 10
 11 FUNCTION() setvar LINE_BEFORE_${1} $(( LINE_BEFORE_${2} + $( AT $2 $3 ) ))
 12
 13 FUNCTION main script $LINENO
 14 main()
 15 {
 16     echo -e
 17     print 10 main $( AT main $LINENO )
 18
 19     FUNCTION A1 main $LINENO
 20     A1()
 21     {
 22         print 20 A1 $( AT A1 $LINENO )
 23
 24         A11()
 25         {
 26             print 30 A11 $LINENO
 27         }
 28
 29         print 20 A1 $( AT A1 $LINENO )
 30
 31         A12()
 32         {
 33             print 30 A12 $LINENO
 34         }
 35
 36         print 20 A1 $( AT A1 $LINENO )
 37
 38         A12
 39         A11
 40
 41         print 20 A1 $( AT A1 $LINENO )
 42     }
 43
 44     print 10 main $( AT main $LINENO )
 45
 46     FUNCTION A2 main $LINENO
 47     A2()
 48     {
 49         print 20 A2 $( AT A2 $LINENO )
 50
 51         A21()
 52         {
 53             print 30 A21 $LINENO
 54         }
 55
 56         print 20 A2 $( AT A2 $LINENO )
 57     }
 58
 59     A2
 60
 61     print 10 main $( AT main $LINENO )
 62
 63     FUNCTION A3 main $LINENO
 64     A3()
 65     {
 66         print 20 A3 $( AT A3 $LINENO )
 67     }
 68
 69     A1
 70     A3
 71
 72     print 10 main $( AT main $LINENO )
 73 }
 74
 75 main
 76

And here's its output:

Code: 
# ./line_numbers

      main : 004
      main : 031
                  A2 : 003
                  A2 : 010
      main : 048
                  A1 : 003
                  A1 : 010
                  A1 : 017
                           A12 : 003
                           A11 : 003
                  A1 : 022
                  A3 : 003
      main : 059

Of course, it's up to the programmer to use FUNCTION and AT as expected in the way shown above.

I think this is it, at least for me, so far ;).
🟩
 
ralphbsz said:
POSIX says that LINENO counts, starting at 1, within the script or function. Which implies that you just found a bug, assuming that the shell tries to be POSIX compliant. Have you opened a PR?
Click to expand...
Please, where can I find a how-to for opening a PR?
 
Interestingly on SH(1) the only mentions to function nesting and scope seems to be:
... The shell uses dynamic scoping, so that if the variable x is made local to function f, which then calls function g, references to the variable x made inside g will refer to the variable x declared inside f, not to the global variable named x. ...
Click to expand...
and
... It terminates the current executional scope, returning from the closest nested function or sourced script; ...
Click to expand...
 
UPDATE:

Of course I believe that the original machinery I've have proposed above should be improved; for instance:
  1. The function names can contain special characters, such as, driver:create:dataset().
    This helps minimize the possibility of name clashing on a large script project composed of multiple source-files.
  2. There should be some help in finding errors during "active development" when frequent changes take place.
    It could happen to forget updating a function "decoration" after changing the function name.
Unfortunately, there seems to be no optimum/spectacular solution, nevertheless I believe the following improvements could be considered:

Code: 
__BEFORE_script=0

__BEFORE()
{
    local before=$1

    filter#01() printf "%s_%s"  $1 $(shift; echo $*)
    filter#02() printf "%s__%s" $1 $(shift; echo $*)
    # and so on...

    [ "$before" != "${before##*:}" ] && before=$( IFS=': ' filter#01 $before )
    [ "$before" != "${before##*.}" ] && before=$( IFS='. ' filter#02 $before )
    # and so on...

    echo __BEFORE_$before
}

__CHECK()
{
    : ${1:?AT() argument #1 should be set to a function name in scope.}
    : ${2:?AT() argument #2 should be set to \$LINENO.}

    eval ": \${$before:?AT() argument #1 <$1> seems suspicious.}"
}

AT()
{
    local before=$( __BEFORE $1 )

    __CHECK $*
    echo $(( $2 < $before ? $2 : $2 - $before ))
}

FUNCTION() readonly $( __BEFORE $1 )=$(( $( __BEFORE $2 ) + $( AT $2 $3 ) ))
🟩
 
UPDATE 2:

Looking back I think that __BEFORE() could be further improved, specially in regard to its efficiency, so I revised it as follows:

Code: 
__BEFORE()
{
    filter()
    {
        name=$1
        shift

        while [ $1 ]
        do
            name=$name$FIX$1
            shift
        done
    }

    expand() filter $*

    fix()
    {
        [ "$1" != "${1##*$2}" ] &&
            IFS="$2" FIX="$3" expand $1
    }

    local name=$1

    fix "$name" ':' '_'
    fix "$name" '.' '__'

    echo __BEFORE_$name
}

I'm inclined to believe this implementation is better because I think (without having profiled) it's more efficient for not making supposedly more expensive calls to printf and it seems more maintainable (specially because fix() makes it easier and clearer to notice what fixes are being applied for deriving a variable-name from a function-name).
🟩
 
jambock.84 said:
Please, where can I find a how-to for opening a PR?
Click to expand...

I could see that another $LINENO bug which I already knew about has already been filed; it seems a solution hasn't already been found or agreed upon.

With respect to how to report a bug I think that the right thing to do has already been explained:
  1. Search if something exact or similar has already been submitted.
  2. If really needed, then file the new bug.
There are two references to read before submitting anything:
  1. Writing FreeBSD Problem Reports
  2. Problem Report Handling Guidelines
 
You must log in or register to reply here.
Back
Top