Shell Bash escape and command substitution

[marypoppins]

Dear All,

I have a hard struggle with understanding how it works. From the bash manual

3.1.1 Shell Operation

The following is a brief description of the shell’s operation when it reads and executes a command. Basically, the shell does the following:

1. Reads its input from a file (see Shell Scripts), from a string supplied as an argument to the -c invocation option (see Invoking Bash), or from the user’s terminal.
2. Breaks the input into words and operators, obeying the quoting rules described in Quoting. These tokens are separated by metacharacters. Alias expansion is performed by this step (see Aliases).
3. Parses the tokens into simple and compound commands (see Shell Commands).
4. Performs the various shell expansions (see Shell Expansions), breaking the expanded tokens into lists of filenames (see Filename Expansion) and commands and arguments.
5. Performs any necessary redirections (see Redirections) and removes the redirection operators and their operands from the argument list.
6. Executes the command (see Executing Commands).
7. Optionally waits for the command to complete and collects its exit status (see Exit Status)

1. 
   and from Command Substitution:
   

   [*]When the old-style backquote form of substitution is used, backslash retains its literal meaning except when followed by ‘$’, ‘`’, or ‘\’.
   

I have two commands from the Advanced Bash-Scripting Guide, which I can not understand how
echo `echo \\z`

1) From shell operation 2. point it breaks tokens first:
echo + `echo \\z`
2) Command Substitution inside text into a separate shell:
echo \\z
3) Again the 2. point from the shell operation tokenizing (with quoting):
echo + \\z (\+\ = \ ; plus the z remains)
4)This results:
feed echo with \z => \z
5) Back to the substitution, but point 8 said that in the old style quoting the backslash remains if it follows letter z
6) so after substitution it looks:
echo \z
7) After command substitution yet another tokenizing (with quoting):
echo + z
8) results: feed echo with z, which is good
z
However the

echo `echo \\\z`
1) From shell operation 2. point it breaks tokens first:
echo + `echo \\\z`
2) Command Substitution inside text into a separate shell:
echo \\\z
3) Again the 2. point from the shell operation tokenizing (with quoting):
echo + \\\z (left two \+\ = \ ; plus the \z remains just z)
4)This results:
echo feed with \z => \z

from this point is this the same as above, however its result all in all is:
\z

Could somebody correct me what is wrong with my knowledge?
thank you very much

 

[Martin Paredes]

In the second example echo `echo \\\z` , the firts 2 backslash => \, the rest \z => \z (backslash retains its literal meaning)

So, `echo \\\z` => \\z

The second echo is echo \\z, the 2 backslash => \, and the command print \z

 

[marypoppins]

Thank you for the answer. So if I'm right the steb by step execution is this: (sorry this will be longer with a little doc)


Shell Operation
Basically, the shell does the following:
SO1) Reads its input from a file (see Shell Scripts), from a string supplied as an argument to the -c invocation option (see Invoking Bash), or from the user’s terminal.
SO2) Breaks the input into words and operators, obeying the quoting rules described in Quoting. These tokens are separated by metacharacters. Alias expansion is performed by this step (see Aliases).
SO3) Parses the tokens into simple and compound commands (see Shell Commands).
SO4) Performs the various shell expansions (see Shell Expansions SE), breaking the expanded tokens into lists of filenames (see Filename Expansion) and commands and arguments.
SO5) Performs any necessary redirections (see Redirections) and removes the redirection operators and their operands from the argument list.
SO6) Executes the command (see Executing Commands).
Optionally waits for the command to complete and collects its exit status (see Exit Status).

Shell Expansions
The order of expansions is:
SE1 )brace expansion;
SE2) tilde expansion, parameter and variable expansion, arithmetic expansion, and command substitution (if os supported process expansion) (done in a left-to-right fashion);
SE3) word splitting;
SE4) and filename expansion.
SE5) After these expansions are performed, quote characters present in the original word are removed unless they have been quoted themselves (quote removal).

For example step by step:
(- When the old-style backquote form of substitution is used, backslash retains its literal meaning except when followed by ‘$’, ‘`’, or ‘\’.
- \z is not a control character )

So: echo `echo \\\z`

1) SO2: echo + `echo \\\z` (two token, token1: echo; token2: `echo \\\z`, no quoting transformations in the command substitution)
2) Command substitution start with subshell and with SO2 again on echo \\\z
3) In the subshell SO2: the input 'echo + \\\z' become two another tokens 'echo + \z' (token1: echo ; token2 \z [thanks to quoting rule in SO2 : \\\z => \+\ and \+z => \z] )
4) In the subshell the results will be \z which goes back to the main shell as \z (because the old-style-backquote rule)
if the subshell result would be \\z it would go back as \z ?
5) The main shell tokens change to echo + \z (token1:echo; token2: \z)
6) There is no extra quoting transformations, it is already done in the point 1), so \z remains \z
7) Results \z


In case of: echo `echo \\\\z`
1) SO2: echo + `echo \\\\z` (two token, token1: echo; token2: `echo \\\\z`, no quoting transformations in the command substitution)
2) Command substitution start with subshell and with SO2 again on echo \\\\z
3) In the subshell SO2: the input 'echo + \\\\z' become two another tokens 'echo + \\z' (token1: echo ; token2 \\z [thanks to quoting rule in SO2 : \\\\z => \+\ and \+\ and z => \\z] )
4) In the subshell the results will be \\z which goes back to the main shell as \z (\\z =>\+\ and z => \z)
5) The main shell tokens change to echo + \z (token1:echo; token2: \z)
6) There is no extra quoting transformations, it is already done in the point 1), so \z remains \z
7) Results \z

While in case of new style the differences in the point 4)
In case of: echo $(echo \\\\z)
1) SO2: echo + $(echo \\\\z) (two token, token1: echo; token2: $(echo \\\\z), no quoting transformations in the command substitution)
2) Command substitution start with subshell and with SO2 again on echo \\\\z
3) In the subshell SO2: the input 'echo + \\\\z' become two another tokens 'echo + \\z' (token1: echo ; token2 \\z [thanks to quoting rule in SO2 : \\\\z => \+\ and \+\ and z => \\z] )
4) In the subshell the results will be \\z which goes back to the main shell as \\z (because the new style does change nothing)
5) The main shell tokens change to echo + \\z (token1:echo; token2: \\z)
6) There is no extra quoting transformations, it is already done in the point 1), so \\z remains \\z
7) Results \\z

Is there any misstake in my train of thought?
thank you very much

 

[Martin Paredes]

The backquote is doing some estrange things, it isn't doing what I expect

Maybe it is more clear if you run bash -xv and type your examples

The meaning of -xv is in The Set Builtin

$ echo `echo \\\\\\z`
echo `echo \\\z`
+ echo '\z' <= six backslash get one
+ echo '\z'
\z

 

[aht0]

Are you sure you are dealing with bash shell to start with and not (t)csh shell? Because unlike Linux, to get bash on FreeBSD you would have to specifically install it.. it may be stupid question but ages a go, after migrating from Linux, I managed to make such an error (tried to run bash scripts on a C-shell)

 

[marypoppins]

It is bash. I installed it because it has a lot of nice features. I can forge scripts, but sometimes I stuck in a problem and now I got fed up to search the solutions on the net. Instead google, I read the manual character by character and want to understand how it works inside.
The three examples in my previous post have correct results (it is from the advanced bash scripting guide page 48 quoting ). I just would like to know if my step by step details is logically correct or not.
Thank you for the shell built-in tips.