Shell Override shebang from rc script?

S

SamKook

I've been trying to override the shebang when starting a service that's a python script but the shebang takes priority over command_interpreter when using run_rc_command apparently.

I get a warning that the command_interpreter value is not equal to the shebang and the shebang version is used(which fails because of the too recent version of sqlite3 and the way the script was made, which requires python 3.12 to fix).

Is that intended behavior and can I get around it while still using the built in rc stuff?

For now I just changed the shebang to the specific non-default python version I need to use for it but I'll have to repeat that change every time that file gets updated which isn't ideal and I'd rather not have to write a full rc script to handle things manually.
 
Didn't think it mattered since I'm asking for the general behavior of rc scripts relying on rc.subr and not my service specifically, but it's sickgear.

Has "#!/usr/bin/env python" as the first line of the python script and I have "command_interpreter="/usr/local/pyenv/versions/3.12.2/bin/python"" in the rc.d script but it's using system python instead of the 3.12.2 version I specify when it executes "run_rc_command "$1"".
 
Here's an easy example of what I mean that anyone can try.

Here's a script called EchoShell that display the interpreter used to run it:
sh: 
#!/bin/sh
pid=$$
interpreter=$(ps -o comm= -p $pid)
echo "The script is executed by: $interpreter"

And here's the echo_shell startup script in rc.d:
sh: 
#!/bin/sh

# PROVIDE: EchoShell
# REQUIRE: LOGIN
# KEYWORD: shutdown

. /etc/rc.subr

name="EchoShell"
rcvar="${name}_enable"

load_rc_config $name

: ${EchoShell_enable:="NO"}
: ${EchoShell_path:="/usr/local"}

command="${EchoShell_path}/EchoShell"
command_interpreter="/usr/local/bin/zsh"

run_rc_command "$1"

If I run this command in a shell:
Code: 
service echo_shell onestart
I get this output:
Code: 
/usr/local/etc/rc.d/echo_shell: WARNING: $command_interpreter /usr/local/bin/zsh != /bin/sh
Starting EchoShell.
The script is executed by: sh

The output I'm hoping/expecting to get is(zsh instead of sh):
Code: 
/usr/local/etc/rc.d/echo_shell: WARNING: $command_interpreter /usr/local/bin/zsh != /bin/sh
Starting EchoShell.
The script is executed by: zsh


What I'm asking is if it is normal to get sh instead of zsh when the interpreter is specified like above. Seems like an inconsistent behavior to me since when you specify the interpreter in a shell, it uses that over the internal shebang line but there might be a good reason why it doesn't.
If it is indeed the expected behavior, I'm then asking if there is a way to get the behavior I expect with rc.subr by setting a switch or something.

I can get my expected behavior either by changing the first line of EchoShell to this:
sh: 
#!/usr/local/bin/zsh

or by changing the run_rc_command line in the startup script to this:
sh: 
${command_interpreter} ${EchoShell_path}/EchoShell

but I would prefer to stick with the rc.subr convention.
 
Reading the rc.subr script (on stable/14), in the comment of run_rc_command there is:
Code: 
# run_rc_command argument
  (snip)
#       command_interpreter n   If not empty, command is interpreted, so
#                               call check_{pidfile,process}() appropriately.

It seems that command_interpreter variable is only used in checking pid file and process. If you want to invoke the script with the interpreter specified, it seems that you need to explicitly write to do so.
 
SamKook said:
Has #!/usr/bin/env python as the first line of the python script
Click to expand...
That would be wrong. That should be #!/usr/bin/env python3.9 or python3.12, or whichever Python version is set as the default or what Python flavor was used.
 
SirDice said:
That would be wrong. That should be #!/usr/bin/env python3.9 or python3.12, or whichever Python version is set as the default or what Python flavor was used.
Click to expand...
But what do you do when multiple versions are supported like 3.9 to 3.12?
Might be proper to specify an exact version in theory but in practice, the OS is more likely to have a symlink of python to whatever version is default and that is likely to be one of the many supported so for people who don't know how things work, it'll work fine without them having to do anything or install another version of python they do not need.
 
Actually, when this is about porting and you have to fiddle with a shebang anyways, adding this /usr/bin/env-hack there makes little sense. USES=shebangfix actually has it in its default patterns to just replace it with the hardcoded path.
 
Hiroo Ono said:
Reading the rc.subr script (on stable/14), in the comment of run_rc_command there is:
Code: 
# run_rc_command argument
  (snip)
#       command_interpreter n   If not empty, command is interpreted, so
#                               call check_{pidfile,process}() appropriately.

It seems that command_interpreter variable is only used in checking pid file and process. If you want to invoke the script with the interpreter specified, it seems that you need to explicitly write to do so.
Click to expand...
Seemed weird so I had a look and yeah, that's basically it. Seems to be a string instead of a bool just in case it's a pearl thing:
Code: 
#    If interpreter != ".", read the first line of procname, remove the
#    leading #!, normalise whitespace, append procname, and attempt to
#    match that against each command, either as is, or with extra words
#    at the end.  As an alternative, to deal with interpreted daemons
#    using perl, the basename of the interpreter plus a colon is also
#    tried as the prefix to procname.
 
SirDice said:
That's what flavors are for.

docs.freebsd.org

Chapter 7. Flavors

Flavors are a way to have multiple variations of a port
docs.freebsd.org docs.freebsd.org
Click to expand...

zirias@ said:
Actually, when this is about porting and you have to fiddle with a shebang anyways, adding this /usr/bin/env-hack there makes little sense. USES=shebangfix actually has it in its default patterns to just replace it with the hardcoded path.
Click to expand...
This isn't a port though, it's just a github source so you can't really do that from just there unless you actually port it.
 
SamKook said:
This isn't a port though
Click to expand...
Your first post seemed to suggest this. That's why I asked which service it specifically was. Because the issue with regards to the Python version should be taken care of by the port and flavors. If the port doesn't work with Python 3.12 for example you should be able to install the Python 3.11 flavor of that port. The 'automatic' shebang fix adjusts the line accordingly.
 
In this particular case, the problem is the opposite, it works with 3.12 and nothing else on freebsd because of the system sqlite3 version that drops supports for double quotes for something(forgot the details). There's a bypass for that with python 3.12 but freebsd doesn't have python 3.12 yet so I can't switch the system default to it and had to install it with pyenv.

Even if I made it into a port myself, I wouldn't be able to make it work.

I expected to be able to override the default interpreter from the rc script but it's apparently impossible without rewriting a lot of the script since it relies on rc.subr which I find odd but seems to be intended to be that way.
 
SamKook said:
I expected to be able to override the default interpreter from the rc script but it's apparently impossible without rewriting a lot of the script since it relies on rc.subr which I find odd but seems to be intended to be that way.
Click to expand...
I've seen the XY-problem mentioned and this is indeed one.

There's no way to ever "override" a shebang. It's interpreted by the kernel when trying to exec() a file. But this most likely isn't what you want anyways. Shebangs are for scripts. All they do is cause the kernel to load and run the script interpreter instead and feed it the original arguments. So, you can just directly invoke the script interpreter you want instead, putting the script and its arguments, well, as arguments.
 
zirias@ said:
So, you can just directly invoke the script interpreter you want instead, putting the script and its arguments, well, as arguments.
Click to expand...
That is what I meant by override the shebang, probably a poor choice of word on my part, but I can't invoke the interpreter directly and still use rc.subr from what I understand which is the issue I'm having.

I assumed setting command_interpreter would make rc.subr invoke that specific interpreter directly but it's not the case.
 
SamKook well the obvious thing to do would still be to fix that shebang, I don't understand the issue yet why this isn't an option here. But then, you can also "misuse" rc.subr and put your python interpreter as the command and the script as the first entry in args ...
 
It's a public git repo(that I'm not a dev for) so every time the script gets updated and I do a pull request to upgrade, the shebang line will get reset or prevent me from upgrading(I forget how git behaves in this case).
It's what I did since I discovered the command_interpreter var didn't behave as I or the devs of that project expected and started looking into a way to fix it that would work for the project without constant manual modifications on the user part.

As far as I can tell, that means ditching rc.subr entirely or misusing it like you explained above which does work but I'm not sure it's a great option for something official(especially since the python interpreter in the rc script should be optional).
 
There are lots of ways to automate that change, you could e.g. just script it, or if you use a git clone directly, you could just use your own local branch and rebase that onto upstream (which won't ever conflict unless THEY change the shebang).

I'm also not sure what you mean by "official". Doesn't sound very "official" to me. Maybe they should fix whatever issue makes them depend on one specific python version...
 
There are lots of ways for a user who know what they're doing to deal with it, yes, but I was looking for a solution that would work for everyone and not just me.

By official I meant something good for general public release. Modifying the interpreter is a temporary solution and people only install the init script once so it would cause issues in the future to do it that way.

I'm sure they will fix the issue at some point, but for now, freebsd is pretty much the only OS affected by the problem(from what I read at least) since python doesn't ship(yet at least) with a problematic(for them) sqlite3 version and it is a very recent problem(even though it's been expected this would happen for about a year or so on freebsd if I remember right). The quick fix is to use python 3.12+ but with an older sqlite3 version, it'll support many more.

I started this thread because I(and the dev I was talking to about this) was very surprised that rc.subr has a specific parameter for the interpreter but completely ignores it when it's time to execute the service. It seemed like a bug to me so I wanted to verify if that was the case or not and find the right way to do it and I now see it is the expected behavior and there is no implemented way around it in rc.subr that keeps specifying it optional.
 
You must log in or register to reply here.
Back
Top