Solved How to call an external .sh file from within python in FreeBSD?

iSh0w

Member


Messages: 34

Hey

My first post here and hoping to get a solution to my problem.

I have a query regarding launching an external bash script file (.sh) in FreeBSD from within a Python script.

For Linux I used:
Code:
os.system(‘sh ‘+ filepath)
For Mac:
Code:
os.system(‘open ‘+ filepath)
And for windows:
Code:
os.startfile(filepath)

I am unable to get any of these to work for FreeBSD. I know startfile is only for Windows, however was wondering if there was an equivalent for FreeBSD without using subprocess.

Or if not possible at all how to use subprocess to call a external script.

Any help would be appreciated

What I have tried thus far:

Code:
process = subprocess.Popen(['/bin/sh','-c',str(filepath)], stdout=subprocess.PIPE)out= process.communicate()

Which runs without throwing any errors but does not call the .sh file!

Tried the code below as well, runs without errors but does not launch the file
Code:
subprocess.call(str(filepath), shell=True)

Double checked the file that is to be called has been given chmod 755 permissions, making it executable, furthermore, removed the .sh and tried to call, no joy. Although double clicking the open or open.sh file does bring up a dialog stating: "Do you want to run open.sh, or display its contents? open.sh is an executable text file." At which point I hit "Run in Terminal" or "Run" producing the same effect or executing the script flawlessly. (Have MATE GUI installed on FreeBSD)

Yes, I did initially use the hash-bang
Code:
#!/bin/env bash
however whilst researching FreeBSD found a solution stating that
Code:
#!/bin/sh
needs to be used, changed it, no difference.

The plugin(where the Python code resides and runs) has been given(along with all pre-existing plugins that already have the permission) drwxr-xr-x. (
Code:
first run chown plex:plex xyz.bundle and then chmod 755 xyz.bundle
)

When debugging the same for Linux, found the permissions to be the culprit, until the correct permissions were given to the plugin.bundle and the .sh file, nothing would run, hence double checking permissions but maybe I am doing something wrong?

Just to confirm the path/to/file.sh was correct and it in fact could be called, used import io and wrote something into the .sh file
Code:
f = io.open(filepath,'wb')
f.write('bingo')

It wrote bingo into the open.sh file successfully. So the path and permissions seem ok...
 

tobik@

Daemon
Developer

Reaction score: 1,423
Messages: 1,909

Code:
subprocess.Popen(["bash", "/path/to/file.sh"])
Add your own options. This should work on Linux, Mac OS X and FreeBSD.

Try to avoid os.system if you can. Read OpenBSD's man page for system(3) as to why (look at the CAVEATS section).
 
OP
iSh0w

iSh0w

Member


Messages: 34

Thanks, trying to debug your
Code:
subprocess.Popen(["bash", "/path/to/file.sh"])
Add your own options. This should work on Linux, Mac OS X and FreeBSD.

Try to avoid os.system if you can. Read OpenBSD's man page for system(3) as to why (look at the CAVEATS section).

Thanks, but trying to debug the code you suggested keep getting
Code:
OSError: [Errno 2] No such file or directory

Although, double checked the path.
Maybe bash cannot be found? As suggested by wblock@

/iSh0w
 

wblock@

Beastie Himself
Developer

Reaction score: 3,740
Messages: 13,851

Does pkg info -ix bash show that it is installed?

For that matter, don't use Bash by default. Use /bin/sh unless you really need Bash functions. And have installed it, of course.
 
OP
iSh0w

iSh0w

Member


Messages: 34

Does pkg info -ix bash show that it is installed?

For that matter, don't use Bash by default. Use /bin/sh unless you really need Bash functions. And have installed it, of course.

Thanks, that command returned
Code:
bash-4.3.39_2
which means it is installed, I don't need bash functions so will try to stick to /bin/sh however no joy thus far :/

/iSh0w
 

wblock@

Beastie Himself
Developer

Reaction score: 3,740
Messages: 13,851

Works here in the interactive Python shell:
Code:
% python
Python 2.7.10 (default, Jul  4 2015, 19:44:00)
[GCC 4.2.1 Compatible FreeBSD Clang 3.4.1 (tags/RELEASE_34/dot1-final 208032)] on freebsd10
Type "help", "copyright", "credits" or "license" for more information.
>>> import os
>>> os.system('sh ' + '~/tester.sh')
f.src
0
>>>

subprocess.Popen works also, but requires an explicit path rather than the ~.
 

tobik@

Daemon
Developer

Reaction score: 1,423
Messages: 1,909

With subprocess.Popen this would be something like:
Code:
import subprocess
import os
subprocess.Popen(["sh", os.path.expanduser("~/tester.sh")]).wait()
 
OP
iSh0w

iSh0w

Member


Messages: 34

Works here in the interactive Python shell:
Code:
% python
Python 2.7.10 (default, Jul  4 2015, 19:44:00)
[GCC 4.2.1 Compatible FreeBSD Clang 3.4.1 (tags/RELEASE_34/dot1-final 208032)] on freebsd10
Type "help", "copyright", "credits" or "license" for more information.
>>> import os
>>> os.system('sh ' + '~/tester.sh')
f.src
0
>>>

subprocess.Popen works also, but requires an explicit path rather than the ~.

Strange right, the open.sh when opened in terminal runs flawlessly even on double click and running the file.
Somehow, when the code is used within a python file it does not call/run the file!

/iSh0w
 
OP
iSh0w

iSh0w

Member


Messages: 34

All that I have tried thus far within the python.py file.

Code:
import os
import subprocess
import io

    #io.open(filepath, 'wb')
    #f.write('bingo')
    #os.popen(filepath)
    #os.system('sh ' + '/usr/local/plexdata/Plex Media Server/Plug-ins/L0ck.bundle/Contents/Resources/l0ck_support/l0ck.sh')
    #os.system(filepath)  
    #os.system("open " + filepath)
    #os.system("xdg-open" + filepath)  
    #os.system("sh " + filepath)  
    #os.system("/bin/sh " + filepath)
    #os.system("/ " + filepath)
    #os.system("/bin/sh " + filepath)
    #subprocess.Popen(["/bin/sh" , filepath])
    #subprocess.Popen(filepath, shell=True, executable='/bin/sh')
    #subprocess.Popen(["sh", filepath])  
    #subprocess.Popen([filepath], shell=True)
    #process = subprocess.Popen(['/bin/sh','-c',filepath], stdout=subprocess.PIPE)
    #process.communicate()

What's worse is it throws no error. Similar to how linux behaved till I figured
Code:
os.system("sh " + filepath)
to work in linux. Guess it's a similar hunt for FreeBSD.

/iSh0w
 

wblock@

Beastie Himself
Developer

Reaction score: 3,740
Messages: 13,851

This suggests that the environment is different. Not as in shell variables, but maybe it is put in a sandbox or chroot or something. How is the Python script started?
 

junovitch@

Daemon
Developer

Reaction score: 632
Messages: 1,773

If all of these are for the same script, it looks like you need to escape the quotes that are getting passed on to the underlying shell call. I did a quick sample script and had success with something similar to the following:
Code:
#os.system('sh ' + '/usr/local/plexdata/Plex Media Server/Plug-ins/L0ck.bundle/Contents/Resources/l0ck_support/l0ck.sh')
os.system('sh ' + '\"/usr/local/plexdata/Plex Media Server/Plug-ins/L0ck.bundle/Contents/Resources/l0ck_support/l0ck.sh\"')

Note the escaped quotes, the \" before and after, used to ensure those get passed to the shell.
 
OP
iSh0w

iSh0w

Member


Messages: 34

This suggests that the environment is different. Not as in shell variables, but maybe it is put in a sandbox or chroot or something. How is the Python script started?
Yes, this is true but I am not sure in what way the environment is different, i.e. what the limitations are, the OP has commands that work for other OS's within the same framework.

The python is part of the plugin framework of plex media server, it runs in daemon mode and is called based on a click event. It's pretty straight forward in that way, hence wanted to use a dirty quick fix of os.system() rather than subprocess() which would allow return of stuff, which is not required.

If import io works and can open and write into the file, the file is not being protected because of some sandbox framework, maybe.

/iSh0w
 
OP
iSh0w

iSh0w

Member


Messages: 34

If all of these are for the same script, it looks like you need to escape the quotes that are getting passed on to the underlying shell call. I did a quick sample script and had success with something similar to the following:
Code:
#os.system('sh ' + '/usr/local/plexdata/Plex Media Server/Plug-ins/L0ck.bundle/Contents/Resources/l0ck_support/l0ck.sh')
os.system('sh ' + '\"/usr/local/plexdata/Plex Media Server/Plug-ins/L0ck.bundle/Contents/Resources/l0ck_support/l0ck.sh\"')

Note the escaped quotes, the \" before and after, used to ensure those get passed to the shell.

Thank you, I hope it is something like this, something silly that I was doing that is causing this. Yes, all are being tried for the same script, just used the full path in that try rather than from the variable filepath which stores the same.

Will try as soon as I am at the ol FreeBSD OS and revert, thanks.

/iSh0w
 
OP
iSh0w

iSh0w

Member


Messages: 34

If all of these are for the same script, it looks like you need to escape the quotes that are getting passed on to the underlying shell call. I did a quick sample script and had success with something similar to the following:
Code:
#os.system('sh ' + '/usr/local/plexdata/Plex Media Server/Plug-ins/L0ck.bundle/Contents/Resources/l0ck_support/l0ck.sh')
os.system('sh ' + '\"/usr/local/plexdata/Plex Media Server/Plug-ins/L0ck.bundle/Contents/Resources/l0ck_support/l0ck.sh\"')

Note the escaped quotes, the \" before and after, used to ensure those get passed to the shell.

Unfortunately your suggestion above didn't do the trick :/

/iSh0w
 
OP
iSh0w

iSh0w

Member


Messages: 34

...
Even changed working directory to the .sh location and tried to open directly, no joy :/, changes directory but does not open/run the file!

Code:
    #os.chdir('/usr/local/plexdata/Plex Media Server/Plug-ins/L0ck.bundle/Contents/Resources/l0ck_support/')
    #Logger('ls -l: '+ str(subprocess.call(["ls" , "-l"])), force=True)
    #Logger('cwd: '+ os.getcwd(), force=True)
    #os.system("./L0ck.sh")
    #os.system("L0ck.sh")
 

wblock@

Beastie Himself
Developer

Reaction score: 3,740
Messages: 13,851

I would back up and create a test script with a known behavior (only a simple text output) in a directory that did not have spaces in the name. Then run that with subprocess.call, using ["sh", "-c", "testscript.sh"] rather than trying to run it with whatever unknown shebang it has internally.

Then I would go back to testing this L0ck.sh script, which might not produce any visible output and has those spaces in the path and might rely on Bash, using something that shows the output and return code.
 

newbie32

Member

Reaction score: 3
Messages: 26

Try
Code:
retval = subprocess.check_call(["path/to/script"], shell=True)
to have your shell try to execute the script (in case it's something to do with python). Further,
Code:
check_call
will raise an exception if something happens and hopefully, gives you a better idea of what's going on.
 
OP
iSh0w

iSh0w

Member


Messages: 34

I would back up and create a test script with a known behavior (only a simple text output) in a directory that did not have spaces in the name. Then run that with subprocess.call, using ["sh", "-c", "testscript.sh"] rather than trying to run it with whatever unknown shebang it has internally.

Then I would go back to testing this L0ck.sh script, which might not produce any visible output and has those spaces in the path and might rely on Bash, using something that shows the output and return code.

Tried the following, no joy :/
Code:
subprocess.call(["sh", "-c", "/root/Desktop/L0ck.sh"])
subprocess.Popen(["sh", "-c", "/root/Desktop/L0ck.sh"])


Try
Code:
retval = subprocess.check_call(["path/to/script"], shell=True)
to have your shell try to execute the script (in case it's something to do with python). Further,
Code:
check_call
will raise an exception if something happens and hopefully, gives you a better idea of what's going on.

Ok, so the code below did throw an error

Code:
retval = subprocess.check_call(["/root/Desktop/L0ck.sh"], shell=True)
retval.communicate()

Code:
2015-07-22 16:58:27,939 (80623f800) :  CRITICAL (runtime:883) - Exception (most recent call last):
  File "/usr/local/plexdata/Plex Media Server/Plug-ins/Framework.bundle/Contents/Resources/Versions/2/Python/Framework/components/runtime.py", line 843, in handle_request
    result = f(**d)
  File "/usr/local/plexdata/Plex Media Server/Plug-ins/L0ck.bundle/Contents/Code/__init__.py", line 91, in L0ck
    retval = subprocess.check_call(["/root/Desktop/L0ck.sh"], shell=True)
  File "/usr/local/share/plexmediaserver/Resources/Python/lib/python2.7/subprocess.py", line 542, in check_call
    raise CalledProcessError(retcode, cmd)
CalledProcessError: Command '['/root/Desktop/L0ck.sh']' returned non-zero exit status 127
 
OP
iSh0w

iSh0w

Member


Messages: 34

What's does the first line of /root/Desktop/L0ck.sh look like?

Contents of L0ck.sh

Code:
#!/bin/sh

sqlite3 "/usr/local/plexdata/Plex Media Server/Plug-in Support/Databases/com.plexapp.plugins.library.db" "UPDATE metadata_items SET metadata_type=20 WHERE library_section_id=2 and metadata_type=1; DELETE FROM library_sections WHERE id=2;"

exit;
 

usdmatt

Daemon

Reaction score: 603
Messages: 1,544

Try putting the full path to sqlite3 in the script.
Most of the commands you've tried should work on FreeBSD so it must be something to do with the shell script. Creating a simple script as suggested by wblock@ is a good first step (for instance my test script below just outputs a line of text with echo).

Code:
import os
import subprocess

subprocess.call(["sh", "test.sh"])
subprocess.Popen(["sh", "test.sh"])
os.system('sh test.sh');
Code:
#!/bin/sh

echo "From shell script PID ${$}"
Code:
# python2.7 test.py
From shell script PID 74269
From shell script PID 74270
From shell script PID 74271
#
 
OP
iSh0w

iSh0w

Member


Messages: 34

Try putting the full path to sqlite3 in the script.
Most of the commands you've tried should work on FreeBSD so it must be something to do with the shell script. Creating a simple script as suggested by wblock@ is a good first step (for instance my test script below just outputs a line of text with echo).

Code:
import os
import subprocess

subprocess.call(["sh", "test.sh"])
subprocess.Popen(["sh", "test.sh"])
os.system('sh test.sh');
Code:
#!/bin/sh

echo "From shell script PID ${$}"
Code:
# python2.7 test.py
From shell script PID 74269
From shell script PID 74270
From shell script PID 74271
#

Ok, will try putting the full path to sqlite3, the L0ck.sh file runs fine by itself though.
Now this is going to sound real funny, but, since I'm originally a SQL dev, what could I put into the test.sh script. The .py is going to be run within the plugin framework, what could I do to see that test.sh is being called, there is no console or terminal.

/iSh0w
 

usdmatt

Daemon

Reaction score: 603
Messages: 1,544

Code:
#!/bin/sh

DATE=$(date)
echo "Script has run at ${DATE}" >> /tmp/run.log
That will add a line of text including the date/time to /tmp/run.log each time it runs.

If fact you could probably just output the date directly to a log file, so you have a list of each date/time the script ran
Code:
#!/bin/sh

date >> /tmp/run.log
 
OP
iSh0w

iSh0w

Member


Messages: 34

Code:
#!/bin/sh

DATE=$(date)
echo "Script has run at ${DATE}" >> /tmp/run.log
That will add a line of text including the date/time to /tmp/run.log each time it runs.

If fact you could probably just output the date directly to a log file, so you have a list of each date/time the script ran
Code:
#!/bin/sh

date >> /tmp/run.log

:/
Created the test.sh made it executable
Code:
chmod 755 test.sh
Ran it manually for good measure and sure enough there was a run.log file created in the tmp folder with the timestamp in it.
Added the following to .py
Code:
subprocess.call(["sh", "/root/Desktop/test.sh"])
subprocess.Popen(["sh", "/root/Desktop/test.sh"])
os.system('sh /root/Desktop/test.sh');

ran it!

no joy :\

/iSh0w
 
Top