Oracle, python cx_Oracle for amd64

Code:
bsd# uname -a
FreeBSD bsd.localdomain 8.3-RELEASE FreeBSD 8.3-RELEASE #0: Mon Apr  9 21:23:18 UTC 2012     [email]root@mason.cse.buffalo.edu[/email]:/usr/obj/usr/src/sys/GENERIC  amd64
bsd# python
Python 2.7.2 (default, Jul  9 2012, 13:49:49) 
[GCC 4.2.2 20070831 prerelease [FreeBSD]] on freebsd8
Type "help", "copyright", "credits" or "license" for more information.
>>> import cx_Oracle
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
ImportError: /usr/lib/libdl.so.2: unsupported file layout
>>> 
bsd# ldd /usr/local/lib/python2.7/site-packages/cx_Oracle.so
/usr/local/lib/python2.7/site-packages/cx_Oracle.so:
	libclntsh.so.10.1 => /usr/lib/oracle/instantclient10_1/libclntsh.so.10.1 (0x800c00000)
	libthr.so.3 => /lib/libthr.so.3 (0x801ac1000)
	libc.so.7 => /lib/libc.so.7 (0x800648000)
	libnnz10.so => /usr/lib/oracle/instantclient10_1/libnnz10.so (0x801bda000)
	libdl.so.2 => not found (0x0)
	libm.so.6 => not found (0x0)
	libpthread.so.0 => not found (0x0)
	libnsl.so.1 => not found (0x0)
	libc.so.6 => not found (0x0)
	libc.so.6 => not found (0x0)
 
Code:
bsd# ll /usr/lib/libdl.so.2
lrwxr-xr-x  1 root  wheel  32 Jul 10 17:54 /usr/lib/libdl.so.2 -> /usr/compat/linux/lib/libdl.so.2
bsd# ll /usr/compat/linux/lib/libdl.so.2
lrwxrwxrwx  1 root  wheel  12 Jul 10 14:07 /usr/compat/linux/lib/libdl.so.2 -> libdl-2.9.so
bsd# ll /usr/compat/linux/lib/libdl-2.9.so 
-rwxr-xr-x  1 root  wheel  18568 Jul 10 15:49 /usr/compat/linux/lib/libdl-2.9.so
 
Code:
bsd# file /usr/local/lib/python2.7/site-packages/cx_Oracle.so
/usr/local/lib/python2.7/site-packages/cx_Oracle.so: ELF 64-bit LSB shared object, x86-64, version 1 (FreeBSD), dynamically linked, not stripped
 
Code:
bsd# file /usr/compat/linux/lib/libdl-2.9.so
/usr/compat/linux/lib/libdl-2.9.so: ELF 32-bit LSB shared object, Intel 80386, version 1 (SYSV), dynamically linked (uses shared libs), for GNU/Linux 2.6.9, not stripped
 
zhuangyan: I have tried to get cx_Oracle working on 64-bit FreeBSD with no luck either.

The problem I had is that linux_base-f10 is available for amd64, but runs in i386/32 bit mode. This didn't seem all that bad, since the linux-oracle-instantclient-* ports are 32-bit only.

Then, however, I tried to build cx_Oracle and it fails, because it's trying to build it against a 64-bit version of libclntsh.so, and we installed the 32-bit instantclient.

I exported the following flag, to tell cx_Oracle to build in 32-bit:
Code:
# export CFLAGS=-m32

But then it complained about not finding 32-bit version of libgcc.a


Currently, we have cx_Oracle working in 32-bit FreeBSD, so I am going to wait until a 64-bit linux_base package is released before trying again. It seems like a big hassle to have to run oracle instant client, cx_Oracle, python, and whatever else in 32-bit while the rest of your system is 64-bit.
 
I'm giving this a try again, now that linux64 emulation landed in FreeBSD 10.3 and emulators/linux_base-c6 is (experimentally) available as 64-bit. I installed a 64-bit instantclient SDK and installed a libaio RPM, and got to the point where "pip install cx_Oracle" actually completes. But Python3.4 segfaults when I try to import the cx_Oracle module.
Any suggestions where to go from here? I'm thinking maybe the FreeBSD libc.so and the Linux libc.so get mixed up?

Code:
$ cat cx.py
import cx_Oracle
$
$ /usr/local/bin/gdb python3.4
GNU gdb (GDB) 7.11.1 [GDB v7.11.1 for FreeBSD]
Copyright (C) 2016 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.  Type "show copying"
and "show warranty" for details.
This GDB was configured as "x86_64-portbld-freebsd10.3".
Type "show configuration" for configuration details.
For bug reporting instructions, please see:
<http://www.gnu.org/software/gdb/bugs/>.
Find the GDB manual and other documentation resources online at:
<http://www.gnu.org/software/gdb/documentation/>.
For help, type "help".
Type "apropos word" to search for commands related to "word"...
Reading symbols from python3.4...done.
(gdb) run cx.py
Starting program: /usr/local/bin/python3.4 cx.py

Program received signal SIGSEGV, Segmentation fault.
0x0000000805f0dc8a in __strrchr_sse2 () from /usr/local/oracle/instantclient_11_2/libc.so.6
(gdb) bt
#0  0x0000000805f0dc8a in __strrchr_sse2 () from /usr/local/oracle/instantclient_11_2/libc.so.6
#1  0x0000000805f737cb in __init_misc () from /usr/local/oracle/instantclient_11_2/libc.so.6
#2  0x0000000805ea9c00 in _init () from /usr/local/oracle/instantclient_11_2/libc.so.6
#3  0x0000000800605a9f in ?? () from /libexec/ld-elf.so.1
#4  0x000000080060a437 in ?? () from /libexec/ld-elf.so.1
#5  0x0000000800606ea0 in ?? () from /libexec/ld-elf.so.1
#6  0x0000000800eaf89e in _PyImport_GetDynLoadFunc (shortname=0x80078b370 "cx_Oracle", pathname=0x800a7a8f0 "/usr/local/lib/python3.4/site-packages/cx_Oracle.so", fp=0x0) at ./Python/dynload_shlib.c:95
#7  0x0000000800e66389 in _PyImport_LoadDynamicModule (name=0x800a7b890, path=0x8007a5170, fp=0x0) at ./Python/importdl.c:69
#8  0x0000000800e655b2 in _imp_load_dynamic_impl (module=0x8006e92d8, name=0x800a7b890, path=0x8007a5170, file=0x0) at Python/import.c:2271
#9  0x0000000800e6513d in _imp_load_dynamic (module=0x8006e92d8, args=0x800a7f2f8) at Python/import.c:2247
#10 0x0000000800cfa8b8 in PyCFunction_Call (func=0x800713758, arg=0x800a7f2f8, kw=0x800a7f288) at Objects/methodobject.c:93
#11 0x0000000800e32256 in ext_do_call (func=0x800713758, pp_stack=0x7ffffffebe80, flags=3, na=0, nk=0) at Python/ceval.c:4558
#12 0x0000000800e2a4b1 in PyEval_EvalFrameEx (f=0x8007c8838, throwflag=0) at Python/ceval.c:2878
#13 0x0000000800e1778d in PyEval_EvalCodeEx (_co=0x8006c7040, globals=0x8006ff838, locals=0x0, args=0x8024b1770, argcount=3, kws=0x8024b1788, kwcount=0, defs=0x0, defcount=0, kwdefs=0x0, closure=0x0) at Python/ceval.c:3588
#14 0x0000000800e343dd in fast_function (func=0x8007043f0, pp_stack=0x7ffffffee560, n=3, na=3, nk=0) at Python/ceval.c:4344
#15 0x0000000800e3193b in call_function (pp_stack=0x7ffffffee560, oparg=3) at Python/ceval.c:4262
#16 0x0000000800e29f80 in PyEval_EvalFrameEx (f=0x8024b15b8, throwflag=0) at Python/ceval.c:2838
#17 0x0000000800e1778d in PyEval_EvalCodeEx (_co=0x8006ec100, globals=0x8006ff838, locals=0x0, args=0x800a7bc50, argcount=2, kws=0x80064f080, kwcount=0, defs=0x0, defcount=0, kwdefs=0x0, closure=0x0) at Python/ceval.c:3588
#18 0x0000000800cc284c in function_call (func=0x80070f788, arg=0x800a7bc28, kw=0x800a6f058) at Objects/funcobject.c:632
#19 0x0000000800c6e6ca in PyObject_Call (func=0x80070f788, arg=0x800a7bc28, kw=0x800a6f058) at Objects/abstract.c:2058
#20 0x0000000800e32270 in ext_do_call (func=0x80070f788, pp_stack=0x7fffffff0be0, flags=3, na=2, nk=0) at Python/ceval.c:4561
#21 0x0000000800e2a4b1 in PyEval_EvalFrameEx (f=0x80247f238, throwflag=0) at Python/ceval.c:2878
#22 0x0000000800e1778d in PyEval_EvalCodeEx (_co=0x8006c7580, globals=0x8006ff838, locals=0x0, args=0x8024579d8, argcount=2, kws=0x8024579e8, kwcount=0, defs=0x80070d700, defcount=1, kwdefs=0x0, closure=0x80070d670) at Python/ceval.c:3588
#23 0x0000000800e343dd in fast_function (func=0x80070f840, pp_stack=0x7fffffff32c0, n=2, na=2, nk=0) at Python/ceval.c:4344
#24 0x0000000800e3193b in call_function (pp_stack=0x7fffffff32c0, oparg=1) at Python/ceval.c:4262
#25 0x0000000800e29f80 in PyEval_EvalFrameEx (f=0x802457838, throwflag=0) at Python/ceval.c:2838
#26 0x0000000800e34245 in fast_function (func=0x800705eb8, pp_stack=0x7fffffff5710, n=1, na=1, nk=0) at Python/ceval.c:4334
#27 0x0000000800e3193b in call_function (pp_stack=0x7fffffff5710, oparg=0) at Python/ceval.c:4262
#28 0x0000000800e29f80 in PyEval_EvalFrameEx (f=0x8024b8b38, throwflag=0) at Python/ceval.c:2838
#29 0x0000000800e34245 in fast_function (func=0x800705f70, pp_stack=0x7fffffff7b60, n=1, na=1, nk=0) at Python/ceval.c:4334
#30 0x0000000800e3193b in call_function (pp_stack=0x7fffffff7b60, oparg=0) at Python/ceval.c:4262
#31 0x0000000800e29f80 in PyEval_EvalFrameEx (f=0x80245db38, throwflag=0) at Python/ceval.c:2838
#32 0x0000000800e34245 in fast_function (func=0x8007146d0, pp_stack=0x7fffffff9fb0, n=2, na=2, nk=0) at Python/ceval.c:4334
#33 0x0000000800e3193b in call_function (pp_stack=0x7fffffff9fb0, oparg=2) at Python/ceval.c:4262
#34 0x0000000800e29f80 in PyEval_EvalFrameEx (f=0x8024b88b8, throwflag=0) at Python/ceval.c:2838
#35 0x0000000800e1778d in PyEval_EvalCodeEx (_co=0x8006f8b80, globals=0x8006ff838, locals=0x0, args=0x8007a6080, argcount=2, kws=0x0, kwcount=0, defs=0x0, defcount=0, kwdefs=0x0, closure=0x0) at Python/ceval.c:3588
#36 0x0000000800cc284c in function_call (func=0x800714788, arg=0x8007a6058, kw=0x0) at Objects/funcobject.c:632
#37 0x0000000800c6e6ca in PyObject_Call (func=0x800714788, arg=0x8007a6058, kw=0x0) at Objects/abstract.c:2058
#38 0x0000000800c6fc5e in _PyObject_CallMethodIdObjArgs (callable=0x800714788, name=0x8011f6ad8 <PyImport_ImportModuleLevelObject.PyId__find_and_load>) at Objects/abstract.c:2328
#39 0x0000000800e638d1 in PyImport_ImportModuleLevelObject (name=0x800a7b890, given_globals=0x800735f38, locals=0x800735f38, given_fromlist=0x8011d1ae0 <_Py_NoneStruct>, level=0) at Python/import.c:1641
#40 0x0000000800e0cb57 in builtin___import__ (self=0x800692358, args=0x8006b88d8, kwds=0x0) at Python/bltinmodule.c:210
#41 0x0000000800cfa923 in PyCFunction_Call (func=0x8006942f8, arg=0x8006b88d8, kw=0x0) at Objects/methodobject.c:99
#42 0x0000000800c6e6ca in PyObject_Call (func=0x8006942f8, arg=0x8006b88d8, kw=0x0) at Objects/abstract.c:2058
#43 0x0000000800e30133 in PyEval_CallObjectWithKeywords (func=0x8006942f8, arg=0x8006b88d8, kw=0x0) at Python/ceval.c:4114
#44 0x0000000800e26bf5 in PyEval_EvalFrameEx (f=0x8024b1338, throwflag=0) at Python/ceval.c:2478
#45 0x0000000800e1778d in PyEval_EvalCodeEx (_co=0x800792b80, globals=0x800735f38, locals=0x800735f38, args=0x0, argcount=0, kws=0x0, kwcount=0, defs=0x0, defcount=0, kwdefs=0x0, closure=0x0) at Python/ceval.c:3588
#46 0x0000000800e16415 in PyEval_EvalCode (co=0x800792b80, globals=0x800735f38, locals=0x800735f38) at Python/ceval.c:775
#47 0x0000000800e7e62f in run_mod (mod=0x80259a0a8, filename=0x800a6c588, globals=0x800735f38, locals=0x800735f38, flags=0x7fffffffe918, arena=0x8025001b0) at Python/pythonrun.c:2175
#48 0x0000000800e7f060 in PyRun_FileExFlags (fp=0x801c20170, filename_str=0x80075d598 "cx.py", start=257, globals=0x800735f38, locals=0x800735f38, closeit=1, flags=0x7fffffffe918) at Python/pythonrun.c:2128
#49 0x0000000800e7da6f in PyRun_SimpleFileExFlags (fp=0x801c20170, filename=0x80075d598 "cx.py", closeit=1, flags=0x7fffffffe918) at Python/pythonrun.c:1601
#50 0x0000000800e7d1d4 in PyRun_AnyFileExFlags (fp=0x801c20170, filename=0x80075d598 "cx.py", closeit=1, flags=0x7fffffffe918) at Python/pythonrun.c:1287
#51 0x0000000800eb4e46 in run_file (fp=0x801c20170, filename=0x802416110 L"cx.py", p_cf=0x7fffffffe918) at Modules/main.c:319
#52 0x0000000800eb3a77 in Py_Main (argc=2, argv=0x802416090) at Modules/main.c:751
#53 0x0000000000400cbe in main (argc=2, argv=0x7fffffffea48) at ./Modules/python.c:69
(gdb)

Here's my notes until now:
Code:
# /etc/rc.conf
#    linux64_enable="YES"
# /etc/make.conf
#    OVERRIDE_LINUX_BASE_PORT=c6_64
#    OVERRIDE_LINUX_NONBASE_PORTS=c6_64
# kldload linux64
# cd /usr/ports/emulators/linux_base-c6 & make install clean

# download 64-bit basic & sdk client for Linux from http://www.oracle.com/technetwork/topics/linuxx86-64soft-092277.html and place in /usr/ports/distfiles/oracle/
# mkdir /usr/local/oracle
# cd /usr/local/oracle
# unzip /usr/ports/distfiles/oracle/instantclient-sdk-linux.x64-11.2.0.4.0.zip
# unzip /usr/ports/distfiles/oracle/instantclient-basic-linux.x64-11.2.0.4.0.zip
# export ORACLE_HOME="/usr/local/oracle/instantclient_11_2"
# export LD_LIBRARY_PATH="/usr/lib/compat:/usr/local/lib:/usr/local/lib/compat/pkg:${ORACLE_HOME}"
# export PATH="${PATH}:${ORACLE_HOME}"
# cd /usr/local/oracle/instantclient_11_2
# cp -a libclntsh.so.11.2 libclntsh.so
# brandelf -t FreeBSD libclntsh.so
# cp -a libocci.so.11.2 libocci.so
# brandelf -t FreeBSD libocci.so
# Download libaio.so.1 for CentOS 6_64 from somewhere, e.g. an RPM https://www.rpmfind.net/linux/rpm2html/search.php?query=libaio.so.1()(64bit)
# Extract the RPM (tar -zxvf libaio-0.3.107-10.el6.x86_64.rpm)
# cp lib64/libaio.so.1* /compat/linux/lib64
# cd /compat/linux/lib64 && ln -s libaio.so.1.0.1 libaio.so.1
# ln -s /compat/linux/lib64/libdl.so.2 /usr/local/oracle/instantclient_11_2/
# ln -s /compat/linux/lib64/libm.so.6 /usr/local/oracle/instantclient_11_2/
# ln -s /compat/linux/lib64/libpthread.so.0 /usr/local/oracle/instantclient_11_2/
# ln -s /compat/linux/lib64/libnsl.so.1 /usr/local/oracle/instantclient_11_2/
# ln -s /compat/linux/lib64/libc.so.6 /usr/local/oracle/instantclient_11_2/
# ln -s /compat/linux/lib64/libaio.so.1 /usr/local/oracle/instantclient_11_2/
# ln -s /compat/linux/lib64/ld-linux-x86-64.so.2  /usr/local/oracle/instantclient_11_2/

# sudo pip install cx_Oracle
 
Last edited:
I got sqlplus working after copying /etc/hosts entries to /compat/linux/etc/hosts. But according to dchagin@ cx_Oracle will only be able to work with a Linux version of Python. So if you want cx_Oracle on FreeBSD, you need to install a Python RPM and put everything under Linux emulation.

I ended up making a simple wrapper for sqlplus instead:

Code:
import sh
sqlplus = sh.Command('/usr/local/instantclient/sqlplus')
for row in sqlplus(
    '-S', 'my_connect_string',
    _env=dict(
        LD_LIBRARY_PATH='/usr/lib/compat:/usr/local/lib:/usr/local/lib/compat/pkg:/usr/local/instantclient',
        NLS_LANG='DANISH_DENMARK.UTF8',
    ),
    _in='SELECT * FROM foo;',
    _bg=False,
    _iter=True,
    _timeout=60,
    _encoding='utf-8',
):
    print(row)
 
What I tried:
-running 11.1-RELEASE
-installed "pip install cx_Oracle" without any hiccups
-grabbed databases/linux-oracle-12-instantclient-basic from https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=198082
-grabbed instantclient-basic-linux.x64-12.1.0.2.0.zip and instantclient-basic-linux-12.1.0.2.0.zip
-compiled 32bit. then fiddled with the SHA256 and SIZE in distinfo to compile 64bit. no errors there

then in python:
swagner@builderMan:~ % python
Python 3.6.3 (default, Oct 10 2017, 16:18:36)
[GCC 4.2.1 Compatible FreeBSD Clang 4.0.0 (tags/RELEASE_400/final 297347)] on freebsd11
Type "help", "copyright", "credits" or "license" for more information.
>>> import cx_Oracle
>>> connection = cx_Oracle.connect("user", "pass", "server.me/DB")
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
cx_Oracle.DatabaseError: DPI-1047: 64-bit Oracle Client library cannot be loaded: "Shared object "libclntsh.so" not found, required by "python"". See https://oracle.github.io/odpi/doc/installation.html#linux for help
>>>
...

and:
>>> import os
>>> print (os.environ)
environ({...... 'LD_LIBRARY_PATH': '/compat/linux/usr/lib/oracle/12.1.0.2.0/client/lib'}


Then I read, that if at all, it would only work with a linux version of Python.

I don't know if I'm posting in the right forum. But I see no really usable tutorial to get an oracle client working for Python on FreeBSD. Which I need to connect to a corporate DB. And I REALLY want to move from linux (systemd....) to FreeBSD on my new server.

Is there a proper place to ask? Or a proper place to look? Am I missing something fundamental? Is there a solution at all?

Thanks a lot for pointing me in the right direction
Best regards
simon
 
For others who might find this page, hoping to get cx_Oracle working in Python on FreeBSD, it's not going to work, except possibly if Python is installed in the Linux compact layer. HOWEVER, you can install sqlplus and use that from native Python. I'm dumping my notes on installing sqlplus here. If you are not installing to a jail, just run all commands on the same host:

Code:
# On host:
# ----------
# Load linux64 kernel module
kldload linux64
# Load linux64 kernel module at startup
echo 'linux64_load="YES"' >> /boot/loader.conf

# In jail:
# ----------
# Build CentOS base package in 64 bit mode
echo 'DEFAULT_VERSIONS+=linux=c6_64' >> /etc/make.conf
cd /usr/ports/emulators/linux_base-c6 && make install clean

# Enable linux emulation at startup (is this necessary?)
echo 'linux_enable="YES"' >> /etc/rc.conf

# download 64-bit basiclite & sqlplus zipfiles for 64bit Linux from http://www.oracle.com/technetwork/topics/linuxx86-64soft-092277.html
# Download libaio.so.1 for CentOS 6_64 from somewhere, e.g. an RPM https://www.rpmfind.net/linux/rpm2html/search.php?query=libaio.so.1()(64bit)
fetch ftp://195.220.108.108/linux/centos/6.9/os/x86_64/Packages/libaio-0.3.107-10.el6.x86_64.rpm
# Copy all downloaded files to /usr/ports/distfiles/oracle/

# Install libaio
cd /usr/ports/distfiles/oracle/
tar -zxvf /usr/ports/distfiles/oracle/libaio-0.3.107-10.el6.x86_64.rpm
cp lib64/libaio.so.1.0.* /compat/linux/lib64/
rm -r lib64 usr
cd /compat/linux/lib64 && ln -s libaio.so.1.0.1 libaio.so.1

#Install instantclient
mkdir /compat/linux/usr/oracle
cd /compat/linux/usr/oracle
unzip /usr/ports/distfiles/oracle/instantclient-basiclite-linux.x64-12.2.0.1.0.zip
unzip /usr/ports/distfiles/oracle/instantclient-sqlplus-linux.x64-12.2.0.1.0.zip

# Set a global LD_CONIFG_PATH for the linux compat layer and run ldconfig
echo '/compat/linux/usr/oracle/instantclient_12_2' >> /compat/linux/etc/ld.so.conf
/compat/linux/sbin/ldconfig

# Make sqlplus available from a standard PATH
ln -s /compat/linux/usr/oracle/instantclient_12_2/sqlplus /usr/local/bin/sqlplus

# Add entries from /etc/hosts to /compat/linux/etc/hosts
grep -v '^#' /etc/hosts > /compat/linux/etc/hosts

When you have a working sqlplus, you can call that from Python. I previously posted code to do that, but I also found this nifty Python package to make working with sqlplus a bit easier: https://github.com/c4s4/sqlplus_commando
 
Back
Top