Linux Binary compatibility - Nvidia Drivers and CUDA for Blender

pbp_jackd

Member

Reaction score: 7
Messages: 45

It is, but that's beside the point. Although, now that I think of it, it might have some problems working with the newest AMD cards.
1 step closer.
Got Linux Blender 2.91 working with OpenGL after I update Mesa to 20 in /compat/linux.

not it looks like so:

Code:
server glx version string: 1.4
client glx version string: 1.4
GLX version: 1.4
    Max core profile version: 4.6
    Max compat profile version: 4.6
    Max GLES1 profile version: 1.1
    Max GLES[23] profile version: 3.2
OpenGL core profile version string: 4.6 (Core Profile) Mesa 20.0.4
OpenGL core profile shading language version string: 4.60
OpenGL version string: 4.6 (Compatibility Profile) Mesa 20.0.4
OpenGL shading language version string: 4.60
OpenGL ES profile version string: OpenGL ES 3.2 Mesa 20.0.4
OpenGL ES profile shading language version string: OpenGL ES GLSL ES 3.20
    GL_EXT_shader_implicit_conversions, GL_EXT_shader_integer_mix,
 

pbp_jackd

Member

Reaction score: 7
Messages: 45

If you are so inclined, you can try AMDGPU-Pro OpenCL libraries (use https://aur.archlinux.org/cgit/aur.git/tree/PKGBUILD?h=opencl-amd as a reference to what download/extract) with my glibc shim. You'll need to set OCL_ICD_VENDORS and LD_LIBRARY_PATH appropriately and then run with-glibc-shim clinfo.
Thx. No luck so far.
Not sure what is missing that clinfo will find a device.

My setup looks like so:
- installed ubuntu 20.04 to /compat/linux ( https://forums.freebsd.org/threads/...-google-chrome-linux-binary-on-freebsd.77559/)
- installed AMD OpenCL libs with this script: https://gist.github.com/tuxutku/79daa2edca131c1525a136b650cdbe0a
- run clinfo in various ways e.g.
Bash:
 OCL_ICD_VENDORS="/compat/linux/etc/OpenCL/vendors/" LD_LIBRARY_PATH="/compat/linux/opt/amdgpu/share/libdrm" LIBGL_DRI3_DISABLE=1 ./with-glibc-shim clinfo
Number of platforms                               0

or
Bash:
OCL_ICD_VENDORS="/compat/linux/etc/OpenCL/vendors/" LD_LIBRARY_PATH="/compat/linux/opt/amdgpu/share/libdrm" LIBGL_DRI3_DISABLE=1 /compat/linux/usr/bin/clinfo

Number of platforms                               1
  Platform Name                                   AMD Accelerated Parallel Processing
  Platform Vendor                                 Advanced Micro Devices, Inc.
  Platform Version                                OpenCL 2.1 AMD-APP (3188.4)
  Platform Profile                                FULL_PROFILE
  Platform Extensions                             cl_khr_icd cl_amd_event_callback cl_amd_offline_devices
  Platform Host timer resolution                  1ns
  Platform Extensions function suffix             AMD

  Platform Name                                   AMD Accelerated Parallel Processing
Number of devices                                 0

NULL platform behavior
  clGetPlatformInfo(NULL, CL_PLATFORM_NAME, ...)  AMD Accelerated Parallel Processing
  clGetDeviceIDs(NULL, CL_DEVICE_TYPE_ALL, ...)   
  clCreateContext(NULL, ...) [default]            No devices found in platform
  clCreateContextFromType(NULL, CL_DEVICE_TYPE_DEFAULT)  No devices found in platform
  clCreateContextFromType(NULL, CL_DEVICE_TYPE_CPU)  No devices found in platform
  clCreateContextFromType(NULL, CL_DEVICE_TYPE_GPU)  No devices found in platform
  clCreateContextFromType(NULL, CL_DEVICE_TYPE_ACCELERATOR)  No devices found in platform
  clCreateContextFromType(NULL, CL_DEVICE_TYPE_CUSTOM)  No devices found in platform
  clCreateContextFromType(NULL, CL_DEVICE_TYPE_ALL)  No devices found in platform

ICD loader properties
  ICD loader Name                                 OpenCL ICD Loader
  ICD loader Vendor                               OCL Icd free software
  ICD loader Version                              2.2.11
  ICD loader Profile                              OpenCL 2.1
 

pbp_jackd

Member

Reaction score: 7
Messages: 45

I specifically said extract and not install with a script. That is, extract files by hand into some local directory; you just messed up your installation.
Did everything again without installing ;-)

Questions:
Guessing based on the output below in 1.1. I think clinfo can see my CPU after using the amd libs but not the GPU.
So what about this file: amdgpu.ids ? It does contain the id for my 5700 XT. Any ideas what to do with it ?
Is part of https://aur.archlinux.org/cgit/aur.git/tree/PKGBUILD?h=opencl-amd but I do not point to it in any way right now.

Observations:
1. If OCL_ICD_VENDORS or LD_LIBRARY_PATH is not set clinfo always gives
Code:
Number of platforms                               0
1.1 running clinfo inside the ubuntu chroot but exported OCL_ICD_VENDORS and LD_LIBRARY_PATH
or from outside (/compat/linux/usr/bin/clinfo) also with exported env variables leads to the output below. If one is not set, the output is same with above.
So it seems like a tiny step forward.
Code:
Number of platforms                               1
  Platform Name                                   AMD Accelerated Parallel Processing
  Platform Vendor                                 Advanced Micro Devices, Inc.
  Platform Version                                OpenCL 2.1 AMD-APP (3188.4)
  Platform Profile                                FULL_PROFILE
  Platform Extensions                             cl_khr_icd cl_amd_event_callback cl_amd_offline_devices
  Platform Host timer resolution                  1ns
  Platform Extensions function suffix             AMD

  Platform Name                                   AMD Accelerated Parallel Processing
Number of devices                                 0

NULL platform behavior
  clGetPlatformInfo(NULL, CL_PLATFORM_NAME, ...)  AMD Accelerated Parallel Processing
  clGetDeviceIDs(NULL, CL_DEVICE_TYPE_ALL, ...)  
  clCreateContext(NULL, ...) [default]            No devices found in platform
  clCreateContextFromType(NULL, CL_DEVICE_TYPE_DEFAULT)  No devices found in platform
  clCreateContextFromType(NULL, CL_DEVICE_TYPE_CPU)  No devices found in platform
  clCreateContextFromType(NULL, CL_DEVICE_TYPE_GPU)  No devices found in platform
  clCreateContextFromType(NULL, CL_DEVICE_TYPE_ACCELERATOR)  No devices found in platform
  clCreateContextFromType(NULL, CL_DEVICE_TYPE_CUSTOM)  No devices found in platform
  clCreateContextFromType(NULL, CL_DEVICE_TYPE_ALL)  No devices found in platform

ICD loader properties
  ICD loader Name                                 OpenCL ICD Loader
  ICD loader Vendor                               OCL Icd free software
  ICD loader Version                              2.2.11
  ICD loader Profile                              OpenCL 2.1

2. running clinfo from host with
OCL_ICD_VENDORS="/tmp/pkgdir/etc/OpenCL/vendors/" LD_LIBRARY_PATH="/tmp/pkgdir/usr/lib" LIBGL_DRI3_DISABLE=1 ./with-glibc-shim clinfo
Code:
➜  bin git:(master) ✗
OCL_ICD_VENDORS="/tmp/pkgdir/etc/OpenCL/vendors/" LD_LIBRARY_PATH="/tmp/pkgdir/usr/lib" LIBGL_DRI3_DISABLE=1 ./with-glibc-shim clinfo
shim init
shim_fcntl64 is not implemented
Assertion failed: (0), function shim_fcntl64, file build/wrappers64.c, line 16037.
[1]    15491 abort (core dumped)  OCL_ICD_VENDORS="/tmp/pkgdir/etc/OpenCL/vendors/" LD_LIBRARY_PATH= =1  clinfo

3. did run ldd clinfo ang got:
Code:
ldd clinfo
        linux_vdso.so.1 (0x00007ffffffff000)
        libOpenCL.so.1 => /usr/lib/x86_64-linux-gnu/libOpenCL.so.1 (0x0000000801200000)
        libdl.so.2 => /lib/x86_64-linux-gnu/libdl.so.2 (0x0000000801070000)
        libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6 (0x000000080140b000)
        /lib64/ld-linux-x86-64.so.2 (0x0000000001021000)
 

pbp_jackd

Member

Reaction score: 7
Messages: 45

Just some notes for myself and in case someone else wants to give the approach a try.

- I did not get OpenCL to work with the Linuxlator ubuntu 18.04.5 LTS + 5700 XT + AMD OpenCL libs (yet). The results are the same as show already above. The 5700 XT is not seen by clinfo.
- I attached a strace log I took from inside the ubuntu chroot. I'm not understanding what is going on there in detail ( maybe someone else can see what is wrong ?) but checking it against the hints from https://github.com/bashbaug/OpenCLP...inux.asciidoc#troubleshooting-opencl-on-linux it seems things are actual setup "OK"
- Finding a Ubuntu version + a version of AMD OpenCL libs which actually work together seems crucial.
- The only combination which worked for me on an actual Ubuntu installation was Ubuntu 18.04.5 HWE + AMD amdgpu-pro-20.40-1147287-ubuntu-18.04.tar.xz . With this the 5700 XT + Ryzen CPU are recognized by clinfo, blender, darktable as OpenCL devices.
- Hints about the OS + AMD OpenCL version: https://devtalk.blender.org/t/guide-install-amd-opencl-on-pop-os-ubuntu-some-derivates/13458
- like shkhln said, its enough to extract the libs and the icd files
- the icd files you need are:
Code:
-rw-r--r--  1 user  user    20B Feb  2 22:26 amdocl-orca64.icd
-rw-r--r--  1 user  user    15B Feb  2 22:26 amdocl64.icd
- the libs you need are:
Code:
lrwxr-xr-x  1 user  user    21B Feb  2 22:26 libamd_comgr.so -> libamd_comgr.so.1.7.0
lrwxr-xr-x  1 user  user    21B Feb  2 22:26 libamd_comgr.so.1 -> libamd_comgr.so.1.7.0
-rw-r--r--  1 user  user   104M Feb  2 22:26 libamd_comgr.so.1.7.0
-rw-r--r--  1 user  user    81M Feb  2 22:26 libamdocl-orca64.so
-rw-r--r--  1 user  user    35M Feb  2 22:26 libamdocl12cl64.so
-rw-r--r--  1 user  user    90M Feb  2 22:26 libamdocl64.so
lrwxr-xr-x  1 user  user    16B Feb  2 22:26 libOpenCL.so.1 -> libOpenCL.so.1.2
-rw-r--r--  1 user  user    30K Feb  2 22:26 libOpenCL.so.1.2

- I replicated the setup as good as I could on FreeBSD (13-Alpha3) like so:
- setup Ubuntu BIONIC Linuxlator according to https://wiki.freebsd.org/LinuxJails
- NOTE: this only gives you Ubuntu version 18.4 LTS. To get 18.04.5 I had to add the following to /compat/ubuntu/etc/apt/sources.list ( run apt update/upgrade. Check OS version e.g. with lsb_release -a)

deb http://archive.ubuntu.com/ubuntu bionic-updates main restricted universe multiverse

- The upgrade is important. Otherwise the mesa version is too old to recognize the 5700 XT and second it would not match the Ubuntu version I used for testing.
- I also tested to upgrade just mesa to latest but seems to make no difference either.


 

Attachments

  • strace.txt
    52 bytes · Views: 13

shkhln

Daemon

Reaction score: 872
Messages: 2,053

With your update the failure is gone. Thx.
That was just an assertion check (stub). I'm rather interested in what it is doing now. What result clinfo prints, differences with Linuxulator and so on.
 

BawdyAnarchist

Member

Reaction score: 5
Messages: 71

Pretty bummed to find out that stupid nvidia doesn't allow CUDA or CL to work on FreeBSD. What a bunch of jerks. Surely they have some backroom deal with Linux??

I need it for Blender, which seems to be the overal best free and open source 3d design software around. And now I have to go buy an AMD GPU just to get this to work. Too bad I didn't know about FreeBSD when I bought this machine, because I specifically bought the nvidia for CUDA support; otherwise would've just gone with AMD.
 

shkhln

Daemon

Reaction score: 872
Messages: 2,053

I need it for Blender, which seems to be the overal best free and open source 3d design software around. And now I have to go buy an AMD GPU just to get this to work.
FYI, you are posting this directly under a string of posts documenting how good AMD is at running Blender on FreeBSD.

What a bunch of jerks. Surely they have some backroom deal with Linux??
Do you have any non-dumb things to say on the topic?
 

BawdyAnarchist

Member

Reaction score: 5
Messages: 71

FYI, you are posting this directly under a string of posts documenting how good AMD is at running Blender on FreeBSD.


Do you have any non-dumb things to say on the topic?
Sorry, I apologize, it was late at night and I was frustrated at not being able to get my nice shiny new GPU to do a basic thing that I needed. I'm just a dabbler and I don't really understand the nuts and bolts of how this works.

Could I make a small request? I'm not going to be running Blender in Linuxulator; just in a FreeBSD jail. Are you saying that I will have problems with OpenCL on AMD GPU as well? Because those are the two options I see in Blender for ray tracing and GPU support (CUDA and OpenCL). I thought this thread was primarily about trying to get OpenCL libraries working on an nvidia GPU.
 

shkhln

Daemon

Reaction score: 872
Messages: 2,053

I thought this thread was primarily about trying to get OpenCL libraries working on an nvidia GPU.
It is supposed to be, but it was hijacked.

Are you saying that I will have problems with OpenCL on AMD GPU as well?
With AMD there are 4 alternative OpenCL drivers: Clover, Orca, PAL and ROCm.
  1. Clover is an open-source OpenCL driver from Mesa, that's what you normally get on FreeBSD.
  2. Orca and PAL are official AMD closed-source OpenCL drivers for Linux.
  3. ROCm is an official AMD open-source GPU computing stack (it's not limited to OpenCL), which is in theory supposed to replace all other AMD GPU computing stuff. However, inexplicably, AMD decided to not support regular consumer GPUs in ROCm. To make this as much of a PITA as possible they also decided to limit support to specific CPUs and motherboard features: https://github.com/RadeonOpenCompute/ROCm#Hardware-and-Software-Support. ROCm is not ported to FreeBSD.
 
Top