OpenCL on Ubuntu 13.04

Unfortunately two years after my post about getting Intel's OpenCL to work on Ubuntu, the out-of-box situation for using OpenCL isn't much better - you can install the OpenCL library/headers/package config files with a nice and quick `apt-get install ocl-icd-opencl-dev`, but doing that will probably just get you to CL_PLATFORM_NOT_FOUND_KHR (-1001) error from clGetPlatformIDs. That's because you do have the dispatcher, but no actual OpenCL drivers. Therefore I decided to play with the three major OpenCL implementations (Intel, AMD, NVidia).

The reason why one can do the simple apt-get install is a free software project called ocl-icd - this provides libOpenCL.so and implements the dispatcher so that every icd-compatible OpenCL implementation can use it and therefore provides nice basis for using OpenCL, but to get to building and running CL kernels you need to have at least one driver, so let's look at them.

Intel's OpenCL

Intel recently released Intel SDK for OpenCL Applications XE 2013 - but Ubuntu/Debain users are out of luck, you won't find an official .deb on Intel's web. Fortunately that is not a deal breaker and with a few commands you can turn the rpm into a deb package (and it is what I recommended in my old post). But since we have ocl-icd these days, I'd go for a different approach this time - install just the bare minimum to have ocl-icd pick up the Intel driver (and add proper dependencies on ocl-icd-libopencl1 and libnuma1), so I created the debs manually this time.

So, what I did was:

1. Grab the tgz from Intel's web.
2. Extract it into a temporary location where you'll find license, readme file, a bunch of scripts and five rpms:
• opencl-1.2-base-[version] - contains the libOpenCL.so, which we don't need cause we have ocl-icd-libopencl1
• opencl-1.2-devel-[version] - contains OpenCL headers, which also aren't needed, as those are in opencl-headers package which is a dependency of ocl-icd-opencl-dev
• opencl-1.2-intel-cpu-[version] - bingo, the ICD
• opencl-1.2-intel-devel-[version] - a few development tools - offline compiler and Qt-based KernelBuilder application
• opencl-1.2-intel-mic-[version] - additional libraries to support also Xeon Phi coprocessor besides Core processors - I ignored this one, cause I don't have access to such processor.
3. Now, lets make binary debian packages from the unzipped files (note that only the intel-cpu package is required to use OpenCL apps, the devel tools are optional), this is pretty easy, following a simple debian package building HOWTO, I put the files from the intel-cpu and intel-devel rpms into the following directory structure:
   

    |-opencl-driver-intel-cpu
    | \-DEBIAN
    | \-etc
    |   \-OpenCL
    |     \-vendors
    | \-usr
    |   \-lib
    |     \-x86_64-linux-gnu
    |       \-OpenCL
    |         \-vendors
    |           \-intel
    |   \-share
    |     \-doc
    |       \-opencl-driver-intel-cpu
    |-opencl-driver-intel-tools
    | \-bin
    | \-DEBIAN
    | \-usr
    |   \-lib
    |     \-x86_64-linux-gnu
    |       \-intel
    |         \-opencl-1.2-3.0.67279
    |   \-share
    |     \-doc
    |       \-opencl-driver-intel-tools

   
   The DEBIAN directories contain the control files:
   

   $ opencl-driver-intel-cpu/DEBIAN/control:
   Package: opencl-driver-intel-cpu
   Version: 3.0.67279-1
   Section: libs
   Priority: optional
   Architecture: amd64
   Depends: ocl-icd-libopencl1 (>= 2.0), libnuma1
   Maintainer: Your Name
   Description: Intel OpenCL CPU implementation
    This package provides Intel OpenCL implementation which can utilize Intel Core processors.
   
   $ opencl-driver-intel-tools/DEBIAN/control:
   
   Package: opencl-driver-intel-tools
   Version: 3.0.67279-1
   Section: libs
   Priority: optional
   Architecture: amd64
   Depends: ocl-icd-libopencl1 (>= 2.0)
   Maintainer: Your Name
   Description: Intel SDK for OpenCL Applications development tools
    This package contains the following tools:
     - Intel SDK for OpenCL - Kernel Builder, which enables building and analyzing OpenCL kernels and provides full offline OpenCL language compilation.
     - Intel SDK for OpenCL - Offline Compiler, a command-line utility, which enables offline compilation and building of OpenCL kernels.

   The leaf directory opencl-driver-intel-cpu/usr/lib/x86_64-linux-gnu/OpenCL/vendors/intel contains all the object files from the intel-cpu rpm, and opencl-driver-intel-tools/usr/lib/x86_64-linux-gnu/intel/opencl-1.2-3.0.67279/ contains the binaries from intel-devel rpm with tiny changes so the bash scripts point to a correct location.
   
   Once this is done, the only remaining file to tamper with is the actual .icd in opencl-driver-intel-cpu/etc/OpenCL/vendors/intel64.icd, which contains just one line with path to the library:
   /usr/lib/x86_64-linux-gnu/OpenCL/vendors/intel/libintelocl.so
4. Now just run `dpkg-deb --build opencl-driver-intel-cpu` and `dpkg-deb --build opencl-driver-intel-tools` and voila your debs with Intel's OpenCL are ready.

You can also download my opencl-driver-intel-cpu.deb and opencl-driver-intel-tools.deb.

Note that these are here just for reference, you should delete them after making sure that your package looks the same - ie I'm not redistributing these.

Once installed, a simple OpenCL app that lists the available platforms should output something like this:

PLATFORM_NAME: Intel(R) OpenCL
  VERSION: OpenCL 1.2 LINUX
  VENDOR: Intel(R) Corporation
  PROFILE: FULL_PROFILE
    DEVICE:       Intel(R) Core(TM) i7-3632QM CPU @ 2.20GHz
    DEVICE VENDOR: Intel(R) Corporation
    MAX COMPUTE UNITS: 8
    DEVICE VERSION: OpenCL 1.2 (Build 67279)
    DRIVER VERSION: 1.2

NVidia's OpenCL

In Ubuntu 13.04 there are now multiple nvidia packages which contain various versions of nvidia's driver, but I couldn't find anywhere a package with the icd file and therefore even if all the nvidia-3* packages do have the driver, the ICD loader isn't able to find it. Nonetheless, the fix for that is easy:

Run `locate libnvidia-opencl.so`, this will probably find a few files, in my case these are found:

/usr/lib/nvidia-304/libnvidia-opencl.so.304.88
/usr/lib/nvidia-current/libnvidia-opencl.so.1
/usr/lib32/nvidia-304/libnvidia-opencl.so.304.88
/usr/lib32/nvidia-current/libnvidia-opencl.so.1

For some reason though, the *.so.1 are broken symlinks in my case (maybe because I have Optimus laptop), therefore I created the .icd in the following manner:

echo /usr/lib/nvidia-304/libnvidia-opencl.so.304.88 > /etc/OpenCL/vendors/nvidia64.icd

The problem with this is that if the driver gets updated, you'll have to modify the .icd again, so if your *.so.1 symlink is not broken, you should use that instead.

Note that the driver also contains the OpenCL loader library (libOpenCL.so) and depending on your LD_LIBRARY_PATH settings, it might get used instead of the one provided by ocl-icd. That is not necessarily terrible, but keep in mind that NVidia's implementation is OpenCL 1.1, so even if other ICDs support 1.2, you'll be stuck with 1.1. Solution? Just remove the extra libOpenCL.so* from /usr/lib/nvidia-*/

Once done, listing CL platforms and devices should also list NVidia:

PLATFORM_NAME: NVIDIA CUDA
  VERSION: OpenCL 1.1 CUDA 4.2.1
  VENDOR: NVIDIA Corporation
  PROFILE: FULL_PROFILE
    DEVICE: GeForce GT 635M
    DEVICE VENDOR: NVIDIA Corporation
    MAX COMPUTE UNITS: 2
    DEVICE VERSION: OpenCL 1.1 CUDA
    DRIVER VERSION: 304.88

OpenCL and Bumblebee

As previously mentioned, I have an Optimus laptop (with integrated Intel GPU as well as NVidia which is used for more demanding applications). If the NVidia GPU is shut down, you won't see the NVidia platform as available (which I find a bit strange, should be a platform with 0 available devices, no?), but once turned on with a `optirun bash`, things should be working properly, although I haven't tried for example CL-GL interop, I can imagine that might not work with Bumblebee.

One issue I noticed though is that if you have both NVidia and Intel's drivers, Intel's driver will crash any OpenCL app run inside the optirun shell, which seems to be caused by the LD_PRELOAD libraries that VirtualGL uses. So either run your apps inside the shell with `LD_PRELOAD= ./myApp`, or just don't run them inside the optirun shell, use a regular one. As long as the GPU is active it is perfectly able to perform calculations even without VirtualGL set up.

AMD's driver

AMD enables to use both CPU and GPU devices with their driver. I don't have a GPU by AMD, so I only got to try the CPU implementation (luckily Intel's and AMD's CPUs are still compatible enough). The installer that they provide installs the whole SDK into /opt/AMDAPP, changes your /etc/profile to include the directories in LD_LIBRARY_PATH, and installs the icd to /etc/OpenCL/vendors.

What I don't like about this is that their SDK also contains libOpenCL.so, so it will be used instead of ocl-icd's. In this case this is less of a problem than in NVidia's case, cause AMD's implementation isn't limited to just OpenCL 1.1, but if you want to use ocl-icd, just remove the libOpenCL.so* from /opt/AMDAPP/lib/x86_64.

It would be nice to have a similar deb package for the AMD's CL driver, but I didn't get to that, maybe someone else wants to? ;) Anyway:

PLATFORM_NAME: AMD Accelerated Parallel Processing
  VERSION: OpenCL 1.2 AMD-APP (1113.2)
  VENDOR: Advanced Micro Devices, Inc.
  PROFILE: FULL_PROFILE
    DEVICE: Intel(R) Core(TM) i7-3632QM CPU @ 2.20GHz
    DEVICE VENDOR: GenuineIntel
    MAX COMPUTE UNITS: 8
    DEVICE VERSION: OpenCL 1.2 AMD-APP (1113.2)
    DRIVER VERSION: 1113.2 (sse2,avx)

Conclusion

As I was testing the various drivers I encountered quite a few issues - Intel's implementation crashes on ratGPU tests, AMD's pretends to work with my OpenCL face detection but doesn't detect anything (Intel's and NVidia's work fine), on top of that the crashes with Intel and Bumblebee/VirtualGL LD_PRELOAD shell. Samples from AMD's SDK crash when used with ocl-icd ICD loader because they call clReleaseContext(NULL), works with AMD's loader though. But in the end there is also a lot more that actually is working - for example a year ago my face detection didn't work at all with Intel's implementation, now it's fine, many of the SDK samples did work with all three drivers. I'd say there was some good progress.

So that's about the current OpenCL state, it's usable, just not out-of-the-box, I do hope that a year from now at least this post will be just saying "To use OpenCL just run `apt-get install opencl-driver-*`".