3

I'm a CUDA newbe I would greatly appreciate help/comment regarding CUDA installation. I've been fighting for a long time with installation on my Ubuntu 18.04 HP Z420 machine: with kernel: 5.3.0-46-generic, Xeon E5, nVidia Quadro K600, and nvidia-driver-440.
I cannot remember for sure in which order I installed what (apt install nvidia-cuda-toolkit vs apt install cuda), as I was installing and uninstalling for a few times due to constant errors. I think in the last trial I installed cuda, uninstalled it and installed nvidia-cuda-toolkit, what seemed to work. Finally I'm getting such outputs with two different versions (10.2 and 9.1).
Will it work? Which version will be used by external applications? Any comments and help will be most appreciated!

nvcc --version 
nvcc: NVIDIA (R) Cuda compiler driver
Copyright (c) 2005-2017 NVIDIA Corporation
Built on Fri_Nov__3_21:07:56_CDT_2017
Cuda compilation tools, release 9.1, V9.1.85


nvidia-smi
Sun Apr 19 19:28:29 2020       
+-----------------------------------------------------------------------------+
| NVIDIA-SMI 440.64.00    Driver Version: 440.64.00    CUDA Version: 10.2  

dpkg -l | grep cuda
rc  cuda-cudart-10-2                                            10.2.89-1                                                   amd64        CUDA Runtime native Libraries
rc  cuda-cudart-dev-10-2                                        10.2.89-1                                                   amd64        CUDA Runtime native dev links, headers
rc  cuda-cufft-10-2                                             10.2.89-1                                                   amd64        CUFFT native runtime libraries
rc  cuda-cupti-10-2                                             10.2.89-1                                                   amd64        CUDA profiling tools runtime libs.
rc  cuda-curand-10-2                                            10.2.89-1                                                   amd64        CURAND native runtime libraries
rc  cuda-cusolver-10-2                                          10.2.89-1                                                   amd64        CUDA solver native runtime libraries
rc  cuda-cusparse-10-2                                          10.2.89-1                                                   amd64        CUSPARSE native runtime libraries
rc  cuda-npp-10-2                                               10.2.89-1                                                   amd64        NPP native runtime libraries
rc  cuda-nvcc-10-2                                              10.2.89-1                                                   amd64        CUDA nvcc
rc  cuda-nvgraph-10-2                                           10.2.89-1                                                   amd64        NVGRAPH native runtime libraries
rc  cuda-nvjpeg-10-2                                            10.2.89-1                                                   amd64        NVJPEG native runtime libraries
rc  cuda-nvprof-10-2                                            10.2.89-1                                                   amd64        CUDA Profiler tools
rc  cuda-nvrtc-10-2                                             10.2.89-1                                                   amd64        NVRTC native runtime libraries
rc  cuda-nvtx-10-2                                              10.2.89-1                                                   amd64        NVIDIA Tools Extension
rc  cuda-sanitizer-api-10-2                                     10.2.89-1                                                   amd64        CUDA Sanitizer API
rc  cuda-toolkit-10-2                                           10.2.89-1                                                   amd64        CUDA Toolkit 10.2 meta-package
rc  cuda-visual-tools-10-2                                      10.2.89-1                                                   amd64        CUDA visual tools
ii  libcudart9.1:amd64                                          9.1.85-3ubuntu1                                             amd64        NVIDIA CUDA Runtime Library
ii  nvidia-cuda-dev                                             9.1.85-3ubuntu1                                             amd64        NVIDIA CUDA development files
ii  nvidia-cuda-doc                                             9.1.85-3ubuntu1                                             all          NVIDIA CUDA and OpenCL documentation
ii  nvidia-cuda-gdb                                             9.1.85-3ubuntu1                                             amd64        NVIDIA CUDA Debugger (GDB)
ii  nvidia-cuda-toolkit                                         9.1.85-3ubuntu1                                             amd64        NVIDIA CUDA development toolkit
djneuron
  • 31
  • Check https://askubuntu.com/questions/1227653/install-latest-nvidia-drivers-unmet-dependencies/1227742#1227742 You are probably in the dangerous state of having all your cuda files subject to autoremove, so don't do that until you fix things. – ubfan1 Apr 19 '20 at 19:03

1 Answers1

1

Credit: The following answer comes from StackOverflow:

CUDA has 2 primary APIs, the runtime and the driver API. Both have a corresponding version (e.g. 8.0, 9.0, etc.)

The necessary support for the driver API (e.g. libcuda.so on linux) is installed by the GPU driver installer.

The necessary support for the runtime API (e.g. libcudart.so on linux, and also nvcc) is installed by the CUDA toolkit installer (which may also have a GPU driver installer bundled in it).

In any event, the (installed) driver API version may not always match the (installed) runtime API version, especially if you install a GPU driver independently from installing CUDA (i.e. the CUDA toolkit).

The nvidia-smi tool gets installed by the GPU driver installer, and generally has the GPU driver in view, not anything installed by the CUDA toolkit installer.

Recently (somewhere between 410.48 and 410.73 driver version on linux) the powers-that-be at NVIDIA decided to add reporting of the CUDA Driver API version installed by the driver, in the output from nvidia-smi.

This has no connection to the installed CUDA runtime version.

nvcc, the CUDA compiler-driver tool that is installed with the CUDA toolkit, will always report the CUDA runtime version that it was built to recognize. It doesn't know anything about what driver version is installed, or even if a GPU driver is installed.

Therefore, by design, these two numbers don't necessarily match, as they are reflective of two different things.

If you are wondering why nvcc -V displays a version of CUDA you weren't expecting (e.g. it displays a version other than the one you think you installed), it may be because you haven't followed the mandatory instructions in step 7 of the cuda linux install guide.

Note that although this question mostly has linux in view, the same concepts apply to windows CUDA installs. The driver has a CUDA driver version associated with it (which can be queried with nvidia-smi, for example). The CUDA runtime also has a CUDA runtime version associated with it. The two will not necessarily match in all cases.

In most cases, if nvidia-smi reports a CUDA version that is numerically equal to or higher than the one reported by nvcc -v, this is not a cause for concern. That is a defined compatibility path in CUDA (newer drivers/driver API support "older" CUDA toolkits/runtime API). For example if nvidia-smi reports CUDA 10.2, and nvcc -v reports CUDA 10.1, that is generally not cause for concern. It should just work, and it does not necessarily mean that you "actually installed CUDA 10.2 when you meant to install CUDA 10.1"

If_You_Say_So
  • 111
  • 3