11

I have installed cuda 7 toolkit with newest driver, and it passed all the tests, it seems that driver is loaded, but every time when I just put my laptop to sleep, and wake it up from sleep, cuda doesn't work anymore, deviceQuery

./deviceQuery Starting...

CUDA Device Query (Runtime API) version (CUDART static linking)

cudaGetDeviceCount returned 38
-> no CUDA-capable device is detected
Result = FAIL
Karpov
  • 111

2 Answers2

16

http://forums.fast.ai/t/cuda-lib-not-working-after-suspend-on-ubuntu-16-04/3546/4

You can reload the nvidia kernel module and it will work fine:

sudo rmmod nvidia_uvm
sudo modprobe nvidia_uvm

Notice though that this won't work unless all applications using that module are closed.

piegames
  • 268
  • 2
    This solved my problem! I'm running with a Quadro K2100M on fedora28. Before trying this, I tried "sudo /usr/bin/nvidia-smi -pm ENABLED; sudo /usr/bin/nvidia-smi -c EXCLUSIVE_PROCESS" without success. I've always had to reboot in the past. – Aaron Swan Nov 08 '18 at 20:28
  • Closing all applications using nvidia_uvm by dropping into shell with init 3 or ctrl+Fx plus restarting the display manager has worked consistently for me. sudo systemctl restart display-manager or sudo service gdm restart

    Related Q&A: https://askubuntu.com/questions/1128057/weid-cuda-driver-state-after-forgetting-to-clean-up-resources-before-suspending and https://askubuntu.com/questions/1228423/how-do-i-fix-cuda-breaking-after-suspend/

    I'm running 22.04, X11, GNOME V 42.9, nvidia-driver-535, CUDA Toolkit 12.2

     – smcnally Mar 08 '24 at 21:11
6

https://chuanwen.wordpress.com/2015/07/19/run-cuda-on-ubuntu-14-04-2/

What if cuda does not work after laptop wake up from sleep:

Put following 2 lines in your /etc/rc.local:

/usr/bin/nvidia-smi -pm ENABLED
/usr/bin/nvidia-smi -c EXCLUSIVE_PROCESS
snoop
  • 4,040
  • 9
  • 40
  • 58
Dog_And_Cat
  • 76