3

I am trying to measure nvidia GPU utilization on Ubuntu, as explained in How to measure GPU usage? with results as accurate as possible. I set the nvidia-smi loop interval to 5 ms, and indeed I am getting about 200 samples a second. Unfortunately it seems that the nvidia-smi can only produce results in resolution of ~170ms (so for each such interval I'm getting about 35 samples which are exactly the same).

Is there any way to increase the nvidia-smi sampling resolution?

enter image description here

Elad Weiss
  • 209

1 Answers1

2

No, if using nvidia-smi or any bindings (Python/Perl/Go) with NVML. The highest resolution is recorded in NVML API Guide that,

Utilization information for a device. Each sample period may be between 1 second and 1/6 second, depending on the product being queried.

Therefore, your observation that "resolution of ~170ms" is quite precise. For further accuracy, it is recommended to use DCGM (only suitable for certain series of GPUs).

Luke
  • 36
  • 3