Tracking temperature of GPU/CPU

[cedricbonhomme]

First, thank you for this project! I really love it.

Do you plan to provide a way to track GPU/CPU temperatures?

If not, could you recommend a method to do this? I’ve already found a few possibilities but am open to other ideas. Ideally, it would be great to have the temperature stored in the emissions.csv file—perhaps as the average temperature per device during a training process.

I’m asking because I recently noticed some performance “degradation” during summer and warm spring temperatures. You can see an example in this report:
GPU Efficiency in VLAI Model Training.
(or see the joined picture)

In the section "Evolution of Experiments in Environment A", the charts were generated directly from CodeCarbon data. It’s really interesting, but I’m missing temperature data.

Again, if there is no way or you are not planning to add this possibility, all good. In this case I will find other ways.

[benoit-cty]

It's a good idea, we plan to add CPU load #911 we could do the same for temperature.

pynvml.nvmlDeviceGetTemperature will give GPU temp.

For CPU maybe psutil.sensors_temperatures() -> 'coretemp'