This is not scientific but I took an old Xeon V4 workstation and an old Quadro GPU and compared the Whisper times for the same approximately 4 minute audio, transcribing in English.

Here are the times which is slightly surpising.

I used the small model with this command: 

whisper vid.mp4 --model small --output_format srt --language en

Of course to test the GPU I added the --device cuda flag and confirmed GPU was being used as utilization on GPU was 100%

CPU Time

real    3m49.033s
 

GPU Time

real    3m55.140s
 

Conclusion

I observed it took some time to load the model into GPU memory, for shorter audio this increased the overall processing time slightly.  I believe if I have time to test a longer audio that the result will be different and GPU will show a better result.

Latest Articles