Low voltage GPU-based AI accelerator

Abstract

During the past decades, the performance and the power efficiency of computer processors has increased drastically. However, the amount of computers present in the world and the demand for calculations run has risen at an equally rapid pace. An estimated 4% of all electricity being consumed is used by computers, largely in massive data centers. With the rise of very large artificial intelligence models, this trend is likely to continue.

Majority of the digital processors on the market leave a significant unused safety voltage margin on top of their "best operating voltage", to ensure error-free operation despite individual differences in the processor, different levels of aging and different environmental factors, e.g, ambient temperature. Removing this safety margin would reduce the power consumption of the processors while maintaining their performance. Multiple hardware solutions have been proposed for this purpose, however, none of them have made their way into mass market, mainly due to prohibitive cost of implementation or lacking enough reliability.

This thesis explores the viability of using a completely software-based solution for detecting timing errors, which arise when the processor’s operating voltage is reduced below the safe limits advised by the chip manufacturer. The method is tested in context of deep convolutional neural network models that are accelerated using an off-the-shelf graphics card. The results show that lowering the operating voltage to the minimum safe levels can save power by up to 21%, while the energy overhead caused by the error detection is under 3.5%.