| Title |
Layer-adaptive Gradient Clipping in Quantization-Aware Training |
| Authors |
박지훈(Jihoon Park) ; 송병철(Byung Cheol Song) |
| DOI |
https://doi.org/10.5573/ieie.2023.60.8.37 |
| Keywords |
Quantization-Aware training; Gradient clipping; Layer-adaptive |
| Abstract |
Quantization in neural network folds the batch normalization layer, so the normalization effect cannot be used and accuracy can be degraded. To avoid the accuracy degradation, this paper achieves stabilized training by applying Adaptive Gradient Clipping (AGC) to Quantization-Aware Training (QAT). Also, we propose a neural network with MLP that takes random clipping vector as inputs for gradient clipping adaptively for each layer. |