From 5da4c816f9ddfef8c89e82bde8fcf7f9ded1547f Mon Sep 17 00:00:00 2001 From: Manu Mathew Date: Wed, 29 Apr 2020 17:51:27 +0530 Subject: [PATCH] quantization doc update --- README.md | 8 ++++---- docs/Quantization.md | 16 ++++++++-------- 2 files changed, 12 insertions(+), 12 deletions(-) diff --git a/README.md b/README.md index abc93ff..f8994f6 100644 --- a/README.md +++ b/README.md @@ -29,20 +29,20 @@ This code also includes tools for **Quantization Aware Training** that can outpu ## Examples The following examples are currently available. Click on each of the links below to go into the full description of the example.

-* **Image Classification**
+- **Image Classification**
- [**Image Classification**](docs/Image_Classification.md)

-* **Pixel2Pixel Prediction**
+- **Pixel2Pixel Prediction**
- [**Semantic Segmentation**](docs/Semantic_Segmentation.md)
- [Depth Estimation](docs/Depth_Estimation.md)
- [Motion Segmentation](docs/Motion_Segmentation.md)
- [**Multi Task Estimation**](docs/Multi_Task_Learning.md)

-* **Object Detection**
+- **Object Detection**
- Object Detection - coming soon..
- Object Keypoint Estimation - coming soon..

-* **Quantization**
+- **Quantization**
- [**Quantization Aware Training**](docs/Quantization.md)

diff --git a/docs/Quantization.md b/docs/Quantization.md index bd01774..a11e346 100644 --- a/docs/Quantization.md +++ b/docs/Quantization.md @@ -10,14 +10,14 @@ TI Deep Learning Library (TIDL) is a highly optimized runtime for Deep Learning - Quantization Aware Training (QAT): This is needed if accuracy obtained with Calibration is not satisfactory (eg. Quantization Accuracy Drop >2%). QAT operates as a second phase after the initial training in floating point is done. We have provide this PyTorch Jacinto AI DevKit to enable QAT with PyTorch. There also a plan to make TensorFlow Jacinto AI DevKit available. Further Details are available at: [https://github.com/TexasInstruments/jacinto-ai-devkit](https://github.com/TexasInstruments/jacinto-ai-devkit)
## Guidelines For Training To Get Best Accuracy With Quantization -To get best accuracy at the quantization stage, it is important that the model is trained carefully, following the guidelines (even during the floating point training). Having spent a lot of time solving quantization issues, we would like to highlight that following these guidelines is of at most importance. If these guidelines are not followed, there is a high risk that the tools and techniques described here may not solve the accuracy drop due to quantization. -* We recommend that the training uses sufficient amount of regularization / weight decay. Regularization / weight decay ensures that the weights, biases and other parameters (if any) are small and compact - this is good for quantization. These features are supported in most of the popular training framework. -* We have noticed that some training code bases do not use weight decay for biases. Some other code bases do not use weight decay for the parameters in Depthwise convolution layers. All these are bad strategies for quantization. These poor choices done (probably to get a 0.1% accuracy lift with floating point) will result in a huge degradation in fixed point - sometimes several percentage points. The weight decay factor should not be too small. We have used a weight decay factor of 1e-4 for training several networks and we highly recommend a similar value. Please do no use small values such as 1e-5. -* We also highly recommend to use Batch Normalization immediately after every Convolution layer. This helps the feature map to be properly regularized/normalized. If this is not done, there can be accuracy degradation with quantization. This especially true for Depthwise Convolution layers. However applying Batch Normalization to the very last Convolution layer (for example, the prediction layer in segmentation/object detection network) may hurt accuracy and can be avoided. - -To summarize, if you are getting poor accuracy with quntization, please check the following: -* Weight decay is applied to all layers / parameters and that weight decay factor is good. -* Ensure that the Convolution layers in the network have Batch Normalization layers immediately after that. The only exception allowed to this rule is for the very last Convolution layer in the network (for example the prediction layer in a segmentation network or detection network, where adding Batch normalization might hurt the floating point accuracy). +To get best accuracy at the quantization stage, it is important that the model is trained carefully, following the guidelines (even during the floating point training). Having spent a lot of time solving quantization issues, we would like to highlight that following these guidelines is of at most importance. If these guidelines are not followed, there is a high risk that the tools and techniques described here may not solve the accuracy drop due to quantization.
+- We recommend that the training uses sufficient amount of regularization / weight decay. Regularization / weight decay ensures that the weights, biases and other parameters (if any) are small and compact - this is good for quantization. These features are supported in most of the popular training framework.
+- We have noticed that some training code bases do not use weight decay for biases. Some other code bases do not use weight decay for the parameters in Depthwise convolution layers. All these are bad strategies for quantization. These poor choices done (probably to get a 0.1% accuracy lift with floating point) will result in a huge degradation in fixed point - sometimes several percentage points. The weight decay factor should not be too small. We have used a weight decay factor of 1e-4 for training several networks and we highly recommend a similar value. Please do no use small values such as 1e-5.
+- We also highly recommend to use Batch Normalization immediately after every Convolution layer. This helps the feature map to be properly regularized/normalized. If this is not done, there can be accuracy degradation with quantization. This especially true for Depthwise Convolution layers. However applying Batch Normalization to the very last Convolution layer (for example, the prediction layer in segmentation/object detection network) may hurt accuracy and can be avoided.
+ +To summarize, if you are getting poor accuracy with quntization, please check the following:
+- Weight decay is applied to all layers / parameters and that weight decay factor is good.
+- Ensure that the Convolution layers in the network have Batch Normalization layers immediately after that. The only exception allowed to this rule is for the very last Convolution layer in the network (for example the prediction layer in a segmentation network or detection network, where adding Batch normalization might hurt the floating point accuracy).
## Post Training Calibration For Quantization (PTQ a.k.a. Calibration) **Note: this is not our recommended method in PyTorch.**
-- 2.39.2