summary | shortlog | log | commit | commitdiff | tree
raw | patch | inline | side by side (parent: 0fa7649)
raw | patch | inline | side by side (parent: 0fa7649)
| author | Manu Mathew <a0393608@ti.com> | |
| Mon, 6 Jan 2020 05:44:00 +0000 (11:14 +0530) | ||
| committer | Manu Mathew <a0393608@ti.com> | |
| Mon, 6 Jan 2020 05:44:00 +0000 (11:14 +0530) |
22 files changed:
diff --git a/README.md b/README.md
index 8eafb6b9db34f3dc410cc9475eaf2ac31e4034fc..3d0a207e9d66b982c010fcb08f46f659da4c631b 100644 (file)
--- a/README.md
+++ b/README.md
* Object Detection - coming soon..<br>
* Object Keypoint Estimation - coming soon..<br>
* Quantization<br>
* Object Detection - coming soon..<br>
* Object Keypoint Estimation - coming soon..<br>
* Quantization<br>
- * [**Post Training Calibration For Quantization**](docs/Quantization.md)<br>
* [**Quantization Aware Training**](docs/Quantization.md)<br>
Some of the common training and validation commands are provided in shell scripts (.sh files) in the root folder.
* [**Quantization Aware Training**](docs/Quantization.md)<br>
Some of the common training and validation commands are provided in shell scripts (.sh files) in the root folder.
index a05cc5d387ff220e9d37bee4aa05d0c4b89cc128..e832ee80ddadf1f9b8f112a72f2b0886733b61cb 100644 (file)
|Dataset |Mode Name |Resize Resolution|Crop Resolution|Complexity (GigaMACS)|MeanIoU% |
|---------|---------- |----------- |---------- |-------- |-------- |
|ImageNet |MobileNetV1 |256x256 |224x224 |0.568 |**71.83**|
|Dataset |Mode Name |Resize Resolution|Crop Resolution|Complexity (GigaMACS)|MeanIoU% |
|---------|---------- |----------- |---------- |-------- |-------- |
|ImageNet |MobileNetV1 |256x256 |224x224 |0.568 |**71.83**|
-|ImageNet |MobileNetV2 |256x256 |224x224 |0.296 |**71.89**|
+|ImageNet |MobileNetV2 |256x256 |224x224 |0.296 |**72.13**|
|ImageNet |ResNet50 |256x256 |224x224 | | |
|.
|ImageNet |MobileNetV1[1]|256x256 |224x224 |0.569 |70.60 |
|ImageNet |ResNet50 |256x256 |224x224 | | |
|.
|ImageNet |MobileNetV1[1]|256x256 |224x224 |0.569 |70.60 |
diff --git a/docs/Quantization.md b/docs/Quantization.md
index ff07b768fb6c4cef113c5b922e095efa8be5abd3..376a29a51c1a7ccbd1ba867a6de4628e49332b16 100644 (file)
--- a/docs/Quantization.md
+++ b/docs/Quantization.md
<p float="left"> <img src="quantization/pact2_activation.png" width="640" hspace="5"/> </p>
This code also contains two wrapper modules called QuantCalibrateModule and QuantTrainModule that will handle such tasks as replacing ReLUs with PACT2, quantizing weights, quantizing activations etc - so the user does not need to do any of these things - the only thing that is required is to wrap the user's module in these wapper modules as explained below.
<p float="left"> <img src="quantization/pact2_activation.png" width="640" hspace="5"/> </p>
This code also contains two wrapper modules called QuantCalibrateModule and QuantTrainModule that will handle such tasks as replacing ReLUs with PACT2, quantizing weights, quantizing activations etc - so the user does not need to do any of these things - the only thing that is required is to wrap the user's module in these wapper modules as explained below.
-There are two primary methods of quantization - Post Training Calibration For Quantization and Trained Quantization (a.k.a Quantization Aware Training).
+There are two primary methods of quantization - Post Training Calibration For Quantization and Trained Quantization (a.k.a Quantization Aware Training). These required the following (only for information - automatically done by QuantCalibrateModule and QuantTrainModule):
+- Replace all the ReLU, ReLU6 layers in the model by PACT2. Insert PACT2 after Convolution+BatchNorm if a ReLU is missing after that. Insert PACT2 anywhere else required - where activation range clipping and range collection is required. For example it can ne after the Fully Connected Layer. We use forward post hooks of PyTorch nn.Modules to call these extra activation functions. Thus we are able to add these extra activations without disturbing the loading of existing pre-trained weights.
+- Clip the weights to an appropriate range if the weight range is very high.
+- Quantize the weights and activations during the forward pass.
+- Other modifications required for Calibration / Trained Quantization
-## Post Training Calibration For Quantization
+Trained Quantization typically provides better accuracy compared to Post Training Calibration. Trained Quantization is also easy to incorporate into your existing PyTorch training code - hence we recommend to use it. Here, we explain both methods for the sake of completeness.
+## Post Training Calibration For Quantization
+Note: Although Calibration methods are described in detail here, this is not our recommended method as this does not yield the best results in some cases. There are specific scenarios where Trained Quantization is not possible (eg. to import a floating point model for embedded inference) and Calibration is mainly meant for such scenarios. However, Calibration methods are improving at a rapid pace and it is possible that Calibration methods will be good enough for all cases in the future - but for now, you can skip to the section on Trained Quantization as it is the recommended method.
+
Post Training Calibration & Quantization can take a model trained in floating point and with a few steps convert it to a model that is friendly for quantized inference. Compared to the alternative (Trained Quantization), the advantages of this method are:
- Calibration is fast - a typical calibration finishes in a few minutes.
- Ground truth is not required - just input images are sufficient.
- Loss function or backward (back-propagation) are not required.
Post Training Calibration & Quantization can take a model trained in floating point and with a few steps convert it to a model that is friendly for quantized inference. Compared to the alternative (Trained Quantization), the advantages of this method are:
- Calibration is fast - a typical calibration finishes in a few minutes.
- Ground truth is not required - just input images are sufficient.
- Loss function or backward (back-propagation) are not required.
-Thus, this is the preferred method of quantization from an ease of use point of view. As explained earlier, in this method, the training happens entirely in floating point. The inference (possibly in an embedded device) happens in fixed point. In between training and fixed point inference, the model goes through the step called Calibration with some sample images. The Calibration happens in PC and Quantized Inference happens in the embedded device. Calibration basically tries to make the quantized output similar to the floating point output - by choosing appropriate activation ranges, weights and biases.
+The disadvantage is:
+- This is a fast and approximate method of quantizing the model and may not always yield the best accuracy.
+
+In Post Training Calibration, the training happens entirely in floating point. The inference (possibly in an embedded device) happens in fixed point. In between training and fixed point inference, the model goes through the step called Calibration with some sample images. The Calibration happens in PC and Quantized Inference happens in the embedded device. Calibration basically tries to make the quantized output similar to the floating point output - by choosing appropriate activation ranges, weights and biases.
A block diagram of Post Training Calibration is shown below:
<p float="left"> <img src="quantization/bias_calibration.png" width="640" hspace="5"/> </p>
A block diagram of Post Training Calibration is shown below:
<p float="left"> <img src="quantization/bias_calibration.png" width="640" hspace="5"/> </p>
Out of these methods, **Advanced Calib** is our recommended Calibration method as of now, as it has the best trade-off between the Accuracy and the features required during fixed point inference. All the Calibration scripts that we have in this page uses "Advanced Calib" by default. Other Calibration methods described here are for information only.
Out of these methods, **Advanced Calib** is our recommended Calibration method as of now, as it has the best trade-off between the Accuracy and the features required during fixed point inference. All the Calibration scripts that we have in this page uses "Advanced Calib" by default. Other Calibration methods described here are for information only.
-#### How to use QuantCalibrateModule
-- The section briefly explains how to make use of our helper/wrapper module to do the calibration of your model. For further details, please see pytorch_jacinto_ai.engine.train_classification.py and pytorch_jacinto_ai.engine.train_pixel2pixel.py. The step by step process is as follows:
-
-###### Model preparation:
-- Replace all the ReLU, ReLU6 layers in the model by PACT2. Insert PACT2 after Convolution+BatchNorm if a ReLU is missing after that. Insert PACT2 anywhere else required - where activation range clipping and range collection is required. For example it can ne after the Fully Connected Layer. We use forward post hooks of PyTorch nn.Modules to call these extra activation functions. Thus we are able to add these extra activations without disturbing the loading of existing pre-trained weights.
-- Clip the weights to an appropriate range if the weight range is very high.
-- Note that this Model preparation is automatically done by QuantCalibrateModule
+In order to do Calibration easily we have a developed a wrapper module called QuantCalibrateModule, which is located in pytorch_jacinto_ai.xnn.quantize.QuantCalibrateModule. We make use of a kind of Parametric Activation called **PACT2** in order to store the calibrated ranges of activations. PACT2 is a improved form of PACT [[1]]. **PACT2 uses power of 2 activation ranges** for activation clipping. PACT2 can learn ranges very quickly (using a statistic method) without back propagation - this feature makes it quite attractive for Calibration. Our wrapper module replaces all the ReLUs in the model with PACT2. It also inserts PACT2 in other places where activation ranges need to be collected. Statistical range clipping in PACT2 improves the Quantized Accuracy over simple min-max range clipping.
-###### Forward iterations:
+#### What happens during Calibration?
- For each iteration perform a forward in floating point using the original weights and biases. During this pass PACT2 layers will collect output ranges using histogram and running average.
- In addition, perform Convolution+BatchNorm merging and quantization of the resulting weights. These quantized and de-quantized weights are used in a forward pass. Ranges collected by PACT2 is used for activation quantization (and de-quantization) to generate quantized output.
- The floating point output and quantized output are compared using statistic measures. Using such statistic measures, we can adjust the weights and biases of Convolutions and Batch Normalization layers - so that the quantized output becomes closer to the floating point output.
- For each iteration perform a forward in floating point using the original weights and biases. During this pass PACT2 layers will collect output ranges using histogram and running average.
- In addition, perform Convolution+BatchNorm merging and quantization of the resulting weights. These quantized and de-quantized weights are used in a forward pass. Ranges collected by PACT2 is used for activation quantization (and de-quantization) to generate quantized output.
- The floating point output and quantized output are compared using statistic measures. Using such statistic measures, we can adjust the weights and biases of Convolutions and Batch Normalization layers - so that the quantized output becomes closer to the floating point output.
-- Within a few iterations, we could get reasonable quantization accuracy for several models that we tried this method on.
-
-In order to do Calibration easily we have a developed a wrapper module called QuantCalibrateModule, which is located in pytorch_jacinto_ai.xnn.quantize.QuantCalibrateModule. We make use of a kind of Parametric Activation called **PACT2** in order to store the calibrated ranges of activations. PACT2 is a improved form of PACT [[1]]. **PACT2 uses power of 2 activation ranges** for activation clipping. PACT2 can learn ranges very quickly (using a statistic method) without back propagation - this feature makes it quite attractive for Calibration. Our wrapper module replaces all the ReLUs in the model with PACT2. It also inserts PACT2 in other places where activation ranges need to be collected. Statistical range clipping in PACT2 improves the Quantized Accuracy over simple min-max range clipping.
+- Within a few iterations, we should get reasonable quantization accuracy.
+#### How to use QuantCalibrateModule
As explained, the method of **Calibration does not need ground truth, loss function or back propagation.** However in the calibration script, we make use of ground truth to measure the loss/accuracy even in the Calibration stage - although that is not necessary.
As explained, the method of **Calibration does not need ground truth, loss function or back propagation.** However in the calibration script, we make use of ground truth to measure the loss/accuracy even in the Calibration stage - although that is not necessary.
-###### Sample Code Snippet:
+The section briefly explains how to make use of our helper/wrapper module to do the calibration of your model. For further details, please see pytorch_jacinto_ai.engine.train_classification.py and pytorch_jacinto_ai.engine.train_pixel2pixel.py. The step by step process is as follows:
+
```
# create your model here:
model = ...
```
# create your model here:
model = ...
torch.onnx.export(model.module, dummy_input, os.path.join(save_path,'model.onnx'), export_params=True, verbose=False)
```
torch.onnx.export(model.module, dummy_input, os.path.join(save_path,'model.onnx'), export_params=True, verbose=False)
```
+Careful attention needs to be given to how the pretrained model is loaded and trained model is saved as shown in the above code snippet.
Few examples of calibration are provided below. These commands are also listed in the file **run_quantization.sh** for convenience.<br>
Few examples of calibration are provided below. These commands are also listed in the file **run_quantization.sh** for convenience.<br>
@@ -93,20 +98,52 @@ python ./scripts/train_segmentation_main.py --phase calibration --dataset_name c
--batch_size 12 --quantize True --epochs 1 --epoch_size 100
```
--batch_size 12 --quantize True --epochs 1 --epoch_size 100
```
-## Trained Quantization (a.k.a Quantization Aware Training)
-As explained in the previous section, Calibration is our preferred method of making a model quantization friendly. However, in exceptional cases, it is possible that the drop in accuracy during calibration is more than acceptable. In this case, Trained Quantization can be used.
+## Trained Quantization a.k.a Quantization Aware Training (recommended method)
+Trained Quantization typically provides better accuracy compared to Post Training Calibration.
-Unlike Calibration, Trained Quantization involves ground truth, loss function and back propagation. The most popular method of trained quantization is [[4]]. It takes care of merging Convolution layers with the adjascent Batch Normalization layers (on-the-fly) during the quantized training (if this merging is not correctly done, quantized training may not improve the accuracy). In addition, we use Straight-Through Estimation (STE) [[2,3]] to improve the gradient flow in back-propagation. Also, the statistical range clipping in PACT2 improves the Quantized Accuracy over simple min-max range clipping.
+Unlike Calibration, Trained Quantization involves ground truth, loss function and back propagation - hence it is very similar to the original floating point training.
+
+The most popular method of trained quantization is [[4]]. It takes care of merging Convolution layers with the adjascent Batch Normalization layers (on-the-fly) during the quantized training (if this merging is not correctly done, quantized training may not improve the accuracy). In addition, we use Straight-Through Estimation (STE) [[2,3]] to improve the gradient flow in back-propagation. Also, the statistical range clipping in PACT2 improves the Quantized Accuracy over simple min-max range clipping.
Note: Instead of STE and statistical ranges for PACT2, we also tried out approximate gradients for scale and trained quantization thresholds proposed in [[5]] (We did not use the gradient nomralization and log-domain training mentioned in the paper). We found that method to be able to learn the clipping thresholds for initial few epochs, but became unstable after a few epochs and loss became high. Compared to that learned thresholds method, our statistical PACT2 ranges/thresholds combined with STE is simple and stable.
A block diagram of Trained Quantization is shown below:
<p float="left"> <img src="quantization/trained_quant_ste.png" width="640" hspace="5"/> </p>
Note: Instead of STE and statistical ranges for PACT2, we also tried out approximate gradients for scale and trained quantization thresholds proposed in [[5]] (We did not use the gradient nomralization and log-domain training mentioned in the paper). We found that method to be able to learn the clipping thresholds for initial few epochs, but became unstable after a few epochs and loss became high. Compared to that learned thresholds method, our statistical PACT2 ranges/thresholds combined with STE is simple and stable.
A block diagram of Trained Quantization is shown below:
<p float="left"> <img src="quantization/trained_quant_ste.png" width="640" hspace="5"/> </p>
+#### What happens during Trained Quantization?
+- For each iteration perform a forward in floating point using the original weights and biases. During this pass PACT2 layers will collect output ranges using histogram and running average.
+- In addition, perform Convolution+BatchNorm merging and quantization of the resulting weights. These quantized and de-quantized weights are used in a forward pass. Ranges collected by PACT2 is used for activation quantization (and de-quantization) to generate quantized output.
+- Backpropagation with STE will update the parameters of the model to reduce the loss with quantization.
+- Within a few epochs, we should get reasonable quantization accuracy.
+
+#### How to use QuantTrainModule
In order to enable quantized training, we have developed the wrapper class pytorch_jacinto_ai.xnn.quantize.QuantTrainModule. The usage of this module can be seen in pytorch_jacinto_ai.engine.train_classification.py and pytorch_jacinto_ai.engine.train_pixel2pixel.py.
```
In order to enable quantized training, we have developed the wrapper class pytorch_jacinto_ai.xnn.quantize.QuantTrainModule. The usage of this module can be seen in pytorch_jacinto_ai.engine.train_classification.py and pytorch_jacinto_ai.engine.train_pixel2pixel.py.
```
+# create your model here:
+model = ...
+
+# create a dummy input - this is required to analyze the model - fill in the input image size expected by your model.
+dummy_input = torch.rand((1,3,384,768))
+
+#wrap your model in QuantTrainModule. Once it is wrapped, the actual model is in model.module
model = pytorch_jacinto_ai.xnn.quantize.QuantTrainModule(model, dummy_input=dummy_input)
model = pytorch_jacinto_ai.xnn.quantize.QuantTrainModule(model, dummy_input=dummy_input)
+
+# load your pretrained weights here into model.module
+pretrained_data = torch.load(pretrained_path)
+model.module.load_state_dict(pretrained_data)
+
+# your training loop here with with loss, backward, optimizer and scheduler.
+# this is the usual training loop - but use a lower learning rate such as 5e-5
+....
+....
+
+# save the model - the calibrated module is in model.module
+torch.save(model.module.state_dict(), os.path.join(save_path,'model.pth'))
+torch.onnx.export(model.module, dummy_input, os.path.join(save_path,'model.onnx'), export_params=True, verbose=False)
```
```
+
+As can be seen, it is very easy to incorporate QuantTrainModule in your existing training code as the only thing required is to wrap your original model in QuantTrainModule. Careful attention needs to be given to how the pretrained model is loaded and trained model is saved as shown in the above code snippet.
+
The Model Preparation steps used for Calibration applies for Trained Quantization as well. - Note that this Model preparation is automatically done by QuantTrainModule. The resultant model can then be used for training as usual and it will take care of quantization constraints during the training forward and backward passes.
One word of caution is that our current implementation of Trained Quantization is a bit slow. The reason for this slowdown is that our implementation is using the top-level python layer of PyTorch and not the underlying C++ layer. But with PyTorch natively supporting the functionality required for quantization under the hood - we hope that this speed issue can be resolved in a future update.
The Model Preparation steps used for Calibration applies for Trained Quantization as well. - Note that this Model preparation is automatically done by QuantTrainModule. The resultant model can then be used for training as usual and it will take care of quantization constraints during the training forward and backward passes.
One word of caution is that our current implementation of Trained Quantization is a bit slow. The reason for this slowdown is that our implementation is using the top-level python layer of PyTorch and not the underlying C++ layer. But with PyTorch natively supporting the functionality required for quantization under the hood - we hope that this speed issue can be resolved in a future update.
index a341c860df8830271bf14525827bd2db727ca1be..b2c86b780aaa5b1e62fdd4085b59e86d101c7f3f 100644 (file)
@@ -83,22 +83,24 @@ Inference can be done as follows (fill in the path to the pretrained model):<br>
### Cityscapes Segmentation
### Cityscapes Segmentation
-|Dataset |Mode Architecture |Backbone Model|Backbone Stride|Resolution |Complexity (GigaMACS)|MeanIoU% |Model Configuration Name |
-|--------- |---------- |----------- |-------------- |-----------|-------- |----------|--------------------------------- |
-|Cityscapes |DeepLabV3Lite |MobileNetV2 |16 |768x384 |3.54 |**69.13** |deeplabv3lite_mobilenetv2_tv |
-|Cityscapes |FPNPixel2Pixel |MobileNetV2 |32 |768x384 |3.84 |**70.39** |fpn_pixel2pixel_mobilenetv2_tv |
-|Cityscapes |FPNPixel2Pixel |MobileNetV2 |64 |1536x768 |3.96 |**71.28** |fpn_pixel2pixel_mobilenetv2_tv_es64|
-|Cityscapes |FPNPixel2Pixel |MobileNetV2 |64 |2048x1024 |7.03 | |fpn_pixel2pixel_mobilenetv2_tv_es64|
-|Cityscapes |DeepLabV3Lite |MobileNetV2 |16 |1536x768 |14.48 |**73.59** |deeplabv3lite_mobilenetv2_tv |
-|Cityscapes |FPNPixel2Pixel |MobileNetV2 |32 |1536x768 |15.37 |**74.98** |fpn_pixel2pixel_mobilenetv2_tv |
-
-|Dataset |Mode Architecture |Backbone Model|Backbone Stride|Resolution |Complexity (GigaMACS)|MeanIoU% |Model Configuration Name |
-|--------- |---------- |----------- |-------------- |-----------|-------- |----------|--------------------------------- |
-|Cityscapes |ERFNet[[4]] | | |1024x512 |27.705 |69.7 |N/A |
-|Cityscapes |SwiftNetMNV2[[5]] |MobileNetV2 | |2048x1024 |41.0 |75.3 |N/A |
-|Cityscapes |DeepLabV3Plus[[6,7]]|MobileNetV2 |16 | |21.27 |70.71 |N/A |
-|Cityscapes |DeepLabV3Plus[[6,7]]|Xception65 |16 | |418.64 |78.79 |N/A |
-
+|Dataset |Mode Architecture |Backbone Model|Backbone Stride|Resolution |Complexity (GigaMACS)|MeanIoU% |Model Configuration Name |
+|--------- |---------- |----------- |-------------- |-----------|-------- |----------|---------------------------------------- |
+|Cityscapes |FPNPixel2Pixel with DWASPP|MobileNetV2 |64 |768x384 |0.99 |62.43 |fpn_pixel2pixel_aspp_mobilenetv2_tv_es64 |
+|Cityscapes |DeepLabV3Lite with DWASPP |MobileNetV2 |16 |768x384 |**3.54** |**69.13** |**deeplabv3lite_mobilenetv2_tv** |
+|Cityscapes |FPNPixel2Pixel |MobileNetV2 |32 |768x384 |3.66 |- |fpn_pixel2pixel_mobilenetv2_tv |
+|Cityscapes |FPNPixel2Pixel with DWASPP|MobileNetV2 |32 |768x384 |3.84 |70.39 |fpn_pixel2pixel_aspp_mobilenetv2_tv |
+|Cityscapes |FPNPixel2Pixel |MobileNetV2 |64 |1536x768 |3.85 |69.82 |fpn_pixel2pixel_mobilenetv2_tv_es64 |
+|Cityscapes |FPNPixel2Pixel with DWASPP|MobileNetV2 |64 |1536x768 |**3.96** |**71.28** |**fpn_pixel2pixel_aspp_mobilenetv2_tv_es64**|
+|Cityscapes |FPNPixel2Pixel with DWASPP|MobileNetV2 |64 |2048x1024 |7.03 |72.67 |fpn_pixel2pixel_aspp_mobilenetv2_tv_es64 |
+|Cityscapes |DeepLabV3Lite with DWASPP |MobileNetV2 |16 |1536x768 |14.48 |73.59 |deeplabv3lite_mobilenetv2_tv |
+|Cityscapes |FPNPixel2Pixel with DWASPP|MobileNetV2 |32 |1536x768 |**15.37** |**74.98** |**fpn_pixel2pixel_aspp_mobilenetv2_tv** |
+
+|Dataset |Mode Architecture |Backbone Model|Backbone Stride|Resolution |Complexity (GigaMACS)|MeanIoU% |Model Configuration Name |
+|--------- |---------- |----------- |-------------- |-----------|-------- |----------|---------------------------------------- |
+|Cityscapes |ERFNet[[4]] | | |1024x512 |27.705 |69.7 |N/A |
+|Cityscapes |SwiftNetMNV2[[5]] |MobileNetV2 | |2048x1024 |41.0 |75.3 |N/A |
+|Cityscapes |DeepLabV3Plus[[6,7]] |MobileNetV2 |16 | |21.27 |70.71 |N/A |
+|Cityscapes |DeepLabV3Plus[[6,7]] |Xception65 |16 | |418.64 |78.79 |N/A |
## References
[1]The Cityscapes Dataset for Semantic Urban Scene Understanding, Marius Cordts, Mohamed Omran, Sebastian Ramos, Timo Rehfeld, Markus Enzweiler, Rodrigo Benenson, Uwe Franke, Stefan Roth, Bernt Schiele, CVPR 2016, https://www.cityscapes-dataset.com/
## References
[1]The Cityscapes Dataset for Semantic Urban Scene Understanding, Marius Cordts, Mohamed Omran, Sebastian Ramos, Timo Rehfeld, Markus Enzweiler, Rodrigo Benenson, Uwe Franke, Stefan Roth, Bernt Schiele, CVPR 2016, https://www.cityscapes-dataset.com/
diff --git a/modules/pytorch_jacinto_ai/engine/infer_pixel2pixel.py b/modules/pytorch_jacinto_ai/engine/infer_pixel2pixel.py
index 585ff3eba3249fc85d15f23463e3683a711c3bde..d61ac15a09b022f57ee474f8b84e8516280db015 100644 (file)
args.model_config.output_channels = None # number of output channels
args.model_config.input_channels = None # number of input channels
args.model_config.num_classes = None # number of classes (for segmentation)
args.model_config.output_channels = None # number of output channels
args.model_config.input_channels = None # number of input channels
args.model_config.num_classes = None # number of classes (for segmentation)
+ args.model_config.output_range = None # max range of output
args.model_config.num_decoders = None # number of decoders to use. [options: 0, 1, None]
args.sky_dir = False
args.model_config.num_decoders = None # number of decoders to use. [options: 0, 1, None]
args.sky_dir = False
assert args.pretrained is not None, 'pretrained path must be provided'
# onnx generation is filing for post quantized module
assert args.pretrained is not None, 'pretrained path must be provided'
# onnx generation is filing for post quantized module
- args.generate_onnx = False if (args.quantize) else args.generate_onnx
-
+ # args.generate_onnx = False if (args.quantize) else args.generate_onnx
#################################################
# set some global flags and initializations
# keep it in args for now - although they don't belong here strictly
#################################################
# set some global flags and initializations
# keep it in args for now - although they don't belong here strictly
# load pretrained weights
xnn.utils.load_weights_check(get_model_orig(model), pretrained=pretrained_data, change_names_dict=change_names_dict)
# load pretrained weights
xnn.utils.load_weights_check(get_model_orig(model), pretrained=pretrained_data, change_names_dict=change_names_dict)
+ if args.generate_onnx:
+ write_onnx_model(args, model, save_path, name='model_best.onnx')
#################################################
# multi gpu mode is not yet supported with quantization in evaluate
#################################################
# multi gpu mode is not yet supported with quantization in evaluate
- if args.gpu_mode and (args.phase=='training'):
+ if args.gpu_mode and ('training' in args.phase):
model = torch.nn.DataParallel(model)
#################################################
model = torch.nn.DataParallel(model)
#################################################
op_file_name = args.data_path.split('/')[-1]
os.system(' ffmpeg -framerate 30 -pattern_type glob -i "{}/*.png" -c:v libx264 -vb 50000k -qmax 20 -r 10 -vf scale=1024:512 -pix_fmt yuv420p {}.MP4'.format(infer_path, op_file_name))
op_file_name = args.data_path.split('/')[-1]
os.system(' ffmpeg -framerate 30 -pattern_type glob -i "{}/*.png" -c:v libx264 -vb 50000k -qmax 20 -r 10 -vf scale=1024:512 -pix_fmt yuv420p {}.MP4'.format(infer_path, op_file_name))
+def write_onnx_model(args, model, save_path, name='checkpoint.onnx'):
+ is_cuda = next(model.parameters()).is_cuda
+ input_list = create_rand_inputs(args, is_cuda=is_cuda)
+ #
+ model.eval()
+ torch.onnx.export(model, input_list, os.path.join(save_path, name), export_params=True, verbose=False)
+ # torch onnx export does not update names. Do it using onnx.save
+
+
+
if __name__ == '__main__':
train_args = get_config()
main(train_args)
if __name__ == '__main__':
train_args = get_config()
main(train_args)
diff --git a/modules/pytorch_jacinto_ai/engine/test_classification.py b/modules/pytorch_jacinto_ai/engine/test_classification.py
index 554cc5e51f95a0a7d67fc712c342c8a6e8aeb783..909f0f5411f786eb6391ab82ad21c02c79666e61 100644 (file)
is_cuda = next(model.parameters()).is_cuda
dummy_input = create_rand_inputs(args, is_cuda=is_cuda)
#
is_cuda = next(model.parameters()).is_cuda
dummy_input = create_rand_inputs(args, is_cuda=is_cuda)
#
- if args.phase == 'training':
+ if 'training' in args.phase:
model = xnn.quantize.QuantTrainModule(model, per_channel_q=args.per_channel_q,
bitwidth_weights=args.bitwidth_weights, bitwidth_activations=args.bitwidth_activations,
dummy_input=dummy_input)
model = xnn.quantize.QuantTrainModule(model, per_channel_q=args.per_channel_q,
bitwidth_weights=args.bitwidth_weights, bitwidth_activations=args.bitwidth_activations,
dummy_input=dummy_input)
- elif args.phase == 'calibration':
+ elif 'calibration' in args.phase:
model = xnn.quantize.QuantCalibrateModule(model, per_channel_q=args.per_channel_q,
bitwidth_weights=args.bitwidth_weights, bitwidth_activations=args.bitwidth_activations,
bias_calibration=args.bias_calibration, lr_calib=args.lr_calib,
dummy_input=dummy_input)
model = xnn.quantize.QuantCalibrateModule(model, per_channel_q=args.per_channel_q,
bitwidth_weights=args.bitwidth_weights, bitwidth_activations=args.bitwidth_activations,
bias_calibration=args.bias_calibration, lr_calib=args.lr_calib,
dummy_input=dummy_input)
- elif args.phase == 'validation':
+ elif 'validation' in args.phase:
# Note: bias_calibration is not enabled in test
model = xnn.quantize.QuantTestModule(model, per_channel_q=args.per_channel_q,
bitwidth_weights=args.bitwidth_weights, bitwidth_activations=args.bitwidth_activations,
# Note: bias_calibration is not enabled in test
model = xnn.quantize.QuantTestModule(model, per_channel_q=args.per_channel_q,
bitwidth_weights=args.bitwidth_weights, bitwidth_activations=args.bitwidth_activations,
assert False, f'invalid phase {args.phase}'
#
assert False, f'invalid phase {args.phase}'
#
-
# load pretrained
xnn.utils.load_weights_check(get_model_orig(model), pretrained=pretrained_data, change_names_dict=change_names_dict)
# load pretrained
xnn.utils.load_weights_check(get_model_orig(model), pretrained=pretrained_data, change_names_dict=change_names_dict)
#################################################
# multi gpu mode is not yet supported with quantization in evaluate
#################################################
# multi gpu mode is not yet supported with quantization in evaluate
- if args.parallel_model and (args.phase=='training'):
+ if args.parallel_model and ('training' in args.phase):
model = torch.nn.DataParallel(model)
#################################################
model = torch.nn.DataParallel(model)
#################################################
diff --git a/modules/pytorch_jacinto_ai/engine/train_classification.py b/modules/pytorch_jacinto_ai/engine/train_classification.py
index 83a0e9d7acd431135426d5e0644d7c73e04e1791..bf9ffec12819b4ea31e4536e8df5510a8c8ba72c 100644 (file)
def main(args):
assert (not hasattr(args, 'evaluate')), 'args.evaluate is deprecated. use args.phase=training or calibration or validation'
def main(args):
assert (not hasattr(args, 'evaluate')), 'args.evaluate is deprecated. use args.phase=training or calibration or validation'
- assert args.phase in ('training', 'calibration', 'validation'), f'invalid phase {args.phase}'
+ assert is_valid_phase(args.phase), f'invalid phase {args.phase}'
assert not hasattr(args, 'model_surgery'), 'the argument model_surgery is deprecated, it is not needed now - remove it'
if (args.phase == 'validation' and args.bias_calibration):
assert not hasattr(args, 'model_surgery'), 'the argument model_surgery is deprecated, it is not needed now - remove it'
if (args.phase == 'validation' and args.bias_calibration):
is_cuda = next(model.parameters()).is_cuda
dummy_input = create_rand_inputs(args, is_cuda=is_cuda)
#
is_cuda = next(model.parameters()).is_cuda
dummy_input = create_rand_inputs(args, is_cuda=is_cuda)
#
- if args.phase == 'training':
+ if 'training' in args.phase:
model = xnn.quantize.QuantTrainModule(model, per_channel_q=args.per_channel_q,
histogram_range=args.histogram_range, bitwidth_weights=args.bitwidth_weights,
bitwidth_activations=args.bitwidth_activations,
dummy_input=dummy_input)
model = xnn.quantize.QuantTrainModule(model, per_channel_q=args.per_channel_q,
histogram_range=args.histogram_range, bitwidth_weights=args.bitwidth_weights,
bitwidth_activations=args.bitwidth_activations,
dummy_input=dummy_input)
- elif args.phase == 'calibration':
+ elif 'calibration' in args.phase:
model = xnn.quantize.QuantCalibrateModule(model, per_channel_q=args.per_channel_q,
bitwidth_weights=args.bitwidth_weights, bitwidth_activations=args.bitwidth_activations,
histogram_range=args.histogram_range, bias_calibration=args.bias_calibration, lr_calib=args.lr_calib,
dummy_input=dummy_input)
model = xnn.quantize.QuantCalibrateModule(model, per_channel_q=args.per_channel_q,
bitwidth_weights=args.bitwidth_weights, bitwidth_activations=args.bitwidth_activations,
histogram_range=args.histogram_range, bias_calibration=args.bias_calibration, lr_calib=args.lr_calib,
dummy_input=dummy_input)
- elif args.phase == 'validation':
+ elif 'validation' in args.phase:
# Note: bias_calibration is not used in test
model = xnn.quantize.QuantTestModule(model, per_channel_q=args.per_channel_q,
bitwidth_weights=args.bitwidth_weights, bitwidth_activations=args.bitwidth_activations,
# Note: bias_calibration is not used in test
model = xnn.quantize.QuantTestModule(model, per_channel_q=args.per_channel_q,
bitwidth_weights=args.bitwidth_weights, bitwidth_activations=args.bitwidth_activations,
count_flops(args, model)
#################################################
count_flops(args, model)
#################################################
- if args.generate_onnx and ((args.phase in ('training','calibration')) or (args.run_soon == False)):
+ if args.generate_onnx and (any(p in args.phase for p in ('training','calibration')) or (args.run_soon == False)):
write_onnx_model(args, get_model_orig(model), save_path)
#
write_onnx_model(args, get_model_orig(model), save_path)
#
model_module = model.module if hasattr(model, 'module') else model
if args.lr_clips is not None:
model_module = model.module if hasattr(model, 'module') else model
if args.lr_clips is not None:
- learning_rate_clips = args.lr_clips if args.phase == 'training' else 0.0
+ learning_rate_clips = args.lr_clips if 'training' in args.phase else 0.0
clips_decay = args.bias_decay if (args.bias_decay is not None and args.bias_decay != 0.0) else args.weight_decay
clips_params = [p for n,p in model_module.named_parameters() if 'clips' in n]
other_params = [p for n,p in model_module.named_parameters() if 'clips' not in n]
clips_decay = args.bias_decay if (args.bias_decay is not None and args.bias_decay != 0.0) else args.weight_decay
clips_params = [p for n,p in model_module.named_parameters() if 'clips' in n]
other_params = [p for n,p in model_module.named_parameters() if 'clips' not in n]
if args.scheduler == 'step':
print("=> milestones : {}".format(args.milestones))
if args.scheduler == 'step':
print("=> milestones : {}".format(args.milestones))
- learning_rate = args.lr if (args.phase == 'training') else 0.0
+ learning_rate = args.lr if ('training'in args.phase) else 0.0
if args.optimizer == 'adam':
optimizer = torch.optim.Adam(param_groups, learning_rate, betas=(args.momentum, args.beta))
elif args.optimizer == 'sgd':
if args.optimizer == 'adam':
optimizer = torch.optim.Adam(param_groups, learning_rate, betas=(args.momentum, args.beta))
elif args.optimizer == 'sgd':
#
###################################################################
#
###################################################################
+def is_valid_phase(phase):
+ phases = ('training', 'calibration', 'validation')
+ return any(p in phase for p in phases)
+
+
def close(args):
if args.logger is not None:
del args.logger
args.logger = None
#
args.best_prec1 = -1
def close(args):
if args.logger is not None:
del args.logger
args.logger = None
#
args.best_prec1 = -1
-#
def get_save_path(args, phase=None):
def get_save_path(args, phase=None):
train_iter = iter(train_loader)
last_update_iter = -1
train_iter = iter(train_loader)
last_update_iter = -1
- progressbar_color = (Fore.YELLOW if args.phase=='calibration' else Fore.WHITE)
+ progressbar_color = (Fore.YELLOW if (('calibration' in args.phase) or ('training' in args.phase and args.quantize)) else Fore.WHITE)
print('{}'.format(progressbar_color), end='')
for iteration in range(args.iters):
print('{}'.format(progressbar_color), end='')
for iteration in range(args.iters):
top1.update(prec1[0], input_size[0])
top5.update(prec5[0], input_size[0])
top1.update(prec1[0], input_size[0])
top5.update(prec5[0], input_size[0])
- if args.phase == 'training':
+ if 'training' in args.phase:
# zero gradients so that we can accumulate gradients
if (iteration % args.iter_size) == 0:
optimizer.zero_grad()
# zero gradients so that we can accumulate gradients
if (iteration % args.iter_size) == 0:
optimizer.zero_grad()
return val_transform
def get_transforms(args):
return val_transform
def get_transforms(args):
- train_transform = get_train_transform(args)
+ # Provision to train with val transform - provide rand_scale as (0, 0)
+ # Fixing the train-test resolution discrepancy, https://arxiv.org/abs/1906.06423
+ always_use_val_transform = (args.rand_scale[0] == 0)
+ train_transform = get_validation_transform(args) if always_use_val_transform else get_train_transform(args)
val_transform = get_validation_transform(args)
return train_transform, val_transform
val_transform = get_validation_transform(args)
return train_transform, val_transform
diff --git a/modules/pytorch_jacinto_ai/engine/train_pixel2pixel.py b/modules/pytorch_jacinto_ai/engine/train_pixel2pixel.py
index 4589a1aff17608da315e13d1812421be14cd7ba2..9273786f3f5d5932bffe82e008d233dfdfd4084c 100644 (file)
'torch version must be 1.1 or higher, due to the change in scheduler.step() and optimiser.step() call order'
assert (not hasattr(args, 'evaluate')), 'args.evaluate is deprecated. use args.phase=training or calibration or validation'
'torch version must be 1.1 or higher, due to the change in scheduler.step() and optimiser.step() call order'
assert (not hasattr(args, 'evaluate')), 'args.evaluate is deprecated. use args.phase=training or calibration or validation'
- assert args.phase in ('training', 'calibration', 'validation'), f'invalid phase {args.phase}'
+ assert is_valid_phase(args.phase), f'invalid phase {args.phase}'
assert not hasattr(args, 'model_surgery'), 'the argument model_surgery is deprecated, it is not needed now - remove it'
if (args.phase == 'validation' and args.bias_calibration):
assert not hasattr(args, 'model_surgery'), 'the argument model_surgery is deprecated, it is not needed now - remove it'
if (args.phase == 'validation' and args.bias_calibration):
is_cuda = next(model.parameters()).is_cuda
dummy_input = create_rand_inputs(args, is_cuda=is_cuda)
#
is_cuda = next(model.parameters()).is_cuda
dummy_input = create_rand_inputs(args, is_cuda=is_cuda)
#
- if args.phase == 'training':
+ if 'training' in args.phase:
model = xnn.quantize.QuantTrainModule(model, per_channel_q=args.per_channel_q,
histogram_range=args.histogram_range, bitwidth_weights=args.bitwidth_weights, bitwidth_activations=args.bitwidth_activations, dummy_input=dummy_input)
model = xnn.quantize.QuantTrainModule(model, per_channel_q=args.per_channel_q,
histogram_range=args.histogram_range, bitwidth_weights=args.bitwidth_weights, bitwidth_activations=args.bitwidth_activations, dummy_input=dummy_input)
- elif args.phase == 'calibration':
+ elif 'calibration' in args.phase:
model = xnn.quantize.QuantCalibrateModule(model, per_channel_q=args.per_channel_q,
bitwidth_weights=args.bitwidth_weights, bitwidth_activations=args.bitwidth_activations,
histogram_range=args.histogram_range, bias_calibration=args.bias_calibration, lr_calib=args.lr_calib, dummy_input=dummy_input)
model = xnn.quantize.QuantCalibrateModule(model, per_channel_q=args.per_channel_q,
bitwidth_weights=args.bitwidth_weights, bitwidth_activations=args.bitwidth_activations,
histogram_range=args.histogram_range, bias_calibration=args.bias_calibration, lr_calib=args.lr_calib, dummy_input=dummy_input)
- elif args.phase == 'validation':
+ elif 'validation' in args.phase:
# Note: bias_calibration is not emabled
model = xnn.quantize.QuantTestModule(model, per_channel_q=args.per_channel_q,
bitwidth_weights=args.bitwidth_weights, bitwidth_activations=args.bitwidth_activations,
# Note: bias_calibration is not emabled
model = xnn.quantize.QuantTestModule(model, per_channel_q=args.per_channel_q,
bitwidth_weights=args.bitwidth_weights, bitwidth_activations=args.bitwidth_activations,
count_flops(args, model)
#################################################
count_flops(args, model)
#################################################
- if args.generate_onnx and ((args.phase in ('training','calibration')) or (args.run_soon == False)):
+ if args.generate_onnx and (any(args.phase in p for p in ('training','calibration')) or (args.run_soon == False)):
write_onnx_model(args, get_model_orig(model), save_path)
#
write_onnx_model(args, get_model_orig(model), save_path)
#
assert(args.solver in ['adam', 'sgd'])
print('=> setting {} solver'.format(args.solver))
if args.lr_clips is not None:
assert(args.solver in ['adam', 'sgd'])
print('=> setting {} solver'.format(args.solver))
if args.lr_clips is not None:
- learning_rate_clips = args.lr_clips if args.phase == 'training' else 0.0
+ learning_rate_clips = args.lr_clips if 'training' in args.phase else 0.0
clips_decay = args.bias_decay if (args.bias_decay is not None and args.bias_decay != 0.0) else args.weight_decay
clips_params = [p for n,p in model.named_parameters() if 'clips' in n]
other_params = [p for n,p in model.named_parameters() if 'clips' not in n]
clips_decay = args.bias_decay if (args.bias_decay is not None and args.bias_decay != 0.0) else args.weight_decay
clips_params = [p for n,p in model.named_parameters() if 'clips' in n]
other_params = [p for n,p in model.named_parameters() if 'clips' not in n]
param_groups = [{'params': filter(lambda p: p.requires_grad, model.parameters()), 'weight_decay': args.weight_decay}]
#
param_groups = [{'params': filter(lambda p: p.requires_grad, model.parameters()), 'weight_decay': args.weight_decay}]
#
- learning_rate = args.lr if (args.phase == 'training') else 0.0
+ learning_rate = args.lr if ('training'in args.phase) else 0.0
if args.solver == 'adam':
optimizer = torch.optim.Adam(param_groups, learning_rate, betas=(args.momentum, args.beta))
elif args.solver == 'sgd':
if args.solver == 'adam':
optimizer = torch.optim.Adam(param_groups, learning_rate, betas=(args.momentum, args.beta))
elif args.solver == 'sgd':
val_writer.file_writer.flush()
# adjust the learning rate using lr scheduler
val_writer.file_writer.flush()
# adjust the learning rate using lr scheduler
- if args.phase == 'training':
+ if 'training' in args.phase:
scheduler.step()
#
#
scheduler.step()
#
#
close(args)
#
close(args)
#
+###################################################################
+def is_valid_phase(phase):
+ phases = ('training', 'calibration', 'validation')
+ return any(p in phase for p in phases)
###################################################################
###################################################################
@@ -583,7 +587,7 @@ def train(args, train_dataset, train_loader, model, optimizer, epoch, train_writ
last_update_iter = -1
# change color to yellow for calibration
last_update_iter = -1
# change color to yellow for calibration
- progressbar_color = (Fore.YELLOW if args.phase=='calibration' else Fore.WHITE)
+ progressbar_color = (Fore.YELLOW if (('calibration' in args.phase) or ('training' in args.phase and args.quantize)) else Fore.WHITE)
print('{}'.format(progressbar_color), end='')
##########################
print('{}'.format(progressbar_color), end='')
##########################
@@ -621,7 +625,7 @@ def train(args, train_dataset, train_loader, model, optimizer, epoch, train_writ
if args.model_config.multi_task_type is not None and len(args.model_config.output_channels) > 1:
xnn.layers.set_losses(model, loss_list_orig)
if args.model_config.multi_task_type is not None and len(args.model_config.output_channels) > 1:
xnn.layers.set_losses(model, loss_list_orig)
- if args.phase == 'training':
+ if 'training' in args.phase:
# zero gradients so that we can accumulate gradients
if (iter % args.iter_size) == 0:
optimizer.zero_grad()
# zero gradients so that we can accumulate gradients
if (iter % args.iter_size) == 0:
optimizer.zero_grad()
def get_transforms(args):
def get_transforms(args):
- train_transform = get_train_transform(args)
+ # Provision to train with val transform - provide rand_scale as (0, 0)
+ # Fixing the train-test resolution discrepancy, https://arxiv.org/abs/1906.06423
+ always_use_val_transform = (args.rand_scale[0] == 0)
+ train_transform = get_validation_transform(args) if always_use_val_transform else get_train_transform(args)
val_transform = get_validation_transform(args)
return train_transform, val_transform
val_transform = get_validation_transform(args)
return train_transform, val_transform
diff --git a/modules/pytorch_jacinto_ai/vision/losses/segmentation_loss.py b/modules/pytorch_jacinto_ai/vision/losses/segmentation_loss.py
index 60ea113a8bdafa91ff0e836aa11ba3b4bcbbc9b1..22e7daf72423e80cf25fb508ec3ba907da29cd0a 100755 (executable)
class SegmentationLoss(torch.nn.Module):
def __init__(self, *args, ignore_index = 255, weight=None, **kwargs):
super().__init__()
class SegmentationLoss(torch.nn.Module):
def __init__(self, *args, ignore_index = 255, weight=None, **kwargs):
super().__init__()
- self.register_buffer('weight', torch.FloatTensor(weight)) if weight is not None else None
+ self.weight = None if weight is None else self.register_buffer('weight', torch.FloatTensor(weight))
self.ignore_index = ignore_index
self.is_avg = False
#
self.ignore_index = ignore_index
self.is_avg = False
#
diff --git a/modules/pytorch_jacinto_ai/vision/models/mobilenetv1.py b/modules/pytorch_jacinto_ai/vision/models/mobilenetv1.py
index eaa0549147e2671ed745f3276d788c38ffe2f451..c0701a1837b1f17eb66433d11334cb9098c5e436 100644 (file)
model_config.num_classes = 1000
model_config.width_mult = 1.
model_config.expand_ratio = 6
model_config.num_classes = 1000
model_config.width_mult = 1.
model_config.expand_ratio = 6
- model_config.strides = (2,2,2,2,2)
+ model_config.strides = None #(2,2,2,2,2)
model_config.activation = xnn.layers.DefaultAct2d
model_config.use_blocks = False
model_config.kernel_size = 3
model_config.activation = xnn.layers.DefaultAct2d
model_config.use_blocks = False
model_config.kernel_size = 3
self.num_classes = self.model_config.num_classes
# strides of various layers
self.num_classes = self.model_config.num_classes
# strides of various layers
- s0 = model_config.strides[0]
- s1 = model_config.strides[1]
- s2 = model_config.strides[2]
- s3 = model_config.strides[3]
- s4 = model_config.strides[4]
+ strides = model_config.strides if (model_config.strides is not None) else (2,2,2,2,2)
+ s0 = strides[0]
+ s1 = strides[1]
+ s2 = strides[2]
+ s3 = strides[3]
+ s4 = strides[4]
if self.model_config.layer_setting is None:
self.model_config.layer_setting = [
if self.model_config.layer_setting is None:
self.model_config.layer_setting = [
diff --git a/modules/pytorch_jacinto_ai/vision/models/mobilenetv2.py b/modules/pytorch_jacinto_ai/vision/models/mobilenetv2.py
index e351affdd53ee35172e2b62fb42d5b01a1e89f1b..a47d4b135fd170801195a55cb6dc92d7590c418a 100644 (file)
model_config.num_classes = 1000
model_config.width_mult = 1.
model_config.expand_ratio = 6
model_config.num_classes = 1000
model_config.width_mult = 1.
model_config.expand_ratio = 6
- model_config.strides = (2,2,2,2,2)
+ model_config.strides = None #(2,2,2,2,2)
model_config.activation = xnn.layers.DefaultAct2d
model_config.use_blocks = False
model_config.kernel_size = 3
model_config.activation = xnn.layers.DefaultAct2d
model_config.use_blocks = False
model_config.kernel_size = 3
}
}
-
-
class InvertedResidual(torch.nn.Module):
def __init__(self, inp, oup, stride, expand_ratio, activation, kernel_size, linear_dw):
super(InvertedResidual, self).__init__()
class InvertedResidual(torch.nn.Module):
def __init__(self, inp, oup, stride, expand_ratio, activation, kernel_size, linear_dw):
super(InvertedResidual, self).__init__()
self.num_classes = self.model_config.num_classes
# strides of various layers
self.num_classes = self.model_config.num_classes
# strides of various layers
- s0 = model_config.strides[0]
+ strides = model_config.strides if (model_config.strides is not None) else (2,2,2,2,2)
+ s0 = strides[0]
sf = 2 if model_config.fastdown else 1 # extra stride if fastdown
sf = 2 if model_config.fastdown else 1 # extra stride if fastdown
- s1 = model_config.strides[1]
- s2 = model_config.strides[2]
- s3 = model_config.strides[3]
- s4 = model_config.strides[4]
+ s1 = strides[1]
+ s2 = strides[2]
+ s3 = strides[3]
+ s4 = strides[4]
if self.model_config.layer_setting is None:
ex = self.model_config.expand_ratio
if self.model_config.layer_setting is None:
ex = self.model_config.expand_ratio
diff --git a/modules/pytorch_jacinto_ai/vision/models/pixel2pixel/fpn_pixel2pixel.py b/modules/pytorch_jacinto_ai/vision/models/pixel2pixel/fpn_pixel2pixel.py
index 31afca9ce3ff7df320d8d3c9ebb5e670ad5d7aee..b84bfb93aff0c1a9716ab07cd1c4bbb8c1bbef28 100644 (file)
__all__ = ['FPNPixel2PixelASPP', 'FPNPixel2PixelDecoder',
__all__ = ['FPNPixel2PixelASPP', 'FPNPixel2PixelDecoder',
- 'fpn_pixel2pixel_aspp_mobilenetv2_tv', 'fpn_pixel2pixel_aspp_resnet50',
- 'fpn_pixel2pixel_aspp_mobilenetv2_tv_es64', 'fpn_pixel2pixel_aspp_resnet50_es64']
+ 'fpn_pixel2pixel_aspp_mobilenetv2_tv', 'fpn_pixel2pixel_aspp_mobilenetv2_tv_es64',
+ # no aspp models
+ 'fpn_pixel2pixel_mobilenetv2_tv', 'fpn_pixel2pixel_mobilenetv2_tv_es64',
+ # resnet models
+ 'fpn_pixel2pixel_aspp_resnet50', 'fpn_pixel2pixel_aspp_resnet50_es64',
+ ]
###########################################
###########################################
activation = self.model_config.activation
self.output_type = model_config.output_type
activation = self.model_config.activation
self.output_type = model_config.output_type
- current_channels = self.model_config.shortcut_channels[-1]
self.decoder_channels = decoder_channels = round(self.model_config.decoder_chan*self.model_config.decoder_factor)
self.decoder_channels = decoder_channels = round(self.model_config.decoder_chan*self.model_config.decoder_factor)
- aspp_channels = round(self.model_config.aspp_chan*self.model_config.decoder_factor)
self.rfblock = None
if self.model_config.use_aspp:
self.rfblock = None
if self.model_config.use_aspp:
+ current_channels = self.model_config.shortcut_channels[-1]
+ aspp_channels = round(self.model_config.aspp_chan * self.model_config.decoder_factor)
self.rfblock = xnn.layers.DWASPPLiteBlock(current_channels, aspp_channels, decoder_channels, dilation=self.model_config.aspp_dil,
avg_pool=False, activation=activation)
elif self.model_config.use_extra_strides:
# a low complexity pyramid
self.rfblock = xnn.layers.DWASPPLiteBlock(current_channels, aspp_channels, decoder_channels, dilation=self.model_config.aspp_dil,
avg_pool=False, activation=activation)
elif self.model_config.use_extra_strides:
# a low complexity pyramid
+ current_channels = self.model_config.shortcut_channels[-3]
self.rfblock = torch.nn.Sequential(xnn.layers.ConvDWSepNormAct2d(current_channels, current_channels, kernel_size=3, stride=2, activation=(activation, activation)),
xnn.layers.ConvDWSepNormAct2d(current_channels, decoder_channels, kernel_size=3, stride=2, activation=(activation, activation)))
self.rfblock = torch.nn.Sequential(xnn.layers.ConvDWSepNormAct2d(current_channels, current_channels, kernel_size=3, stride=2, activation=(activation, activation)),
xnn.layers.ConvDWSepNormAct2d(current_channels, decoder_channels, kernel_size=3, stride=2, activation=(activation, activation)))
- self.model_config.shortcut_strides += [64, 128]
else:
else:
+ current_channels = self.model_config.shortcut_channels[-1]
self.rfblock = xnn.layers.ConvNormAct2d(current_channels, decoder_channels, kernel_size=1, stride=1)
current_channels = decoder_channels
self.rfblock = xnn.layers.ConvNormAct2d(current_channels, decoder_channels, kernel_size=1, stride=1)
current_channels = decoder_channels
return model, change_names_dict
return model, change_names_dict
+# fast down sampling model (encoder stride 64 model)
def fpn_pixel2pixel_aspp_mobilenetv2_tv_es64(model_config, pretrained=None):
model_config = get_config_fpnp2p_mnv2().merge_from(model_config)
model_config.fastdown = True
def fpn_pixel2pixel_aspp_mobilenetv2_tv_es64(model_config, pretrained=None):
model_config = get_config_fpnp2p_mnv2().merge_from(model_config)
model_config.fastdown = True
return fpn_pixel2pixel_aspp_mobilenetv2_tv(model_config, pretrained=pretrained)
return fpn_pixel2pixel_aspp_mobilenetv2_tv(model_config, pretrained=pretrained)
+##################
+# similar to the original fpn model with extra convolutions with strides (no aspp)
+def fpn_pixel2pixel_mobilenetv2_tv(model_config, pretrained=None):
+ model_config = get_config_fpnp2p_mnv2().merge_from(model_config)
+ model_config.use_aspp = False
+ model_config.use_extra_strides = True
+ model_config.shortcut_strides = (4, 8, 16, 32, 64, 128)
+ model_config.shortcut_channels = (24, 32, 96, 320, 320, 256)
+ return fpn_pixel2pixel_aspp_mobilenetv2_tv(model_config, pretrained=pretrained)
+
+
+# similar to the original fpn model with extra convolutions with strides (no aspp) - fast down sampling model (encoder stride 64 model)
+def fpn_pixel2pixel_mobilenetv2_tv_es64(model_config, pretrained=None):
+ model_config = get_config_fpnp2p_mnv2().merge_from(model_config)
+ model_config.use_aspp = False
+ model_config.use_extra_strides = True
+ model_config.fastdown = True
+ model_config.strides = (2,2,2,2,2)
+ model_config.shortcut_strides = (8, 16, 32, 64, 128, 256)
+ model_config.shortcut_channels = (24, 32, 96, 320, 320, 256)
+ model_config.decoder_chan = 256
+ model_config.aspp_chan = 256
+ return fpn_pixel2pixel_aspp_mobilenetv2_tv(model_config, pretrained=pretrained)
+
+
###########################################
def get_config_fpnp2p_resnet50():
# only the delta compared to the one defined for mobilenetv2
###########################################
def get_config_fpnp2p_resnet50():
# only the delta compared to the one defined for mobilenetv2
diff --git a/modules/pytorch_jacinto_ai/vision/models/resnet.py b/modules/pytorch_jacinto_ai/vision/models/resnet.py
index e63b5b5f9b474e122acb5e6a320071374fb0aeb5..0f8e3ee5399ef17bd86ecf37c71c0e4bfbce8a8d 100644 (file)
groups=1, width_per_group=64, replace_stride_with_dilation=None,
norm_layer=None, input_channels=3, strides=None, width_mult=1.0, fastdown=False):
super(ResNet, self).__init__()
groups=1, width_per_group=64, replace_stride_with_dilation=None,
norm_layer=None, input_channels=3, strides=None, width_mult=1.0, fastdown=False):
super(ResNet, self).__init__()
- strides = strides if strides is not None else (2,2,2,2,2)
if norm_layer is None:
norm_layer = nn.BatchNorm2d
self._norm_layer = norm_layer
if norm_layer is None:
norm_layer = nn.BatchNorm2d
self._norm_layer = norm_layer
raise ValueError(f"replace_stride_with_dilation should be None or a 3-element tuple, got {replace_stride_with_dilation}")
# strides of various layers
raise ValueError(f"replace_stride_with_dilation should be None or a 3-element tuple, got {replace_stride_with_dilation}")
# strides of various layers
+ strides = strides if (strides is not None) else (2,2,2,2,2)
s0 = strides[0]
s1 = strides[1]
sf = 2 if fastdown else 1 # additional stride if fast down is true
s0 = strides[0]
s1 = strides[1]
sf = 2 if fastdown else 1 # additional stride if fast down is true
model_config = xnn.utils.ConfigNode()
model_config.input_channels = 3
model_config.num_classes = 1000
model_config = xnn.utils.ConfigNode()
model_config.input_channels = 3
model_config.num_classes = 1000
- model_config.strides = (2,2,2,2,2)
+ model_config.strides = None #(2,2,2,2,2)
model_config.fastdown = False
return model_config
model_config.fastdown = False
return model_config
diff --git a/modules/pytorch_jacinto_ai/xnn/layers/multi_task.py b/modules/pytorch_jacinto_ai/xnn/layers/multi_task.py
index d3c1975fd3a90af81559dd6ede450968dac5e435..e45908d64c4b23804e284936329a2271b875f364 100644 (file)
self.loss_scales[task_idx] = torch.nn.functional.tanh(loss_scale) if clip_scale else loss_scale
#
self.loss_offset = self.uncertainty_factors
self.loss_scales[task_idx] = torch.nn.functional.tanh(loss_scale) if clip_scale else loss_scale
#
self.loss_offset = self.uncertainty_factors
+ elif self.multi_task_type == "dtp": #dynamic task priority
+ pass
#
del dy_norms_smooth_mean, dy_norms_mean
#
del dy_norms_smooth_mean, dy_norms_mean
diff --git a/modules/pytorch_jacinto_ai/xnn/quantize/quant_graph_module.py b/modules/pytorch_jacinto_ai/xnn/quantize/quant_graph_module.py
index 070fb8b0e9f0c8dd18e2f6d79e0b1a68257e55d8..0ce62c868354e1ac85e647d48869999b194972ab 100644 (file)
else:
quantize_out = True
#
else:
quantize_out = True
#
- elif utils.is_conv(module):
+ elif utils.is_conv(module) or utils.is_deconv(module):
if len(next_module)==1 and (utils.is_normalization(next_module[0]) or utils.is_activation(next_module[0])):
quantize_out = False
else:
if len(next_module)==1 and (utils.is_normalization(next_module[0]) or utils.is_activation(next_module[0])):
quantize_out = False
else:
# quantize_out = True
# #
# quantize_out = True
# #
- qparams.quantize_w = isinstance(module, (torch.nn.Conv2d,torch.nn.Linear)) # all conv layers will be quantized
- qparams.quantize_b = isinstance(module, (torch.nn.Conv2d,torch.nn.Linear)) # all conv layers will be quantized
+ qparams.quantize_w = isinstance(module, (torch.nn.Conv2d,torch.nn.ConvTranspose2d,torch.nn.Linear)) # all conv/deconv layers will be quantized
+ qparams.quantize_b = isinstance(module, (torch.nn.Conv2d,torch.nn.ConvTranspose2d,torch.nn.Linear)) # all conv/deconv layers will be quantized
qparams.quantize_out = quantize_out # selectively quantize output
qparams.quantize_in = qparams.is_input # only top modules's input need to be quantized
qparams.align_in = isinstance(module, (layers.AddBlock, layers.CatBlock,torch.nn.AdaptiveAvgPool2d))# all tensors to be made same q at the input
qparams.quantize_out = quantize_out # selectively quantize output
qparams.quantize_in = qparams.is_input # only top modules's input need to be quantized
qparams.align_in = isinstance(module, (layers.AddBlock, layers.CatBlock,torch.nn.AdaptiveAvgPool2d))# all tensors to be made same q at the input
diff --git a/run_classification.sh b/run_classification.sh
index abc67eb52be0fefa2666021e33f2f42012adf370..430f6861b0841147ce43cd6f5183e550f6377894 100755 (executable)
--- a/run_classification.sh
+++ b/run_classification.sh
## =====================================================================================
## Cifar100 Classification (Automatic Download)
#### Training with MobileNetV2
## =====================================================================================
## Cifar100 Classification (Automatic Download)
#### Training with MobileNetV2
-#python ./scripts/train_classification_main.py --dataset_name cifar100_classification --model_name mobilenetv2_tv_x1 --data_path ./data/datasets/cifar100_classification --img_resize 32 --img_crop 32 --rand_scale 0.5 1.0
+#python ./scripts/train_classification_main.py --dataset_name cifar100_classification --model_name mobilenetv2_tv_x1 --data_path ./data/datasets/cifar100_classification --img_resize 32 --img_crop 32 --rand_scale 0.5 1.0 --strides 1 1 1 2 2
## Cifar10 Classification (Automatic Download)
#### Training with MobileNetV2
## Cifar10 Classification (Automatic Download)
#### Training with MobileNetV2
-#python ./scripts/train_classification_main.py --dataset_name cifar10_classification --model_name mobilenetv2_tv_x1 --data_path ./data/datasets/cifar10_classification --img_resize 32 --img_crop 32 --rand_scale 0.5 1.0
+#python ./scripts/train_classification_main.py --dataset_name cifar10_classification --model_name mobilenetv2_tv_x1 --data_path ./data/datasets/cifar10_classification --img_resize 32 --img_crop 32 --rand_scale 0.5 1.0 --strides 1 1 1 2 2
## ImageNet Classification (Automatic Download)
#### Training with MobileNetV2
## ImageNet Classification (Automatic Download)
#### Training with MobileNetV2
## ImageNet Classification (Manual Download)
#### Training with MobileNetV2
#python ./scripts/train_classification_main.py --dataset_name image_folder_classification --model_name mobilenetv2_tv_x1 --data_path ./data/datasets/image_folder_classification
## ImageNet Classification (Manual Download)
#### Training with MobileNetV2
#python ./scripts/train_classification_main.py --dataset_name image_folder_classification --model_name mobilenetv2_tv_x1 --data_path ./data/datasets/image_folder_classification
+#### Training with MobileNetV2 - Small Resolution
+#python ./scripts/train_classification_main.py --dataset_name image_folder_classification --model_name mobilenetv2_tv_x1 --data_path ./data/datasets/image_folder_classification --img_resize 146 --img_crop 128 --batch_size 1024 --lr 0.2 --workers 16
#### Training with MobileNetV2 with 2x channels and expansion factor of 2
#python ./scripts/train_classification_main.py --dataset_name image_folder_classification --model_name mobilenetv2_tv_x2_t2 --data_path ./data/datasets/image_folder_classification --batch_size 256
#### Training with MobileNetV2 with 2x channels and expansion factor of 2
#python ./scripts/train_classification_main.py --dataset_name image_folder_classification --model_name mobilenetv2_tv_x2_t2 --data_path ./data/datasets/image_folder_classification --batch_size 256
diff --git a/run_depth.sh b/run_depth.sh
index b0ceaab234bfa003e179c6a2ca5ec326fe83ed4d..f96d1afefc010b755927e6f788793cffa67937ab 100755 (executable)
--- a/run_depth.sh
+++ b/run_depth.sh
## Training
## =====================================================================================
#### KITTI Depth (Manual Download) - Training with MobileNetV2+DeeplabV3Lite
## Training
## =====================================================================================
#### KITTI Depth (Manual Download) - Training with MobileNetV2+DeeplabV3Lite
-#python ./scripts/train_depth_main.py --dataset_name kitti_depth --model_name deeplabv3lite_mobilenetv2_tv --data_path ./data/datasets/kitti/kitti_depth/data --img_resize 384 768 --output_size 374 1242
+#python ./scripts/train_depth_main.py --dataset_name kitti_depth --model_name deeplabv3lite_mobilenetv2_tv --data_path ./data/datasets/kitti/kitti_depth/data --img_resize 384 768 --output_size 374 1242 \
+#--pretrained https://download.pytorch.org/models/mobilenet_v2-b0353104.pth
#### KITTI Depth (Manual Download) - Training with ResNet50+FPN
#python ./scripts/train_depth_main.py --dataset_name kitti_depth --model_name fpn_pixel2pixel_aspp_resnet50 --data_path ./data/datasets/kitti/kitti_depth/data --img_resize 384 768 --output_size 374 1242 \
#### KITTI Depth (Manual Download) - Training with ResNet50+FPN
#python ./scripts/train_depth_main.py --dataset_name kitti_depth --model_name fpn_pixel2pixel_aspp_resnet50 --data_path ./data/datasets/kitti/kitti_depth/data --img_resize 384 768 --output_size 374 1242 \
diff --git a/run_quantization.sh b/run_quantization.sh
index b7ec51f955f4bfd101ec044b5b4be8273206161a..a867cc9a43b0715f047b72fafccff03f6bf77c2e 100755 (executable)
--- a/run_quantization.sh
+++ b/run_quantization.sh
# Quantization
## =====================================================================================
# Quantization
## =====================================================================================
-## Post Training Calibration & Quantization - this will write out a quantization friendly model very quickly
-## =====================================================================================
-#
-#### Image Classification - Post Training Calibration & Quantization - ResNet50
-#python ./scripts/train_classification_main.py --phase calibration --dataset_name image_folder_classification --model_name resnet50_x1 --data_path ./data/datasets/image_folder_classification \
-#--pretrained https://download.pytorch.org/models/resnet50-19c8e357.pth \
-#--batch_size 64 --quantize True --epochs 1 --epoch_size 100
-#
-#
-#### Image Classification - Post Training Calibration & Quantization - MobileNetV2
-#python ./scripts/train_classification_main.py --phase calibration --dataset_name image_folder_classification --model_name mobilenetv2_tv_x1 --data_path ./data/datasets/image_folder_classification \
-#--pretrained https://download.pytorch.org/models/mobilenet_v2-b0353104.pth \
-#--batch_size 64 --quantize True --epochs 1 --epoch_size 100
-#
-#
-#### Image Classification - Post Training Calibration & Quantization for a TOUGH MobileNetV2 pretrained model
-#python ./scripts/train_classification_main.py --phase calibration --dataset_name image_folder_classification --model_name mobilenetv2_shicai_x1 --data_path ./data/datasets/image_folder_classification \
-#--pretrained ./data/modelzoo/pretrained/pytorch/others/shicai/MobileNet-Caffe/mobilenetv2_shicai_rgb.tar \
-#--batch_size 64 --quantize True --epochs 1 --epoch_size 100
-#
-#
-#### Semantic Segmentation - Post Training Calibration & Quantization for MobileNetV2+DeeplabV3Lite
-#python ./scripts/train_segmentation_main.py --phase calibration --dataset_name cityscapes_segmentation --model_name deeplabv3lite_mobilenetv2_tv --data_path ./data/datasets/cityscapes/data --img_resize 384 768 --output_size 1024 2048 --gpus 0 1 \
-#--pretrained ./data/modelzoo/semantic_segmentation/cityscapes/deeplabv3lite-mobilenetv2/cityscapes_segmentation_deeplabv3lite-mobilenetv2_2019-06-26-08-59-32.pth \
-#--batch_size 12 --quantize True --epochs 1 --epoch_size 100
-#
-#
-## =====================================================================================
-## Trained Quantization - If And Only if Post Training Calibration and Quantization (above) doesn't work
+## Trained Quantization
## =====================================================================================
#
#### Image Classification - Trained Quantization - MobileNetV2
## =====================================================================================
#
#### Image Classification - Trained Quantization - MobileNetV2
#--pretrained ./data/modelzoo/semantic_segmentation/cityscapes/deeplabv3lite-mobilenetv2/cityscapes_segmentation_deeplabv3lite-mobilenetv2_2019-06-26-08-59-32.pth \
#--batch_size 12 --quantize True --epochs 150 --lr 5e-5 --evaluate_start False
#--pretrained ./data/modelzoo/semantic_segmentation/cityscapes/deeplabv3lite-mobilenetv2/cityscapes_segmentation_deeplabv3lite-mobilenetv2_2019-06-26-08-59-32.pth \
#--batch_size 12 --quantize True --epochs 150 --lr 5e-5 --evaluate_start False
+
## =====================================================================================
## Acuracy Evaluation with Post Training Quantization - cannot save quantized model - only accuracy evaluation
## =====================================================================================
## =====================================================================================
## Acuracy Evaluation with Post Training Quantization - cannot save quantized model - only accuracy evaluation
## =====================================================================================
#--batch_size 1 --quantize True
#--batch_size 1 --quantize True
-
+## =====================================================================================
+## Post Training Calibration & Quantization - this is fast, but may not always yield best quantized accuracy (not recommended)
+## =====================================================================================
+#
+#### Image Classification - Post Training Calibration & Quantization - ResNet50
+#python ./scripts/train_classification_main.py --phase calibration --dataset_name image_folder_classification --model_name resnet50_x1 --data_path ./data/datasets/image_folder_classification \
+#--pretrained https://download.pytorch.org/models/resnet50-19c8e357.pth \
+#--batch_size 64 --quantize True --epochs 1 --epoch_size 100
+#
+#
+#### Image Classification - Post Training Calibration & Quantization - MobileNetV2
+#python ./scripts/train_classification_main.py --phase calibration --dataset_name image_folder_classification --model_name mobilenetv2_tv_x1 --data_path ./data/datasets/image_folder_classification \
+#--pretrained https://download.pytorch.org/models/mobilenet_v2-b0353104.pth \
+#--batch_size 64 --quantize True --epochs 1 --epoch_size 100
+#
+#
+#### Image Classification - Post Training Calibration & Quantization for a TOUGH MobileNetV2 pretrained model
+#python ./scripts/train_classification_main.py --phase calibration --dataset_name image_folder_classification --model_name mobilenetv2_shicai_x1 --data_path ./data/datasets/image_folder_classification \
+#--pretrained ./data/modelzoo/pretrained/pytorch/others/shicai/MobileNet-Caffe/mobilenetv2_shicai_rgb.tar \
+#--batch_size 64 --quantize True --epochs 1 --epoch_size 100
+#
+#
+#### Semantic Segmentation - Post Training Calibration & Quantization for MobileNetV2+DeeplabV3Lite
+#python ./scripts/train_segmentation_main.py --phase calibration --dataset_name cityscapes_segmentation --model_name deeplabv3lite_mobilenetv2_tv --data_path ./data/datasets/cityscapes/data --img_resize 384 768 --output_size 1024 2048 --gpus 0 1 \
+#--pretrained ./data/modelzoo/semantic_segmentation/cityscapes/deeplabv3lite-mobilenetv2/cityscapes_segmentation_deeplabv3lite-mobilenetv2_2019-06-26-08-59-32.pth \
+#--batch_size 12 --quantize True --epochs 1 --epoch_size 100
diff --git a/run_segmentation.sh b/run_segmentation.sh
index 9e05986fa30408ae16fb540ad39fc9927f757d23..fef2a5efb62d3e1081bbfd535cf7bd4245b06003 100755 (executable)
--- a/run_segmentation.sh
+++ b/run_segmentation.sh
# Summary of commands - uncomment one and run this script
# Summary of commands - uncomment one and run this script
-#### For the datasets in sections marked as "Automatic Download", dataset will be downloaded automatically downloaded before training begins. For "Manual Download", it is expected that it is manually downloaded and kept in the folder specified agaianst the --data_path option.
+#### Manual Download: It is expected that the dataset is manually downloaded and kept in the folder specified agaianst the --data_path option.
## =====================================================================================
## Training
## =====================================================================================
## =====================================================================================
## Training
## =====================================================================================
-#### Cityscapes Semantic Segmentation (Manual Download) - Training with MobileNetV2+DeeplabV3Lite
-#python ./scripts/train_segmentation_main.py --dataset_name cityscapes_segmentation --model_name deeplabv3lite_mobilenetv2_tv --data_path ./data/datasets/cityscapes/data --img_resize 384 768 --output_size 1024 2048 --gpus 0 1
+#### Cityscapes Semantic Segmentation - Training with MobileNetV2+DeeplabV3Lite
+#python ./scripts/train_segmentation_main.py --dataset_name cityscapes_segmentation --model_name deeplabv3lite_mobilenetv2_tv --data_path ./data/datasets/cityscapes/data --img_resize 384 768 --output_size 1024 2048 --gpus 0 1 \
+#--pretrained https://download.pytorch.org/models/mobilenet_v2-b0353104.pth
-#### Cityscapes Semantic Segmentation (Manual Download) - Training with MobileNetV2+DeeplabV3Lite, Higher Resolution
-#python ./scripts/train_segmentation_main.py --dataset_name cityscapes_segmentation --model_name deeplabv3lite_mobilenetv2_tv --data_path ./data/datasets/cityscapes/data --img_resize 768 1536 --rand_crop 512 1024 --output_size 1024 2048 --gpus 0 1
+#### Cityscapes Semantic Segmentation - original fpn - no aspp model, stride 64 model - Low Complexity Model
+#python ./scripts/train_segmentation_main.py --dataset_name cityscapes_segmentation --model_name fpn_pixel2pixel_aspp_mobilenetv2_tv_es64 --data_path ./data/datasets/cityscapes/data --img_resize 384 768 --output_size 1024 2048 --gpus 0 1 \
+#--pretrained https://download.pytorch.org/models/mobilenet_v2-b0353104.pth
-#### Cityscapes Semantic Segmentation (Manual Download) - Training with ResNet50+FPN
+#### Cityscapes Semantic Segmentation - Training with MobileNetV2+DeeplabV3Lite, Higher Resolution
+#python ./scripts/train_segmentation_main.py --dataset_name cityscapes_segmentation --model_name deeplabv3lite_mobilenetv2_tv --data_path ./data/datasets/cityscapes/data --img_resize 768 1536 --rand_crop 512 1024 --output_size 1024 2048 --gpus 0 1 \
+#--pretrained https://download.pytorch.org/models/mobilenet_v2-b0353104.pth
+
+#### Cityscapes Semantic Segmentation - original fpn - no aspp model, stride 64 model, Higher Resolution - Low Complexity Model
+#python ./scripts/train_segmentation_main.py --dataset_name cityscapes_segmentation --model_name fpn_pixel2pixel_aspp_mobilenetv2_tv_es64 --data_path ./data/datasets/cityscapes/data --img_resize 768 1536 --rand_crop 512 1024 --output_size 1024 2048 --gpus 0 1 \
+#--pretrained https://download.pytorch.org/models/mobilenet_v2-b0353104.pth
+
+#### Cityscapes Semantic Segmentation - Training with ResNet50+FPN
#python ./scripts/train_segmentation_main.py --dataset_name cityscapes_segmentation --model_name fpn_pixel2pixel_aspp_resnet50 --data_path ./data/datasets/cityscapes/data --img_resize 768 1536 --rand_crop 512 1024 --output_size 1024 2048 --gpus 0 1 \
#--pretrained https://download.pytorch.org/models/resnet50-19c8e357.pth
#python ./scripts/train_segmentation_main.py --dataset_name cityscapes_segmentation --model_name fpn_pixel2pixel_aspp_resnet50 --data_path ./data/datasets/cityscapes/data --img_resize 768 1536 --rand_crop 512 1024 --output_size 1024 2048 --gpus 0 1 \
#--pretrained https://download.pytorch.org/models/resnet50-19c8e357.pth
-#### VOC Segmentation (Manual Download) - Training with MobileNetV2+DeeplabV3Lite
-#python ./scripts/train_segmentation_main.py --dataset_name voc_segmentation --model_name deeplabv3lite_mobilenetv2_tv --data_path ./data/datasets/voc --img_resize 512 512 --output_size 512 512 --gpus 0 1
+#### VOC Segmentation - Training with MobileNetV2+DeeplabV3Lite
+#python ./scripts/train_segmentation_main.py --dataset_name voc_segmentation --model_name deeplabv3lite_mobilenetv2_tv --data_path ./data/datasets/voc --img_resize 512 512 --output_size 512 512 --gpus 0 1 \
+#--pretrained https://download.pytorch.org/models/mobilenet_v2-b0353104.pth
## =====================================================================================
## Validation
## =====================================================================================
## =====================================================================================
## Validation
## =====================================================================================
-#### Validation - Cityscapes Semantic Segmentation (Manual Download) - Validation with MobileNetV2+DeeplabV3Lite - populate the pretrained filename in ??
+#### Validation - Cityscapes Semantic Segmentation - Validation with MobileNetV2+DeeplabV3Lite - populate the pretrained filename in ??
#python ./scripts/train_segmentation_main.py --evaluate True --dataset_name cityscapes_segmentation --model_name deeplabv3lite_mobilenetv2_tv --data_path ./data/datasets/cityscapes/data --img_resize 384 768 --output_size 1024 2048 --gpus 0 1 \
#--pretrained ??
#python ./scripts/train_segmentation_main.py --evaluate True --dataset_name cityscapes_segmentation --model_name deeplabv3lite_mobilenetv2_tv --data_path ./data/datasets/cityscapes/data --img_resize 384 768 --output_size 1024 2048 --gpus 0 1 \
#--pretrained ??
-#### Inference - Cityscapes Semantic Segmentation (Manual Download) - Inference with MobileNetV2+DeeplabV3Lite - populate the pretrained filename in ??
+#### Inference - Cityscapes Semantic Segmentation - Inference with MobileNetV2+DeeplabV3Lite - populate the pretrained filename in ??
#python ./scripts/infer_segmentation_main.py --dataset_name cityscapes_segmentation_measure --model_name deeplabv3lite_mobilenetv2_tv --data_path ./data/datasets/cityscapes/data --img_resize 384 768 --output_size 1024 2048 --gpus 0 1 \
#python ./scripts/infer_segmentation_main.py --dataset_name cityscapes_segmentation_measure --model_name deeplabv3lite_mobilenetv2_tv --data_path ./data/datasets/cityscapes/data --img_resize 384 768 --output_size 1024 2048 --gpus 0 1 \
-#--pretrained ./data/modelzoo/semantic_segmentation/cityscapes/deeplabv3lite-mobilenetv2/cityscapes_segmentation_deeplabv3lite-mobilenetv2_2019-06-26-08-59-32.pth
+#--pretrained ???
-#### Validation - VOC Segmentation (Manual Download) - Validation with MobileNetV2+DeeplabV3Lite - populate the pretrained filename in ??
+#### Validation - VOC Segmentation - Validation with MobileNetV2+DeeplabV3Lite - populate the pretrained filename in ??
#python ./scripts/train_segmentation_main.py --evaluate True --dataset_name voc_segmentation_measure --model_name deeplabv3lite_mobilenetv2_tv --data_path ./data/datasets/voc --img_resize 512 512 --output_size 512 512 --gpus 0 1
#--evaluate True --pretrained ???
#python ./scripts/train_segmentation_main.py --evaluate True --dataset_name voc_segmentation_measure --model_name deeplabv3lite_mobilenetv2_tv --data_path ./data/datasets/voc --img_resize 512 512 --output_size 512 512 --gpus 0 1
#--evaluate True --pretrained ???
index 81498249cddba559e0a08f87df40ea65a24c352d..2887be1eda02844a6cc924d6b1a9a40694289380 100755 (executable)
parser.add_argument('--save_path', type=str, default=None, help='checkpoint save folder')
parser.add_argument('--gpus', type=int, nargs='*', default=None, help='Base learning rate')
parser.add_argument('--batch_size', type=int, default=None, help='Batch size')
parser.add_argument('--save_path', type=str, default=None, help='checkpoint save folder')
parser.add_argument('--gpus', type=int, nargs='*', default=None, help='Base learning rate')
parser.add_argument('--batch_size', type=int, default=None, help='Batch size')
+parser.add_argument('--strides', type=int, nargs='*', default=None, help='strides in the model')
parser.add_argument('--lr', type=float, default=None, help='Base learning rate')
parser.add_argument('--lr_calib', type=float, default=None, help='Learning rate for calibration')
parser.add_argument('--model_name', type=str, default=None, help='model name')
parser.add_argument('--lr', type=float, default=None, help='Base learning rate')
parser.add_argument('--lr_calib', type=float, default=None, help='Learning rate for calibration')
parser.add_argument('--model_name', type=str, default=None, help='model name')
parser.add_argument('--resume', type=str, default=None, help='resume an unfinished training from this model')
parser.add_argument('--phase', type=str, default=None, help='training/calibration/validation')
parser.add_argument('--evaluate_start', type=str2bool, default=None, help='Whether to run validation before the training')
parser.add_argument('--resume', type=str, default=None, help='resume an unfinished training from this model')
parser.add_argument('--phase', type=str, default=None, help='training/calibration/validation')
parser.add_argument('--evaluate_start', type=str2bool, default=None, help='Whether to run validation before the training')
+parser.add_argument('--workers', type=int, default=None, help='number of workers for dataloading')
#
parser.add_argument('--quantize', type=str2bool, default=None, help='Quantize the model')
parser.add_argument('--histogram_range', type=str2bool, default=None, help='run only evaluation and no training')
#
parser.add_argument('--quantize', type=str2bool, default=None, help='Quantize the model')
parser.add_argument('--histogram_range', type=str2bool, default=None, help='run only evaluation and no training')
args.model_config.input_channels = 3
args.model_config.output_type = 'classification'
args.model_config.output_channels = None
args.model_config.input_channels = 3
args.model_config.output_type = 'classification'
args.model_config.output_channels = None
+args.model_config.strides = None #(2,2,2,2,2)
+args.img_resize = 256
+args.img_crop = 224
args.solver = 'sgd' #'sgd' #'adam'
args.epochs = 150 #150 #120
args.start_epoch = 0 #0
args.solver = 'sgd' #'sgd' #'adam'
args.epochs = 150 #150 #120
args.start_epoch = 0 #0
################################
for key in vars(cmds):
################################
for key in vars(cmds):
- if key == 'gpus':
- pass # already taken care above, since this has to be done before importing pytorch
+ if key == 'gpus': # already taken care above, since this has to be done before importing pytorch
+ pass
+ elif key == 'strides': # strides is in model_config
+ value = getattr(cmds, key)
+ if value != 'None' and value is not None:
+ setattr(args.model_config, key, value)
+ #
elif hasattr(args, key):
value = getattr(cmds, key)
if value != 'None' and value is not None:
elif hasattr(args, key):
value = getattr(cmds, key)
if value != 'None' and value is not None:
################################
# these dependent on the dataset chosen
################################
# these dependent on the dataset chosen
-args.img_resize = (args.img_resize if args.img_resize else 256)
-args.img_crop = (args.img_crop if args.img_crop else 224)
args.model_config.num_classes = (100 if 'cifar100' in args.dataset_name else (10 if 'cifar10' in args.dataset_name else 1000))
args.model_config.num_classes = (100 if 'cifar100' in args.dataset_name else (10 if 'cifar10' in args.dataset_name else 1000))
-args.model_config.strides = (1,1,1,2,2) if args.img_crop<56 else ((1,1,2,2,2) if args.img_crop<112 else ((1,2,2,2,2) if args.img_crop<224 else (2,2,2,2,2)))
+
################################
################################
################################
# In addition run a quantized calibration, starting from the trained model
################################
# In addition run a quantized calibration, starting from the trained model
-if args.phase == 'training' and (not args.quantize):
+if 'training' in args.phase and (not args.quantize):
save_path = train_classification.get_save_path(args)
args.pretrained = os.path.join(save_path, 'model_best.pth.tar')
save_path = train_classification.get_save_path(args)
args.pretrained = os.path.join(save_path, 'model_best.pth.tar')
- args.phase = 'calibration'
+ args.phase = 'training_quantize'
args.quantize = True
args.quantize = True
- args.epochs = 1
- args.epoch_size = 100
+ args.epochs = 25
+ args.lr = 5e-5
train_classification.main(args)
#
################################
# In addition run a separate validation, starting from the calibrated model - to estimate the quantized accuracy accurately
train_classification.main(args)
#
################################
# In addition run a separate validation, starting from the calibrated model - to estimate the quantized accuracy accurately
-if args.phase == 'training' or args.phase == 'calibration':
+if 'training' in args.phase or 'calibration' in args.phase:
save_path = train_classification.get_save_path(args)
args.pretrained = os.path.join(save_path, 'model_best.pth.tar')
args.phase = 'validation'
save_path = train_classification.get_save_path(args)
args.pretrained = os.path.join(save_path, 'model_best.pth.tar')
args.phase = 'validation'
index 63ce6177ecbe1f073c01d1a2b46977ab675c8c68..286c084f56eb8a73e543c15f03a2b2c9a81c9bb7 100755 (executable)
################################
# In addition run a quantized calibration, starting from the trained model
################################
# In addition run a quantized calibration, starting from the trained model
-if args.phase == 'training' and (not args.quantize):
+if 'training'in args.phase and (not args.quantize):
save_path = train_pixel2pixel.get_save_path(args)
args.pretrained = os.path.join(save_path, 'model_best.pth.tar')
save_path = train_pixel2pixel.get_save_path(args)
args.pretrained = os.path.join(save_path, 'model_best.pth.tar')
- args.phase = 'calibration'
+ args.phase = 'training_quantize'
args.quantize = True
args.quantize = True
- args.epochs = 1
- args.epoch_size = 100
+ args.epochs = 25
+ args.lr = 5e-5
train_pixel2pixel.main(args)
#
################################
# In addition run a separate validation, starting from the calibrated model - to estimate the quantized accuracy accurately
train_pixel2pixel.main(args)
#
################################
# In addition run a separate validation, starting from the calibrated model - to estimate the quantized accuracy accurately
-if args.phase == 'training' or args.phase == 'calibration':
+if 'training' in args.phase or 'calibration' in args.phase:
save_path = train_pixel2pixel.get_save_path(args)
args.pretrained = os.path.join(save_path, 'model_best.pth.tar')
args.phase = 'validation'
save_path = train_pixel2pixel.get_save_path(args)
args.pretrained = os.path.join(save_path, 'model_best.pth.tar')
args.phase = 'validation'
index c1e54aaa9643f605908a2bf863be93c67e2ea27c..e2297a30d771b48136df5f6887688ea39774ede9 100755 (executable)
################################
# In addition run a quantized calibration, starting from the trained model
################################
# In addition run a quantized calibration, starting from the trained model
-if args.phase == 'training' and (not args.quantize):
+if 'training' in args.phase and (not args.quantize):
save_path = train_pixel2pixel.get_save_path(args)
args.pretrained = os.path.join(save_path, 'model_best.pth.tar')
save_path = train_pixel2pixel.get_save_path(args)
args.pretrained = os.path.join(save_path, 'model_best.pth.tar')
- args.phase = 'calibration'
+ args.phase = 'training_quantize'
args.quantize = True
args.quantize = True
- args.epochs = 1
- args.epoch_size = 100
+ args.epochs = 25
+ args.lr = 5e-5
train_pixel2pixel.main(args)
#
################################
# In addition run a separate validation, starting from the calibrated model - to estimate the quantized accuracy accurately
train_pixel2pixel.main(args)
#
################################
# In addition run a separate validation, starting from the calibrated model - to estimate the quantized accuracy accurately
-if args.phase == 'training' or args.phase == 'calibration':
+if 'training' in args.phase or 'calibration' in args.phase:
save_path = train_pixel2pixel.get_save_path(args)
args.pretrained = os.path.join(save_path, 'model_best.pth.tar')
args.phase = 'validation'
save_path = train_pixel2pixel.get_save_path(args)
args.pretrained = os.path.join(save_path, 'model_best.pth.tar')
args.phase = 'validation'