modelzoo doc update

diff --git a/README.md b/README.md
index 81b319b8e461979559adb87a6fcaa7fb74ebfc94..6a5933c53d8c2eef42147baeef575aab7b905866 100644 (file)
--- a/README.md
+++ b/README.md
@@ -3,7 +3,7 @@
 
 This repository is an extension of the popular [mmdetection](https://github.com/open-mmlab/mmdetection) open source repository for object detection training. While mmdetection focuses on a wide variety of models, typically at high complexity, we focus on models that are optimized for speed and accuracy so that they run efficiently on embedded devices. 
 
-Kindly take time to read through the original documentation of the original [mmdetection](https://github.com/open-mmlab/mmdetection) before attempting to use this repository.
+Kindly take time to read through the documentation of the original [mmdetection](https://github.com/open-mmlab/mmdetection) before attempting to use this repository.
 
 
 ## License
@@ -13,7 +13,9 @@ Please see [LICENSE](./LICENSE) and [LICENSE.SPDX](./LICENSE.SPDX)
 
 ## Installation
 
-This repository requires mmdetection (mmdet Python package) to be installed. Please refer to [installation instructions for mmdetection](https://github.com/open-mmlab/mmdetection/blob/master/docs/install.md) for installation and dataset preparation. If you get any issues with the master branch of mmdetection, please try after checking out the latest release tag. Note mmdetection also requreis mmcv and several other dependencies to be installed as described in the above URL.
+This repository requires mmdet Python package from mmdetection to be installed. Please refer to [installation instructions for mmdetection](https://github.com/open-mmlab/mmdetection/blob/master/docs/install.md) for installation and also for dataset preparation. If you get any issues with the master branch of mmdetection, please try after checking out the latest release tag. 
+
+Note: mmdetection also requreis mmcv and several other dependencies to be installed as described in the above URL.
 
 After installing mmdetection, please install [PyTorch-Jacinto-AI-DevKit](https://bitbucket.itg.ti.com/projects/JACINTO-AI/repos/pytorch-jacinto-ai-devkit/browse/) as this repository uses several components from there - especially to define low complexity models and to do Quantization Aware Training (QAT) and Calibration.
 
@@ -32,13 +34,14 @@ Several trained models with accuracy report is available at [Jacinto-AI-Detectio
 
 ## Quantization
 
-Tutorial on how to do [Quantization Aware Training](./docs/det_quantization.md) in Jacinto-AI-MMDetection. 
+This tutorial explaines how to do [Quantization Aware Training](./docs/det_quantization.md) of detection models. We also provide sample QAT models.
 
 
 ## Acknowledgement
 
 This is an open source project that is contributed by researchers and engineers from various colleges and companies. We appreciate all the contributors who implement their methods or add new features, as well as users who give valuable feedbacks.
-We wish that the toolbox and benchmark could serve the growing research community by providing a flexible toolkit to reimplement existing methods and develop their own new detectors.
+
+We wish that the toolbox and benchmark could serve the growing research community by providing a flexible toolkit to train existing detectors and also to develop their own new detectors.
 
 
 ## Citation

diff --git a/docs/det_modelzoo.md b/docs/det_modelzoo.md
index 383e5b8f6c68b790d77562e45fe39c8655f5715d..67493f9e1007390b5328b9e709fae0f238fbc86e 100644 (file)
--- a/docs/det_modelzoo.md
+++ b/docs/det_modelzoo.md
@@ -2,23 +2,34 @@
 
 MMDetection has a huge Model Zoo, supporting a lot of models. Many of them are high complexity models that are not suitable for embedded scenarios that require high throughput. (Please refer to the mmdetection documentation link above for details). However, in this fork, we list only speed/accuracy optimized models that we have trained ourselves or is recommending from another location.
 
+
 ## Features
 
-|                    | ResNet   | Reget    | MobileNet|
-|--------------------|:--------:|:--------:|:--------:|
-| SSD                | ✓        | ✓        | ✓        |
-| RetinaNet          | ☐        | ☐        | ☐        |
-| FCOS               | ✗        | ✗        | ✗        |
-| Faster R-CNN       | ✗        | ✗        | ✗        |
-| Mask R-CNN         | ✗        | ✗        | ✗        |
+|                             | ResNet   | RegNetX  | MobileNet|
+|-----------------------------|:--------:|:--------:|:--------:|
+| SSD                         | ✓        | ✓        | ☐        |
+| RetinaNet                   | ☐        | ☐        | ☐        |
+| CenterNet(Objects As Points)| ✗        | ✗        | ✗        |
+| EfficientDet                | ✗        | ✗        | ✗        |
+| YOLOv3                      | ✗        | ✗        | ✗        |
+| Faster R-CNN                | ✗        | ✗        | ✗        |
+| Mask R-CNN                  | ✗        | ✗        | ✗        |
 
 ✓ Available, ☐ In progress or partially available, ✗ TBD
 
 
+We have config files for ResNet [5], RegNetX [6] and MobileNet [7] backbone architectures. However RegNetX family of backbone architectures strike a good balance between complexity, accuracy and easiness to quantize. Hence we strongly recommend to use that.
+
+The suffix **Lite** or **lite** in our model names or config files indicate that the model uses either Depthwise Convolutions (like in MobileNet models) or grouped convolutions (like in RegNetX models). When the backbone is a MobileNet, Depthwise convolutions are used even in FPN and the detector heads. When the backbone is a RegNetX model, Grouped convolutions are used with the same group size that the RegNetX backbone uses. But for backbones that use regular convolutions (such as ResNet) Depthwise or Grouped convolutions are not used as expected, and they are mostly compatible with the original mmdetection configurations.<br>
+
+We shall add support for additional low complexity models as mmdetection adds support for those architectures. Additional detectors that we are looking forward to see in mmdetection are CenterNet(Objects As Points), EfficientDet and YOLOv3. For more information, see the [roadmap of mmdetection](https://github.com/open-mmlab/mmdetection/issues/2931).
+
+
 #### COCO 2017 Dataset
 - Train on COCO 2017 Train Set
 - Test on COCO 2017 Validation Set
 
+
 ###### Single Shot Mult-Box Detector (SSD) 
 Please see the reference [1] for algorithmic details of the detector.
 
@@ -37,7 +48,7 @@ Please see the reference [2] for algorithmic details of the detector.
 
 |Model Arch       |Backbone Model|Resolution |AP [0.5:0.95]%|Giga MACS |Model Config File                |Download |
 |----------       |--------------|-----------|--------------|----------|---------------------------------|---------|
-|RetinaNetLite+FPN|RegNet800MF   |512x512    |              |**6.04**  |retinanet-lite_regnet_fpn_bgr.py |         |
+|RetinaNetLite+FPN|RegNetX800MF  |512x512    |              |**6.04**  |retinanet-lite_regnet_fpn_bgr.py |         |
 |RetinaNetLite+FPN|RegNetX1.6GF  |768x768    |              |          |retinanet-lite_regnet_fpn.py     |         |
 |.
 |RetinaNet+FPN*   |ResNet50      |512x512    |**29.7**      |**68.88** |                                 |[link](https://github.com/open-mmlab/mmdetection/tree/master/configs/retinanet) |
@@ -45,14 +56,31 @@ Please see the reference [2] for algorithmic details of the detector.
 |RetinaNet+FPN*   |ResNet50      |(1536,768) |**37.0**      |**275.5** |                                 |[link](https://github.com/open-mmlab/mmdetection/tree/master/configs/retinanet) |
 <br>
 
-- The suffix **Lite** indicates that the model uses either Depthwise Convolutions (like in MobileNet models) or grouped convolutions (like in RegNetX models). When the backbone is a MobileNet, we use Depthwise convolutions even in FPN and the detector heads. When the backbone is a RegNet model, we use Grouped convolutions with the same group size that the RegNet backbone uses. But for backbones that use regular convolutions (such as ResNet) we do not use Depthwise or Grouped convolutions.
-- A square resolution such as 512x512 indicates that the inputs are resized to that resolution without respecting the aspect ration of the image (keep_ratio=False in config files)<br>
-- A non-square resolution indicated with comma (1536,768) or dash (1536-768) indicates that images are resized to fit within this maximum and minimum size - but the aspect ratio of the image is preserved (keep_ratio=True in config files). This means that each image may have a different size after it is resized and hence is not suitable for embedded inference. But the interesting thing is that such a model can also be inferred or evaluated using a square aspect ratio.<br>
-- The models with a \* were not trained by us, but rather taken from mmdetection model zoo and inference is run at teh said resolution.<br>
+
+- A fixed resolution is indicated by width x height and it means that the inputs are resized to that resolution without preserving the aspect ratio of the image (keep_ratio=False in config files). **A fixed resolution is recommended for embedded inference.** The resolution after resize can be square (example: 512x512) or non-square (example: 768x384). Please note that embedded friendly detectors such as SSD [1] and EfficientDet (arXiv:1911.09070) [3] uses this kind of fixed resizing.<br>
+
+- A resolution range is indicated with comma (1536,768) or dash (1536-768) - it means that images are resized to fit within this maximum and minimum size - but the aspect ratio of the image is preserved (keep_ratio=True in config files). **This means that each image may have a different size after it is resized and hence is not suitable for embedded inference.** For example ONNXRuntime inference software [8] does not allow to do inference with variable size inputs for a given model. 
+
+- *Mmdetection typically uses a resolution range to train models for most models except SSD. An interesting observation that we had is that such  models trained for non-square resolutions can also be inferred or evaluated using square aspect ratios (with a bit of accuracy drop, of course). This leads to the possibility of reusing the models provided in the [mmdetection model zoo](https://github.com/open-mmlab/mmdetection/blob/master/docs/model_zoo.md) for an embedded scenario.* <br>
+
+- The models with a \* were not trained by us, but rather taken from mmdetection model zoo and inference is run at the said resolution.<br>
 
 
 ## References
 
 [1] SSD: Single Shot MultiBox Detector, https://arxiv.org/abs/1512.02325, Wei Liu, Dragomir Anguelov, Dumitru Erhan, Christian Szegedy, Scott Reed, Cheng-Yang Fu, Alexander C. Berg
 
-[2] Focal Loss for Dense Object Detection, https://arxiv.org/abs/1708.02002, Tsung-Yi Lin, Priya Goyal, Ross Girshick, Kaiming He, Piotr Dollár
\ No newline at end of file
+[2] Focal Loss for Dense Object Detection, https://arxiv.org/abs/1708.02002, Tsung-Yi Lin, Priya Goyal, Ross Girshick, Kaiming He, Piotr Dollár
+
+[3] EfficientDet: Scalable and Efficient Object Detection, https://arxiv.org/abs/1911.09070, Mingxing Tan, Ruoming Pang, Quoc V. Le
+
+[4] Objects as Points, https://arxiv.org/abs/1904.07850, Xingyi Zhou, Dequan Wang, Philipp Krähenbühl
+
+[5] Deep Residual Learning for Image Recognition, https://arxiv.org/abs/1512.03385, Kaiming He, Xiangyu Zhang, Shaoqing Ren, Jian Sun
+
+[6] MobileNetV2: Inverted Residuals and Linear Bottlenecks, https://arxiv.org/abs/1801.04381, Mark Sandler, Andrew Howard, Menglong Zhu, Andrey Zhmoginov, Liang-Chieh Chen<br>
+MobileNets: Efficient Convolutional Neural Networks for Mobile Vision Applications, https://arxiv.org/abs/1704.04861, Andrew G. Howard, Menglong Zhu, Bo Chen, Dmitry Kalenichenko, Weijun Wang, Tobias Weyand, Marco Andreetto, Hartwig Adam
+
+[7] Designing Network Design Spaces, https://arxiv.org/abs/2003.13678, Ilija Radosavovic, Raj Prateek Kosaraju, Ross Girshick, Kaiming He, Piotr Dollár
+
+[8] ONNX Runtime cross-platform inferencing software, https://github.com/microsoft/onnxruntime
\ No newline at end of file