Remove the compiled binary files

[ti-machine-learning/ti-machine-learning.git] / doc / latex / index.tex

\hypertarget{index_sectionOverview}{}\section{Overview}\label{index_sectionOverview}
The Texas Instruments Machine Learning (T\-I\-M\-L) library is C implementation of common machine learning functions optimized for T\-I embedded devices. Initially, the library supports convolutional neural networks (C\-N\-Ns), but will grow with time in scope. The library has a minimal set of dependencies upon other libraries to simplify the installation and use.\hypertarget{index_sectionInstallation}{}\section{Installation}\label{index_sectionInstallation}
\hypertarget{index_subsectionDependencies}{}\subsection{Dependencies}\label{index_subsectionDependencies}
{\bfseries  T\-I\-M\-L library dependencies }\par
 The T\-I\-M\-L library requires a high performance C basic linear algebra subprogram (C\-B\-L\-A\-S) library and a library for reading Joint Photographic Experts Group (J\-P\-E\-G) images. An optimized C\-B\-L\-A\-S implementation is included with T\-I embedded devices. For pre development work on a desktop, a reference (unoptimized) C\-B\-L\-A\-S implementation can be downloaded from \href{http://www.netlib.org/blas/blast-forum/cblas.tgz}{\tt http\-://www.\-netlib.\-org/blas/blast-\/forum/cblas.\-tgz} or in Ubuntu using the command

{\ttfamily  sudo apt-\/get install libatlas-\/base-\/dev }

A J\-P\-E\-G library, libjpeg, can be downloaded from \par
\href{http://sourceforge.net/projects/libjpeg/files/latest/download?source=files}{\tt http\-://sourceforge.\-net/projects/libjpeg/files/latest/download?source=files} \par
or in Ubuntu using the command

{\ttfamily  sudo apt-\/get install libjpeg-\/dev }

{\bfseries  Application dependencies }\par
 The Image\-Net database conversion application resizes all the images to size 256$\ast$256. The resizing is done using open\-C\-V. In Ubuntu, use the following command to install open\-C\-V

{\ttfamily  sudo apt-\/get install libopencv-\/dev }

The interop application converts a C\-N\-N model in Caffe format to T\-I\-M\-L C\-N\-N format. The Caffe format depends on google's protocol buffer library. To install the library, use command

{\ttfamily  sudo apt-\/get install libprotobuf-\/dev }\hypertarget{index_subsectionDirectoryStructure}{}\subsection{Directory Structure}\label{index_subsectionDirectoryStructure}
Install the T\-I\-M\-L library by unzipping the zip file to a directory referred to as $<$install\-\_\-directory$>$. At a terminal prompt, change the working directory to {\ttfamily $<$install\-\_\-directory$>$/build} and use the command {\ttfamily make all} to compile the library. The library file will be placed inside the {\ttfamily $<$install\-\_\-directory$>$/bin} folder.

The directory structure of the compiled library is shown as follows\-:
\begin{DoxyItemize}
\item {\ttfamily $<$install\-\_\-directory$>$/bin}\-: library binary file directory
\item {\ttfamily $<$install\-\_\-directory$>$/build}\-: makefile directory
\item {\ttfamily $<$install\-\_\-directory$>$/doc}\-: doxygen file directory
\item {\ttfamily $<$install\-\_\-directory$>$/src/common}\-: common library directory
\begin{DoxyItemize}
\item {\ttfamily $<$install\-\_\-directory$>$/src/common/api}\-: library A\-P\-I source file
\item {\ttfamily $<$install\-\_\-directory$>$/src/common/cnn}\-: cnn moudule source file
\item {\ttfamily $<$install\-\_\-directory$>$/src/common/util}\-: util moudule source file
\end{DoxyItemize}
\item {\ttfamily $<$install\-\_\-directory$>$/src/test}\-: test module directory
\begin{DoxyItemize}
\item {\ttfamily $<$install\-\_\-directory$>$/src/test/cnn}\-: cnn test source file
\item {\ttfamily $<$install\-\_\-directory$>$/src/test/util}\-: util test source file
\end{DoxyItemize}
\item {\ttfamily $<$install\-\_\-directory$>$/src/benchmark}\-: benchmark module directory
\begin{DoxyItemize}
\item {\ttfamily $<$install\-\_\-directory$>$/src/benchmark/cnn}\-: cnn benchmark source file
\begin{DoxyItemize}
\item {\ttfamily $<$install\-\_\-directory$>$/src/benchmark/cnn/class}\-: cnn classification benchmark source file
\end{DoxyItemize}
\end{DoxyItemize}
\item {\ttfamily $<$install\-\_\-directory$>$/src/app}\-: application module directory
\begin{DoxyItemize}
\item {\ttfamily $<$install\-\_\-directory$>$/src/app/cnn}\-: cnn application source file
\begin{DoxyItemize}
\item {\ttfamily $<$install\-\_\-directory$>$/src/app/cnn/class}\-: cnn classification application source file
\begin{DoxyItemize}
\item {\ttfamily $<$install\-\_\-directory$>$/src/app/cnn/class/cifar10}\-: cnn C\-I\-F\-A\-R10 database classification application source file
\item {\ttfamily $<$install\-\_\-directory$>$/src/app/cnn/class/imagenet}\-: cnn Image\-Net database classification application source file
\item {\ttfamily $<$install\-\_\-directory$>$/src/app/cnn/class/mnist}\-: cnn M\-N\-I\-S\-T database classification application source file
\end{DoxyItemize}
\item {\ttfamily $<$install\-\_\-directory$>$/src/app/cnn/scene}\-: cnn scene labeling application source file
\begin{DoxyItemize}
\item {\ttfamily $<$install\-\_\-directory$>$/src/app/cnn/scene/sbd}\-: cnn stanford background dataset scene labeling application source file
\end{DoxyItemize}
\item {\ttfamily $<$install\-\_\-directory$>$/src/app/cnn/interop}\-: cnn interoperation application source file
\begin{DoxyItemize}
\item {\ttfamily $<$install\-\_\-directory$>$/src/app/cnn/interop/caffe}\-: cnn-\/caffe interoperation application source file
\end{DoxyItemize}
\item {\ttfamily $<$install\-\_\-directory$>$/src/app/cnn/convert}\-: cnn database conversion application source file
\begin{DoxyItemize}
\item {\ttfamily $<$install\-\_\-directory$>$/src/app/cnn/convert/imagenet}\-: cnn Image\-Net database conversion application source file
\item {\ttfamily $<$install\-\_\-directory$>$/src/app/cnn/convert/sbd}\-: cnn S\-B\-D database conversion application source file
\end{DoxyItemize}
\end{DoxyItemize}
\end{DoxyItemize}
\item {\ttfamily $<$install\-\_\-directory$>$/src/database}\-: database directory
\begin{DoxyItemize}
\item cifar10\-: Canadian Institute for Advanced Research-\/10 Class
\item imagenet\-: Image\-Net 2012
\item mnist\-: Mixed National Institute of Standards and Technology
\item sbd\-: Stanford Background Dataset
\item model\-: pretrained models
\begin{DoxyItemize}
\item cifar10
\item alexnet
\item caffenet
\item vggnet
\item mnist
\item sbd
\end{DoxyItemize}
\end{DoxyItemize}
\end{DoxyItemize}\hypertarget{index_subsectionDoc}{}\subsection{Document Genearation}\label{index_subsectionDoc}
Documents of html and pdf formats can be generated using Doxygen. Make sure you have both latex and Doxygen installed. In Ubuntu, the installation commands are


\begin{DoxyItemize}
\item {\ttfamily sudo apt-\/get install doxygen}
\item {\ttfamily sudo apt-\/get install texlive-\/full}
\end{DoxyItemize}

Use the command {\ttfamily doxygen $<$install-\/ {\ttfamily $<$install\-\_\-directory$>$/src/app/cnn/scene}\-: cnn scene labeling application source file\-\_\-directory$>$/doc/timl.Doxyfile} to generate the documents. The html version is located at {\ttfamily doxygen $<$install\-\_\-directory$>$/doc/html}. To generate the pdf version, change the directory to {\ttfamily doxygen $<$install\-\_\-directory$>$/doc/latex} and run the command {\ttfamily make}. A filed named refman.\-pdf will be generated in the currnet folder.\hypertarget{index_subsectionImageDatabases}{}\subsection{Image Databases}\label{index_subsectionImageDatabases}
In order to run the examples provided by the library, it is required to download additional databases of test images.

{\bfseries M\-N\-I\-S\-T}

The M\-N\-I\-S\-T database of handwritten digits, available from this page, has a training set of 60,000 examples, and a test set of 10,000 examples. It is a subset of a larger set available from N\-I\-S\-T. The digits have been size-\/normalized and centered in a fixed-\/size image.

You can download the M\-N\-I\-S\-T database from \href{http://yann.lecun.com/exdb/mnist/}{\tt http\-://yann.\-lecun.\-com/exdb/mnist/} or simpliy change the directory to {\ttfamily $<$install\-\_\-directory$>$/src/database/mnist} and run the script {\ttfamily ./database\-M\-N\-I\-S\-T\-Download.sh}. After the download, there should be 4 files in the folder\-:
\begin{DoxyItemize}
\item {\ttfamily t10k-\/images.\-idx3-\/ubyte}
\item {\ttfamily t10k-\/labels.\-idx1-\/ubyte}
\item {\ttfamily train-\/images.\-idx3-\/ubyte}
\item {\ttfamily train-\/labels.\-idx1-\/ubyte}.
\end{DoxyItemize}

{\bfseries C\-I\-F\-A\-R10}

The C\-I\-F\-A\-R-\/10 dataset consists of 60000 32x32 colour images in 10 classes, with 6000 images per class. The dataset is divided into five training batches and one test batch, each with 10000 images. The test batch contains exactly 1000 randomly-\/selected images from each class. The training batches contain the remaining images in random order, but some training batches may contain more images from one class than another. Between them, the training batches contain exactly 5000 images from each class.

You can download the C\-I\-F\-A\-R10 database from \href{http://www.cs.toronto.edu/~kriz/cifar.html}{\tt http\-://www.\-cs.\-toronto.\-edu/$\sim$kriz/cifar.\-html} or simpliy change the directory to {\ttfamily $<$install\-\_\-directory$>$/src/database/cifar10} and run the script {\ttfamily ./database\-C\-I\-F\-A\-R10\-Download.sh}. After the download, there should be 6 files in the folder\-:
\begin{DoxyItemize}
\item {\ttfamily data\-\_\-batch\-\_\-1.\-bin}
\item {\ttfamily data\-\_\-batch\-\_\-2.\-bin}
\item {\ttfamily data\-\_\-batch\-\_\-3.\-bin}
\item {\ttfamily data\-\_\-batch\-\_\-4.\-bin}
\item {\ttfamily data\-\_\-batch\-\_\-5.\-bin}
\item {\ttfamily test\-\_\-batch\-\_\-6.\-bin}
\end{DoxyItemize}

{\bfseries Image\-Net}

Image\-Net is an image dataset organized according to the Word\-Net hierarchy.\-Each meaningful concept in Word\-Net, possibly described by multiple words or word phrases, is called a \char`\"{}synonym set\char`\"{} or \char`\"{}synset\char`\"{}. There are more than 100,000 synsets in Word\-Net, majority of them are nouns (80,000+). Image\-Net aims to provide on average 1000 images to illustrate each synset. Images of each concept are quality-\/controlled and human-\/annotated.

Download the database from \href{http://www.image-net.org/challenges/LSVRC/2012/nonpub-downloads}{\tt http\-://www.\-image-\/net.\-org/challenges/\-L\-S\-V\-R\-C/2012/nonpub-\/downloads} to $<$install\-\_\-directory$>$/database/imagnet. You need to register before you can download the database.\-http\-://dags.stanford.\-edu/projects/scenedataset.html
\begin{DoxyItemize}
\item Download the training images of size 138\-G\-B and M\-D5\-: 1d675b47d978889d74fa0da5fadfb00e
\item Download the validation images of size 6.\-3\-G\-B and M\-D5\-: 29b22e2961454d5413ddabcf34fc5622
\item Download the auxiliary files by changing the working directory to {\ttfamily $<$install\-\_\-directory$>$/src/database/imagnet} and runing the script {\ttfamily  ./database\-Image\-Net\-Download.sh }
\item Extract all the files to the $<$install\-\_\-directory$>$/src/database/imagnet
\item Change run the script ./database\-Image\-Net\-Convert.sh to convert the raw database to a format that is ready to be processed by T\-I\-M\-L. You may need to change the path variables in this script if you choose to store the database in other directories. You can also specify the number of images to be converted in the script.
\end{DoxyItemize}

After the conversion, there should be two folders, test and train, in $<$install\-\_\-directory$>$/src/database/imagnet. There will be a single labels.\-txt file in each of the folder. All the images will be resized to 256$\ast$256.

{\bfseries Stanford Background Dataset}

The Stanford Background Dataset is a new dataset introduced in Gould et al. (I\-C\-C\-V 2009) for evaluating methods for geometric and semantic scene understanding. The dataset contains 715 images chosen from existing public datasets\-: Label\-Me, M\-S\-R\-C, P\-A\-S\-C\-A\-L V\-O\-C and Geometric Context. The selection criteria were for the images to be of outdoor scenes, have approximately 320-\/by-\/240 pixels, contain at least one foreground object, and have the horizon position within the image.

You can download the database from \href{http://dags.stanford.edu/data/iccv09Data.tar.gz}{\tt http\-://dags.\-stanford.\-edu/data/iccv09\-Data.\-tar.\-gz} or simply change the working directory to {\ttfamily $<$install\-\_\-directory$>$/src/database/sbd} and run the script {\ttfamily database\-S\-B\-D\-Download.\-sh} To convert the database to T\-I\-M\-L compatible format, run the script ./database\-S\-B\-D\-Convert.sh. You can specify the number of images used for training and testing by changing the corresponding variables in the script. After running the script, there should be two folders named \char`\"{}train\char`\"{} and \char`\"{}test\char`\"{} in the current folder.\hypertarget{index_subsectionCNNPretrainedModels}{}\subsection{C\-N\-N Pretrained Models}\label{index_subsectionCNNPretrainedModels}
T\-I\-M\-L can convert models pretrained by Caffe\cite{Caffe} to a format that is compatible with T\-I\-M\-L. Caffe is a deep learning framework developed by the Berkeley Vision and Learning Center (B\-V\-L\-C). You can train your C\-N\-N on Caffe on C\-P\-U or G\-P\-U, save the parameters and then convert it to a format supported by T\-I\-M\-L.

{\bfseries  Alex\-Net }\cite{AlexNet}

Alex\-Net is a deep convolutional neural network to classify the 1.\-3 million high-\/resolution images in the L\-S\-V\-R\-C-\/2010 Image\-Net training set into the 1000 different classes. On the test data, Alex\-Net achieved top-\/1 and top-\/5 error rates of 39.\-7\% and 18.\-9\% which is considerably better than the previous state-\/of-\/the-\/art results. The neural network, which has 60 million parameters and 500,000 neurons, consists of five convolutional layers, some of which are followed by max-\/pooling layers, and two globally connected layers with a final 1000-\/way softmax.
\begin{DoxyItemize}
\item Download the Caffe Alex\-Net binary file from \href{http://dl.caffe.berkeleyvision.org/bvlc_alexnet.caffemodel}{\tt http\-://dl.\-caffe.\-berkeleyvision.\-org/bvlc\-\_\-alexnet.\-caffemodel} to $<$install\-\_\-directory$>$/src/database/model/alexnet
\item Download the Caffe Alex\-Net depoly file from \href{https://github.com/BVLC/caffe/blob/master/models/bvlc_alexnet/deploy.prototxt}{\tt https\-://github.\-com/\-B\-V\-L\-C/caffe/blob/master/models/bvlc\-\_\-alexnet/deploy.\-prototxt} to $<$install\-\_\-directory$>$/src/database/model/alexnet
\item Run the script ./database\-Model\-Alex\-Net\-Interop.sh to perform the conversion
\end{DoxyItemize}

After the conversion, there should two files in $<$install\-\_\-directory$>$/database/model/alexnet\-:
\begin{DoxyItemize}
\item database\-Model\-Alex\-Net.\-m {\itshape A text file that defines the structure of the C\-N\-N}
\item database\-Model\-Alex\-Net.\-m.\-params {\itshape A binary file that stores the parameters of the C\-N\-N}
\end{DoxyItemize}

{\bfseries  Caffe\-Net }

The Caffe\-Net is a replication of the model described in the Alex\-Net publication with some differences\-:
\begin{DoxyItemize}
\item not training with the relighting data-\/augmentation;
\item the order of pooling and normalization layers is switched (in Caffe\-Net, pooling is done before normalization).
\end{DoxyItemize}

This model obtains a top-\/1 accuracy 57.\-4\% and a top-\/5 accuracy 80.\-4\% on the validation set, using just the center crop.


\begin{DoxyItemize}
\item Download the Caffe Caffe\-Net binary file from \href{http://dl.caffe.berkeleyvision.org/bvlc_reference_caffenet.caffemodel}{\tt http\-://dl.\-caffe.\-berkeleyvision.\-org/bvlc\-\_\-reference\-\_\-caffenet.\-caffemodel} to $<$install\-\_\-directory$>$/database/model/caffenet
\item Download the Caffe Caffe\-Net depoly file from \href{https://github.com/BVLC/caffe/blob/master/models/bvlc_reference_caffenet/deploy.prototxt}{\tt https\-://github.\-com/\-B\-V\-L\-C/caffe/blob/master/models/bvlc\-\_\-reference\-\_\-caffenet/deploy.\-prototxt} to $<$install\-\_\-directory$>$/src/database/model/caffenet
\item Run the script ./database\-Model\-Caffe\-Net\-Interop.sh to perform the conversion
\end{DoxyItemize}

After the conversion, there should two files in $<$install\-\_\-directory$>$/src/database/model/caffenet\-:
\begin{DoxyItemize}
\item database\-Model\-Caffe\-Net.\-m
\item database\-Model\-Caffe\-Net.\-m.\-params
\end{DoxyItemize}

{\bfseries  V\-G\-G\-Net } \cite{VGG}

V\-G\-G\-Net shows that a significant improvement on the prior-\/art configurations can be achieved by increasing the depth to 16-\/19 weight layers, which is substantially deeper than what has been used in the prior art. To reduce the number of parameters in such very deep networks, the researchers use very small 3×3 filters in all convolutional layers.


\begin{DoxyItemize}
\item Download the V\-G\-G\-Net binary file from \href{http://www.robots.ox.ac.uk/~vgg/software/very_deep/caffe/VGG_ILSVRC_16_layers.caffemodel}{\tt http\-://www.\-robots.\-ox.\-ac.\-uk/$\sim$vgg/software/very\-\_\-deep/caffe/\-V\-G\-G\-\_\-\-I\-L\-S\-V\-R\-C\-\_\-16\-\_\-layers.\-caffemodel} to $<$install\-\_\-directory$>$/src/database/model/vggnet
\item Download the V\-G\-G\-Net depoly file from \href{https://gist.githubusercontent.com/ksimonyan/211839e770f7b538e2d8/raw/0067c9b32f60362c74f4c445a080beed06b07eb3/VGG_ILSVRC_16_layers_deploy.prototxt}{\tt https\-://gist.\-githubusercontent.\-com/ksimonyan/211839e770f7b538e2d8/raw/0067c9b32f60362c74f4c445a080beed06b07eb3/\-V\-G\-G\-\_\-\-I\-L\-S\-V\-R\-C\-\_\-16\-\_\-layers\-\_\-deploy.\-prototxt} to $<$install\-\_\-directory$>$/src/database/model/vggnet
\item Run the script ./database\-Model\-V\-G\-G\-Net\-Interop.sh to perform the conversion
\end{DoxyItemize}

After the conversion, there should two files in $<$install\-\_\-directory$>$/src/database/model/caffenet\-:
\begin{DoxyItemize}
\item database\-Model\-V\-G\-G\-Net.\-m
\item database\-Model\-V\-G\-G\-Net.\-m.\-params
\end{DoxyItemize}

{\bfseries  M\-N\-I\-S\-T } A pretrained C\-N\-N for M\-N\-I\-S\-T database is located at $<$install\-\_\-directory$>$/src/database/model/mnist\-:
\begin{DoxyItemize}
\item database\-Model\-M\-N\-I\-S\-T.\-m
\item database\-Model\-M\-N\-I\-S\-T.\-m.\-params
\end{DoxyItemize}

{\bfseries  C\-I\-F\-A\-R10 } A pretrained C\-N\-N for C\-I\-F\-A\-R10 database is located at $<$install\-\_\-directory$>$/src/database/model/cifar10\-:
\begin{DoxyItemize}
\item database\-Model\-C\-I\-F\-A\-R10.\-m
\item database\-Model\-C\-I\-F\-A\-R10.\-m.\-params
\end{DoxyItemize}\hypertarget{index_sectionCNNs}{}\section{Convolutional Neural Networks (\-C\-N\-Ns)}\label{index_sectionCNNs}
The T\-I\-M\-L library provides a set of A\-P\-Is that allow a user to implement a C\-N\-N architecture and perform training and testing. Common C\-N\-N layer types are supported in the current version and more will be added in future versions. Both image classification and scene labeling examples are provided.

This section briefly describes the use of C\-N\-N library A\-P\-Is. For complete examples, refer to the folder {\ttfamily $<$install\-\_\-directory$>$/src/app/cnn}\hypertarget{index_subsectionTrainingParameters}{}\subsection{Training Parameters}\label{index_subsectionTrainingParameters}
The default training parameters structure is obtained by calling {\ttfamily \hyperlink{group__cnn_gaaf7a5b272ae2ba5d0453f214afaab747}{timl\-C\-N\-N\-Training\-Params\-Default()}}. Default parameters can be overridden by setting specific field values (refer to {\ttfamily \hyperlink{structtimlCNNTrainingParams}{timl\-C\-N\-N\-Training\-Params}} for details).\hypertarget{index_subsectionLayers}{}\subsection{Layers}\label{index_subsectionLayers}
This section describes the different C\-N\-N layers supported by the T\-I\-M\-L library. To create a C\-N\-N, call {\ttfamily \hyperlink{group__cnn_gaf75b4687f4b493b8a85bc7fcc37bdf18}{timl\-C\-N\-N\-Create\-Conv\-Neural\-Network()}}. Here is an example code block in {\ttfamily timl\-Test\-C\-N\-N\-Simple\-Training()} that generates a simple C\-N\-N with 8 layer types\-:


\begin{DoxyCode}
trainingParams              = \hyperlink{group__cnn_gaaf7a5b272ae2ba5d0453f214afaab747}{timlCNNTrainingParamsDefault}();
trainingParams.batchSize    = BATCH\_SIZE;
trainingParams.learningRate = 0.1;
cnn = \hyperlink{group__cnn_gaf75b4687f4b493b8a85bc7fcc37bdf18}{timlCNNCreateConvNeuralNetwork}(trainingParams, 0);
inputParams       = \hyperlink{group__cnn_ga71b91e772f1e63528e53a6284ba7b879}{timlCNNInputParamsDefault}();
inputParams.scale = 1.0/256.0;
\hyperlink{group__cnn_ga253033ab06b21f2bf28301bc1e53c419}{timlCNNAddInputLayer}(cnn, IMAGE\_ROW, IMAGE\_COL, IMAGE\_CHANNEL, inputParams);           
       \textcolor{comment}{// input layer}
\hyperlink{group__cnn_ga772bc4b21c2f800ee8ac5248a732f116}{timlCNNAddConvLayer}(cnn, 5, 5, 1, 1, 6, 
      \hyperlink{group__cnn_ga33ddb7145ae4e19ea52e12576f4871aa}{timlCNNConvParamsDefault}());                    \textcolor{comment}{// conv layer}
\hyperlink{group__cnn_gaddfd98d23613318bdc368eac98fddf0b}{timlCNNAddNonlinearLayer}(cnn, Util\_Relu);                                          
           \textcolor{comment}{// relu layer}
\hyperlink{group__cnn_gab5dcc5ac470e15bea85f0a5570eec37d}{timlCNNAddPoolingLayer}(cnn, 4, 4, 4, 4, CNN\_MaxPooling, 
      \hyperlink{group__cnn_gacd163e268b3ccc9805bcca59689a36f5}{timlCNNPoolingParamsDefault}()); \textcolor{comment}{// max pooling layer}
\hyperlink{group__cnn_ga7627fe7a5294d7c8cac75d974b78a27a}{timlCNNAddNormLayer}(cnn, \hyperlink{group__cnn_ga03ee34f7d893af02536bcbf2d071f721}{timlCNNNormParamsDefault}());           
                              \textcolor{comment}{// norm layer}
\hyperlink{group__cnn_gae988ef172bde701db59b1cfc33dd50ab}{timlCNNAddDropoutLayer}(cnn, 0.5);                                                    
         \textcolor{comment}{// dropout layer}
\hyperlink{group__cnn_ga0219ba397301c5fb3eedfb64e62e33bc}{timlCNNAddLinearLayer}(cnn, 10, \hyperlink{group__cnn_gad5c8fd10f11ccaa8142a665fc8ee93b8}{timlCNNLinearParamsDefault}())
      ;                           \textcolor{comment}{// linear layer}
\hyperlink{group__cnn_gaddfd98d23613318bdc368eac98fddf0b}{timlCNNAddNonlinearLayer}(cnn, Util\_Softmax);                                       
           \textcolor{comment}{// softmax layer}
\hyperlink{group__cnn_ga666fa0a68b725aa1e35dba659ba7c5f9}{timlCNNInitialize}(cnn);
\hyperlink{group__cnn_ga21bc0c855cb658f51fe5d6911acddb54}{timlCNNReset}(cnn);
\end{DoxyCode}
\hypertarget{index_subsubsectionInputLayer}{}\subsubsection{Input}\label{index_subsubsectionInputLayer}
The input layer is responsible for preprocessing the raw input data. The user can choose to crop, scale, mirror, or subtract the mean form the raw image. The required parameters are the dimensions (row, column, channel) of the feature maps in the input layer. Refer to {\ttfamily \hyperlink{group__cnn_ga253033ab06b21f2bf28301bc1e53c419}{timl\-C\-N\-N\-Add\-Input\-Layer()}} for more details.\hypertarget{index_subsubsectionConvolutionalLayer}{}\subsubsection{Convolutional}\label{index_subsubsectionConvolutionalLayer}
The convolutional layer performs 2\-D convolution or correlation between feature maps and kernels. The required parameters are the 2\-D kernel dimension (row, column), kernel strides along row and column and the output feature map channel. For each feature map in the previous layer, there will be a corresponding 2\-D kernel applied to it and links to each feature map in the next layer. Refer to {\ttfamily \hyperlink{group__cnn_ga772bc4b21c2f800ee8ac5248a732f116}{timl\-C\-N\-N\-Add\-Conv\-Layer()}} for more detail.\hypertarget{index_subsubsectionNonlinearLayer}{}\subsubsection{Nonlinear}\label{index_subsubsectionNonlinearLayer}
The nonlinear layer implements 1 nonlinearity, supported types are\-:


\begin{DoxyItemize}
\item rectified linear unit\par
 $ f(z) = \max(0, z) $
\item sigmoid\par
 $ f(z) = \frac{1}{1+e^{-z}} $
\item softmax\par
 $ f(z)_j = \frac{e^{-z_j}}{\sum_{i=0}^{K}e^{-z_i}} $
\item tanh\par
 $ f(z) = \frac{e^{z}-e^{-z}}{e^{z}+e^{-z}} $
\end{DoxyItemize}To add a nonlinear layer to the cnn structure, call the function {\ttfamily \hyperlink{group__cnn_gaddfd98d23613318bdc368eac98fddf0b}{timl\-C\-N\-N\-Add\-Nonlinear\-Layer()}}. Note that for image classification applications, the last layer is most commonly chosen to be a softmax nonlinear layer that output an array of class probabilities that sum up to 1.\hypertarget{index_subsubsectionPoolingLayer}{}\subsubsection{Pooling}\label{index_subsubsectionPoolingLayer}
The pooling layer performs a local maxing or averaging operation on the feature maps. The required parameters are the pooling kernel dimension (row, col), kernel strides along column and row and the pooling method. Refer to {\ttfamily \hyperlink{group__cnn_gab5dcc5ac470e15bea85f0a5570eec37d}{timl\-C\-N\-N\-Add\-Pooling\-Layer()}} for more details.\hypertarget{index_subsubsectionNormalizationLayer}{}\subsubsection{Normalization}\label{index_subsubsectionNormalizationLayer}
The normalization layer performs local contrast normalization across channels\-:\par
 $ y^i = \frac{x^i}{(1+\frac{\alpha}{N}\sum^{\min(N-1, i+N/2)}_{j=\max(0, i-N/2)}x^j)^\beta} $ ,where $x^i$ stands for the feature map in the i-\/th channel. The channel span N defaults to 5. $\alpha$ and $\beta$ default to 0.\-001 and 0.\-75, respectively. Refer to {\ttfamily \hyperlink{group__cnn_ga7627fe7a5294d7c8cac75d974b78a27a}{timl\-C\-N\-N\-Add\-Norm\-Layer()}} for more details.\hypertarget{index_subsubsectionDropoutLayer}{}\subsubsection{Dropout}\label{index_subsubsectionDropoutLayer}
The dropout layer is used during training to help prevent overfitting. Each element in the feature map is set to 0 (deactivated) according to a preset probability in the training phase. Note the dropout layer is simply a pass through in the testing phase. Refer to {\ttfamily \hyperlink{group__cnn_gae988ef172bde701db59b1cfc33dd50ab}{timl\-C\-N\-N\-Add\-Dropout\-Layer()}} for more details.\hypertarget{index_subsubsectionLinearLayer}{}\subsubsection{Linear}\label{index_subsubsectionLinearLayer}
The linear layer is also referred to as the inner product layer or fully connected layer. A traditional neural network layer, the feature map in the previous layer is first vectorized and then multiplied by a weight matrix to obtain the feature map of next layer. The required parameter is the dimension of the output feature map. Refer to {\ttfamily \hyperlink{group__cnn_ga0219ba397301c5fb3eedfb64e62e33bc}{timl\-C\-N\-N\-Add\-Linear\-Layer()}} for more details.\hypertarget{index_subsectionMemory}{}\subsection{Memory}\label{index_subsectionMemory}
Once the setup of the cnn structure is complete, {\ttfamily \hyperlink{group__cnn_ga666fa0a68b725aa1e35dba659ba7c5f9}{timl\-C\-N\-N\-Initialize()}} is called to allocate the memory. There are 3 levels of memory allocation specified inside {\ttfamily \hyperlink{structtimlCNNTrainingParams}{timl\-C\-N\-N\-Training\-Params}}.
\begin{DoxyItemize}
\item Level 1 can be used both for training and testing and requires the most memory.
\item Level 2 requires less memory but can only be used for testing.
\item Level 3 uses even less memory by operating on a memory pool and can also only be used for testing.
\end{DoxyItemize}

Note that this function does not initialize the kernels or weights in the convolutional and linear layers. To perform that initialization the function {\ttfamily \hyperlink{group__cnn_ga21bc0c855cb658f51fe5d6911acddb54}{timl\-C\-N\-N\-Reset()}} is used.

The exact memory allocated in bytes can be obtained by calling {\ttfamily \hyperlink{group__cnn_ga2e8a7e765f885a236e40dfaed59913c6}{timl\-C\-N\-N\-Memory()}}.

{\ttfamily \hyperlink{group__cnn_ga78addba765ea7dd8bf958d19d42be931}{timl\-C\-N\-N\-Resize()}} is used to resize the dimension of a C\-N\-N. Once the input layer dimension is changed, the dimension of the following layers will be re-\/calcualted and re-\/allocated.

{\ttfamily \hyperlink{group__cnn_ga409c28e6bb3633f1de65647b790aa320}{timl\-C\-N\-N\-Clone()}} creates a independent copy of a C\-N\-N. The function {\ttfamily \hyperlink{group__cnn_ga52826c14f9f1913947f5ca6937b5afd5}{timl\-C\-N\-N\-Share\-Params()}} differs from {\ttfamily \hyperlink{group__cnn_ga409c28e6bb3633f1de65647b790aa320}{timl\-C\-N\-N\-Clone()}} in that the C\-N\-N structure returned by this function allocates its own feature map memory but points to the parameter memory of its target. Therefore, {\ttfamily \hyperlink{group__cnn_ga52826c14f9f1913947f5ca6937b5afd5}{timl\-C\-N\-N\-Share\-Params()}} returns a structure that takes less memory compared with the structure returned by {\ttfamily \hyperlink{group__cnn_ga409c28e6bb3633f1de65647b790aa320}{timl\-C\-N\-N\-Clone()}}, which allocates its own parameter memory. The user should be careful when manipulating the shared copy of a C\-N\-N as it may write to the parameters of its target. {\ttfamily \hyperlink{group__cnn_ga52826c14f9f1913947f5ca6937b5afd5}{timl\-C\-N\-N\-Share\-Params()}} is used primarily to accelerate the training and testing of a C\-N\-N. Multiple shared copies of a C\-N\-N can work in parallel by using Open\-M\-P.

To free the space allocated by the C\-N\-N, call the function {\ttfamily \hyperlink{group__cnn_ga5af48b599237f48f016ce2060c6053e1}{timl\-C\-N\-N\-Delete()}}.\hypertarget{index_subsectionUtilityFunctions}{}\subsection{Utility Functions}\label{index_subsectionUtilityFunctions}
Utility functions aim to perform miscellaneous functions that related to I\-O.

The phase of a C\-N\-N can be set to training or testing by calling {\ttfamily \hyperlink{group__cnn_ga04d9a7e3725ce777402e1e5c4d5d4ce0}{timl\-C\-N\-N\-Set\-Mode()}}. A C\-N\-N allocated with level 2 or 3 is not allowed to be set to the training mode as no memory is allocated to run the back propagation algorithm. In the testing mode, only forward propagation will be performed.

C\-N\-N structure information can be printed to the console by using {\ttfamily \hyperlink{group__cnn_ga123557b8025497fe266d58d80ee9983b}{timl\-C\-N\-N\-Print()}}. {\ttfamily \hyperlink{group__cnn_gaf65f8d45c892f70eeee34b47791bdcb5}{timl\-C\-N\-N\-Get\-Layer\-Num()}} returns the number of layers of the C\-N\-N. {\ttfamily \hyperlink{group__cnn_ga72df82405eb43be8bb75373aaf8ec101}{timl\-C\-N\-N\-Get\-Params\-Num()}} returns the total number of parameters in the C\-N\-N.

{\ttfamily \hyperlink{group__cnn_gad7f3f988c763fe5a80826439c9fb08bd}{timl\-C\-N\-N\-Write\-To\-File()}} writes the cnn structure into a combination of text and binary files. The key parameter of this function is the {\ttfamily timl\-Util\-Params\-Level}, which specifies the level of details of the writing.
\begin{DoxyItemize}
\item Level 1 only writes the network structure to a text file without specifying the parameters or feature maps of the cnn. The text file is formatted in a syntax that is compatible with Matlab script.
\item Level 2 writes both the network structure text file and the binary parameter file.
\item Level 3 writes one more state binary files. Level 3 is only used for debugging purpose. Note that the path of the binary files to read is written into the text file.
\end{DoxyItemize}

Similarly, {\ttfamily \hyperlink{group__cnn_gabab7c3c18a2fa4a3a9985da42662682c}{timl\-C\-N\-N\-Read\-From\-File()}} can be used to read a C\-N\-N from text or binary files.

One example of the generated text file is shown below\-: 
\begin{DoxyCode}
paramsBinaryFileName = \textcolor{stringliteral}{'./database/test/cnn/timl\_cnn\_simple\_config.m.params'};
stateBinaryFileName  = \textcolor{stringliteral}{'./database/test/cnn/timl\_cnn\_simple\_config.m.state'};

cnn.params.\hyperlink{structtimlCNNTrainingParams_ae54e9bf53e695070047e6fe4711d42e2}{count}          = 0;
cnn.params.\hyperlink{structtimlCNNTrainingParams_a92c1f02fac7360a8aa865736f27b684a}{batchSize}      = 100;
cnn.params.\hyperlink{structtimlCNNTrainingParams_a8c02933b3f59a4b7086c9446e0d67297}{epoch}          = 1;
cnn.params.learningRate   = 0.1000;
cnn.params.momentum       = 0.0000;
cnn.params.phase          = 0;
cnn.params.allocatorLevel = 0;
cnn.params.\hyperlink{structtimlCNNTrainingParams_a7037ed9b5aa77cb921f21fa338abf26e}{costType}       = 0;

layerNum                                    = 8;
cnn.layer(1).id                             = 1;
cnn.layer(1).type                           = 0;
cnn.layer(1).row                            = 28;
cnn.layer(1).col                            = 28;
cnn.layer(1).channel                        = 1;
cnn.layer(1).inputParams.row                = 28;
cnn.layer(1).inputParams.col                = 28;
cnn.layer(1).inputParams.channel            = 1;
cnn.layer(1).inputParams.scale              = 1.0000;
cnn.layer(1).inputParams.trainingCropType   = 0;
cnn.layer(1).inputParams.trainingMirrorType = 1;
cnn.layer(1).inputParams.testingCropType    = 0;
cnn.layer(1).inputParams.testingMirrorType  = 1;

cnn.layer(2).id                                 = 2;
cnn.layer(2).type                               = 1;
cnn.layer(2).row                                = 24;
cnn.layer(2).col                                = 24;
cnn.layer(2).channel                            = 6;
cnn.layer(2).convParams.kernelRow               = 5;
cnn.layer(2).convParams.kernelCol               = 5;
cnn.layer(2).convParams.padUp                   = 0;
cnn.layer(2).convParams.padDown                 = 0;
cnn.layer(2).convParams.padLeft                 = 0;
cnn.layer(2).convParams.padRight                = 0;
cnn.layer(2).convParams.strideX                 = 1;
cnn.layer(2).convParams.strideY                 = 1;
cnn.layer(2).convParams.inputFeatureMapChannel  = 1;
cnn.layer(2).convParams.outputFeatureMapChannel = 6;
cnn.layer(2).convParams.type                    = 0;
cnn.layer(2).convParams.kernelDecayFactor       = 1.0000;
cnn.layer(2).convParams.kernelInit.type         = 3;
cnn.layer(2).convParams.kernelLearningFactor    = 1.0000;
cnn.layer(2).convParams.biasInit.type           = 0;
cnn.layer(2).convParams.biasLearningFactor      = 1.0000;

cnn.layer(3).id                   = 3;
cnn.layer(3).type                 = 3;
cnn.layer(3).row                  = 24;
cnn.layer(3).col                  = 24;
cnn.layer(3).channel              = 6;
cnn.layer(3).nonlinearParams.type = 0;

cnn.layer(4).id                     = 4;
cnn.layer(4).type                   = 2;
cnn.layer(4).row                    = 12;
cnn.layer(4).col                    = 12;
cnn.layer(4).channel                = 6;
cnn.layer(4).poolingParams.type     = 0;
cnn.layer(4).poolingParams.scaleRow = 2;
cnn.layer(4).poolingParams.scaleCol = 2;
cnn.layer(4).poolingParams.padUp    = 0;
cnn.layer(4).poolingParams.padDown  = 0;
cnn.layer(4).poolingParams.padLeft  = 0;
cnn.layer(4).poolingParams.padRight = 0;
cnn.layer(4).poolingParams.strideX  = 2;
cnn.layer(4).poolingParams.strideY  = 2;

cnn.layer(5).id                 = 5;
cnn.layer(5).type               = 6;
cnn.layer(5).row                = 12;
cnn.layer(5).col                = 12;
cnn.layer(5).channel            = 6;
cnn.layer(5).dropoutParams.prob = 0.5000

cnn.layer(6).id               = 6;
cnn.layer(6).type             = 5;
cnn.layer(6).row              = 12;
cnn.layer(6).col              = 12;
cnn.layer(6).channel          = 6;
cnn.layer(6).normParams.type  = 0;
cnn.layer(6).normParams.N     = 5;
cnn.layer(6).normParams.alpha = 0.0010;
cnn.layer(6).normParams.beta  = 0.7500;

cnn.layer(7).id                                = 7;
cnn.layer(7).type                              = 4;
cnn.layer(7).row                               = 1;
cnn.layer(7).col                               = 1;
cnn.layer(7).channel                           = 10;
cnn.layer(7).linearParams.dim                  = 10;
cnn.layer(7).linearParams.prevDim              = 864;
cnn.layer(7).linearParams.weightDecayFactor    = 1.0000;
cnn.layer(7).linearParams.weightInit.type      = 3;
cnn.layer(7).linearParams.weightLearningFactor = 1.0000;
cnn.layer(7).linearParams.biasInit.type        = 0;
cnn.layer(7).linearParams.biasLearningFactor   = 1.0000;

cnn.layer(8).id                   = 8;
cnn.layer(8).type                 = 3;
cnn.layer(8).row                  = 1;
cnn.layer(8).col                  = 1;
cnn.layer(8).channel              = 10;
cnn.layer(8).nonlinearParams.type = 1;
\end{DoxyCode}


Note that the text file is formatted using Matlab syntax such that the text file can be used as a script to obtain a cnn structure object.\hypertarget{index_cnnTraining}{}\subsection{Training}\label{index_cnnTraining}
The following code block in {\ttfamily \hyperlink{group__testCNN_gaae822c926bf3623fccbb8196504df2a1}{test\-C\-N\-N\-Simple\-Training()}} performs training\-:


\begin{DoxyCode}
\textcolor{comment}{// read MNIST database}
 printf(\textcolor{stringliteral}{"2. Read the MNIST database\(\backslash\)n"});
 \hyperlink{group__util_ga218c4e1b952545e65e70b46fc087c0fa}{timlUtilReadMNIST}(DATABASE\_PATH, &training, &testing);

 \textcolor{comment}{// training}
 printf(\textcolor{stringliteral}{"3. Start training\(\backslash\)n"});
 clock\_gettime(CLOCK\_REALTIME, &startTime);
 \textcolor{keywordflow}{for} (i = 0; i < batchNum; i++) \{
    \hyperlink{group__cnn_ga9198f6379fad0e9e45dd8f64cc030ec3}{timlCNNSupervisedTrainingWithLabelBatchMode}(cnn, training.
      data + i*batchSize*dim, training.label + i*batchSize, dim, batchSize);
 \}
 clock\_gettime(CLOCK\_REALTIME, &endTime);
 trainingTime = \hyperlink{group__util_gad332b3fc9cca905cb073bb950c27d9a3}{timlUtilDiffTime}(startTime, endTime);
 printf(\textcolor{stringliteral}{"Training time = %.2f s.\(\backslash\)n"}, trainingTime/1000000.0);
\end{DoxyCode}


In this example, the training and testing image data set structures are read from the M\-N\-I\-S\-T database and then passed into {\ttfamily \hyperlink{group__cnn_ga9198f6379fad0e9e45dd8f64cc030ec3}{timl\-C\-N\-N\-Supervised\-Training\-With\-Label\-Batch\-Mode()}}. Note that there is also a multi-\/thread version of this function that called {\ttfamily \hyperlink{group__cnn_ga9198f6379fad0e9e45dd8f64cc030ec3}{timl\-C\-N\-N\-Supervised\-Training\-With\-Label\-Batch\-Mode()}}. The training time is returned by {\ttfamily \hyperlink{group__util_gad332b3fc9cca905cb073bb950c27d9a3}{timl\-Util\-Diff\-Time()}} in microsecond precision.\hypertarget{index_subsectionTesting}{}\subsection{Testing}\label{index_subsectionTesting}
The following code block in {\ttfamily test\-C\-N\-N\-Simple\-Testing()} performs testing\-:


\begin{DoxyCode}
\textcolor{keywordflow}{for}(i = 0; i < testing.num; i++)
\{
    label = \hyperlink{group__cnn_gafda5077b0d39c278469278b37f845aed}{timlCNNClassifyTop1SingleMode}(cnn, testing.data + i*dim, dim);
    \textcolor{keywordflow}{if} (label != testing.label[i]) misClassifyNum++;
\}
\end{DoxyCode}


{\ttfamily \hyperlink{group__cnn_gafda5077b0d39c278469278b37f845aed}{timl\-C\-N\-N\-Classify\-Top1\-Single\-Mode()}} returns the top 1 label generated by the C\-N\-N. A similar function {\ttfamily \hyperlink{group__cnn_ga8b59ecbca36e54cfa1df0766ece686b5}{timl\-C\-N\-N\-Classify\-Top\-N\-Batch\-Mode()}} that returns the top N labels together with their corresponding probabilities for a batch of data. Note that there is also a multi-\/thread version of this function called {\ttfamily \hyperlink{group__cnn_ga4bb1537483995068ac4d250e9e8a7615}{timl\-C\-N\-N\-Classify\-Top\-N\-Batch\-Mode\-Open\-M\-P()}} that classifies a batch of images using Open\-M\-P.\hypertarget{index_sectionApplications}{}\section{Applications}\label{index_sectionApplications}
Application source code is located at {\ttfamily $<$install\-\_\-directory$>$/src/app}. To run the applications, change the directory to the corresponding binary files and run the applications.\par
 There are 1 application example for Caffe to T\-I\-M\-L C\-N\-N model interoperation\-:
\begin{DoxyItemize}
\item {\ttfamily app\-C\-N\-N\-Interop\-Caffe}\-: Caffe interoperation
\end{DoxyItemize}

There are 2 application example for Caffe to T\-I\-M\-L C\-N\-N database conversion\-:
\begin{DoxyItemize}
\item {\ttfamily app\-C\-N\-N\-Convert\-Image\-Net}\-: Imag\-Net conversion
\item {\ttfamily app\-C\-N\-N\-Convert\-S\-B\-D}\-: S\-B\-D conversion
\end{DoxyItemize}

There are 8 application examples for C\-N\-N classification\-:
\begin{DoxyItemize}
\item {\ttfamily \hyperlink{group__appCNNClass_ga74ae1f2ec0bb9e1cfc344cdfd204f4da}{app\-C\-N\-N\-Class\-M\-N\-I\-S\-T\-Training()}}\-: M\-N\-I\-S\-T database training
\item {\ttfamily \hyperlink{group__appCNNClass_gafa188db78d9c7794ae1bac7a8c93cb3a}{app\-C\-N\-N\-Class\-M\-N\-I\-S\-T\-Testing()}}\-: M\-N\-I\-S\-T database testing
\item {\ttfamily \hyperlink{group__appCNNClass_gac0aaedb9badf585311124c33751ee778}{app\-C\-N\-N\-Class\-C\-I\-F\-A\-R10\-Training()}}\-: C\-I\-F\-A\-R10 database training
\item {\ttfamily \hyperlink{group__appCNNClass_gaff0f3476d60a46eeb68eae09d6df964a}{app\-C\-N\-N\-Class\-C\-I\-F\-A\-R10\-Testing()}}\-: C\-I\-F\-A\-R10 database testing
\item {\ttfamily app\-C\-N\-N\-Class\-Caffe\-Net\-Training()}\-: Caffe\-Net database training
\item {\ttfamily app\-C\-N\-N\-Class\-Caffe\-Net\-Testing()}\-: Caffe\-Net database testing
\item {\ttfamily app\-C\-N\-N\-Class\-Alex\-Net\-Testing()}\-: Alex\-Net database training
\item {\ttfamily app\-C\-N\-N\-Class\-V\-G\-G\-Net\-Testing()}\-: V\-G\-G\-Net database testing
\end{DoxyItemize}

There are 2 application examples for C\-N\-N scene labeling\-:
\begin{DoxyItemize}
\item {\ttfamily \hyperlink{group__appCNNScene_ga38cb0fe9d4fb06709a7e9330911c9431}{app\-C\-N\-N\-Scene\-S\-B\-D\-Training()}}\-: scene labeling database training
\item {\ttfamily \hyperlink{group__appCNNScene_gaa2f9ce79a0aaf4a15ef414ac09f0ca7d}{app\-C\-N\-N\-Scene\-S\-B\-D\-Testing()}}\-: scene labeling database testing
\end{DoxyItemize}\hypertarget{index_subsectionClassification}{}\subsection{Classification}\label{index_subsectionClassification}
Training and testing a C\-N\-N for image classification is very straightforward to implement using the T\-I\-M\-L library. Let's take the M\-N\-I\-S\-T database for example. Here is one code block in {\ttfamily \hyperlink{group__appCNNClass_ga74ae1f2ec0bb9e1cfc344cdfd204f4da}{app\-C\-N\-N\-Class\-M\-N\-I\-S\-T\-Training()}}. There are 3 major steps. First, build up the C\-N\-N structure using the library A\-P\-I. Next, read the database. Third, apply the training function on the database. After the training, you can write the trained network to file(s) using {\ttfamily \hyperlink{group__cnn_gad7f3f988c763fe5a80826439c9fb08bd}{timl\-C\-N\-N\-Write\-To\-File()}}. 
\begin{DoxyCode}
\textcolor{comment}{// setup CNN}
printf(\textcolor{stringliteral}{"1. Build up the CNN\(\backslash\)n"});
\hyperlink{struct__timlConvNeuralNetwork__}{timlConvNeuralNetwork} *cnn = \hyperlink{group__cnn_gaf75b4687f4b493b8a85bc7fcc37bdf18}{timlCNNCreateConvNeuralNetwork}
      (\hyperlink{group__cnn_gaaf7a5b272ae2ba5d0453f214afaab747}{timlCNNTrainingParamsDefault}(), 0);
cnn->params.learningRate = LEARN\_RATE;
cnn->params.\hyperlink{structtimlCNNTrainingParams_a92c1f02fac7360a8aa865736f27b684a}{batchSize}    = BATCH\_SIZE;
inputParams = \hyperlink{group__cnn_ga71b91e772f1e63528e53a6284ba7b879}{timlCNNInputParamsDefault}();
inputParams.scale = 1.0/256.0;
\hyperlink{group__cnn_ga253033ab06b21f2bf28301bc1e53c419}{timlCNNAddInputLayer}(cnn, IMAGE\_ROW, IMAGE\_COL, IMAGE\_CHANNEL, inputParams);           
        \textcolor{comment}{// input layer}
convParams = \hyperlink{group__cnn_ga33ddb7145ae4e19ea52e12576f4871aa}{timlCNNConvParamsDefault}();
convParams.kernelInit.type = Util\_Xavier;
\hyperlink{group__cnn_ga772bc4b21c2f800ee8ac5248a732f116}{timlCNNAddConvLayer}(cnn, 5, 5, 1, 1, 20, convParams);                                   
       \textcolor{comment}{// conv layer}
poolingParams = \hyperlink{group__cnn_gacd163e268b3ccc9805bcca59689a36f5}{timlCNNPoolingParamsDefault}();
\hyperlink{group__cnn_gab5dcc5ac470e15bea85f0a5570eec37d}{timlCNNAddPoolingLayer}(cnn, 2, 2, 2, 2, CNN\_MaxPooling, poolingParams);              
          \textcolor{comment}{// max pooling layer}
convParams = \hyperlink{group__cnn_ga33ddb7145ae4e19ea52e12576f4871aa}{timlCNNConvParamsDefault}();
convParams.kernelInit.type = Util\_Xavier;
\hyperlink{group__cnn_ga772bc4b21c2f800ee8ac5248a732f116}{timlCNNAddConvLayer}(cnn, 5, 5, 1, 1, 50, convParams);                                   
       \textcolor{comment}{// conv layer}
\hyperlink{group__cnn_gab5dcc5ac470e15bea85f0a5570eec37d}{timlCNNAddPoolingLayer}(cnn, 2, 2, 2, 2, CNN\_MaxPooling, 
      \hyperlink{group__cnn_gacd163e268b3ccc9805bcca59689a36f5}{timlCNNPoolingParamsDefault}());  \textcolor{comment}{// max pooling layer}
\hyperlink{group__cnn_ga0219ba397301c5fb3eedfb64e62e33bc}{timlCNNAddLinearLayer}(cnn, 500, \hyperlink{group__cnn_gad5c8fd10f11ccaa8142a665fc8ee93b8}{timlCNNLinearParamsDefault}()
      );                           \textcolor{comment}{// linear layer}
\hyperlink{group__cnn_gaddfd98d23613318bdc368eac98fddf0b}{timlCNNAddNonlinearLayer}(cnn, Util\_Relu);                                          
            \textcolor{comment}{// relu layer}
\hyperlink{group__cnn_ga0219ba397301c5fb3eedfb64e62e33bc}{timlCNNAddLinearLayer}(cnn, 10, \hyperlink{group__cnn_gad5c8fd10f11ccaa8142a665fc8ee93b8}{timlCNNLinearParamsDefault}())
      ;                            \textcolor{comment}{// linear layer}
\hyperlink{group__cnn_gaddfd98d23613318bdc368eac98fddf0b}{timlCNNAddNonlinearLayer}(cnn, Util\_Softmax);                                       
            \textcolor{comment}{// softmax layer}
\hyperlink{group__cnn_ga666fa0a68b725aa1e35dba659ba7c5f9}{timlCNNInitialize}(cnn);
\hyperlink{group__cnn_ga21bc0c855cb658f51fe5d6911acddb54}{timlCNNReset}(cnn);
mem = \hyperlink{group__cnn_ga2e8a7e765f885a236e40dfaed59913c6}{timlCNNMemory}(cnn);
\hyperlink{group__cnn_ga123557b8025497fe266d58d80ee9983b}{timlCNNPrint}(cnn);
printf(\textcolor{stringliteral}{"CNN memory allocation = %.10f MB.\(\backslash\)n"}, (\textcolor{keywordtype}{float}) mem/1024.0/1024.0);
printf(\textcolor{stringliteral}{"CNN parameter #       = %ld.\(\backslash\)n"}, \hyperlink{group__cnn_ga72df82405eb43be8bb75373aaf8ec101}{timlCNNGetParamsNum}(cnn));

\textcolor{comment}{// read MNIST database}
printf(\textcolor{stringliteral}{"2. Read the MNIST database\(\backslash\)n"});
\hyperlink{group__util_ga218c4e1b952545e65e70b46fc087c0fa}{timlUtilReadMNIST}(DATABASE\_PATH, &training, &testing);

\textcolor{comment}{// training}
printf(\textcolor{stringliteral}{"3. Start training\(\backslash\)n"});
clock\_gettime(CLOCK\_REALTIME, &startTime);
\textcolor{keywordflow}{for} (i = 0; i < batchNum; i++) \{
   \hyperlink{group__cnn_ga9198f6379fad0e9e45dd8f64cc030ec3}{timlCNNSupervisedTrainingWithLabelBatchMode}(cnn, training.
      data + i*batchSize*dim, training.label + i*batchSize, dim, batchSize);
\}
clock\_gettime(CLOCK\_REALTIME, &endTime);
trainingTime = \hyperlink{group__util_gad332b3fc9cca905cb073bb950c27d9a3}{timlUtilDiffTime}(startTime, endTime);
printf(\textcolor{stringliteral}{"Training time = %.2f s.\(\backslash\)n"}, trainingTime/1000000.0);
\end{DoxyCode}
 Deploying or testing the C\-N\-N is even simpler. Here is one code block in {\ttfamily \hyperlink{group__appCNNClass_gafa188db78d9c7794ae1bac7a8c93cb3a}{app\-C\-N\-N\-Class\-M\-N\-I\-S\-T\-Testing()}}. Again, there are 3 major steps. First, read the C\-N\-N structure from file(s). Next, read the testing database. Third, apply the testing function to produced the labels generated by the C\-N\-N. 
\begin{DoxyCode}
\textcolor{comment}{// read CNN config}
printf(\textcolor{stringliteral}{"1. Read CNN config\(\backslash\)n"});
\hyperlink{struct__timlConvNeuralNetwork__}{timlConvNeuralNetwork} *cnn = \hyperlink{group__cnn_gabab7c3c18a2fa4a3a9985da42662682c}{timlCNNReadFromFile}(MODEL\_PATH, 0);
\hyperlink{group__cnn_ga04d9a7e3725ce777402e1e5c4d5d4ce0}{timlCNNSetMode}(cnn, Util\_Tes@subsection subsectionSceneLabeling Scene Labelingt);
mem = \hyperlink{group__cnn_ga2e8a7e765f885a236e40dfaed59913c6}{timlCNNMemory}(cnn);
\hyperlink{group__cnn_ga123557b8025497fe266d58d80ee9983b}{timlCNNPrint}(cnn);
printf(\textcolor{stringliteral}{"CNN memory allocation = %.10f MB.\(\backslash\)n"}, (\textcolor{keywordtype}{float})mem/1024.0/1024.0);
printf(\textcolor{stringliteral}{"CNN parameter #       = %lu.\(\backslash\)n"}, \hyperlink{group__cnn_ga72df82405eb43be8bb75373aaf8ec101}{timlCNNGetParamsNum}(cnn));

\textcolor{comment}{// read MNIST database}
printf(\textcolor{stringliteral}{"2. Read MNIST database\(\backslash\)n"});
\hyperlink{group__util_ga218c4e1b952545e65e70b46fc087c0fa}{timlUtilReadMNIST}(DATABASE\_PATH, &training, &testing);

\textcolor{comment}{// testing}
printf(\textcolor{stringliteral}{"3. Start testing\(\backslash\)n"});
clock\_gettime(CLOCK\_REALTIME, &startTime);
\hyperlink{group__cnn_ga8b59ecbca36e54cfa1df0766ece686b5}{timlCNNClassifyTopNBatchMode}(cnn, testing.data, dim, testing.num, label, NULL, 
      topN);
clock\_gettime(CLOCK\_REALTIME, &endTime);
testingTime = \hyperlink{group__util_gad332b3fc9cca905cb073bb950c27d9a3}{timlUtilDiffTime}(startTime, endTime);
classifyNum = \hyperlink{group__util_ga64e5197e9d131870e97cd510b3f499ac}{timlUtilClassifyAccuracy}(label, topN, testing.num, testing.label);
classifyPercent = (float)classifyNum/(\textcolor{keywordtype}{float})testing.num;
printf(\textcolor{stringliteral}{"Testing time      = %.3f s\(\backslash\)n"}, testingTime/1000000.0);
printf(\textcolor{stringliteral}{"Classify accuracy = %.3f %%\(\backslash\)n"}, classifyPercent*100.00);
\end{DoxyCode}
\hypertarget{index_subsectionSceneLabeling}{}\subsection{Scene Labeling}\label{index_subsectionSceneLabeling}
Scene labeling consists in labeling each pixel in an image with the category of the object it belongs to. Instread of producing one lable for the entire image, scene labeling produces one label for each pixel in the image. Therefore, training a C\-N\-N for scene labeling is slightly different from training a C\-N\-N for classification purpose. The setup of the C\-N\-N follows the same procedure\-:


\begin{DoxyCode}
\textcolor{comment}{// build up the CNN}
printf(\textcolor{stringliteral}{"1. Build up the CNN\(\backslash\)n"});
cnn = \hyperlink{group__cnn_gaf75b4687f4b493b8a85bc7fcc37bdf18}{timlCNNCreateConvNeuralNetwork}(
      \hyperlink{group__cnn_gaaf7a5b272ae2ba5d0453f214afaab747}{timlCNNTrainingParamsDefault}(), 0);
\hyperlink{group__cnn_ga253033ab06b21f2bf28301bc1e53c419}{timlCNNAddInputLayer}(cnn, PATCH\_SIZE, PATCH\_SIZE, IMAGE\_CHANNEL, 
      \hyperlink{group__cnn_ga71b91e772f1e63528e53a6284ba7b879}{timlCNNInputParamsDefault}());
\hyperlink{group__cnn_ga772bc4b21c2f800ee8ac5248a732f116}{timlCNNAddConvLayer}(cnn, 6, 6, 1, 1, 25, 
      \hyperlink{group__cnn_ga33ddb7145ae4e19ea52e12576f4871aa}{timlCNNConvParamsDefault}());                    \textcolor{comment}{// conv layer}
\hyperlink{group__cnn_gaddfd98d23613318bdc368eac98fddf0b}{timlCNNAddNonlinearLayer}(cnn, Util\_Tanh);                                          
            \textcolor{comment}{// tanh layer}
\hyperlink{group__cnn_gab5dcc5ac470e15bea85f0a5570eec37d}{timlCNNAddPoolingLayer}(cnn, 8, 8, 8, 8, CNN\_MaxPooling, 
      \hyperlink{group__cnn_gacd163e268b3ccc9805bcca59689a36f5}{timlCNNPoolingParamsDefault}());  \textcolor{comment}{// max pooling layer}
\hyperlink{group__cnn_ga772bc4b21c2f800ee8ac5248a732f116}{timlCNNAddConvLayer}(cnn, 3, 3, 1, 1, 50, 
      \hyperlink{group__cnn_ga33ddb7145ae4e19ea52e12576f4871aa}{timlCNNConvParamsDefault}());                    \textcolor{comment}{// conv layer}
\hyperlink{group__cnn_gaddfd98d23613318bdc368eac98fddf0b}{timlCNNAddNonlinearLayer}(cnn, Util\_Tanh);                                          
            \textcolor{comment}{// tanh layer}
\hyperlink{group__cnn_gab5dcc5ac470e15bea85f0a5570eec37d}{timlCNNAddPoolingLayer}(cnn, 2, 2, 2, 2, CNN\_MaxPooling, 
      \hyperlink{group__cnn_gacd163e268b3ccc9805bcca59689a36f5}{timlCNNPoolingParamsDefault}());  \textcolor{comment}{// max pooling layer}
\hyperlink{group__cnn_ga0219ba397301c5fb3eedfb64e62e33bc}{timlCNNAddLinearLayer}(cnn, 8, \hyperlink{group__cnn_gad5c8fd10f11ccaa8142a665fc8ee93b8}{timlCNNLinearParamsDefault}());
                                   \textcolor{comment}{// linear layer}
\hyperlink{group__cnn_gaddfd98d23613318bdc368eac98fddf0b}{timlCNNAddNonlinearLayer}(cnn, Util\_Softmax);                                       
            \textcolor{comment}{// softmax layer}
\end{DoxyCode}


The second step is to setup the training database\-: 
\begin{DoxyCode}
slTraining.num              = TRAIN\_IMAGE\_NUM;
slTraining.row              = IMAGE\_ROW;
slTraining.col              = IMAGE\_COL;
slTraining.channel          = IMAGE\_CHANNEL;
slTraining.patchSize        = PATCH\_SIZE;
slTraining.imageFileNameStr = TRAIN\_IMAGE\_PATH;
slTraining.labelFileNameStr = TRAIN\_LABEL\_PATH;
\end{DoxyCode}


In this example, 450 images are used for training and 143 images are used for testing. The dimension of the images are 240$\ast$320$\ast$3. There are 450 text file labels in the training database, each with a 240$\ast$320 label matrix. The label integer ranges from -\/1 to 7 (8 classes) with -\/1 indicating an unlabeled pixel. The image and label text file name must follow the format as indicated in the image\-File\-Name\-Str and label\-File\-Name\-Str. For each pixel in the image, a 133$\ast$133 patch centered at the pixel is passed into the C\-N\-N to output a label.

To train the database, the C\-N\-N first randomly selects an image and then randomly selects a pixel. Next, a patch centered at that pixel is passed to the C\-N\-N for training. The pixels are randomly selected without replacement, with 240$\ast$320$\ast$450 iterations needed to sweep through the entire database 1x. There is a specialized training function\-: 
\begin{DoxyCode}
\hyperlink{group__appCNNScene_gaf308ce265a25eed146051684e2da2671}{appCNNSceneSupervisedTraining}(cnn, &slTraining);
\end{DoxyCode}


To test one image, a natural way is to generate a patch for each pixel which requires 240$\ast$320 forward passes through the C\-N\-N. However, there is more efficient way to do this which exploits the convolutional structure. First, pad zeros on the test image and shift the image in all four directions. The number of shifted and zero padded images is proportional to the number of pooling layers in the C\-N\-N. Next, pass the shifted and zero padded image to the trained C\-N\-N. Finally, merge the output feature maps into a label matrix for the image. Using this approach, the input feature map size is no longer the patch size which is 133$\ast$133 in the above example. We need to re-\/calculate the input feature map dimensions and use function {\ttfamily \hyperlink{group__cnn_ga78addba765ea7dd8bf958d19d42be931}{timl\-C\-N\-N\-Resize()}} to resize the C\-N\-N. Note that the above cnn has gone through 2 max pooling layers which scales down the feature map by a factor of 2$\ast$8 in the row and column dimensions. The formula to calculate the resized row and col for the input layer is\-: 
\begin{DoxyCode}
printf(\textcolor{stringliteral}{"3. Resize the feature maps\(\backslash\)n"});
resolutionLossRow = 8*2; \textcolor{comment}{// resolution loss due to max pooling}
resolutionLossCol = 8*2; \textcolor{comment}{// resolution loss due to max pooling}
resizeRow = slTraining.row + (slTraining.patchSize/2)*2 - (resolutionLossRow - 1);
resizeCol = slTraining.col + (slTraining.patchSize/2)*2 - (resolutionLossCol - 1);
\hyperlink{group__cnn_ga78addba765ea7dd8bf958d19d42be931}{timlCNNResize}(cnn, resizeRow, resizeCol, slTraining.channel);
\end{DoxyCode}


Multiple forward passes through the same C\-N\-N can be performed in parallel. First, create multilple shared copies of the C\-N\-N to form a team called cnn\-Team. Then we can call the function {\ttfamily \hyperlink{group__appCNNScene_gad77aeeb63649011f260d9c475bef945d}{app\-C\-N\-N\-Scene\-Classify\-Open\-M\-P()}} to perform the labeling operation. A single-\/thread version of the function also exists as {\ttfamily \hyperlink{group__appCNNScene_ga4924a13edce94afaba52667d78ef7256}{app\-C\-N\-N\-Scene\-Classify()}}; The parameter scale in {\ttfamily \hyperlink{group__appCNNScene_gad77aeeb63649011f260d9c475bef945d}{app\-C\-N\-N\-Scene\-Classify\-Open\-M\-P()}} stands for the scale down factor of the generated label matrix. When scale == 1, no downscaling is performed. When scale == 4, the generated label matrix would be of size 60$\ast$80 and then upscaled to 240$\ast$320 to match the size of the image. Downscaling can effectively reduce the labeling time.


\begin{DoxyCode}
\textcolor{comment}{// create cnnTeam}
cnnTeam[0] = cnn;
\textcolor{keywordflow}{for} (i = 1; i < thread; i++) \{
   cnnTeam[i] = \hyperlink{group__cnn_ga52826c14f9f1913947f5ca6937b5afd5}{timlCNNShareParams}(cnn, 0);
\}

\textcolor{comment}{// testing}
printf(\textcolor{stringliteral}{"2. Start testing\(\backslash\)n"});
\textcolor{keywordflow}{for} (i = 0; i < slTesting.num; i++) \{
   printf(\textcolor{stringliteral}{"Read image %03d.jpg\(\backslash\)n"}, i);
   sprintf(str, slTesting.imageFileNameStr, i);
   \hyperlink{group__util_ga4e3a2beb9a301e6c71f5c1df09d6099e}{timlUtilReadFixedSizeJPEG}(str, image, slTesting.row, slTesting.col, slTesting.
      channel);
   clock\_gettime(CLOCK\_REALTIME, &startTime);
   \hyperlink{group__appCNNScene_gad77aeeb63649011f260d9c475bef945d}{appCNNSceneClassifyOpenMP}(cnnTeam, thread, image, slTesting.row, slTesting.col,
       slTesting.channel, labelMatrix, scale);
   clock\_gettime(CLOCK\_REALTIME, &endTime);
   testingTime = \hyperlink{group__util_gad332b3fc9cca905cb073bb950c27d9a3}{timlUtilDiffTime}(startTime, endTime);

   \textcolor{comment}{// read true label}
   sprintf(str, slTesting.labelFileNameStr, i);
   fp = fopen(str, \textcolor{stringliteral}{"rt"});
   \textcolor{keywordflow}{for} (n = 0; n < slTesting.row; n++) \{
      \textcolor{keywordflow}{for} (m = 0; m < slTesting.col; m++) \{
         fscanf(fp, \textcolor{stringliteral}{"%d"}, trueLabelMatrix + n*slTesting.col + m);
      \}
      fscanf(fp, \textcolor{stringliteral}{"\(\backslash\)n"});
   \}
   fclose(fp);

   \textcolor{comment}{// calculate accuracy}
   labelAccuracy = \hyperlink{group__appCNNScene_ga656def3c972943463bee38f746074b79}{appCNNSceneAccuracy}(labelMatrix, trueLabelMatrix, slTesting.row*
      slTesting.col);
   printf(\textcolor{stringliteral}{"Test image %03d label accuracy = %.2f %%\(\backslash\)n"}, i, 100.0*labelAccuracy);
   printf(\textcolor{stringliteral}{"Test image %03d time           = %.3f s\(\backslash\)n"}, i, testingTime/1000000.0);
\}
\end{DoxyCode}
\hypertarget{index_sectionBenchmarks}{}\section{Benchmarks}\label{index_sectionBenchmarks}
The benchmark source code is located at {\ttfamily $<$install\-\_\-directory$>$/src/benchmark}.

There are 2 examples for C\-N\-N classification benchmark\-:
\begin{DoxyItemize}
\item {\ttfamily \hyperlink{group__benchmarkCNNClass_ga9417402a234aae9eb184bda7d9c293a7}{benchmark\-C\-N\-N\-Class\-Caffe\-Net\-Testing()}} \-: Caffe\-Net benchmark
\item {\ttfamily \hyperlink{group__benchmarkCNNClass_ga83c37557afa0f2361fd24c3f4f01603c}{benchmark\-C\-N\-N\-Class\-V\-G\-G\-Net\-Testing()}} \-: V\-G\-G\-Net benchmark
\end{DoxyItemize}

These two functions will benchmark the memory usage and processing time for each individual layers in the network.