1 %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\r
2 %% %%\r
3 %% Copyright (C) 2013 Texas Instruments Incorporated - http://www.ti.com %%\r
4 %% %%\r
5 %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%\r
6 \chapter{Testing}\r
7 \label{chap:lcpd-tests}\r
8 \r
9 \paragraph{}\r
10 As was discussed in chapter \ref{chap:definition-of-done}, a backlog item \r
11 must be \textbf{tested} and the test code should be \textbf{submitted} to\r
12 system test before we can call the backlog item done.\r
13 Every backlog item should have a test entry (i.e. \textit{add test} functionality) in VersionOne.\r
14 If the developer needs help from system test developing/integrating the test\r
15 code then he/she \textbf{MUST} assign the test entry to him/herself and a member\r
16 of the system test team.\r
17 \r
18 \section{Test Code}\r
19 \paragraph{}\r
20 LCPD's core-sdk filesystem must be used for kernel testing. LCPD's core-sdk \r
21 image comes with various test-related applications pre-installed:\r
22 \begin{itemize}\r
23 \item ltp-ddt\r
24 \item iperf\r
25 \item bonnie++ \r
26 \item hdparm\r
27 \item iozone3\r
28 \item lmbench\r
29 \item rt-tests\r
30 \item evtest\r
31 \item libdrm-tests\r
32 \item and others...\r
33 \end{itemize}\r
34 So users are encouraged to check if there is already a test case or test \r
35 utility available in the core-sdk filesystem before they start developing a new one.\r
36 It is recommended that new test cases are developed in LTP-DDT \ref{sec:ltp-ddt} project.\r
37 \r
38 \subsection{Submitting test code}\r
39 \label{sec:sub-test-code}\r
40 \paragraph{}\r
41 The preferred way to submit test code to system test is to submit ltp-ddt patches to\r
42 opentest@arago-project.org mailing list. \r
43 \r
44 \paragraph{}\r
45 Once a test have been submitted, the systest team will take care of integrating it\r
46 into Opentest.\r
47 \r
48 \subsection{LTP-DDT}\r
49 \label{sec:ltp-ddt}\r
50 \r
51 \subsection{Getting LTP-DDT}\r
52 \paragraph{}\r
53 There are couple of ways to get LTP-DDT sources:\r
54 \begin{enumerate}\r
55 \item Clone it from http://arago-project.org/git/projects/test-automation/ltp-ddt.git\r
56 \item Use \textit{linux-devtest.py -U} to get it cloned automatically\r
57 \end{enumerate}\r
58 \r
59 \subsection{Creating new LTP-DDT tests}\r
60 \paragraph{}\r
61 The recommended LTP-DDT development flow is to clone ltp-ddt, build and install it\r
62 on a nfs filesystem, so you can quickly check your LTP-DDT changes on a real target.\r
63 You can get a core-sdk filesystem tarball that you can use to setup your nfs at\r
64 \href{http://lcpd.dal.design.ti.com/core-sdk/}{LCPD core sdk}\r
65 \r
66 \paragraph{}\r
67 Please refer to \href{http://arago-project.org/git/projects/?p=test-automation/ltp-ddt.git;a=blob;f=README-DDT#l19}{LTP-DDT README} sections 3 and 4 for detailed information about creating new LTP-DDT tests.\r
68 \r
69 \paragraph{}\r
70 Please refer to \href{http://arago-project.org/git/projects/?p=test-automation/ltp-ddt.git;a=blob;f=README-DDT#l163}{LTP-DDT README} sections 6 and 7 for LTP-DDT build instructions.\r
71 \r
72 \subsection{Running LTP-DDT tests}\r
73 \paragraph{}\r
74 Please refer to \href{http://arago-project.org/git/projects/test-automation/ltp-ddt.git?p=projects/test-automation/ltp-ddt.git;a=blob;f=README-DDT;#l223}{LTP-DDT README} section 8.\r
75 \r
76 \section{Test Automation Frameworks}\r
77 \subsection{Opentest}\r
78 \label{sec:Opentest}\r
79 \paragraph{}\r
80 \href{http://arago-project.org/wiki/index.php/Opentest}{Opentest} test automation\r
81 framework is used to manage test plans, test results, test hardware and test \r
82 execution.\r
83 \r
84 \paragraph{}\r
85 You don't need Opentest to develop LTP-DDT test cases. You don't need Opentest \r
86 to run LTP-DDT tests either. We anticipate that most kernel developers won't \r
87 need to use Opentest on a regular basis, but instead they will run LTP-DDT tests\r
88 directly on their Device Under Test (Board or EVM). \r
89 \r
90 \paragraph{}\r
91 You will only need Opentest when you:\r
92 \begin{itemize}\r
93 \item want to run tests on boards that you don't have.\r
94 \item want to run complex tests that require test equipment.\r
95 \item want to run testplans defined in Testlink\r
96 \item want to store results in Testlink\r
97 \item want to request multiple test runs and don't want to baby-sit them, you just want to \textbf{Click-and-Forget}\r
98 \end{itemize}\r
99 \r
100 \paragraph{}\r
101 System test is responsible for integrating LTP-DDT test cases into Opentest \r
102 test plans. \r
103 System test will also develop/integrate complex test cases.\r
104 \r
105 \subsection{Installing Opentest}\r
106 \paragraph{}\r
107 Check \href{http://arago-project.org/wiki/index.php/OpentestOtherSrvLinuxDevtest}{Opentest Installation}\r
108 \paragraph{}\r
109 \textbf{IMPORTANT:} Typical LCPD users should not install complete Opentest framework\r
110 in their desktop but instead we recommend just installing Options 2 (Staf) and\r
111 8 (Command Line Tools).\r
112 \r