diff options
Diffstat (limited to 'README_Streamline.txt')
| -rw-r--r-- | README_Streamline.txt | 250 |
1 files changed, 142 insertions, 108 deletions
diff --git a/README_Streamline.txt b/README_Streamline.txt index 0445e3f..5472c07 100644 --- a/README_Streamline.txt +++ b/README_Streamline.txt | |||
| @@ -1,108 +1,142 @@ | |||
| 1 | 1 | ||
| 2 | *** Purpose *** | 2 | *** Purpose *** |
| 3 | 3 | ||
| 4 | Instructions on setting up ARM Streamline on the target. | 4 | Instructions on setting up ARM Streamline on the target. |
| 5 | The gator driver and gator daemon are required to run on the ARM linux target in order for ARM Streamline to operate. | 5 | The gator driver and gator daemon are required to run on the ARM linux target in order for ARM Streamline to operate. |
| 6 | The driver should be built as a module and the daemon must run with root permissions on the target. | 6 | The driver should be built as a module and the daemon must run with root permissions on the target. |
| 7 | 7 | ||
| 8 | *** Introduction *** | 8 | *** Introduction *** |
| 9 | 9 | ||
| 10 | A linux development environment with cross compiling tools is most likely required, depending on what is already created and provided. | 10 | A linux development environment with cross compiling tools is most likely required, depending on what is already created and provided. |
| 11 | -For users, the ideal environment is to be given a BSP with gatord and gator.ko already running on a properly configured kernel. In such a scenario, a development environment is not needed, root permission may or may not be needed (gatord must be executed with root permissions but can be automatically started, see below), and the user can run Streamline and profile the system without any setup. | 11 | -For users, the ideal environment is to be given a BSP with gatord and gator.ko already running on a properly configured kernel. In such a scenario, a development environment is not needed, root permission may or may not be needed (gatord must be executed with root permissions but can be automatically started, see below), and the user can run Streamline and profile the system without any setup. |
| 12 | -The ideal development environment has the kernel source code available to be rebuilt and executed on the target. This environment allows the greatest flexibility in configuring the kernel and building the gator driver module. | 12 | -The ideal development environment has the kernel source code available to be rebuilt, usually by cross-compiling on a host machine. This environment allows the greatest flexibility in configuring the kernel and building the gator driver module. |
| 13 | -However, it is possible that a user/developer has a kernel but does not have the source code. In this scenario it may or may not be possible to obtain a valid profile. | 13 | -However, it is possible that a user/developer has a kernel but does not have the source code. In this scenario it may or may not be possible to obtain a valid profile. |
| 14 | -First, check if the kernel has the proper configuration options (see below). Profiling cannot occur using a kernel that is not configured properly, a new kernel must be created. | 14 | -First, check if the kernel has the proper configuration options (see below). Profiling cannot occur using a kernel that is not configured properly, a new kernel must be created. See if /proc/config.gz exists on the target. |
| 15 | -Second, given a properly configured kernel, check if the filesystem contains the kernel source/headers, which can be used to re-create the gator driver. | 15 | -Second, given a properly configured kernel, check if the filesystem contains the kernel source/headers, which can be used to re-create the gator driver. These files may be located in different areas, but common locations are /lib/modules/ and /usr/src. |
| 16 | -If the kernel is not properly configured or sources/headers are not available, the developer is on their own and kernel creation is beyond the scope of this document. Note: It is possible for a module to work when compiled against a similar kernel source code, though this is not guaranteed to work due to differences in kernel structures, exported symbols and incompatible configuration parameters. | 16 | -If the kernel is not properly configured or sources/headers are not available, the developer is on their own and kernel creation is beyond the scope of this document. Note: It is possible for a module to work when compiled against a similar kernel source code, though this is not guaranteed to work due to differences in kernel structures, exported symbols and incompatible configuration parameters. |
| 17 | 17 | ||
| 18 | *** Preparing and building the kernel *** | 18 | *** Kernel configuration *** |
| 19 | 19 | ||
| 20 | cd into the root source dir of the linux kernel | 20 | menuconfig options (depending on the kernel version, the location of these configuration settings within menuconfig may differ) |
| 21 | if your target has never been configured, choose the appropriate configuration for your target | 21 | - General Setup |
| 22 | make ARCH=arm CROSS_COMPILE=${CROSS_TOOLS}/bin/arm-none-linux-gnueabi- <platform_defconfig> | 22 | - Kernel Performance Events And Counters |
| 23 | make ARCH=arm CROSS_COMPILE=${CROSS_TOOLS}/bin/arm-none-linux-gnueabi- menuconfig | 23 | - [*] Kernel performance events and counters (enables CONFIG_PERF_EVENTS) |
| 24 | 24 | - [*] Profiling Support (enables CONFIG_PROFILING) | |
| 25 | Required Kernel Changes (depending on the kernel version, the location of these configuration settings within menuconfig may be different) | 25 | - Kernel Features |
| 26 | - General Setup | 26 | - [*] High Resolution Timer Support (enables CONFIG_HIGH_RES_TIMERS) |
| 27 | - [*] Profiling Support | 27 | - [*] Use local timer interrupts (only required for SMP, enables CONFIG_LOCAL_TIMERS) |
| 28 | - Kernel hacking | 28 | - [*] Enable hardware performance counter support for perf events (enables CONFIG_HW_PERF_EVENTS) |
| 29 | - [*] Tracers | 29 | - CPU Power Management |
| 30 | - [*] Trace process context switches and events | 30 | - CPU Frequency scaling |
| 31 | - Kernel Features | 31 | - [*] CPU Frequency scaling (enables CONFIG_CPU_FREQ) |
| 32 | - [*] High Resolution Timer Support | 32 | - Kernel hacking |
| 33 | - [*] Use local timer interrupts (only required for SMP) | 33 | - [*] Mutex debugging: basic checks (optional, enables CONFIG_DEBUG_MUTEXES) |
| 34 | 34 | - [*] Compile the kernel with debug info (optional, enables CONFIG_DEBUG_INFO) | |
| 35 | The "context switches and events" option will not be available if other trace configurations are enabled. Other trace configurations being enabled is sufficient to turn on context switches and events. | 35 | - [*] Tracers |
| 36 | 36 | - [*] Trace process context switches and events (#) | |
| 37 | Optional Kernel Changes (depending on the kernel version, the location of these configuration settings within menuconfig may be different) | 37 | |
| 38 | Note: Configurations may not be supported on all targets | 38 | (#) The "Trace process context switches and events" is not the only option that enables tracing (CONFIG_GENERIC_TRACER or CONFIG_TRACING) and may not be visible in menuconfig as an option if other trace configurations are enabled. Other trace configurations being enabled is sufficient to turn on tracing. |
| 39 | - System Type | 39 | |
| 40 | - [*] <SoC name> debugging peripherals (enable core performance counters on supported SoCs) /* kernels before 2.6.35 */ | 40 | The configuration options: |
| 41 | 41 | CONFIG_GENERIC_TRACER or CONFIG_TRACING | |
| 42 | make -j5 ARCH=arm CROSS_COMPILE=${CROSS_TOOLS}/bin/arm-none-linux-gnueabi- uImage | 42 | CONFIG_PROFILING |
| 43 | 43 | CONFIG_HIGH_RES_TIMERS | |
| 44 | *** Checking the gator requirements *** | 44 | CONFIG_LOCAL_TIMERS (for SMP systems) |
| 45 | 45 | CONFIG_PERF_EVENTS and CONFIG_HW_PERF_EVENTS (kernel versions 3.0 and greater) | |
| 46 | (optional) Use the hrtimer_module utility to validate the kernel High Resolution Timer requirement. | 46 | CONFIG_DEBUG_MUTEXES (optional, provides 'mutex' as a reason code when a thread stops running) |
| 47 | 47 | CONFIG_DEBUG_INFO (optional, used for analyzing the kernel) | |
| 48 | *** Building the gator module *** | 48 | CONFIG_CPU_FREQ (optional, provides frequency setting of the CPU) |
| 49 | 49 | ||
| 50 | To create the gator.ko module, | 50 | These may be verified on a running system using /proc/config.gz (if this file exists) by running 'zcat /proc/config.gz | grep <option>'. For example, confirming that CONFIG_PROFILING is enabled |
| 51 | cd /path/to/gator/driver-src | 51 | > zcat /proc/config.gz | grep CONFIG_PROFILING |
| 52 | tar xzf gator-driver.tar.gz | 52 | CONFIG_PROFILING=y |
| 53 | cd gator-driver | 53 | |
| 54 | make -C <kernel_build_dir> M=`pwd` ARCH=arm CROSS_COMPILE=<...> modules | 54 | *** Checking the gator requirements *** |
| 55 | for example | 55 | |
| 56 | make -C /home/username/kernel_2.6.32/ M=`pwd` ARCH=arm CROSS_COMPILE=/home/username/CodeSourcery/Sourcery_G++_Lite/bin/arm-none-linux-gnueabi- modules | 56 | (optional) Use the hrtimer_module utility to validate the kernel High Resolution Timer requirement. |
| 57 | If successful, a gator.ko module should be generated | 57 | |
| 58 | 58 | *** Building the gator module *** | |
| 59 | *** Building the gator daemon *** | 59 | |
| 60 | 60 | To create the gator.ko module, | |
| 61 | cd /path/to/gator/daemon-src | 61 | cd /path/to/gator/driver-src |
| 62 | tar -xzf gator-daemon.tar.gz | 62 | tar xzf gator-driver.tar.gz |
| 63 | For Linux, | 63 | cd gator-driver |
| 64 | build with 'make' | 64 | make -C <kernel_build_dir> M=`pwd` ARCH=arm CROSS_COMPILE=<...> modules |
| 65 | For Android, | 65 | for example when using the linaro-toolchain-binaries |
| 66 | mv gator-daemon jni | 66 | make -C /home/username/kernel_2.6.32/ M=`pwd` ARCH=arm CROSS_COMPILE=/home/username/gcc-linaro-arm-linux-gnueabihf-4.7-2013.01-20130125_linux/bin/arm-linux-gnueabihf- modules |
| 67 | install the android ndk, see developer.android.com | 67 | If successful, a gator.ko module should be generated |
| 68 | /path/to/ndk/ndk-build | 68 | |
| 69 | gatord should now be created and located in libs/armeabi | 69 | *** Building the gator daemon *** |
| 70 | 70 | ||
| 71 | *** Running gator *** | 71 | cd /path/to/gator/daemon-src |
| 72 | 72 | tar -xzf gator-daemon.tar.gz (may need to issue with 'sudo') | |
| 73 | Load the kernel onto the target and copy gatord and gator.ko into the target's filesystem. | 73 | For Linux targets, |
| 74 | Ensure gatord has execute permissions | 74 | cd gator-daemon |
| 75 | chmod +x gatord | 75 | make CROSS_COMPILE=<...> # For ARMv7 targets |
| 76 | gator.ko must be located in the same directory as gatord on the target. | 76 | make -f Makefile_aarch64 CROSS_COMPILE=<...> # For ARMv8 targets |
| 77 | With root privileges, run the daemon | 77 | gatord should now be created |
| 78 | sudo ./gatord & | 78 | For Android targets (install the android ndk, see developer.android.com) |
| 79 | 79 | mv gator-daemon jni | |
| 80 | *** Compiling an application or shared library *** | 80 | ndk-build |
| 81 | 81 | or execute /path/to/ndk/ndk-build if the ndk is not on your path | |
| 82 | Recommended compiler settings: | 82 | gatord should now be created and located in libs/armeabi |
| 83 | "-g": Debug symbols needed for best analysis results. | 83 | |
| 84 | "-fno-inline": Speed improvement when processing the image files and most accurate analysis results. | 84 | *** Running gator *** |
| 85 | "-fno-omit-frame-pointer": ARM EABI frame pointers (Code Sourcery cross compiler) allow the call stack to be recorded with each sample taken when in ARM state (i.e. not -mthumb). | 85 | |
| 86 | 86 | Load the kernel onto the target and copy gatord and gator.ko into the target's filesystem. | |
| 87 | *** Profiling the kernel (optional) *** | 87 | Ensure gatord has execute permissions |
| 88 | 88 | chmod +x gatord | |
| 89 | make ARCH=arm CROSS_COMPILE=$(CROSS_TOOLS}/bin/arm-none-linux-gnueabi- menuconfig | 89 | gator.ko must be located in the same directory as gatord on the target or the location specified with the -m option or already insmod'ed. |
| 90 | - Kernel Hacking | 90 | With root privileges, run the daemon |
| 91 | - [*] Compile the kernel with debug info | 91 | sudo ./gatord & |
| 92 | 92 | Note: gatord requires libstdc++.so.6 which is usually supplied by the Linux distribution on the target. A copy of libstdc++.so.6 is available in the DS-5 Linux example distribution. | |
| 93 | make -j5 ARCH=arm CROSS_COMPILE=${CROSS_TOOLS}/bin/arm-none-linux-gnueabi- uImage | 93 | |
| 94 | Use vmlinux as the image for debug symbols in Streamline. | 94 | *** Customizing the l2c-310 Counter *** |
| 95 | Drivers may be profiled using this method by statically linking the driver into the kernel image or adding the module as an image. | 95 | |
| 96 | To perform kernel stack unwinding and module unwinding, edit the Makefile to enable GATOR_KERNEL_STACK_UNWINDING and rebuild gator.ko. | 96 | The l2c-310 counter in gator_events_l2c-310.c contains hard coded offsets where the L2 cache counter registers are located. This offset can also be configured via a module parameter specified when gator.ko is loaded, ex: |
| 97 | 97 | insmod gator.ko l2c310_addr=<offset> | |
| 98 | *** Automatically start gator on boot (optional) *** | 98 | Further, the l2c-310 counter can be disabled by providing an offset of zero, ex: |
| 99 | 99 | insmod gator.ko l2c310_addr=0 | |
| 100 | cd /etc/init.d | 100 | |
| 101 | vi rungator.sh | 101 | *** Compiling an application or shared library *** |
| 102 | #!/bin/bash | 102 | |
| 103 | /path/to/gatord & | 103 | Recommended compiler settings: |
| 104 | update-rc.d rungator.sh defaults | 104 | "-g": Debug symbols needed for best analysis results. |
| 105 | 105 | "-fno-inline": Speed improvement when processing the image files and most accurate analysis results. | |
| 106 | *** GPL License *** | 106 | "-fno-omit-frame-pointer": ARM EABI frame pointers (Code Sourcery cross compiler) allow recording of the call stack with each sample taken when in ARM state (i.e. not -mthumb). |
| 107 | 107 | "-marm": This option is required if your compiler is configured with --with-mode=thumb, otherwise call stack unwinding will not work. | |
| 108 | For license information, please see the file LICENSE after unzipping driver-src/gator-driver.tar.gz. | 108 | |
| 109 | *** Hardfloat EABI *** | ||
| 110 | Binary applications built for the soft or softfp ABI are not compatible on a hardfloat system. All soft/softfp applications need to be rebuilt for hardfloat. To see if your ARM compiler supports hardfloat, run "gcc -v" and look for --with-float=hard. | ||
| 111 | To compile for non-hardfloat targets it is necessary to add options '-marm -march=armv4t -mfloat-abi=soft'. It may also be necessary to provide a softfloat filesystem by adding the option --sysroot, ex: '--sysroot=../DS-5Examples/distribution/filesystem/armv5t_mtx'. The gatord makefile will do this when run as 'make SOFTFLOAT=1 SYSROOT=/path/to/sysroot' | ||
| 112 | The armv5t_mtx filesystem is provided as part of the "DS-5 Linux Example Distribution" package which can be downloaded from the DS-5 Downloads page. | ||
| 113 | Attempting to run an incompatible binary often results in the confusing error message "No such file or directory" when clearly the file exists. | ||
| 114 | |||
| 115 | *** Bugs *** | ||
| 116 | |||
| 117 | There is a bug in some Linux kernels where perf misidentifies the CPU type. To see if you are affected by this, run ls /sys/bus/event_source/devices/ and verify the listed processor type matches what is expected. For example, an A9 should show the following. | ||
| 118 | |||
| 119 | # ls /sys/bus/event_source/devices/ | ||
| 120 | ARMv7_Cortex_A9 breakpoint software tracepoint | ||
| 121 | |||
| 122 | To workaround the issue try upgrading to a later kernel or comment out the gator_events_perf_pmu_cpu_init(gator_cpu, type); cal in gator_events_perf_pmu.c | ||
| 123 | |||
| 124 | *** Profiling the kernel (optional) *** | ||
| 125 | |||
| 126 | CONFIG_DEBUG_INFO must be enabled, see "Kernel configuration" section above. | ||
| 127 | Use vmlinux as the image for debug symbols in Streamline. | ||
| 128 | Drivers may be profiled using this method by statically linking the driver into the kernel image or adding the driver as an image to Streamline. | ||
| 129 | To perform kernel stack unwinding and module unwinding, edit the Makefile to enable GATOR_KERNEL_STACK_UNWINDING and rebuild gator.ko. | ||
| 130 | |||
| 131 | *** Automatically start gator on boot (optional) *** | ||
| 132 | |||
| 133 | cd /etc/init.d | ||
| 134 | vi rungator.sh | ||
| 135 | #!/bin/bash | ||
| 136 | /path/to/gatord & | ||
| 137 | update-rc.d rungator.sh defaults | ||
| 138 | |||
| 139 | *** GPL License *** | ||
| 140 | |||
| 141 | For license information, please see the file LICENSE after unzipping driver-src/gator-driver.tar.gz. | ||
| 142 | |||