Added graph viewer for TIDL API timestamp data 1. Created a python matplotlib based utility for viewing timestamp data generated from the TIDL API (viewer/execution_graph.py) 2. Minor updates to API internals to add ExecutionObject type, index to timestamp output (MCT-1073)
Simplify API for multiple contexts 1. Simplify context API in ExecutionObject. Replace context_id variants to multiple existing APIs with these two APIs: bool AcquireAndRunContext(uint32_t& context_idx, int frame_idx, const IODeviceArgInfo& in, const IODeviceArgInfo& out); bool WaitAndReleaseContext(uint32_t context_idx); 2. The timing methods for host execution in EOPs and EOs: * GetProcessTimeInMilliSeconds() * GetHostProcessTimeInMilliSeconds() are no longer accurate with multiple contexts and pipelining. Replace these methods and replace with a generic timestamp based approach. There is a single API call to enable time stamps in an application: //! Enable time stamp generation for TIDL API events bool EnableTimeStamps(const std::string& file = "timestamp.log", size_t num_frames=32); If this method is called before TIDL API frame processing, the API will generate timestamps for events corresponding to each frame (e.g. EOP::ProcessFrameStartAsync, EOP::ProcessFrameWait, etc.). These timestamps are then written to file when the user's application completes. A separate script is used for post-processing the time stamps and generating data for the user. (MCT-1073, MCT-1074)
Enqueue multiple frames at device side - Previous implementation won't send/enqueue next frame to device until the host has received completion message for current frame. The improvement is to create multiple sets/contexts of internal TIDL input/output buffers at device side, and to send/enqueue next frame using a different set/context of internal TIDL input/output buffers to device while device is still processing the current frame. When device finishes current frame, it can immediately read its messageQ and start processing the next frame, without waiting for the completion message reaching the host and the hosting sending the next frame. - In pipelined processing of multiple frames, this optimization can effectively hide the round-trip communication between host and device. - Removed deprecated enableInternalInput feature - MCT-1059
Report memory usage when device allocation fails TIDL API creates 2 device side heaps: 1. Parameter heap 2. Network heap The sizes of these heaps are specified in the Configuration object, via PARAM_HEAP_SIZE and NETWORK_HEAP_SIZE. Existing behavior: If the heaps are not large enough, allocation on the device triggers an assertion failure with no indication of how large the heaps need to be for successfull allocation. To improve the usability of the API, provide feedback to the user on the heap sizes required to satisfy device side allocations when any allocation fails. Also added `-Wall -Werror` when building examples and fixed failures. (MCT-1035)
ExecutionObjectPipeline for executing layersGroups - Add top level ExecutionObjectPipeline class to execute multiple layersGroups. - An ExecutionObjectPipeline is constructed from multiple ExecutionObjects, each ExecutionObject executes one layersGroup in the network, together they execute consecutive layersGroups. - Same look and feel as ExecutionObject, e.g. ProcessFrameStartAsync, ProcessFrameWait, GetInputBufferPointer, GetOutputBufferPointer - MCT-1017, MCT-1029
Modified IODeviceArgInfo to enable pipelining EOs (MCT-1030)
Remove implementation details from ArgInfo Implementation details such as argument kind and PipeInfo should not be a part of the user facing ArgInfo class. Also, PipeInfo is relevant only for input/output arguments. Moved implementation details out of ArgInfo and created 2 new classes: DeviceArgInfo and IODeviceArgInfo. DeviceArgInfo inherits from ArgInfo and adds an argument kind (buffer, local or scalar). IODeviceArgInfo consists of DeviceArgInfo and PipeInfo. (MCT-1030)
Add support for accessing layer output data * Added API methods to enable tracing and dump layer output buffers to file 1. ExecutionObject::WriteLayerOutputsToFile 2. ExecutionObject::GetOutputsFromAllLayers 3. ExecutionObject::GetOutputFromLayer See examples/layer_output/main.cpp, ProcessTrace() for examples of using these tracing APIs. * Added a 'enableTrace' field to the configuration file. Set to true to enable tracing. * Update configuration file parser to allow comments, blank lines Note: Tracing requires ti-opencl, SHA-ID 188cc38627315d3041b80ee86e6ca33087348062 or newer. (MCT-1023)
license headers: updated year, added where missing (MCT-961)
Rename to TIDL * Replace references to TINN in the sources and directory structure with TIDL or TIDL API * Moved the TIDL network viewer (tidl_viewer) from utils to viewer (MCT-983)