| author | Xusong Wang <xusongw@google.com> | |
| Tue, 28 Aug 2018 23:50:01 +0000 (16:50 -0700) | ||
| committer | Miao Wang <miaowang@google.com> | |
| Thu, 6 Sep 2018 21:41:06 +0000 (21:41 +0000) | ||
| commit | 39c865b3aa8587d426e726380e5e6f70ac4b6f6d | |
| tree | fa15d895c67143c0e578351d87bdf0808506b279 | tree | snapshot (tar.xz tar.gz zip) |
| parent | 391eb588cf3132b5de8d290154d0ffbc4135a981 | commit | diff |
Fix logics for floating-point comparision in VTS test.
Set the acceptable error range based on both absolute tolerance and
relative tolerance.
Currently, absolute tolerance is set to 1e-5 for FP32 and 5 epsilon
(~5e-3) for FP16 relaxed computation. The relative tolerance is set to
5ULP of the corresponding precision. Add a TODO mark for potential
future adjustment on error limit based on testing.
Bug: 111768023
Test: none
Change-Id: Idedcec3e09fd7de9696811b93c81d0f180e896ef
Set the acceptable error range based on both absolute tolerance and
relative tolerance.
Currently, absolute tolerance is set to 1e-5 for FP32 and 5 epsilon
(~5e-3) for FP16 relaxed computation. The relative tolerance is set to
5ULP of the corresponding precision. Add a TODO mark for potential
future adjustment on error limit based on testing.
Bug: 111768023
Test: none
Change-Id: Idedcec3e09fd7de9696811b93c81d0f180e896ef
| neuralnetworks/1.0/vts/functional/GeneratedTestHarness.cpp | diff | blob | history |