Modify data-validation script and dictionary-validation script to disallow exotic...
authorJan "yenda" Trmal <jtrmal@gmail.com>
Thu, 28 Sep 2017 04:10:42 +0000 (00:10 -0400)
committerGitHub <noreply@github.com>
Thu, 28 Sep 2017 04:10:42 +0000 (00:10 -0400)
commit6cab750e87fa8affd51ef96b244bf6d06e37ac76
tree79c856a462e7470d1e49d0796474f715c61fa070
parentba00b18c290f4ccaba92aba11e45ac7da2d96396
Modify data-validation script and dictionary-validation script to disallow exotic space characters (#1910)

* validate_lang checks for incompatible UTF-8 whitespaces

* adding validate_dict_dir as well

* include utf-8 whitespaces validation for data/<name>/text files

* fix perl syntax error
egs/wsj/s5/utils/validate_data_dir.sh
egs/wsj/s5/utils/validate_dict_dir.pl
egs/wsj/s5/utils/validate_lang.pl
egs/wsj/s5/utils/validate_text.pl [new file with mode: 0755]