Toward Reliable NLP Systems via Software Testing

Speaker:
Dr. Pinjia HE
Postdoctoral researcher, Computer Science Department
ETH Zurich

Abstract:
NLP systems such as machine translation have been increasingly utilized in our daily lives. Thus, their reliability becomes critical; mistranslations by Google Translate, for example, can lead to misunderstanding, financial loss, threats to personal safety and health, etc. On the other hand, due to their complexity, such systems are difficult to get right. Because of their nature (i.e., based on large, complex neural networks), traditional reliability techniques are challenging to be applied. In this talk, I will present my recent work that has spearheaded the testing of machine translation systems, toward building reliable NLP systems. In particular, I will describe three complementary approaches which collectively found 1,000+ diverse translation errors in the widely-used Google Translate and Bing Microsoft Translator. I will also describe my work on LogPAI, an end-to-end log management framework powered by AI algorithms for traditional software reliability, and conclude the talk with my vision for making both traditional and intelligent software such as NLP systems more reliable.

Biography:
Pinjia HE has been a postdoctoral researcher in the Computer Science Department at ETH Zurich after receiving his PhD degree from The Chinese University of Hong Kong (CUHK) in 2018. He has research expertise in software engineering and artificial intelligence, and is particularly passionate about making both traditional and intelligent software reliable. His research on automated log analysis and machine translation testing appeared in top computer science venues, such as ICSE, ESEC/FSE, ASE, and TDSC. The LogPAI project led by him has been starred 2,000+ times on GitHub and downloaded 30,000+ times by 380+ organizations, and won a Most Influential Paper (MIP) award at ISSRE. He also won a 2016 Excellent Teaching Assistantship at CUHK. He has served on program committees of MET’21, DSML’21, ECOOP’20 Artifact, and ASE’19 Demo, and reviewed for top journals and conferences (e.g., TSE, TOSEM, ICSE, KDD, and IJCAI). According to Google Scholar, he has an h-index of 14 and 1,200+ citations.

Join Zoom Meeting:
https://cuhk.zoom.us/j/98498351623?pwd=UHFFUU1QbExYTXAxTWxCMk9BbW9mUT09

Enquiries: Miss Caroline TAI at Tel. 3943 8440
For more information, please refer to http://www.cse.cuhk.edu.hk/seminar-archive/