Trinity: Trust, Resilience and Interpretability of Machine Learning Models

Abstract

Despite the remarkable strides over the last decade in the performance of machine learning techniques, their applications are typically limited to nonadversarial benign environments. The use of deep learning in applications such as biometric recognition, and intrusion detection, require them to operate in adversarial environments. But the overwhelming empirical studies and theoretical results have shown that these methods are extremely fragile and susceptible to adversarial attacks. The rationale for why these methods make the decisions they do are also notoriously difficult to interpret; understanding such rationale may be crucial for the aforementioned applications. In this chapter, we discuss the connections between these related challenges, and describe a novel integrated approach, Trinity (Trust, Resilience and INterpretabilITY ), for analyzing these models.