AASS seminar: When to trust an explanation?

Who: WASP Distinguished Guest Professor Professor Barbara Hammer, University of Bielefeld, Germany  
What: Seminar: "When to trust an explanation?"
When: Thursday, November 26, at 13:00. 
Where: Hörsal T
Remote attendance: https://oru-se.zoom.us/j/65096623293

Abstract:
Explainable AI (XAI) aims at the integration, extension, or substitution of otherwise black-box machine learning models by components, which are comprehensible for humans. One purpose is to enable humans to understand relevant components of AI methods , such that a critical inspection and the possibility to deal with errors is offered, among others. EU’S AI Act even implements a right for explanations for critical domains.  As a consequence, many XAI methods have been developed for different purposes. Yet, how to evaluate those? Why do different methods lead to different results? In which sense humans can trust XAI?

Within the talk, I will give a short introduction into XAI methods and examples of modern XAI schemes and their suitability for applications in critical infrastructure. More precisely, I will show their suitability to support the detection of faults in water distribution systems such as sensor faults or leakages.

Afterwards, I will have a look at vulnerabilities, such as attacks on XAI methods and limited plausibility of classical methods. Moreover, some limitations are inherent in typical modeling of XAI methods such as SHAP: they typically model single feature attribution, while important effects come from feature interactions. I will discuss extensions which also take feature interactions into account. I will demonstrate their suitability to provide XAI pipelines for frontier models including language models or language-vision embedding.