Advanced School for Computing and Imaging (ASCI)

ASCI office
Delft University of Technology
Building 28, room 04.E120
Van Mourik Broekmanweg 6
2628 XE – DELFT, The Netherlands

P: +31 15 27 88032

Visiting hours office
Monday, Tuesday, Thursday: 10:00 – 15:00


The ASCI office is located at the Delft University of Technology campus.  It is easily accessible by bicycle, public transport and car. The numbers of buildings can help you find your way around the campus. Make sure you remember the name and building number of your destination.

Contact us at +31 15 278 8032 or send us an email at

Interactive Visualization for Interpretable Machine Learning

Interactive Visualization for Interpretable Machine Learning

Author : Dennis Collaris
Promotor(s) : van Wijk, Jack J., Promotor  Pechenizkiy, Mykola, Promotor
University : Technische Universiteit Eindhoven
Year of publication : 2022
Link to repository Link to thesis


Machine learning has firmly established itself as a valuable and ubiquitous technique in commercial applications. It enables businesses to make sense of their data and make predictions about future events. Besides increasing accuracy, currently there is a strong demand for understanding how specific models operate and how certain decisions are made. Understanding models is particularly important in high-impact domains such as credit, employment, and housing, where the decisions made using machine learning impact the lives of real people. The field of eXplainable Artificial Intelligence (XAI) aims to help experts understand complex machine learning models. In recent years, various techniques have been proposed to open up the black box of machine learning. However, because interpretability is an inherently subjective concept it remains challenging to define what a good explanation is.

We argue we should actively involve data scientists in the process of generating explanations, and leverage their expertise in the domain and machine learning. Interactive visualization provides an excellent opportunity to both involve and empower experts. In this dissertation, we explore interactive visualization for machine learning interpretation from different perspectives, ranging from local explanation of single predictions to global explanation of the entire model.

We first introduce ExplainExplore: an interactive explanation system to locally explore explanations of individual predictions. Next, we leverage the domain knowledge of the data scientist to determine whether the explanation makes sense and is appropriate. In a use case with data scientists from the debtor management department at Achmea, we show the participants could effectively use explanations to diagnose a model and find problems, identify areas where the model can be improved, and support their everyday decision-making process.

Next, we propose the Contribution-Value plot as a new building block for interpretability visualization. It shows how much differents features in the data contribution towards a prediction, but also which feature values are most relevant. In an online survey with 22 machine learning professionals and visualization experts, we show our visualization increases correctness and confidence and reduces the time needed to obtain an insight compared to previous techniques.

Finally, we introduce StrategyAtlas: an interactive system which provides a global understanding of complex machine learning models by showing if a model treats different customers differently (in other words, the model has multiple prediction strategies). We show that our different visualizations enable data scientists to check for and ascertain the validity of these strategies. Next, we applied the system in a real-world project for automatic insurance acceptance at Achmea, which showed that professional data scientists were able to understand a complex model and improve their production model based on these insights.

On the website more information can be found about our visualization contributions, including showcase videos, conference presentations and online interactive demos.