Comparing Convolutional Neural Network in a Rashomon Set using Explanations

Abstract

Deep learning neural networks achieve great performance in image classification tasks. To measure the performance, a validation set is used to estimate the performance on unseen test data. A set of different models with similar performance on the validation set is called Rashomon set. Even though the performance of the validation set are similar the reasoning behind the decision may differ. Unfortunately, deep neural networks are a black box models, where the reasoning behind a decision is not clear. In this thesis we compare these black box models and aim to differentiate models which are right for the right reasons and models which are right for the wrong reasons.

We examine whether different reasons between models may be found by using extremal perturbations masks, which highlight the most important part behind a models predictions. We compare the similarity of masks from different models on the same instance. We find that images with decoy can be found using this method if we compare a decoy model with non-decoy models. However, some images without decoys have similar properties as images with decoys. Another method explored in this thesis is by using influential instance. Influential instances are training instances which are important behind a decision for a validation instance. By comparing these influential instances across models we want to show the difference behind the reasoning behind a decision. Similar to the previous approach, images with decoys can be detected, but images without a decoy can have similar properties as images with decoys. We conclude that in certain scenarios explanations are useful to differentiate models that are right for the right reasons from models that are right for the wrong reasons.