ExplainReduce is a global, model-agnostic, and post-hoc explanation method that extracts a small set of 'proxies' from a large collection of local models generated by local explanation methods (e.g., LIME and SHAP). Each proxy is a representative local model that effectively captures the prediction behaviour of the black-box model within a specific neighbourhood. The set of proxies summarizes the black-box model's decision pattern across different regions of the input space, facilitating global interpretability.
Preprint of the ExplainReduce paper
Seppäläinen, Lauri, Mudong Guo, and Puolamäki, Kai (2025).
ExplainReduce: Summarising Local Explanations via ProxiesArxiv preprint https://arxiv.org/abs/2502.10311.
ExplainReduce requires Python 3, version 3.11 or greater. To install python, follow the instructions from the official website of Python. For package management, we recommend the uv package manager. You can install the dependencies by running
uv sync
after cloning the repository. Alternatively, install the requirements via pip with
pip3 install -r requirements.txt
Next, create a copy of config.template named config.ini at the root of this repository.
Clone the GLocalX repository using
git clone git@github.com:msetzu/glocalx.git
and enter the path to that directory to config.ini:
[Paths]
GLOCALX_PATH = /your/path/to/glocalx
Finally, for downloading OpenML datasets, add your OpenML API key to config.ini:
[Keys]
OPENML_APIKEY = your_openml_api_key
A simple example of the idea behind ExplainReduce. A black-box model (left) can have many local explanations (middle), but ExplainReduce can reduce the size of the local explanation set to get a global explanation consisting of two simple models (right).
