Feature-flow Interpretation of Deep Convolutional Neural Networks

Xinrui Cui , Dan Wang, Z. Jane Wang
University of British Columbia
PDF
Model visualization

The feature-flow interpretation for the VGG16 network decision. For the bird class prediction of the input image, the feature-flow provides the interpretation from deep to shallow layers. For the Pool5 and Pool4 layers, the sunburst charts present feature-flow units (the outer circle) and their corresponding interpretable body parts (the inner circle). Meantime, we also show the most critical feature-flow unit for each object part on the right side of the corresponding sunburst chart. For the Pool3 layer, the feature-flow units and their contributions to the network prediction are displayed in the doughnut chart. Besides, we show the top-4 feature-flow units in the Pool3 layer.

Abstract

Despite the great success of deep convolutional neural networks (DCNNs) in computer vision tasks, their black-box aspect remains a critical concern. The interpretability of DCNN models has been attracting increasing attention. In this work, we propose a novel model, Feature-fLOW INterpretation (FLOWIN) model, to interpret a DCNN by its feature-flow. The FLOWIN can express deep-layer features as a sparse representation of shallow-layer features. Based on that, it distills the optimal feature-flow for the prediction of a given instance, starting from deep layers to shallow layers. Therefore, the FLOWIN can provide an instance-specific interpretation, which presents its feature-flow units and their interpretable meanings for its network decision. The FLOWIN can also give the quantitative interpretation in which the contribution of each flow unit in different layers is used to interpret the net decision. From the class-level view, we can further understand networks by studying feature-flows within and between classes. The FLOWIN not only provides the visualization of the feature-flow but also studies feature-flow quantitatively by investigating its density and similarity metrics. In our experiments, the FLOWIN is evaluated on different datasets and networks by quantitative and qualitative ways to show its interpretability.

BibTeX

@ARTICLE{9019647,
  author={Cui, Xinrui and Wang, Dan and Wang, Z. Jane},
  journal={IEEE Transactions on Multimedia}, 
  title={Feature-Flow Interpretation of Deep Convolutional Neural Networks}, 
  year={2020},
  volume={22},
  number={7},
  pages={1847-1861},
  doi={10.1109/TMM.2020.2976985}
}