Explainable Neural Networks: Recent Advancements, Part 2
Looking back a decade (2010–2020), a four part series
Where are we?
This blog focusses on developments on explainability of neural networks. We divide our presentation into a four part blog series:
- Part 1 talks about the effectiveness of Visualizing Gradients of the image pixels for explaining the pre-softmax class score of CNNs.
- Part 2 talks about some more advanced/modified gradient based methods like DeConvolution, Guided Back Propagation for explaining CNNs.
- Part 3 talks about some short comings of gradient based approaches and discusses alternate axiomatic approaches like Layer-wise Relevance Propagation, Taylor Decomposition, Deep LiFT.
- Part 4 talks about some recent developments like Integrated Gradients (continuing from part 3) and recent novelties in CNN architecture like Class Activation Maps developed to make the the feature maps more interpretable.
Gradient Based Approaches
Continuing from Part 1, we discuss more sophisticated gradient based techniques developed for explaining neural networks.
Deconvolution Networks (2014)
One of the earliest efforts in explaining the feature maps learnt by a CNN can be traced back to the works of Matthew Zeiler on Deconvolution Networks. In their work on “Visualizing and Understanding Convolutional Networks, (ECCV 2014)”, the authors describe a method to approximately project the activations of an intermediate hidden layer back to the input layer. Such projections can provide an insight into what details the hidden layer has captured from the input image. By projecting successive layers (back to the input layer), the authors show that the CNN learns successively more complicated patterns in the image like edges, simple shapes, more complicated shapes, textures etc. The authors show their results by visualizing the hidden layers of the famous AlexNet and from their insights, are also able to tune the layers to achieve better performance!
Here is a brief description of the projection method. Starting from the desired layer, the activation signal is passed down through the layers (similar to back-propagation), though max pooling layer, ReLU, weight multiplication:
Initialize: Start with the layer that you want to project down and initialize the reconstructed signal equal to the activations of that layer. Back propagate the reconstructed signal down.
- MaxPool: When u encounter a MaxPooling layer, look for indices from where the inputs were pooled and passed up in the forward pass. In the backward pass, pass the reconstructed signal values to these indices, zeroing out the other positions.
- ReLU: When u encounter the ReLU layer, pass the reconstructed signal only if it is positive, else zero it out.
- Weights: When u encounter the CNN layer or any weight multiplication, multiply the transpose of the weights to the reconstructed signal and pass it down.
A couple of points to note:
- This method doesn’t invert the CNN exactly. It only projects the pixels which favor the activation of a hidden layer.
- Though it is called deconvolution, this method doesn’t actually deconvolve the CNN. Deconvolution is a misnomer which has stuck around for historical reasons. It can/may be better called transposed convolution.
- Most of the operations in the deconvolution are close to gradient back propagation. We would discuss more of this in the coming sections.
Guided BackPropagation (2015)
Given that gradients can be used as a saliency map for understanding the decisions of a neural network, how different is the deconvolution method from gradient visualization? In their original paper on saliency maps (from previous section), the authors had alluded to the striking similarities of deconvolution and gradient back-propagation. This idea was further investigated and advanced by Jost Tobias Springenberg, Alexey Dosovitskiy, Thomas Brox, Martin Riedmiller in their work “Striving for Simplicity: The All Convolutional Net, (ICLR 2015)”. It is noteworthy to mention that this paper conducted an ablation study on CNN architectures and made a number of other important contributions other than explainability, which is beyond the scope of this blog.
When we compare the equations of deconvolution and gradient we back propagation, we notice that all the equations are exactly the same except at the ReLU stage. In Deconvolution, the (gradient-like) reconstructed signal is passed down only when it is positive, i.e we only pass down the signals that help boost the activation. While in regular gradient propagation, the gradient is passed down through the ReLU whenever the ReLU passed the inputs up in forward pass. There is a fine distinction between the two.
The authors combine these two conflicting approaches into one approach called Guided Back Propagation. In this approach, the authors, suggest using gradient back propagation as it is except at the ReLU stages. At ReLU stages, we back propagate the gradient only if the gradient is positive. Here are the equations:
Guided BackPropagation differs from the “vanilla” gradient back propagation only at the ReLU stage.
Here are some results presented by the authors on the ImageNet dataset. The authors notice that the visualizations provided by Guided BackPropagation is cleaner than “vanilla” gradient back propagation and deconvolution.
Whats Next?
Gradient based methods though easy to understand, have some important shortcomings. We discuss some of these shortcomings in the next part. We also discuss about some axiomatic approaches to relevance introduced to combat these short comings like Layer-wise Relevance Propagation, DeepLiFT etc. We discuss these cool techniques in the next part.
To read more about such exciting works on explainability of neural networks, you can catch the next part here: Link to Part 3
