Activation maximisation

Overview

Activation maximisation is a locally explainable method that focuses on input patterns which maximise a given hidden unit activation. Activation maximisation helps understand the layer-wise feature importance to an input instance.

How convolutional neural networks see the world

Let $\theta$ be the parameters of the model, and $z_{i,j}(\theta ,x)$ be the activation of a particular unit $i$ from layer $j$. The activation map can be defined as the problem:

$$x^{\ast }=\underset{\Vert x\Vert =\rho }{\mathrm{argmax}}{z}_{i,j}(\theta ,x)$$

where $\theta$ is fixed.

Algorithm

The above process consists of four steps:

1. An image $x=x_0$ with random pixel values is set to be the input to the activation computation.
2. The gradients with respect to the noise image, $\frac{\partial z_{i,j}}{\partial x}$, are computed through backpropagation.
3. Each pixel of the noise image is changed iteratively to maximise the activation of the neuron, guided by the direction of the gradient$\left(\frac{\partial z_{i,j}}{\partial x}\right)$:

$$x\leftarrow x+\eta \frac{\partial z_{i,j}}{\partial x}$$

4. This process terminates at a specific pattern image $x^{\ast }$, which can be seen as the preferred input for this neuron.

References

• Erhan, D., Courville, A., & Bengio, Y. (2010). Understanding representations learned in deep architectures.
• Qin, Z., Yu, F., Liu, C., & Chen, X. (2018). How convolutional neural network see the world-A survey of convolutional neural network visualization methods. arXiv preprint arXiv:1804.11191.
• https://blog.keras.io/how-convolutional-neural-networks-see-the-world.html