📜

제가 개인적으로 읽은 논문들을 정리해놓는 공간입니다! 대부분은 물리 기반 렌더링(Physically-based Rendering)과 비전 분야의 논문들로 채워질 예정입니다. 언어는 편한 정리를 위해 영어와 한글을 혼용합니다.

Search

Non-local Neural Network was one of the outstanding model for video understanding and classification. The main contributions of the paper is to introduce a non-local operator that can...

efficiently capturing spatial features

Easily inserted to any network architecture

Non-local Mean

The idea of non-local neural network is from non-local means(NLM) algorithm, which is one of the edge-preserving denoise algorithms.

The paper compares the non-local mean operation to conv and fc layer. The advantage that the NLM has is that all pixels (or channels) of the image are weighed for each pixel denoising. This is different from the conv layer which only search a local pixels(e.g. kernel size) which is its common limitation. Also NLM is different to conv and fc layer in perspective of learning. When conv and fc layer have to learn weights (and biases), NLM returns a fixed value of relation of pixels. Also note the fact that NLM can deal with variable-size inputs. By this NLM's advantages, NLM is expected to enhance the receptive field for vision models.

Non-local Block and Self-attention

NLM requires a distance metric to determine the weight for each pixel. There are multiple candidates but ends up the candidates gives similar contribution to the performance improvement. However, the most interesting part when using the Embedding Gaussian as the metric.

f(x_i, x_j) = e^{\theta(x_i) \phi(x_j)}

Above is the formula for the embedding gaussian. Each pixels $x_i$ and $x_j$ are embedded into each embedding. The fact is, this formula can be expressed as a self-attention module. This is possible by thinking the exponential as the softmax function.

Thanks to the residual connection, this module can be inserted inbetween the layers. The paper calls this a Non-local Block and provides various tests to provide the way of the efficient use of the self-attention module.

Non-local Neural Networks

TOP