PIXELBANKv8.2.1
Menu

Gram Matrix for Style

Implement a function to compute the Gram matrix for neural style transfer, which is a crucial component in capturing the style of an image. This task involves understanding how to represent the style of an image using feature maps extracted from a Convolutional Neural Network (CNN).

The Gram matrix is used to capture the style by computing correlations between feature maps, where the feature map FF is a 3D tensor with shape (C,H×W)(C, H \times W), representing the output of a CNN layer with CC channels, height HH, and width WW. The Gram matrix GG is a 2D tensor that represents the correlations between these feature maps.

Here are the steps to compute the Gram matrix:

  1. Extract the feature map FF from a CNN layer.
  2. Compute the correlations between the feature maps. The Gram matrix GG can be computed using the formula Gij=kFikFjkG_{ij} = \sum_k F_{ik} F_{jk}, which represents the dot product of the ithi^{th} and jthj^{th} feature maps.
Gij=kFikFjkG_{ij} = \sum_k F_{ik} F_{jk}

The style loss LstyleL_{style} is then computed as the squared difference between the Gram matrix of the content image and the Gram matrix of the style image: Lstyle=GcontentGstyle2L_{style} = \|G_{content} - G_{style}\|^2.

This technique is widely used in image generation and editing tasks.

Example:

Input:
features = CNN activations shape (64, 32, 32)
Output:
Gram matrix shape (64, 64)
Reasoning:
  1. Reshape features to (C, H*W)
  2. Gram = F @ F.T
  3. Normalize by number of elements

Constraints:

  • features: Feature maps from CNN (C, H, W)
  • Return: Gram matrix (C, C)
Editor

Test Results

0/0
Run code to see test results.