📘

Gram Matrix for Style

HardStyle Transfer 📚 CV: Computational Photography

Implement a function to compute the Gram matrix for neural style transfer, which is a crucial component in capturing the style of an image. This task involves understanding how to represent the style of an image using feature maps extracted from a Convolutional Neural Network (CNN).

The Gram matrix is used to capture the style by computing correlations between feature maps, where the feature map $F$ is a 3D tensor with shape $(C, H \times W)$ , representing the output of a CNN layer with $C$ channels, height $H$ , and width $W$ . The Gram matrix $G$ is a 2D tensor that represents the correlations between these feature maps.

Here are the steps to compute the Gram matrix:

Extract the feature map $F$ from a CNN layer.
Compute the correlations between the feature maps. The Gram matrix $G$ can be computed using the formula $G_{ij} = \sum_k F_{ik} F_{jk}$ , which represents the dot product of the $i^{th}$ and $j^{th}$ feature maps.

G_{ij} = \sum_k F_{ik} F_{jk}

The style loss $L_{style}$ is then computed as the squared difference between the Gram matrix of the content image and the Gram matrix of the style image: $L_{style} = \|G_{content} - G_{style}\|^2$ .

This technique is widely used in image generation and editing tasks.

Example:

Input:

features = CNN activations shape (64, 32, 32)

Output:

Gram matrix shape (64, 64)

Reasoning:

Reshape features to (C, H*W)
Gram = F @ F.T
Normalize by number of elements

Constraints:

features: Feature maps from CNN (C, H, W)
Return: Gram matrix (C, C)

More from CV: Computational Photography

Trimap-Based Alpha EstimationHard Drago Adaptive Log MappingHard Alpha Matte CompositingMedium Reinhard Global Tone MappingMedium Exposure FusionHard

Background Knowledge

Neural style transfer (NST) separates content and style using a pretrained CNN (e.g., VGG).

Content is captured by feature activations at higher layers (spatial structure, objects).
Style is captured by statistics of features (textures, colors, brush strokes), not their exact spatial locations.

Given a feature map $F \in \mathbb{R}^{C \times (H \times W)}$ , each row is a channel and each column is a spatial location. The Gram matrix $G \in \mathbb{R}^{C \times C}$ is defined as

G_{ij} = \sum_k F_{ik} F_{jk}

It computes inner products between feature channels, measuring how often channels activate together across the image. This is invariant to spatial arrangement (order of columns $k$ ), so it captures texture / style rather than layout. Style loss then compares Gram matrices of style vs generated image:

L_{style} = \|G^{(gen)} - G^{(style)}\|_F^2

Algorithm / Approach

General pattern to compute a Gram matrix for style:

Take a CNN feature map of shape $(C, H, W)$ .
Flatten spatial dimensions to get shape $(C, H \times W)$ .
Compute Gram matrix as a channel–channel correlation: $G = F \cdot F^\top$ , giving shape $(C, C)$ .
Optionally normalize (e.g., divide by $H \times W$ or $C \cdot H \cdot W$ ) so style loss scales reasonably across layers.

In NST, this Gram computation is applied to multiple style layers, and their Gram matrices are compared to the style image’s Gram matrices to form the total style loss.

Step-by-Step Strategy

Assume input feature map tensor F has shape (C, H, W) (or (N, C, H, W) with batch):

Handle batch (if present)

If F is (N, C, H, W), usually you:
Either process a single image (N=1) and drop batch dimension.
Or compute Gram per sample independently.

👑

Continue the full explanation

You're reading the free preview. Unlock the complete walkthrough, the code editor, test runner and reference solution with Premium.

🔒

Editor locked

The code editor is locked for Pro problems. It is only available for free problems. Please upgrade to gain access to the code editor for all problems.

solution.py

Python 3.13

Test Results

0/0

Run code to see test results.