Bidirectional Scan for Vision
Vision Mamba processes image patches bidirectionally to capture both forward and backward context, unlike the causal (left-to-right only) scanning in language models.
For a 1D sequence x=[x1​,x2​,...,xN​]:
- Forward scan: Cumulative sum from left to right
- Backward scan: Cumulative sum from right to left
- Output: Element-wise sum of forward and backward scans
This bidirectional approach ensures each patch attends to information from both directions, crucial for non-causal tasks like image classification.
Task: Implement the bidirectional scan fusion.
Example:
x = [1.0, 2.0, 3.0, 4.0]
[11.0, 12.0, 13.0, 14.0]
Forward: [1, 3, 6, 10]. Backward: [10, 9, 7, 4]. Sum: [11, 12, 13, 14]. Each position aggregates information from all positions.
Constraints:
- Sequence length N: 1≤N≤1000
- Values are floats
Bidirectional Scan for Vision
Background Knowledge
In language models, causal (unidirectional) scanning ensures the model only sees past tokens when predicting the next one. However, images have no inherent directionality - every pixel should be able to attend to every other pixel.
Why Bidirectional?
| Domain | Scanning | Reason |
|---|---|---|
| Language | Causal (→) | Can't see future tokens during generation |
| Vision | Bidirectional (↔) | All patches are available simultaneously |
Vision Mamba's Approach
Vision Mamba adapts the Mamba architecture for vision by:
- Running a forward scan (left-to-right)
- Running a backward scan (right-to-left)
- Fusing both scan outputs (typically addition)
This gives each patch access to global context while maintaining O(N) complexity.
Continue the full explanation
You're reading the free preview. Unlock the complete walkthrough, the code editor, test runner and reference solution with Premium.
Editor locked
The code editor is locked for Pro problems. It is only available for free problems. Please upgrade to gain access to the code editor for all problems.