Word2Vec Skip-Gram Pairs
Generate Skip-Gram training pairs for Word2Vec.
Given a list of words (a sentence) and a window size, generate all (center, context) pairs where the context word is within the window of the center word.
For each center word at index , the context words are at indices (where valid).
Return a list of (center_word, context_word) tuples, in order.
Example
Input
words = ["the", "quick", "brown", "fox"] window = 1
Output
[('the', 'quick'), ('quick', 'the'), ('quick', 'brown'), ('brown', 'quick'), ('brown', 'fox'), ('fox', 'brown')]Reasoning
- We start by iterating over each word in the input list
wordswith its index . - For each word at index , we generate context words within the window size , which means we consider the words at indices (where valid).
- We create pairs of (center_word, context_word) tuples for each valid context word, resulting in the following pairs:
- For "the" at index 0, the context word is "quick" at index 1, giving ('the', 'quick').
- For "quick" at index 1, the context words are "the" at index 0 and "brown" at index 2, giving ('quick', 'the') and ('quick', 'brown').
- For "brown" at index 2, the context words are "quick" at index 1 and "fox" at index 3, giving ('brown', 'quick') and ('brown', 'fox').
- For "fox" at index 3, the context word is "brown" at index 2, giving ('fox', 'brown').
- The final output is a list of these generated pairs in order: [('the', 'quick'), ('quick', 'the'), ('quick', 'brown'), ('brown', 'quick'), ('brown', 'fox'), ('fox', 'brown')].
Constraints
- words: list of strings (already tokenized)
- window: integer >= 1
- Return list of (center, context) string tuples
- Process center words left-to-right, context words left-to-right