Word2Vec Skip-Gram Pairs
Generate Skip-Gram training pairs for Word2Vec.
Given a list of words (a sentence) and a window size, generate all (center, context) pairs where the context word is within the window of the center word.
For each center word at index i, the context words are at indices [i−w,...,i−1,i+1,...,i+w] (where valid).
Return a list of (center_word, context_word) tuples, in order.
Example:
words = ["the", "quick", "brown", "fox"] window = 1
[('the', 'quick'), ('quick', 'the'), ('quick', 'brown'), ('brown', 'quick'), ('brown', 'fox'), ('fox', 'brown')]- We start by iterating over each word in the input list
wordswith its index i. - For each word at index i, we generate context words within the window size w=1, which means we consider the words at indices [i−1,i+1] (where valid).
- We create pairs of (center_word, context_word) tuples for each valid context word, resulting in the following pairs:
- For "the" at index 0, the context word is "quick" at index 1, giving ('the', 'quick').
- For "quick" at index 1, the context words are "the" at index 0 and "brown" at index 2, giving ('quick', 'the') and ('quick', 'brown').
- For "brown" at index 2, the context words are "quick" at index 1 and "fox" at index 3, giving ('brown', 'quick') and ('brown', 'fox').
- For "fox" at index 3, the context word is "brown" at index 2, giving ('fox', 'brown').
- The final output is a list of these generated pairs in order: [('the', 'quick'), ('quick', 'the'), ('quick', 'brown'), ('brown', 'quick'), ('brown', 'fox'), ('fox', 'brown')].
Constraints:
- words: list of strings (already tokenized)
- window: integer >= 1
- Return list of (center, context) string tuples
- Process center words left-to-right, context words left-to-right
Background Knowledge
The problem revolves around Word2Vec, a popular Natural Language Processing (NLP) technique used for learning vector representations of words, also known as word embeddings. Word2Vec is based on the idea that words with similar meanings tend to appear in similar contexts. The Skip-Gram model is a specific architecture used in Word2Vec, where the goal is to predict the context words given a center word. This is in contrast to the Continuous Bag of Words (CBOW) model, which predicts the center word given the context words.
In the context of Word2Vec, a window size is a hyperparameter that determines how many words to consider as context for a given center word. For example, if the window size is 2, the context words for a center word at index i would be the words at indices [i−2,i−1,i+1,i+2], where valid. The Skip-Gram training pairs are the pairs of center words and their corresponding context words, which are used to train the Word2Vec model.
Understanding the concept of word embeddings is crucial to solving this problem. Word embeddings are vector representations of words in a high-dimensional space, where semantically similar words are closer together. The Word2Vec model learns these embeddings by predicting the context words given a center word, or vice versa. The Skip-Gram model is particularly useful for capturing the nuances of word meanings and relationships, as it considers the context words in a flexible and dynamic way.
Continue the full explanation
You're reading the free preview. Unlock the complete walkthrough, the code editor, test runner and reference solution with Premium.
Editor locked
The code editor is locked for Pro problems. It is only available for free problems. Please upgrade to gain access to the code editor for all problems.