Text Feature Extractor
Extract numerical features from text for classification.
Given a text, compute these features:
- word_count — number of words
- char_count — number of characters (excluding spaces)
- avg_word_length — average word length, rounded to 4 decimal places
- vocab_richness — number of unique words / total words, rounded to 4 decimal places
- longest_word — length of the longest word
- digit_ratio — fraction of characters that are digits (of all non-space chars), rounded to 4 decimal places
Input: A single line of text (may contain letters, digits, spaces)
Output: A dictionary with the six features, printed with keys in the order listed above.
Example:
hello world hello 123
{'word_count': 4, 'char_count': 17, 'avg_word_length': 4.25, 'vocab_richness': 0.75, 'longest_word': 5, 'digit_ratio': 0.1765}word_count: 4 words: ["hello", "world", "hello", "123"]
char_count: h-e-l-l-o-w-o-r-l-d-h-e-l-l-o-1-2-3 = 17 chars (no spaces)
avg_word_length: (5 + 5 + 5 + 3) / 4 = 18/4 = 4.25
vocab_richness: 3 unique words / 4 total = 0.75
longest_word: max(5, 5, 5, 3) = 5
digit_ratio: 3 digits out of 17 non-space chars = 3/17 = 0.1765
Constraints:
- Words are split on whitespace
- Character count excludes spaces
- Word comparisons for vocab_richness are case-sensitive
- Round float values to 4 decimal places
Background Knowledge
The problem of extracting numerical features from text is a fundamental task in Natural Language Processing (NLP), particularly in Text Classification. To tackle this problem, it's essential to understand basic text processing concepts, such as tokenization, which is the process of splitting text into individual words or tokens. Additionally, familiarity with string manipulation and basic statistical calculations is necessary.
In NLP, word count and character count are basic features that can provide insights into the length and complexity of a text. The average word length can indicate the level of formality or simplicity of the language used. Vocabulary richness, calculated as the ratio of unique words to total words, is a measure of the diversity of the language. The longest word can be an indicator of the text's complexity or the presence of specialized terminology. Finally, the digit ratio can help identify texts that contain a significant amount of numerical data.
Understanding these features and how they are calculated is crucial for solving the problem. It involves applying basic mathematical operations, such as counting, averaging, and calculating ratios, to the text data. The features extracted can then be used as inputs to machine learning models for text classification tasks.
Algorithm/Approach
The general approach to solving this problem involves a series of steps that include text preprocessing, feature calculation, and output formatting. The algorithm pattern can be summarized as follows:
- Preprocess the input text by removing unnecessary characters and splitting it into words.
- Calculate each of the required features using the preprocessed text.
- Store the calculated features in a data structure, such as a dictionary, for easy access and output.
Continue the full explanation
You're reading the free preview. Unlock the complete walkthrough, the code editor, test runner and reference solution with Premium.
Editor locked
The code editor is locked for Pro problems. It is only available for free problems. Please upgrade to gain access to the code editor for all problems.