Text Feature Extractor

mediumPROText ClassificationNLP 2: Applications

Extract numerical features from text for classification.

Given a text, compute these features:

word_count — number of words
char_count — number of characters (excluding spaces)
avg_word_length — average word length, rounded to 4 decimal places
vocab_richness — number of unique words / total words, rounded to 4 decimal places
longest_word — length of the longest word
digit_ratio — fraction of characters that are digits (of all non-space chars), rounded to 4 decimal places

Input: A single line of text (may contain letters, digits, spaces)

Output: A dictionary with the six features, printed with keys in the order listed above.

Example

Input

hello world hello 123

Output

{'word_count': 4, 'char_count': 17, 'avg_word_length': 4.25, 'vocab_richness': 0.75, 'longest_word': 5, 'digit_ratio': 0.1765}

Reasoning

word_count: 4 words: ["hello", "world", "hello", "123"]

char_count: h-e-l-l-o-w-o-r-l-d-h-e-l-l-o-1-2-3 = 17 chars (no spaces)

avg_word_length: (5 + 5 + 5 + 3) / 4 = 18/4 = 4.25

vocab_richness: 3 unique words / 4 total = 0.75

longest_word: max(5, 5, 5, 3) = 5

digit_ratio: 3 digits out of 17 non-space chars = 3/17 = 0.1765

Constraints

Words are split on whitespace
Character count excludes spaces
Word comparisons for vocab_richness are case-sensitive
Round float values to 4 decimal places

More from NLP 2: Applications

BIO Entity Span Extractoreasy Simple Entity Taggereasy Sentiment Word Countereasy Keyword Classifiereasy One-Hot Sequence Encodereasy

Concept primerPREVIEW

Background Knowledge

The problem of extracting numerical features from text is a fundamental task in Natural Language Processing (NLP), particularly in Text Classification. To tackle this problem, it's essential to understand basic text processing concepts, such as tokenization, which is the process of splitting text into individual words or tokens. Additionally, familiarity with string manipulation and basic statistical calculations is necessary.

In NLP, word count and character count are basic features that can provide insights into the length and complexity of a text. The average word length can indicate the level of formality or simplicity of the language used. Vocabulary richness, calculated as the ratio of unique words to total words, is a measure of the diversity of the language. The longest word can be an indicator of the text's complexity or the presence of specialized terminology. Finally, the digit ratio can help identify texts that contain a significant amount of numerical data.

Understanding these features and how they are calculated is crucial for solving the problem. It involves applying basic mathematical operations, such as counting, averaging, and calculating ratios, to the text data. The features extracted can then be used as inputs to machine learning models for text classification tasks.

Algorithm/Approach

The general approach to solving this problem involves a series of steps that include text preprocessing, feature calculation, and output formatting. The algorithm pattern can be summarized as follows:

Preprocess the input text by removing unnecessary characters and splitting it into words.
Calculate each of the required features using the preprocessed text.
Store the calculated features in a data structure, such as a dictionary, for easy access and output.

PRO

Continue the full explanation

You're reading the free preview. Unlock the complete walkthrough, the code editor, test runner and reference solution with Premium.

PRO

Editor locked

The code editor is locked for Pro problems. It is only available for free problems. Please upgrade to gain access to the code editor for all problems.

solution.py

Python 3.13

Test results