ROUGE-L Score
Compute the ROUGE-L F1 score using Longest Common Subsequence.
ROUGE-L uses the length of the LCS between reference and candidate:
- Precision = LCS / len(candidate)
- Recall = LCS / len(reference)
- F1 = 2 × P × R / (P + R)
Input:
- Line 1: reference (space-separated tokens)
- Line 2: candidate (space-separated tokens)
Output: ROUGE-L F1 score, rounded to 4 decimal places.
Example:
a b c d e a c e
0.7500
- First, we find the Longest Common Subsequence (LCS) between the reference and candidate:
a c e - Then, we calculate precision and recall:
- Precision = LCS/len(candidate)=3/3=1
- Recall = LCS/len(reference)=3/5=0.6
- The ROUGE-L F1 score is computed using the formula: F1=2â‹…Pâ‹…R/(P+R)=2â‹…1â‹…0.6/(1+0.6)=0.75
- The final output is the F1 score rounded to 4 decimal places: 0.75=0.7500
Constraints:
- Use dynamic programming for LCS
- If both P and R are 0, F1 = 0
- Round to 4 decimal places
More from LLM 3: Applications & Evaluation
Background Knowledge
The ROUGE-L score is a measure used to evaluate the quality of text summarization systems. It is based on the Longest Common Subsequence (LCS) between the reference summary and the candidate summary. The LCS is the longest sequence of words that appears in both the reference and the candidate, in the same order. The ROUGE-L score uses the length of the LCS to calculate precision, recall, and F1 score. Precision measures the proportion of words in the candidate summary that are also in the reference summary, while recall measures the proportion of words in the reference summary that are also in the candidate summary.
The F1 score is the harmonic mean of precision and recall, and it provides a balanced measure of both. The F1 score is calculated as 2×P×R/(P+R), where P is precision and R is recall. The ROUGE-L score is a widely used metric in natural language processing and information retrieval, and it is often used to evaluate the performance of text summarization systems.
To solve this problem, it is essential to have a good understanding of dynamic programming, as it is used to find the LCS between two sequences. Dynamic programming is a method for solving complex problems by breaking them down into smaller subproblems, solving each subproblem only once, and storing the solutions to subproblems to avoid redundant computation. In the context of the LCS problem, dynamic programming is used to build a 2D table that stores the lengths of common subsequences between the reference and the candidate.
Continue the full explanation
You're reading the free preview. Unlock the complete walkthrough, the code editor, test runner and reference solution with Premium.
Editor locked
The code editor is locked for Pro problems. It is only available for free problems. Please upgrade to gain access to the code editor for all problems.