ROUGE

$$
\text{ROUGE-N} = \frac{\sum_{S \in {\text{Reference summaries}}} \sum_{\text{gram}n \in S} \text{Count}{\text{match}}(\text{gram}n)}{\sum{S \in {\text{Reference summaries}}} \sum_{\text{gram}_n \in S} \text{Count}(\text{gram}_n)}
$$
Or we can write it as an N-gram recall:
$$
\frac{\text{Total matching N-grams}}{\text{Total N-grams}}
$$

• ROUGE-1: Words (tokens)
• ROUGE-2: Bigrams
• ROUGE-L: Longest common subsequence
• ROUGE-Lsum: Summary-level ROUGE-L