Given two snippets of text, Semantic Textual Similarity captures the notion that some texts are more similar than others, measuring their degree of semantic equivalence. The tasks consists in rating the degree of semantic equivalence between two text snippets. The Spanish task introduced two diverse datasets on different genres, namely encyclopedic descriptions extracted from the Spanish Wikipedia and contemporary Spanish newswire. Participants had access to a limited amount of labeled data, consisting of 65 sentence pairs, which they could use for training. The similarity scores were adapted to fit a range from 0 to 4.
Task results
System | Pearson correlation Sort ascending |
---|---|
UMCC-DLSI-run2 | 0.8070 |
Meerkat-Mafia-run2 | 0.8040 |
UNAL-NLP-run1 | 0.8010 |
UMCC-DLSI-run1 | 0.7910 |
Meerkat-Mafia-run3 | 0.7880 |