The goal of the JOKER tasks is to foster progress on the automatic interpretation, generation, and translation of wordplay. This edition focuses on puns. A pun is a form of wordplay in which a word or phrase evokes the meaning of another word or phrase with a similar or identical pronunciation. Task 2 Pun location in Spanish consists in determining which word express a pun in tweets already classified as containing a pun.
Publication
Ermakova, L., Miller, T., Bosser, AG., Palma Preciado, V.M., Sidorov, G., Jatowt, A. (2023). Overview of JOKER – CLEF-2023 Track on Automatic Wordplay Analysis. In: Arampatzis, A., et al. Experimental IR Meets Multilinguality, Multimodality, and Interaction. CLEF 2023. Lecture Notes in Computer Science, vol 14163. Springer, Cham. https://doi.org/10.1007/978-3-031-42448-9_26
Competition
Language
Spanish
NLP topic
Abstract task
Dataset
Year
2023
Publication link
Ranking metric
Accuracy
Task results
| System | Precision | Recall | F1 Sort ascending | CEM | Accuracy | MacroPrecision | MacroRecall | MacroF1 | RMSE | MicroPrecision | MicroRecall | MicroF1 | MAE | MAP | UAS | LAS | MLAS | BLEX | Pearson correlation | Spearman correlation | MeasureC | BERTScore | EMR | Exact Match | F0.5 | Hierarchical F | ICM | MeasureC | Propensity F | Reliability | Sensitivity | Sentiment Graph F1 | WAC | b2 | erde30 | sent | weighted f1 |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| Smroltra_SimpleT5 | 0.7950 | ||||||||||||||||||||||||||||||||||||
| Akranlu_tokenclassification_y | 0.7917 | ||||||||||||||||||||||||||||||||||||
| UBO_SimpleT5 | 0.7767 | ||||||||||||||||||||||||||||||||||||
| Akranlu_tokenclassification_x | 0.7751 | ||||||||||||||||||||||||||||||||||||
| Les_miserables_simplet5 | 0.7618 | ||||||||||||||||||||||||||||||||||||
| Les_miserables_word | 0.4954 | ||||||||||||||||||||||||||||||||||||
| Smroltra_SpaCy | 0.4448 |
