In this task, systems are required to determine whether a short message in Spanish is ironic or not according to a specified context (by assigning a binary value 1 or 0).
Publication
Reynier Ortega-Bueno, Francisco Rangel, Delia Irazú Hernández Farías, Paolo Rosso, Manuel Montes-y-Gómez, José Medina-Pagola (2019) Overview of the Task on Irony Detection in Spanish Variants. Proceedings of the Iberian Languages Evaluation Forum (IberLEF 2019
Competition
Language
Spanish (Cuba)
Spanish (Mexico)
Spanish (Spain)
NLP topic
Abstract task
Dataset
Year
2019
Publication link
Ranking metric
Macro F1
Task results
| System | Precision | Recall | F1 | CEM | Accuracy | MacroPrecision | MacroRecall | MacroF1 Sort ascending | RMSE | MicroPrecision | MicroRecall | MicroF1 | MAE | MAP | UAS | LAS | MLAS | BLEX | Pearson correlation | Spearman correlation | MeasureC | BERTScore | EMR | Exact Match | F0.5 | Hierarchical F | ICM | MeasureC | Propensity F | Reliability | Sensitivity | Sentiment Graph F1 | WAC | b2 | erde30 | sent | weighted f1 |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| ELiRF-UPV | 0.7167 | ||||||||||||||||||||||||||||||||||||
| CICLiku | 0.6875 | ||||||||||||||||||||||||||||||||||||
| VRAIN | 0.6842 | ||||||||||||||||||||||||||||||||||||
| ELiRF-UPV | 0.6803 | ||||||||||||||||||||||||||||||||||||
| CIMAT | 0.6709 | ||||||||||||||||||||||||||||||||||||
| Jzaragoza | 0.6703 | ||||||||||||||||||||||||||||||||||||
| SCoMoDI | 0.6652 | ||||||||||||||||||||||||||||||||||||
| LASTUS-UPF method1 | 0.6606 | ||||||||||||||||||||||||||||||||||||
| CIMAT | 0.6596 | ||||||||||||||||||||||||||||||||||||
| ELiRF-UPV | 0.6527 |
