EXIST-2024: Source Intention in tweets (soft-soft)

This task aims to classify sexist tweets based on the author's intent, helping to understand the role social media plays in the emission and dissemination of sexist messages. In this task, a ternary classification is proposed: direct, reported, judgemental. This task includes a soft-soft evaluation, where the probability of each label predicted by the system is compared with the probability defined based on the annotation disagreement in the gold standard.

Publication
Plaza, L. et al. (2024).EXIST 2024: sEXism Identification in Social neTworks and Memes. Experimental IR Meets Multilinguality, Multimodality, and Interaction. CLEF 2024. Lecture Notes in Computer Science, volume 14612
Language
Spanish
English
NLP topic
Dataset
Year
2024
Ranking metric
ICMSoft

Task results

System ICM Soft Sort ascending
NYCU-NLP_2 -0.2543
NYCU-NLP_1 -0.4059
NYCU-NLP_3 -0.5226
BAZI_1 -1.3468
Victor-UNED_2 -1.6440

If you have published a result better than those on the list, send a message to odesia-comunicacion@lsi.uned.es indicating the result and the DOI of the article, along with a copy of it if it is not published openly.