EXIST-2024: Source Intention in memes (soft-soft)

This task aims to classify sexist memes according to the author's intent, which helps to understand the role social media plays in the issuance and dissemination of sexist content. In this task, a ternary classification is proposed: direct, reported, judgemental. This task includes a soft-soft evaluation, where the probability of each label predicted by the system is compared with the probability defined based on the disagreement in the annotation of the gold standard.

Publication
Plaza, L. et al. (2024).EXIST 2024: sEXism Identification in Social neTworks and Memes. Experimental IR Meets Multilinguality, Multimodality, and Interaction. CLEF 2024. Lecture Notes in Computer Science, volume 14612
Language
Spanish
English
NLP topic
Dataset
Year
2024
Ranking metric
ICMSoft

Task results

System ICM Soft Sort ascending
Victor-UNED_2 -1.2660
MMICI_1 -1.3738
MMICI_2 -1.5329
NICA_1 -1.5907
CNLP-NITS-PP_1 -4.7893

If you have published a result better than those on the list, send a message to odesia-comunicacion@lsi.uned.es indicating the result and the DOI of the article, along with a copy of it if it is not published openly.