EXIST-2025: Source Intention in memes (hard-hard)

This task aims to classify sexist memes according to the author’s intention, allowing for a better understanding of the role social media plays in the expression and dissemination of sexist content. In this task, a ternary classification is proposed: direct, reported, and judgemental. In this task, a hard-hard evaluation is considered, where system-predicted labels are compared with the gold standard labels.

Publication
Plaza, L. et al. (2025). Overview of EXIST 2025: Learning with Disagreement for Sexism Identification and Characterization in Tweets, Memes, and TikTok Videos. In: Carrillo-de-Albornoz, J., et al. Experimental IR Meets Multilinguality, Multimodality, and Interaction. CLEF 2025. Lecture Notes in Computer Science, vol 16089. Springer, Cham.
Language
Spanish
English
NLP topic
Year
2025
Ranking metric
ICM

Task results

System ICM Sort ascending
CogniCIC_1 0.2224
GrootWatch_3 0.1868
ArcosGPT_1 0.0597
GrootWatch_2 -0.0588
GrootWatch_1 -0.3055

If you have published a result better than those on the list, send a message to odesia-comunicacion@lsi.uned.es indicating the result and the DOI of the article, along with a copy of it if it is not published openly.