In this task, systems are required to automatically propose a factual tag for each event in a text. Event are annotated and three possible categories of factuality are considered:
- Facts: current and past situations in the world that are presented as real.
- Counterfacts: current and past situations that the writer presents as not having happened.
- Possibilities, future situations, predictions, hypothesis and other options: situations presented as uncertain since the writer does not commit openly to the truth-value either because they have not happened yet or because the author does not know.
Publication
Aiala Rosá, Laura Alonso, Irene Castellón, Luis Chiruzzo, Hortensia Curell, Ana Fernandez Montraveta, Santiago Góngora, Marisa Malcuori, Glòria Vàzquez and Dina Wonsever (2020) Overview of FACT at IberLEF 2020: Events Detection and Classification. Proceedings of the Iberian Languages Evaluation Forum (IberLEF 2020)
Language
Spanish
NLP topic
Abstract task
Dataset
Year
2020
Publication link
Ranking metric
Macro F1
Task results
System | Precision | Recall | F1 | CEM | Accuracy | MacroPrecision | MacroRecall | MacroF1 Sort ascending | RMSE | MicroPrecision | MicroRecall | MicroF1 | MAE | MAP | UAS | LAS | MLAS | BLEX | Pearson correlation | Spearman correlation | MeasureC | BERTScore | EMR | Exact Match | F0.5 | Hierarchical F | ICM | MeasureC | Propensity F | Reliability | Sensitivity | Sentiment Graph F1 | WAC | b2 | erde30 | sent | weighted f1 |
---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
t.romani | 0.6120 | 0.6040 | 0.6070 | ||||||||||||||||||||||||||||||||||
guster | 0.6210 | 0.5740 | 0.5930 | ||||||||||||||||||||||||||||||||||
accg14 | 0.5560 | 0.5450 | 0.5500 | ||||||||||||||||||||||||||||||||||
trinidadg | 0.5580 | 0.5200 | 0.5360 | ||||||||||||||||||||||||||||||||||
premjithb | 0.4550 | 0.3760 | 0.3930 |