Given a plain text clinical case report document collection, participants must return the exact character offsets of all species mentions, both human and non-human.
Publication
Antonio Miranda-Escalada, Eulàlia Farré-Maduell, Salvador Lima-López, Darryl Estrada, Luis Gascó, Martin Krallinger (2022) Mention detection, normalization & classification of species, pathogens, humans and food in clinical documents: Overview of the LivingNER shared task and resources. Procesamiento del Lenguaje Natural, Revista nº 69, septiembre de 2022, pp. 241-253.
Language
Spanish
NLP topic
Abstract task
Dataset
Year
2022
Publication link
Ranking metric
Micro F
Task results
| System | Precision | Recall | F1 | CEM | Accuracy | MacroPrecision | MacroRecall | MacroF1 | RMSE | MicroPrecision | MicroRecall | MicroF1 Sort ascending | MAE | MAP | UAS | LAS | MLAS | BLEX | Pearson correlation | Spearman correlation | MeasureC | BERTScore | EMR | Exact Match | F0.5 | Hierarchical F | ICM | MeasureC | Propensity F | Reliability | Sensitivity | Sentiment Graph F1 | WAC | b2 | erde30 | sent | weighted f1 |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| Vicomtech NLP | 0.9583 | 0.9438 | 0.9510 | ||||||||||||||||||||||||||||||||||
| racai | 0.9569 | 0.9439 | 0.9503 | ||||||||||||||||||||||||||||||||||
| READ-Biomed | 0.9540 | 0.9411 | 0.9475 | ||||||||||||||||||||||||||||||||||
| SINAI | 0.9571 | 0.9346 | 0.9457 | ||||||||||||||||||||||||||||||||||
| plncmm | 0.9455 | 0.9373 | 0.9414 |
