Named entity recognition

This task requires automatically finding disease mentions in published clinical cases.

All disease mentions are defined by their corresponding character offsets (start character and end character) in UTF-8 plain text clinical cases. 

Publication
Antonio Miranda-Escalada, Luis Gascó, Salvador Lima-López, Eulàlia Farré-Maduell, Darryl Estrada, Anastasios Nentidis, Anastasia Krithara, Georgios Katsimpras, Georgios Paliouras, Martin Krallinger (2022) Overview of DisTEMIST at BioASQ: Automatic detection and normalization of diseases from clinical texts: results, methods, evaluation and multilingual resources. Working Notes of Conference and Labs of the Evaluation (CLEF) Forum. CEUR Workshop Proceedings. http://ceur-ws.org/Vol-3180/paper-11.pdf .
Competition
DisTEMIST at BioASQ: Automatic detection and normalization of diseases from clinical texts
Language
Spanish
NLP topic
(named) entity recognition
Abstract task
Sequence labeling
Dataset
DisTEMIST
Year
2022
Publication link
http://ceur-ws.org/Vol-3180/p…
Ranking metric
Macro F1

Task results

System Precision Recall F1 CEM Accuracy MacroPrecision MacroRecall MacroF1 RMSE MicroPrecision MicroRecall MicroF1 MAE MAP UAS LAS MLAS BLEX Pearson correlation Spearman correlation MeasureC BERTScore EMR Exact Match F0.5 Hierarchical F ICM MeasureC Propensity F Reliability Sensitivity Sentiment Graph F1 WAC b2 erde30 sent weighted f1
PICUSLab-NER Results 0.7915 0.7629 0.7770
HPI-DHC-post-process 0.7434 0.7483 0.7458
HPI-DHC-lr-post-process 0.7417 0.7371 0.7394
SINAI run2-biomedical_model 0.7520 0.7529 0.7387
SINAI-run1-clinical_model 0.7519 0.7221 0.7367
HPI-DHC-postprocess 0.6207 0.5196 0.5657
Better Innovations Lab & Norwegian Centre for E-health Research-run1-snomed 0.5478 0.4577 0.4987
Better Innovations Lab & Norwegian Centre for E-health Research-run2-snomed-limited 0.5497 0.4549 0.4978
HPI-DHC-4-ensemble-reranking 0.5427 0.4513 0.4928
HPI-DHC-3-ensemble 0.4678 0.3890 0.4248

If you have published a result better than those on the list, send a message to odesia-comunicacion@lsi.uned.es indicating the result and the DOI of the article, along with a copy of it if it is not published openly.