Given a plain text clinical case report document collection, participating systems have to return all species mentions, together with their corresponding NCBI taxonomy concept identifiers.
The National Center for Biotechnology Information (NCBI) Taxonomy includes names of organisms classified primarily based on a phylogenetic hierarchy. The NCBI Taxonomy is a universal database, used by the International Nucleotide Sequence Database Collaboration (INSDC), which includes GenBank, the European Molecular Biology Laboratory (EMBL), and DNA Data Bank of Japan (DDBJ) as a single source of taxonomic classification to maintain consistency between databases. In NCBI, each unique code identifies a specific type of organism (e.g., Taxonomy ID: 5476 for Candida Albicans) or groups of organisms (Taxonomy ID: 40674 for mammals).
Task results
System | Precision | Recall | F1 | CEM | Accuracy | MacroPrecision | MacroRecall | MacroF1 | RMSE | MicroPrecision | MicroRecall | MicroF1 Sort ascending | MAE | MAP | UAS | LAS | MLAS | BLEX | Pearson correlation | Spearman correlation | MeasureC | BERTScore | EMR | Exact Match | F0.5 | Hierarchical F | ICM | MeasureC | Propensity F | Reliability | Sensitivity | Sentiment Graph F1 | WAC | b2 | erde30 | sent | weighted f1 |
---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
Vicomtech NLP | 0.9376 | 0.9234 | 0.9304 | ||||||||||||||||||||||||||||||||||
Clac | 0.9495 | 0.8910 | 0.9193 | ||||||||||||||||||||||||||||||||||
plncmm | 0.9139 | 0.9060 | 0.9099 | ||||||||||||||||||||||||||||||||||
IGES | 0.8979 | 0.8512 | 0.8740 | ||||||||||||||||||||||||||||||||||
Pumas | 0.9389 | 0.8075 | 0.8682 |