Table 2 Performance comparison of nine models based on GRUs with different input pipelines. Models are evaluated using F1 and semantic similarity. Each model includes certain inputs (listed as column headers). When a particular input type is included, there is a \(\checkmark\) in the corresponding cell
Model | Input Pipelines | Embeddings | |||||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
\(X^{token}_{test}\) | \(X^{char}_{test}\) | \(X^{repr}_{test}\) | \(X^{POS}_{test}\) | \(X^{BIOT}_{test}\) | \(X^{UMLS}_{test}\) | CRAFT | GloVe | ELMo | |||||||
Prot. | Biom. | Chem. | Macr. | F1 | Sem. | F1 | Sem. | F1 | Sem. | ||||||
\(M_1\) | \(\checkmark\) | 0.78 | 0.79 | 0.82 | 0.83 | 0.81 | 0.81 | ||||||||
\(M_2\) | \(\checkmark\) | \(\checkmark\) | 0.79 | 0.80 | 0.82 | 0.83 | 0.82 | 0.83 | |||||||
\(M_3\) | \(\checkmark\) | \(\checkmark\) | \(\checkmark\) | 0.80 | 0.81 | 0.82 | 0.83 | 0.81 | 0.81 | ||||||
\(M_4\) | \(\checkmark\) | \(\checkmark\) | \(\checkmark\) | \(\checkmark\) | 0.79 | 0.80 | 0.82 | 0.83 | 0.82 | 0.83 | |||||
\(M_5\) | \(\checkmark\) | \(\checkmark\) | \(\checkmark\) | \(\checkmark\) | \(\checkmark\) | 0.81 | 0.82 | 0.81 | 0.82 | 0.84 | 0.84 | ||||
\(M_6\) | \(\checkmark\) | \(\checkmark\) | \(\checkmark\) | \(\checkmark\) | \(\checkmark\) | \(\checkmark\) | 0.79 | 0.80 | 0.82 | 0.83 | 0.83 | 0.83 | |||
\(M_7\) | \(\checkmark\) | \(\checkmark\) | \(\checkmark\) | \(\checkmark\) | \(\checkmark\) | \(\checkmark\) | \(\checkmark\) | 0.81 | 0.82 | 0.82 | 0.84 | 0.84 | 0.84 | ||
\(M_8\) | \(\checkmark\) | \(\checkmark\) | \(\checkmark\) | \(\checkmark\) | \(\checkmark\) | \(\checkmark\) | \(\checkmark\) | \(\checkmark\) | 0.80 | 0.81 | 0.82 | 0.83 | 0.83 | 0.84 | |
\(M_9\) | \(\checkmark\) | \(\checkmark\) | \(\checkmark\) | \(\checkmark\) | \(\checkmark\) | \(\checkmark\) | \(\checkmark\) | \(\checkmark\) | \(\checkmark\) | 0.80 | 0.81 | 0.82 | 0.83 | 0.84 | 0.84 |