This page displays the final official leaderboards of results for the AdMIRe tasks.
Note that participants were required to submit a system description paper to be entered on the final leaderboard. This page will be updated with links to the system description papers when they are published.
Subtask A
Text and Images - English
| Team | Rank (Test set) | Rank (Extended eval set) | Top 1 Accuracy (Test set) | DCG Score (Test set) | Top 1 Accuracy (Extended eval set) | DCG Score (Extended eval set) |
|---|---|---|---|---|---|---|
| PALI-NLP | 1 | 1 | 0.93 | 3.52 | 0.83 | 3.43 |
| dutir914 | 2 | 3 | 0.93 | 3.46 | 0.79 | 3.28 |
| AlexUNLP | 3 | 5 | 0.93 | 3.45 | 0.72 | 3.22 |
| AIMA | 4 | 10 | 0.87 | 3.44 | 0.48 | 2.90 |
| daalft | 5 | 2 | 0.87 | 3.43 | 0.81 | 3.35 |
| PoliTo | 6 | 4 | 0.87 | 3.38 | 0.75 | 3.20 |
| UCSC NLP T6 | 7 | - | 0.87 | 3.36 | ||
| Zhoumou | 8 | 6 | 0.73 | 3.20 | 0.69 | 3.21 |
| HiTZ-Ixa | 9 | 7 | 0.73 | 3.13 | 0.58 | 3.00 |
| TübingenCL | 10 | - | 0.67 | 3.16 | ||
| Howard University-AI4PC | 11 | - | 0.67 | 3.13 | ||
| UoR-NCL | 12 | 8 | 0.67 | 3.10 | 0.57 | 2.96 |
| FJWU_SemEvalSquad | 13 | 11 | 0.60 | 2.90 | 0.47 | 2.85 |
| JNLP | 14 | 9 | 0.53 | 3.14 | 0.55 | 3.13 |
| Modgenix | 15 | - | 0.53 | 2.82 | ||
| YNU-HPCC | 16 | - | 0.47 | 2.85 | ||
| UMUTeam | 17 | 12 | 0.40 | 2.68 | 0.24 | 2.52 |
Text Only - English
| Team | Rank (Test set) | Rank (Extended eval set) | Top 1 Accuracy (Test set) | DCG Score (Test set) | Top 1 Accuracy (Extended eval set) | DCG Score (Extended eval set) |
|---|---|---|---|---|---|---|
| CTYUN-AI | 1 | 1 | 0.87 | 3.51 | 0.64 | 3.10 |
| PoliTo | 2 | 2 | 0.67 | 3.07 | 0.59 | 3.04 |
| daalft | 3 | 5 | 0.67 | 3.07 | 0.33 | 2.61 |
| JNLP | 4 | 4 | 0.67 | 3.04 | 0.51 | 2.86 |
| Transformer25 | 5 | 3 | 0.47 | 2.82 | 0.54 | 3.04 |
| ChuenSumi | 6 | 6 | 0.40 | 2.89 | 0.29 | 2.68 |
Text and Images - Portuguese
| Team | Rank (Test set) | Rank (Extended eval set) | Top 1 Accuracy (Test set) | DCG Score (Test set) | Top 1 Accuracy (Extended eval set) | DCG Score (Extended eval set) |
|---|---|---|---|---|---|---|
| HiTZ-Ixa | 1 | 7 | 1.00 | 3.51 | 0.45 | 2.82 |
| dutir914 | 2 | 2 | 0.92 | 3.43 | 0.69 | 3.06 |
| Zhoumou | 3 | 3 | 0.85 | 3.33 | 0.67 | 3.10 |
| daalft | 4 | 4 | 0.77 | 3.31 | 0.56 | 2.95 |
| PALI-NLP | 5 | 1 | 0.69 | 3.21 | 0.76 | 3.23 |
| AlexUNLP | 6 | 6 | 0.62 | 3.09 | 0.51 | 2.91 |
| UoR-NCL | 7 | 5 | 0.54 | 3.05 | 0.56 | 2.90 |
| YNU-HPCC | 8 | - | 0.38 | 2.92 | ||
| UMUTeam | 9 | 8 | 0.38 | 2.57 | 0.18 | 2.33 |
| HowardUniversity-AI4PC | 10 | - | 0.23 | 2.64 | ||
| Modgenix | 11 | - | 0.23 | 2.57 |
Text Only - Portuguese
| Team | Rank (Test set) | Rank (Extended eval set) | Top 1 Accuracy (Test set) | DCG Score (Test set) | Top 1 Accuracy (Extended eval set) | DCG Score (Extended eval set) |
|---|---|---|---|---|---|---|
| CTYUN-AI | 1 | 1 | 0.92 | 3.43 | 0.56 | 2.97 |
Subtask B
Text and Images - English
| Team | Rank (Test set) | Rank (Extended eval set) | Image Accuracy (Test set) | Sentence Type (Test set) | Image Accuracy (Extended eval set) | Sentence Type (Extended eval set) |
|---|---|---|---|---|---|---|
| daalft | 1 | 2 | 0.60 | 1.00 | 0.23 | 0.77 |
| PALI-NLP | 2 | 1 | 0.60 | 0.80 | 0.93 | 1.00 |
| Modgenix | 3 | - | 0.60 | 0.60 |
Text Only - English
| Team | Rank (Test set) | Rank (Extended eval set) | Image Accuracy (Test set) | Sentence Type (Test set) | Image Accuracy (Extended eval set) | Sentence Type (Extended eval set) |
|---|---|---|---|---|---|---|
| daalft | 1 | 2 | 1.00 | 1.00 | 0.60 | 0.77 |
| Transformer25 | 2 | 1 | 0.80 | 0.60 | 0.60 | 0.90 |