Feature Request: Support for Separate Leaderboards per Task

[ff98li]

Dear Codabench admins and devs,

I'm reaching out for the possibility for implementing separate leaderboards for individual tasks within competitions. As we are considering introducing a new task to our competition, participants may prefer to submit results for specific tasks only. However, Codabench currently puts all task results into a single leaderboard, leading to n/a entries for tasks which participants opt out of. This can be confusing and might not accurately reflect participants' performance in chosen tasks on a single leaderboard.

To better illustrate the problem, say a user choose to opt out of task 1:

Then for task 1, the scores will be all n/a

In addition to the n/a entries, different tasks could also use different column keys for the leaderboard (e.g. different metrics). In the example shown above, two tasks are sharing the same column keys. Therefore, I would ask if separate leaderboards for each task could be integrated into Codabench. Alternatively, are there any suggested workarounds that could help us achieve a clearer distinction between task performances on the leaderboard?

Thanks.

[ihsaan-ullah]

This seems like a good use-case for multiple leaderboards. What do you think @Didayolo

@ff98li at the moment we don't have multiple leaderboards. We are planning to implement this functionality but it may take some time. I don't think there is any way to not show n/a.

EDIT:

There is one way: to separate tasks in phases. But then you can only activate one phase and other phases will start after one phase ends

[Didayolo]

Indeed!

Related:

• Multiple leaderboards #873
• Features suggestions #1334

[Didayolo]

I close this issue to merge and keep track of it here: #1540.