Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Feature/storage/monitoring tool #1191

Closed

Conversation

OhMaley
Copy link
Collaborator

@OhMaley OhMaley commented Oct 18, 2023

Description

This PR aims to offer a monitoring tool similar to the one available in Codalab.
The following features have been developed:

  • The monitoring tool is only accessible by admins
  • It is accessible under the user menu -> analytics -> storage tab
  • 3 metrics are available
    • Usage history
    • Competitions usage history and details
    • Users usage history and details
  • Metrics can be download using the download buttons
  • The metrics only displays what has been computed by the server beforehand
  • 2 background tasks has been coded:
    • a reset task that get executed every 3 month (based on the value that is set in the settings/base.py). It reset the value of all computed file sizes so they will be re-computed again without any shifting on the next run of the storage analytics task
    • a storage analytic snapshot task that get executed every week (based on the value that is set in the settings/base.py). It measures all files with unset size. Then it measures the storage usage per category (competition, user or admin) and per day. It then check for database <-> storage inconsistency (and created a log file in the logs/ folder). And save everything into the database so it becomes query-able by the front-end.

Issues this PR resolves

#713

Checklist

  • Code review by me
  • Hand tested by me
  • I'm proud of my work
  • Code review by reviewer
  • Hand tested by reviewer
  • CircleCi tests are passing
  • Ready to merge

@OhMaley OhMaley added the Enhancement Feature suggestions and improvements label Oct 18, 2023
@Didayolo
Copy link
Collaborator

Should we merge this to a new branch on codabench repository to trigger the automatic tests?

@Didayolo Didayolo mentioned this pull request Nov 28, 2023
7 tasks
@Didayolo Didayolo closed this Nov 28, 2023
@Didayolo Didayolo mentioned this pull request Nov 28, 2023
7 tasks
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Enhancement Feature suggestions and improvements
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants