Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Implementing end-to-end test with Byzantine-Robust Aggregator for cifar10 task #557

Merged
merged 13 commits into from
May 4, 2023

Conversation

francesco98
Copy link
Collaborator

@francesco98 francesco98 commented Dec 12, 2022

Discojs

  • Changed cifar10 task definition to comply the input size required by the network. Indeed, MobileNet is using an input size of 224x224x3, while the example data (and the previous specified input shape) was 32x32x32.
  • Preprocessing function to normalize tabular data has been added

Server

Changed the end-to-end test case for the federated setting to use both standard aggregator and byzantine-robust one. Also, the test is now considering the presence of multiple users, splitting the training data among them and validating the model computing the accuracy after the training phase. As last step, an assertion has been added to expect a reasonable accuracy after training.

@francesco98 francesco98 linked an issue Dec 12, 2022 that may be closed by this pull request
@francesco98 francesco98 self-assigned this Dec 12, 2022
@francesco98 francesco98 added discojs Related to Disco.js server Related to the server testing Related to the testing of Disco.js labels Dec 12, 2022
Copy link
Member

@martinjaggi martinjaggi left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

added some comments. one goal to keep in mind is to make the Byzantine robust clipping such that it can later also be used in decentralized. so let's try to define the clipping center based on local info only (and try to minimize additional communication if any).

we should also write down some thoughts&docu on how it will work with async (whenever the buffer size is reached, we do an aggregation with that group, so that's a round)

@s314cy s314cy force-pushed the 556-missing-unit-test-for-byzantine-robust-aggregator branch from 0322805 to 7b5f30b Compare March 7, 2023 14:43
@s314cy s314cy marked this pull request as ready for review March 7, 2023 14:44
@s314cy s314cy force-pushed the 556-missing-unit-test-for-byzantine-robust-aggregator branch from 7b5f30b to ecaa4fc Compare March 7, 2023 14:47
@s314cy s314cy force-pushed the 556-missing-unit-test-for-byzantine-robust-aggregator branch 2 times, most recently from e545f84 to d3992cb Compare March 23, 2023 12:15
@s314cy s314cy force-pushed the 556-missing-unit-test-for-byzantine-robust-aggregator branch 4 times, most recently from 6283798 to d482335 Compare May 3, 2023 10:22
@s314cy s314cy force-pushed the 556-missing-unit-test-for-byzantine-robust-aggregator branch from 93a6169 to 09a9863 Compare May 4, 2023 08:22
@s314cy s314cy self-requested a review May 4, 2023 08:58
@s314cy s314cy merged commit 4f38d19 into develop May 4, 2023
@s314cy s314cy deleted the 556-missing-unit-test-for-byzantine-robust-aggregator branch May 4, 2023 09:01
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
discojs Related to Disco.js server Related to the server testing Related to the testing of Disco.js
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Missing Unit Test for Byzantine-robust aggregator
4 participants