Skip to content

Latest commit

 

History

History
executable file
·
110 lines (102 loc) · 14.3 KB

logs.rst

File metadata and controls

executable file
·
110 lines (102 loc) · 14.3 KB
Raw

Logs

Results for Knowledge Distillation Algorithms -

Algo Dataset Model(s) used Parameters in the network Epochs Test accuracy(ies)
VanillaKD MNIST ResNet50 - Teacher 454010 5 0.9787
ResNet18 - Student 213178 0.9706
NoisyTeacher MNIST ResNet50 - Teacher 454010 5 0.9771
ResNet18 - Student 213178 0.9769
MessyCollab MNIST ResNet50 - Teacher 454010 5 0.9795
ResNet18 - Student 213178 0.9724
SoftRandom MNIST ResNet50 - Teacher 454010 5 0.9819
ResNet18 - Student 213178 0.9799
TAKD MNIST ResNet50 - Teacher 454010 5 0.9971
ResNet34 - Assistant 1 403194 0.9939
ResNet34 - Assistant 2 403194 0.9940
ResNet18 - Student 213178 0.9960
CSKD CIFAR100 ResNet18 - Student (Normal) 213178 200 0.7529
ResNet18 - Student (CSKD) 213178 0.7801
BANN MNIST ResNet18 - Student (gen 1) 213178 5 0.9726
ResNet18 - Student (gen 2) 213178 0.9695
DML MNIST ResNet18 - Student 213178 5 0.9660
ResNet18 - Student 213178 0.9620
ResNet18 - Student 213178 0.9641
SelfTraining MNIST ResNet18 - Student 213178 5 0.9772
VirtualTeacher MNIST ResNet18 - Student 213178 5 0.9814
RCO MNIST ResNet50 - Teacher 454010 10 0.9827
ResNet18 - Student 213178 0.9735
Attention MNIST ResNet50 - Teacher 10182 10 0.9858
ResNet18 - Student 4254 0.9745
RKDLoss MNIST ResNet50 - Teacher 10182 10 0.9858
ResNet18 - Student 4254 0.9651
KA/ProbShift MNIST ResNet50 - Teacher 10182 10 0.9858
ResNet18 - Student 4254 20 0.9495
KA/LabelSmoothReg MNIST ResNet50 - Teacher 10182 10 0.9858
ResNet18 - Student 4254 20 0.9451
BERT2LSTM IMDB BERT(base-uncased) - Teacher 109483778 1 0.8934
LSTM - Student 1051274 5 0.8522

Results for Pruning Algorithms -

Model Used Dataset Pruning epoch % Model pruned Training epochs Accuracy
ResNet18 MNIST 1/3 0.0 5 98.78 %
2/3 10.0 5 98.91 %
3/3 18.99 5 98.90 %

Results for Quantization -

Algo Used Model Used Dataset Model Size (MB) Training epochs Accuracy Time elapsed
Static Quantization ResNet18 CIFAR10 Base 46.8373 5 0.7208 26.1193
Quantized 11.7366 0.7073 12.9016
Quantization Aware Training ResNet18 CIFAR10 Base 46.8373 5 0.7208 26.1193
Quantized 11.8408 3 0.7128 13.0377
Dynamic Quantization LSTM IMDB Base 5.3829 3 0.6964 56.6991
Quantized 4.3757 0.6953 54.2108