Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Bug]: #816

Closed
1 task done
ikun0124 opened this issue Dec 26, 2022 · 7 comments
Closed
1 task done

[Bug]: #816

ikun0124 opened this issue Dec 26, 2022 · 7 comments

Comments

@ikun0124
Copy link

ikun0124 commented Dec 26, 2022

Describe the bug

test commend result is different from Testing stage result of train commend.

Dataset

Folder

Model

GANomaly

Steps to reproduce the behavior

run test.py train.py respectively

OS information

OS information:

  • OS: [e.g. Ubuntu 18.04]
  • Python version: [e.g. 3.8.10]
  • Anomalib version: [e.g. 0.3.6]
  • PyTorch version: [e.g. 1.9.0]
  • CUDA/cuDNN version: [e.g. 11.1]
  • GPU models and configuration: [e.g. 2x GeForce RTX 3090]
  • Any other relevant information: [e.g. I'm using a custom dataset]

Expected behavior

test commend result is different from Testing stage result of train commend.

Screenshots

No response

Pip/GitHub

pip

What version/branch did you use?

No response

Configuration YAML

none

Logs

none

Code of Conduct

  • I agree to follow this project's Code of Conduct
@ikun0124
Copy link
Author

can someone answer my question? I've been trying for several days.

@ashwinvaidya17
Copy link
Collaborator

@loyal-ikun can you try the main branch and see if results are still different. Also, can you share the commands you used for training and testing the model?

@ikun0124
Copy link
Author

my testing commend is python tools/test.py --config /home/lian/data/code/results/ganomaly/tor/logs/lightning_logs/version_3/hparams.yaml

only ganomaly model can't reproduce the training results.

@ikun0124
Copy link
Author

this my yaml file:

dataset:
name: #options: [mvtec, btech, folder]
format: folder
path:
normal_dir: normal_train
abnormal_dir: abnormal #abnormal
normal_test_dir: normal_test #normal_test
task: classification
mask: null
extensions: null
split_ratio: null
image_size: 64
train_batch_size: 256
test_batch_size: 256

inference_batch_size: 32

num_workers: 8
transform_config:
train: null
val: null
create_validation_set: false
tiling:
apply: true
tile_size: 64
stride: null
remove_border_count: 0
use_random_tiling: False
random_tile_count: 16

model:
name: ganomaly
latent_vec_size: 130 #100 800 also ok
n_features: 64 #64
extra_layers: 0
add_final_conv: true
early_stopping:
patience: 3
metric: image_AUROC
mode: max
lr: 0.0002 #0.0002 0.0001ok
beta1: 0.5 #0.5
beta2: 0.999 #0.999
wadv: 1 #1
wcon: 50 #50
wenc: 1 #1
normalization_method: none

metrics:
image:
- F1Score
- AUROC
- Accuracy
- Precision
- Recall
threshold:
image_default: 0
adaptive: true

visualization:
show_images: False # show images on the screen
save_images: False # save images to the file system
log_images: False # log images to the available loggers (if any)
image_save_path: null # path to which images will be saved
mode: full # options: ["full", "simple"]

project:
seed: 42
path: ./results

logging:
logger: [csv] # options: [comet, tensorboard, wandb, csv] or combinations.
log_graph: false # Logs the model graph to respective logger.

optimization:
export_mode: ""

PL Trainer Args. Don't add extra parameter here.

trainer:
accelerator: auto # <"cpu", "gpu", "tpu", "ipu", "hpu", "auto">
accumulate_grad_batches: 1
amp_backend: native
auto_lr_find: false
auto_scale_batch_size: false
auto_select_gpus: false
benchmark: false
check_val_every_n_epoch: 2
default_root_dir: null
detect_anomaly: false
deterministic: false
devices: 1
enable_checkpointing: true
enable_model_summary: true
enable_progress_bar: true
fast_dev_run: false
gpus: null # Set automatically
gradient_clip_val: 0
ipus: null
limit_predict_batches: 1.0
limit_test_batches: 1.0
limit_train_batches: 1.0
limit_val_batches: 1.0
log_every_n_steps: 50
log_gpu_memory: null
max_epochs: 100 #100
max_steps: null
min_epochs: null
min_steps: null
move_metrics_to_cpu: false
multiple_trainloader_mode: max_size_cycle
num_nodes: 1
num_processes: null
num_sanity_val_steps: 0
overfit_batches: 0.0
plugins: null
precision: 32
profiler: null
reload_dataloaders_every_n_epochs: 0
replace_sampler_ddp: true
strategy: null
sync_batchnorm: false
tpu_cores: null
track_grad_norm: -1
val_check_interval: 1.0

@ikun0124
Copy link
Author

@ashwinvaidya17 my commend and yaml file, as seen above.

@ashwinvaidya17
Copy link
Collaborator

Ideally the lightning_logs folder should not be generated. I just tested on main and explicitly passing the model weights to the test.py script works. Can you try this python tools/test.py --config <path_to_training_config> --weight_file <results/.../model.ckpt> Replace the path based on your model config and weights location.

@ikun0124
Copy link
Author

I have solved this problem according to your method.Thank you.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants