• Boosting Few-Shot Visual Learning with Self-Supervision（代码理解）

    GitHub - valeoai/BF3S: Boosting Few-Shot Visual Learning with Self-Supervision

    Preparation

    Pre-requisites

    • Python 3.7
    • Pytorch >= 1.0.0
    • CUDA 10.0 or higher

    Installation

    (1) Clone the repo:

    $ git clone https://github.com/valeoai/BF3S

    (2) Install this repository and the dependencies using pip:

    $ pip install -e ./BF3S

    With this, you can edit the BF3S code on the fly and import function and classes of BF3S in other projects as well.

    (3) Optional. To uninstall this package, run:

    $ pip uninstall BF3S

    (4) Create dataset and experiment directories:

    $ cd BF3S
$ mkdir ./datasets
$ mkdir ./experiments

    You can take a look at the Dockerfile if you are uncertain the about steps to install this project.

    Necessary datasets.

    (1) MiniImagenet To download the MiniImagenet dataset go to this github page and follow the instructions there. Then, set in bf3s/datasets/mini_imagenet_dataset.py the path to where the dataset resides in your machine.

    (2) tiered-MiniImagenet To download the tiered-MiniImagenet dataset go to this github page and follow the instructions there. Then, set in bf3s/datasets/tiered_mini_imagenet_dataset.py the path to where the dataset resides in your machine.

    (3) ImageNet-FS Download the ImageNet dataset and set in bf3s/datasets/imagenet_dataset.py the path to where the dataset resides in your machine.

    (4) CIFAR-FS The dataset will be automatically downloaded when you run the code. Set in bf3s/datasets/cifar100_fewshot_dataset.py the path to where the dataset should be downloaded.

    这里我下载的数据集是“MiniImagenet”（因为readme中的MiniImagenet 下载地址无效），它的链接为：https://drive.google.com/file/d/12V7qi-AjrYi6OoJdYcN_k502BM_jcP8D/view

    提供该链接的网址为：GitHub - kjunelee/MetaOptNet: Meta-Learning with Differentiable Convex Optimization (CVPR 2019 Oral)

    Download pre-trained models (optional).

    (1) Download the models trained on the MiniImageNet dataset.

    # Run from the BF3S directory
$ mkdir ./experiments/miniImageNet
$ cd ./experiments/miniImageNet

# WRN-28-10-based Cosine Classifier (CC) with rotation prediction self-supervision model.
$ wget https://github.com/valeoai/BF3S/releases/download/0.1.0/WRNd28w10CosineClassifierRotAugRotSelfsupervision.zip
$ unzip WRNd28w10CosineClassifierRotAugRotSelfsupervision.zip

# WRN-28-10-based CC with rotation prediction self-supervision model trained with extra unlabeled images from tiered-MiniImageNet.
$ wget https://github.com/valeoai/BF3S/releases/download/0.1.0/WRNd28w10CosineClassifierRotAugRotSelfsupervision_SemisupervisedTieredUnlabeled.zip
$ unzip WRNd28w10CosineClassifierRotAugRotSelfsupervision_SemisupervisedTieredUnlabeled.zip

# WRN-28-10-based CC with location prediction self-supervision model.
$ wget https://github.com/valeoai/BF3S/releases/download/0.1.0/WRNd28w10CosineClassifierLocSelfsupervision.zip
$ unzip WRNd28w10CosineClassifierLocSelfsupervision.zip

$ cd ../../

    (2) Download the model trained on the CIFAR-FS dataset.

    # Run from the BF3S directory
$ mkdir ./experiments/cifar
$ cd ./experiments/cifar

# WRN-28-10-based CC with rotation prediction self-supervision model.
$ wget https://github.com/valeoai/BF3S/releases/download/0.2.0/WRNd28w10CosineClassifierRotAugRotSelfsupervision.zip
$ unzip WRNd28w10CosineClassifierRotAugRotSelfsupervision.zip

$ cd ../../

    (3) Download the model trained on the ImageNet-FS dataset.

    # Run from the BF3S directory
$ mkdir ./experiments/ImageNet
$ cd ./experiments/ImageNet

# ResNet10-based CC with rotation prediction self-supervision model.
$ wget https://github.com/valeoai/BF3S/releases/download/0.3.0/ResNet10CosineClassifierRotSelfsupervision.zip
$ unzip ResNet10CosineClassifierRotSelfsupervision.zip

$ cd ../../

    Train and test Cosine Classifier (CC) based few-shot models [3] with self-supervision.

    MiniImageNet: CC model with rotation prediction self-supervision.

    To train and test the WRN-28-10 based Cosine Classifier model with rotation prediction self-supervision run (you can skip the training step if you have downloaded the pre-trained model):

    # Run from the BF3S directory
# Train the model.
$ python scripts/train_fewshot.py --config=miniImageNet/WRNd28w10CosineClassifierRotAugRotSelfsupervision

# Test the model on the 1-shot setting.
$ python scripts/test_fewshot.py --config=miniImageNet/WRNd28w10CosineClassifierRotAugRotSelfsupervision --num_novel=5 --num_train=1 --num_episodes=2000
# Expected 5-way classification accuracy: 62.81% with confidence interval +/- 0.46%

# Test the model on the 5-shot setting.
$ python scripts/test_fewshot.py --config=miniImageNet/WRNd28w10CosineClassifierRotAugRotSelfsupervision --num_novel=5 --num_train=5 --num_episodes=2000
# Expected 5-way classification accuracy: 80.00% with confidence interval +/- 0.34%

    Note that the configuration file (of the above experiment) specified by the config variable is located here: ./config/miniImageNet/WRNd28w10CosineClassifierRotAugRotSelfsupervision.py. All the experiment configuration files are placed in the ./config/ directory.

    1. (imagemol) D:\pycharm_workspace\2\BF3S>python scripts/train_fewshot.py --config=miniImageNet/WRNd28w10CosineClassifierRotAugRotSelfsupervision
    2. Loading experiment miniImageNet/WRNd28w10CosineClassifierRotAugRotSelfsupervision
    3. Generated logs and/or snapshots will be stored on D:\pycharm_workspace\2\BF3S\experiments\miniImageNet/WRNd28w10CosineClassifierRotAugRotSelfsupervision
    4. ==> Download MiniImageNet dataset at ./datasets/MiniImagenet
    5. Loading mini ImageNet dataset - phase train
    6. ==> Download MiniImageNet dataset at ./datasets/MiniImagenet
    7. Loading mini ImageNet dataset - phase val
    8. 2022-08-24 20:34:00,187 - bf3s.algorithms.algorithm - INFO   - ==> Tensorboard file location: D:\pycharm_workspace\2\BF3S\experiments\miniImageNet\WRNd28w10CosineClassifierRotAugRotSelfsupervision\logs\TENSORBOARD
    9. 2022-08-24 20:34:00,187 - bf3s.algorithms.algorithm - INFO   - Algorithm options {'data_train_opt': {'dataset_name': 'MiniImageNet80x80', 'nKnovel': 0, 'nKbase': 64, 'n_exemplars': 0, 'n_test_novel': 0, 'n_test_base': 64, 'batch_size': 1, 'epoch_size': 1000, 'phase': 'train'}, 'data_test_opt': {'dataset_name': 'MiniImageNet80x80', 'nKnovel': 5, 'nKbase': 64, 'n_exemplars': 1, 'n_test_novel': 75, 'n_test_base': 75, 'batch_size': 1, 'epoch_size': 500}, 'max_num_epochs': 26, 'networks': {'feature_extractor': {'def_file': 'feature_extractors.wide_resnet', 'pretrained': None, 'opt': {'depth': 28, 'widen_Factor': 10, 'drop_rate': 0.0, 'pool': 'none'}, 'optim_params': {'optim_type': 'sgd', 'lr': 0.1, 'momentum': 0.9, 'weight_decay': 0.0005, 'nesterov': True, 'LUT_lr': [(20, 0.1), (23, 0.01), (26, 0.001)]}}, 'classifier': {'def_file': 'classifiers.cosine_classifier_with_weight_generator', 'pretrained': None, 'opt': {'num_classes': 64, 'num_features': 640, 'scale_cls': 10, 'learn_scale': True, 'global_pooling': True}, 'optim_params': {'optim_type': 'sgd', 'lr': 0.1, 'momentum': 0.9, 'weight_decay': 0.0005, 'nesterov': True, 'LUT_lr': [(20, 0.1), (23, 0.01), (26, 0.001)]}}, 'classifier_aux': {'def_file': 'classifiers.convnet_plus_classifier', 'pretrained': None, 'opt': {'convnet_type': 'wrn_block', 'convnet_opt': {'num_channels_in': 640, 'num_channels_out': 640, 'num_layers': 4, 'stride': 2}, 'classifier_opt': {'classifier_type': 'cosine', 'num_channels': 640, 'scale_cls': 10.0, 'learn_scale': True, 'num_classes': 4, 'global_pooling': True}}, 'optim_params': {'optim_type': 'sgd', 'lr': 0.1, 'momentum': 0.9, 'weight_decay': 0.0005, 'nesterov': True, 'LUT_lr': [(20, 0.1), (23, 0.01), (26, 0.001)]}}}, 'criterions': {'loss': {'ctype': 'CrossEntropyLoss', 'opt': None}}, 'algorithm_type': 'selfsupervision.fewshot_selfsupervision_rotation', 'auxiliary_rotation_task_coef': 1.0, 'rotation_invariant_classifier': True, 'random_rotation': False, 'exp_dir': 'D:\\pycharm_workspace\\2\\BF3S\\experiments\\miniImageNet/WRNd28w10CosineClassifierRotAugRotSelfsupervision', 'disp_step': 200}
    10. 2022-08-24 20:34:00,188 - bf3s.algorithms.algorithm - INFO   - Set network feature_extractor
    11. 2022-08-24 20:34:00,188 - bf3s.algorithms.algorithm - INFO   - ==> Initiliaze network feature_extractor from file feature_extractors.wide_resnet with opts: {'depth': 28, 'widen_Factor': 10, 'drop_rate': 0.0, 'pool': 'none'}
    12. 2022-08-24 20:34:00,537 - bf3s.algorithms.algorithm - INFO   - ==> Network architecture:
    13. 2022-08-24 20:34:00,538 - bf3s.algorithms.algorithm - INFO   - WideResnet(
    14.   (_feature_blocks): ModuleList(
    15.     (0): Sequential(
    16.       (Conv): Conv2d(3, 16, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
    17.       (BN): BatchNorm2d(16, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
    18.       (ReLU): ReLU(inplace=True)
    19.     )
    20.     (1): Sequential(
    21.       (Block): NetworkBlock(
    22.         (layer): Sequential(
    23.           (0): BasicBlock(
    24.             (convResidual): Sequential(
    25.               (conv1): Conv2d(16, 160, kernel_size=(3, 3), stride=(2, 2), padding=(1, 1), bias=False)
    26.               (bn2): BatchNorm2d(160, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
    27.               (relu2): ReLU(inplace=True)
    28.               (conv2): Conv2d(160, 160, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
    29.             )
    30.             (convShortcut): Conv2d(16, 160, kernel_size=(1, 1), stride=(2, 2), bias=False)
    31.           )
    32.           (1): BasicBlock(
    33.             (convResidual): Sequential(
    34.               (bn1): BatchNorm2d(160, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
    35.               (relu1): ReLU(inplace=True)
    36.               (conv1): Conv2d(160, 160, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
    37.               (bn2): BatchNorm2d(160, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
    38.               (relu2): ReLU(inplace=True)
    39.               (conv2): Conv2d(160, 160, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
    40.             )
    41.             (convShortcut): Sequential()
    42.           )
    43.           (2): BasicBlock(
    44.             (convResidual): Sequential(
    45.               (bn1): BatchNorm2d(160, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
    46.               (relu1): ReLU(inplace=True)
    47.               (conv1): Conv2d(160, 160, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
    48.               (bn2): BatchNorm2d(160, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
    49.               (relu2): ReLU(inplace=True)
    50.               (conv2): Conv2d(160, 160, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
    51.             )
    52.             (convShortcut): Sequential()
    53.           )
    54.           (3): BasicBlock(
    55.             (convResidual): Sequential(
    56.               (bn1): BatchNorm2d(160, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
    57.               (relu1): ReLU(inplace=True)
    58.               (conv1): Conv2d(160, 160, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
    59.               (bn2): BatchNorm2d(160, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
    60.               (relu2): ReLU(inplace=True)
    61.               (conv2): Conv2d(160, 160, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
    62.             )
    63.             (convShortcut): Sequential()
    64.           )
    65.         )
    66.       )
    67.       (BN): BatchNorm2d(160, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
    68.       (ReLU): ReLU(inplace=True)
    69.     )
    70.     (2): Sequential(
    71.       (Block): NetworkBlock(
    72.         (layer): Sequential(
    73.           (0): BasicBlock(
    74.             (convResidual): Sequential(
    75.               (conv1): Conv2d(160, 320, kernel_size=(3, 3), stride=(2, 2), padding=(1, 1), bias=False)
    76.               (bn2): BatchNorm2d(320, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
    77.               (relu2): ReLU(inplace=True)
    78.               (conv2): Conv2d(320, 320, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
    79.             )
    80.             (convShortcut): Conv2d(160, 320, kernel_size=(1, 1), stride=(2, 2), bias=False)
    81.           )
    82.           (1): BasicBlock(
    83.             (convResidual): Sequential(
    84.               (bn1): BatchNorm2d(320, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
    85.               (relu1): ReLU(inplace=True)
    86.               (conv1): Conv2d(320, 320, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
    87.               (bn2): BatchNorm2d(320, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
    88.               (relu2): ReLU(inplace=True)
    89.               (conv2): Conv2d(320, 320, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
    90.             )
    91.             (convShortcut): Sequential()
    92.           )
    93.           (2): BasicBlock(
    94.             (convResidual): Sequential(
    95.               (bn1): BatchNorm2d(320, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
    96.               (relu1): ReLU(inplace=True)
    97.               (conv1): Conv2d(320, 320, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
    98.               (bn2): BatchNorm2d(320, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
    99.               (relu2): ReLU(inplace=True)
    100.               (conv2): Conv2d(320, 320, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
    101.             )
    102.             (convShortcut): Sequential()
    103.           )
    104.           (3): BasicBlock(
    105.             (convResidual): Sequential(
    106.               (bn1): BatchNorm2d(320, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
    107.               (relu1): ReLU(inplace=True)
    108.               (conv1): Conv2d(320, 320, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
    109.               (bn2): BatchNorm2d(320, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
    110.               (relu2): ReLU(inplace=True)
    111.               (conv2): Conv2d(320, 320, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
    112.             )
    113.             (convShortcut): Sequential()
    114.           )
    115.         )
    116.       )
    117.       (BN): BatchNorm2d(320, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
    118.       (ReLU): ReLU(inplace=True)
    119.     )
    120.     (3): Sequential(
    121.       (Block): NetworkBlock(
    122.         (layer): Sequential(
    123.           (0): BasicBlock(
    124.             (convResidual): Sequential(
    125.               (conv1): Conv2d(320, 640, kernel_size=(3, 3), stride=(2, 2), padding=(1, 1), bias=False)
    126.               (bn2): BatchNorm2d(640, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
    127.               (relu2): ReLU(inplace=True)
    128.               (conv2): Conv2d(640, 640, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
    129.             )
    130.             (convShortcut): Conv2d(320, 640, kernel_size=(1, 1), stride=(2, 2), bias=False)
    131.           )
    132.           (1): BasicBlock(
    133.             (convResidual): Sequential(
    134.               (bn1): BatchNorm2d(640, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
    135.               (relu1): ReLU(inplace=True)
    136.               (conv1): Conv2d(640, 640, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
    137.               (bn2): BatchNorm2d(640, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
    138.               (relu2): ReLU(inplace=True)
    139.               (conv2): Conv2d(640, 640, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
    140.             )
    141.             (convShortcut): Sequential()
    142.           )
    143.           (2): BasicBlock(
    144.             (convResidual): Sequential(
    145.               (bn1): BatchNorm2d(640, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
    146.               (relu1): ReLU(inplace=True)
    147.               (conv1): Conv2d(640, 640, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
    148.               (bn2): BatchNorm2d(640, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
    149.               (relu2): ReLU(inplace=True)
    150.               (conv2): Conv2d(640, 640, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
    151.             )
    152.             (convShortcut): Sequential()
    153.           )
    154.           (3): BasicBlock(
    155.             (convResidual): Sequential(
    156.               (bn1): BatchNorm2d(640, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
    157.               (relu1): ReLU(inplace=True)
    158.               (conv1): Conv2d(640, 640, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
    159.               (bn2): BatchNorm2d(640, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
    160.               (relu2): ReLU(inplace=True)
    161.               (conv2): Conv2d(640, 640, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
    162.             )
    163.             (convShortcut): Sequential()
    164.           )
    165.         )
    166.       )
    167.       (BN): BatchNorm2d(640, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
    168.       (ReLU): ReLU(inplace=True)
    169.     )
    170.   )
    171. )
    172. 2022-08-24 20:34:00,540 - bf3s.algorithms.algorithm - INFO   - Set network classifier
    173. 2022-08-24 20:34:00,540 - bf3s.algorithms.algorithm - INFO   - ==> Initiliaze network classifier from file classifiers.cosine_classifier_with_weight_generator with opts: {'num_classes': 64, 'num_features': 640, 'scale_cls': 10, 'learn_scale': True, 'global_pooling': True}
    174. 2022-08-24 20:34:00,546 - bf3s.algorithms.algorithm - INFO   - ==> Network architecture:
    175. 2022-08-24 20:34:00,546 - bf3s.algorithms.algorithm - INFO   - CosineClassifierWithWeightGeneration()
    176. 2022-08-24 20:34:00,546 - bf3s.algorithms.algorithm - INFO   - Set network classifier_aux
    177. 2022-08-24 20:34:00,546 - bf3s.algorithms.algorithm - INFO   - ==> Initiliaze network classifier_aux from file classifiers.convnet_plus_classifier with opts: {'convnet_type': 'wrn_block', 'convnet_opt': {'num_channels_in': 640, 'num_channels_out': 640, 'num_layers': 4, 'stride': 2}, 'classifier_opt': {'classifier_type': 'cosine', 'num_channels': 640, 'scale_cls': 10.0, 'learn_scale': True, 'num_classes': 4, 'global_pooling': True}}
    178. 2022-08-24 20:34:00,816 - bf3s.algorithms.algorithm - INFO   - ==> Network architecture:
    179. 2022-08-24 20:34:00,816 - bf3s.algorithms.algorithm - INFO   - ConvnetPlusClassifier(
    180.   (layers): Sequential(
    181.     (0): WideResnetBlock(
    182.       (block): Sequential(
    183.         (Block): NetworkBlock(
    184.           (layer): Sequential(
    185.             (0): BasicBlock(
    186.               (convResidual): Sequential(
    187.                 (conv1): Conv2d(640, 640, kernel_size=(3, 3), stride=(2, 2), padding=(1, 1), bias=False)
    188.                 (bn2): BatchNorm2d(640, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
    189.                 (relu2): ReLU(inplace=True)
    190.                 (conv2): Conv2d(640, 640, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
    191.               )
    192.               (convShortcut): Conv2d(640, 640, kernel_size=(1, 1), stride=(2, 2), bias=False)
    193.             )
    194.             (1): BasicBlock(
    195.               (convResidual): Sequential(
    196.                 (bn1): BatchNorm2d(640, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
    197.                 (relu1): ReLU(inplace=True)
    198.                 (conv1): Conv2d(640, 640, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
    199.                 (bn2): BatchNorm2d(640, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
    200.                 (relu2): ReLU(inplace=True)
    201.                 (conv2): Conv2d(640, 640, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
    202.               )
    203.               (convShortcut): Sequential()
    204.             )
    205.             (2): BasicBlock(
    206.               (convResidual): Sequential(
    207.                 (bn1): BatchNorm2d(640, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
    208.                 (relu1): ReLU(inplace=True)
    209.                 (conv1): Conv2d(640, 640, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
    210.                 (bn2): BatchNorm2d(640, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
    211.                 (relu2): ReLU(inplace=True)
    212.                 (conv2): Conv2d(640, 640, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
    213.               )
    214.               (convShortcut): Sequential()
    215.             )
    216.             (3): BasicBlock(
    217.               (convResidual): Sequential(
    218.                 (bn1): BatchNorm2d(640, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
    219.                 (relu1): ReLU(inplace=True)
    220.                 (conv1): Conv2d(640, 640, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
    221.                 (bn2): BatchNorm2d(640, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
    222.                 (relu2): ReLU(inplace=True)
    223.                 (conv2): Conv2d(640, 640, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
    224.               )
    225.               (convShortcut): Sequential()
    226.             )
    227.           )
    228.         )
    229.         (BN): BatchNorm2d(640, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
    230.         (ReLU): ReLU(inplace=True)
    231.       )
    232.     )
    233.     (1): Classifier(
    234.       (layers): CosineClassifier(num_channels=640, num_classes=4, scale_cls=10.0 (learnable=True), normalize_x=True, normalize_w=True, bias=False)
    235.     )
    236.   )
    237. )
    238. 2022-08-24 20:34:00,821 - bf3s.algorithms.algorithm - INFO   - Initialize criterion[loss]: CrossEntropyLoss with options: None
    239. 2022-08-24 20:34:00,822 - bf3s.algorithms.algorithm - INFO   - ==> Loading criterions to device: cuda
    240. 2022-08-24 20:34:00,822 - bf3s.algorithms.algorithm - INFO   - ==> Loading tensors to device: cuda
    241. 2022-08-24 20:34:00,962 - bf3s.algorithms.algorithm - INFO   - ==> Loading networks to device: cuda
    242. 2022-08-24 20:42:13,319 - bf3s.algorithms.algorithm - INFO   - Initialize optimizer: sgd with params: {'optim_type': 'sgd', 'lr': 0.1, 'momentum': 0.9, 'weight_decay': 0.0005, 'nesterov': True, 'LUT_lr': [(20, 0.1), (23, 0.01), (26, 0.001)]} for netwotk: feature_extractor
    243. 2022-08-24 20:42:13,319 - bf3s.algorithms.algorithm - INFO   - Initialize optimizer: sgd with params: {'optim_type': 'sgd', 'lr': 0.1, 'momentum': 0.9, 'weight_decay': 0.0005, 'nesterov': True, 'LUT_lr': [(20, 0.1), (23, 0.01), (26, 0.001)]} for netwotk: classifier
    244. 2022-08-24 20:42:13,319 - bf3s.algorithms.algorithm - INFO   - Initialize optimizer: sgd with params: {'optim_type': 'sgd', 'lr': 0.1, 'momentum': 0.9, 'weight_decay': 0.0005, 'nesterov': True, 'LUT_lr': [(20, 0.1), (23, 0.01), (26, 0.001)]} for netwotk: classifier_aux
    245. 2022-08-24 20:42:13,320 - bf3s.algorithms.algorithm - INFO   - Training epoch [  1 /  26]
    246. 2022-08-24 20:42:13,320 - bf3s.algorithms.algorithm - INFO   - ==> Set to feature_extractor optimizer lr = 0.1000000000
    247. 2022-08-24 20:42:13,320 - bf3s.algorithms.algorithm - INFO   - ==> Set to classifier optimizer lr = 0.1000000000
    248. 2022-08-24 20:42:13,320 - bf3s.algorithms.algorithm - INFO   - ==> Set to classifier_aux optimizer lr = 0.1000000000
    249. 2022-08-24 20:42:13,320 - bf3s.algorithms.algorithm - INFO   - Training: WRNd28w10CosineClassifierRotAugRotSelfsupervision
    250. 2022-08-24 20:42:13,320 - bf3s.algorithms.algorithm - INFO   - ==> Dataset: MiniImageNet_train [1000 batches]
    251.  20%|███████████████▎                                                             | 199/1000 [35:37<1:20:28,  6.03s/it]2022-08-24 21:17:56,249 - bf3s.algorithms.algorithm - INFO   - ==> Iteration [  1][ 200 / 1000]: {'loss_cls': '4.1589', 'loss_rot': '1.2066', 'loss_total': '5.3655', 'Accuracy': '1.6797', 'AccuracyRot': '40.5254'}
    252.  40%|███████████████████████████████▌                                               | 399/1000 [55:29<58:49,  5.87s/it]2022-08-24 21:37:48,318 - bf3s.algorithms.algorithm - INFO   - ==> Iteration [  1][ 400 / 1000]: {'loss_cls': '4.1589', 'loss_rot': '1.1953', 'loss_total': '5.3542', 'Accuracy': '1.6172', 'AccuracyRot': '41.2666'}
    253.  60%|██████████████████████████████████████████████                               | 599/1000 [1:15:21<40:07,  6.00s/it]2022-08-24 21:57:40,442 - bf3s.algorithms.algorithm - INFO   - ==> Iteration [  1][ 600 / 1000]: {'loss_cls': '4.1589', 'loss_rot': '1.1900', 'loss_total': '5.3489', 'Accuracy': '1.6198', 'AccuracyRot': '41.5859'}
    254.  80%|█████████████████████████████████████████████████████████████▌               | 799/1000 [1:34:52<19:36,  5.85s/it]2022-08-24 22:17:11,494 - bf3s.algorithms.algorithm - INFO   - ==> Iteration [  1][ 800 / 1000]: {'loss_cls': '4.1589', 'loss_rot': '1.1859', 'loss_total': '5.3448', 'Accuracy': '1.5781', 'AccuracyRot': '41.8672'}
    255. 100%|████████████████████████████████████████████████████████████████████████████▉| 999/1000 [1:54:23<00:05,  5.85s/it]2022-08-24 22:36:42,316 - bf3s.algorithms.algorithm - INFO   - ==> Iteration [  1][1000 / 1000]: {'loss_cls': '4.1589', 'loss_rot': '1.1828', 'loss_total': '5.3417', 'Accuracy': '1.5391', 'AccuracyRot': '42.0699'}
    256. 100%|████████████████████████████████████████████████████████████████████████████| 1000/1000 [1:54:28<00:00,  6.87s/it]
    257. 2022-08-24 22:36:42,321 - bf3s.algorithms.algorithm - INFO   - ==> Training stats: {'loss_cls': '4.1589', 'loss_rot': '1.1828', 'loss_total': '5.3417', 'Accuracy': '1.5391', 'AccuracyRot': '42.0699'}
    258. 2022-08-24 22:36:45,884 - bf3s.algorithms.algorithm - INFO   - Evaluating: WRNd28w10CosineClassifierRotAugRotSelfsupervision
    259. 2022-08-24 22:36:45,884 - bf3s.algorithms.algorithm - INFO   - ==> Dataset: MiniImageNet_val [500 batches]
    260. 100%|███████████████████████████████████████████████████████████████████████████████████| 500/500 [08:44<00:00,  1.05s/it]
    261. 2022-08-24 22:45:30,847 - bf3s.algorithms.algorithm - INFO   - ==> Results: {'loss': '4.2341', 'AccuracyBoth': '0.8107', 'AccuracyBase': '1.6213', 'AccuracyNovel': '20.0000', 'AccuracyNovel_std': '0.0000', 'AccuracyNovel_cnf': '0.0000'}
    262. 2022-08-24 22:45:30,848 - bf3s.algorithms.algorithm - INFO   - ==> Evaluation stats: {'loss': '4.2341', 'AccuracyBoth': '0.8107', 'AccuracyBase': '1.6213', 'AccuracyNovel': '20.0000', 'AccuracyNovel_std': '0.0000', 'AccuracyNovel_cnf': '0.0000'}
    263. 2022-08-24 22:45:31,121 - bf3s.algorithms.algorithm - INFO   - ==> Best results w.r.t. AccuracyNovel metric: epoch: 1 - {'loss': '4.2341', 'AccuracyBoth': '0.8107', 'AccuracyBase': '1.6213', 'AccuracyNovel': '20.0000', 'AccuracyNovel_std': '0.0000', 'AccuracyNovel_cnf': '0.0000'}
    264. 2022-08-24 22:45:31,121 - bf3s.algorithms.algorithm - INFO   - ==> Best results w.r.t. AccuracyNovel metric: epoch: 1 - {'loss': '4.2341', 'AccuracyBoth': '0.8107', 'AccuracyBase': '1.6213', 'AccuracyNovel': '20.0000', 'AccuracyNovel_std': '0.0000', 'AccuracyNovel_cnf': '0.0000'}
    265. 2022-08-24 22:45:31,121 - bf3s.algorithms.algorithm - INFO   - Training epoch [  2 /  26]
    266. 2022-08-24 22:45:31,121 - bf3s.algorithms.algorithm - INFO   - ==> Set to feature_extractor optimizer lr = 0.1000000000
    267. 2022-08-24 22:45:31,121 - bf3s.algorithms.algorithm - INFO   - ==> Set to classifier optimizer lr = 0.1000000000
    268. 2022-08-24 22:45:31,121 - bf3s.algorithms.algorithm - INFO   - ==> Set to classifier_aux optimizer lr = 0.1000000000
    269. 2022-08-24 22:45:31,121 - bf3s.algorithms.algorithm - INFO   - Training: WRNd28w10CosineClassifierRotAugRotSelfsupervision
    270. 2022-08-24 22:45:31,122 - bf3s.algorithms.algorithm - INFO   - ==> Dataset: MiniImageNet_train [1000 batches]
    271.  20%|███████████████▉                                                                | 199/1000 [19:21<1:18:10,  5.86s/it]2022-08-24 23:04:58,476 - bf3s.algorithms.algorithm - INFO   - ==> Iteration [  2][ 200 / 1000]: {'loss_cls': '4.1589', 'loss_rot': '0.8979', 'loss_total': '5.0568', 'Accuracy': '1.8203', 'AccuracyRot': '49.8711'}
    272.  40%|████████████████████████████████▋                                                 | 399/1000 [38:52<58:36,  5.85s/it]2022-08-24 23:24:29,101 - bf3s.algorithms.algorithm - INFO   - ==> Iteration [  2][ 400 / 1000]: {'loss_cls': '4.1589', 'loss_rot': '0.8896', 'loss_total': '5.0485', 'Accuracy': '1.7109', 'AccuracyRot': '50.1729'}
    273.  60%|█████████████████████████████████████████████████                                 | 599/1000 [58:22<39:06,  5.85s/it]2022-08-24 23:43:59,678 - bf3s.algorithms.algorithm - INFO   - ==> Iteration [  2][ 600 / 1000]: {'loss_cls': '4.1589', 'loss_rot': '0.8866', 'loss_total': '5.0454', 'Accuracy': '1.6276', 'AccuracyRot': '50.0618'}
    274.  80%|███████████████████████████████████████████████████████████████▉                | 799/1000 [1:17:53<19:36,  5.85s/it]2022-08-25 00:03:30,281 - bf3s.algorithms.algorithm - INFO   - ==> Iteration [  2][ 800 / 1000]: {'loss_cls': '4.1589', 'loss_rot': '0.8848', 'loss_total': '5.0437', 'Accuracy': '1.6133', 'AccuracyRot': '50.1372'}
    275. 100%|███████████████████████████████████████████████████████████████████████████████▉| 999/1000 [1:37:24<00:05,  5.86s/it]2022-08-25 00:23:00,991 - bf3s.algorithms.algorithm - INFO   - ==> Iteration [  2][1000 / 1000]: {'loss_cls': '4.1589', 'loss_rot': '0.8839', 'loss_total': '5.0428', 'Accuracy': '1.6062', 'AccuracyRot': '50.1906'}
    276. 100%|███████████████████████████████████████████████████████████████████████████████| 1000/1000 [1:37:29<00:00,  5.85s/it]
    277. 2022-08-25 00:23:00,992 - bf3s.algorithms.algorithm - INFO   - ==> Training stats: {'loss_cls': '4.1589', 'loss_rot': '0.8839', 'loss_total': '5.0428', 'Accuracy': '1.6062', 'AccuracyRot': '50.1906'}
    278. 2022-08-25 00:23:04,586 - bf3s.algorithms.algorithm - INFO   - Evaluating: WRNd28w10CosineClassifierRotAugRotSelfsupervision
    279. 2022-08-25 00:23:04,586 - bf3s.algorithms.algorithm - INFO   - ==> Dataset: MiniImageNet_val [500 batches]
    280. 100%|███████████████████████████████████████████████████████████████████████████████████| 500/500 [08:45<00:00,  1.05s/it]
    281. 2022-08-25 00:31:49,689 - bf3s.algorithms.algorithm - INFO   - ==> Results: {'loss': 'nan', 'AccuracyBoth': '5.5160', 'AccuracyBase': '1.6213', 'AccuracyNovel': '20.0000', 'AccuracyNovel_std': '0.0000', 'AccuracyNovel_cnf': '0.0000'}
    282. NaN or Inf found in input tensor.
    283. 2022-08-25 00:31:49,690 - bf3s.algorithms.algorithm - INFO   - ==> Evaluation stats: {'loss': 'nan', 'AccuracyBoth': '5.5160', 'AccuracyBase': '1.6213', 'AccuracyNovel': '20.0000', 'AccuracyNovel_std': '0.0000', 'AccuracyNovel_cnf': '0.0000'}
    284. 2022-08-25 00:31:49,690 - bf3s.algorithms.algorithm - INFO   - ==> Best results w.r.t. AccuracyNovel metric: epoch: 1 - {'loss': '4.2341', 'AccuracyBoth': '0.8107', 'AccuracyBase': '1.6213', 'AccuracyNovel': '20.0000', 'AccuracyNovel_std': '0.0000', 'AccuracyNovel_cnf': '0.0000'}
    285. 2022-08-25 00:31:49,691 - bf3s.algorithms.algorithm - INFO   - Training epoch [  3 /  26]
    286. 2022-08-25 00:31:49,691 - bf3s.algorithms.algorithm - INFO   - ==> Set to feature_extractor optimizer lr = 0.1000000000
    287. 2022-08-25 00:31:49,691 - bf3s.algorithms.algorithm - INFO   - ==> Set to classifier optimizer lr = 0.1000000000
    288. 2022-08-25 00:31:49,691 - bf3s.algorithms.algorithm - INFO   - ==> Set to classifier_aux optimizer lr = 0.1000000000
    289. 2022-08-25 00:31:49,691 - bf3s.algorithms.algorithm - INFO   - Training: WRNd28w10CosineClassifierRotAugRotSelfsupervision
    290. 2022-08-25 00:31:49,692 - bf3s.algorithms.algorithm - INFO   - ==> Dataset: MiniImageNet_train [1000 batches]
    291.  20%|███████████████▉                                                                | 199/1000 [19:27<1:18:32,  5.88s/it]2022-08-25 00:51:23,095 - bf3s.algorithms.algorithm - INFO   - ==> Iteration [  3][ 200 / 1000]: {'loss_cls': '4.1589', 'loss_rot': '0.8803', 'loss_total': '5.0392', 'Accuracy': '1.5703', 'AccuracyRot': '50.6738'}
    292.  40%|████████████████████████████████▋                                                 | 399/1000 [39:04<58:56,  5.88s/it]2022-08-25 01:11:00,168 - bf3s.algorithms.algorithm - INFO   - ==> Iteration [  3][ 400 / 1000]: {'loss_cls': '4.1589', 'loss_rot': '0.8798', 'loss_total': '5.0387', 'Accuracy': '1.5781', 'AccuracyRot': '50.5850'}
    293.  60%|█████████████████████████████████████████████████                                 | 599/1000 [58:41<39:20,  5.89s/it]2022-08-25 01:30:37,293 - bf3s.algorithms.algorithm - INFO   - ==> Iteration [  3][ 600 / 1000]: {'loss_cls': '4.1589', 'loss_rot': '0.8798', 'loss_total': '5.0387', 'Accuracy': '1.5599', 'AccuracyRot': '50.4466'}
    294.  80%|███████████████████████████████████████████████████████████████▉                | 799/1000 [1:18:18<19:43,  5.89s/it]2022-08-25 01:50:14,362 - bf3s.algorithms.algorithm - INFO   - ==> Iteration [  3][ 800 / 1000]: {'loss_cls': '4.1589', 'loss_rot': '0.8798', 'loss_total': '5.0386', 'Accuracy': '1.5391', 'AccuracyRot': '50.4712'}
    295. 100%|███████████████████████████████████████████████████████████████████████████████▉| 999/1000 [1:37:55<00:05,  5.89s/it]2022-08-25 02:09:51,432 - bf3s.algorithms.algorithm - INFO   - ==> Iteration [  3][1000 / 1000]: {'loss_cls': '4.1589', 'loss_rot': '0.8799', 'loss_total': '5.0388', 'Accuracy': '1.5266', 'AccuracyRot': '50.4121'}
    296. 100%|███████████████████████████████████████████████████████████████████████████████| 1000/1000 [1:38:01<00:00,  5.88s/it]
    297. 2022-08-25 02:09:51,433 - bf3s.algorithms.algorithm - INFO   - ==> Training stats: {'loss_cls': '4.1589', 'loss_rot': '0.8799', 'loss_total': '5.0388', 'Accuracy': '1.5266', 'AccuracyRot': '50.4121'}
    298. 2022-08-25 02:09:55,060 - bf3s.algorithms.algorithm - INFO   - Evaluating: WRNd28w10CosineClassifierRotAugRotSelfsupervision
    299. 2022-08-25 02:09:55,060 - bf3s.algorithms.algorithm - INFO   - ==> Dataset: MiniImageNet_val [500 batches]
    300. 100%|███████████████████████████████████████████████████████████████████████████████████| 500/500 [08:43<00:00,  1.05s/it]
    301. 2022-08-25 02:18:38,811 - bf3s.algorithms.algorithm - INFO   - ==> Results: {'loss': 'nan', 'AccuracyBoth': '0.8107', 'AccuracyBase': '1.6213', 'AccuracyNovel': '20.0000', 'AccuracyNovel_std': '0.0000', 'AccuracyNovel_cnf': '0.0000'}
    302. NaN or Inf found in input tensor.
    303. 2022-08-25 02:18:38,812 - bf3s.algorithms.algorithm - INFO   - ==> Evaluation stats: {'loss': 'nan', 'AccuracyBoth': '0.8107', 'AccuracyBase': '1.6213', 'AccuracyNovel': '20.0000', 'AccuracyNovel_std': '0.0000', 'AccuracyNovel_cnf': '0.0000'}
    304. 2022-08-25 02:18:38,812 - bf3s.algorithms.algorithm - INFO   - ==> Best results w.r.t. AccuracyNovel metric: epoch: 1 - {'loss': '4.2341', 'AccuracyBoth': '0.8107', 'AccuracyBase': '1.6213', 'AccuracyNovel': '20.0000', 'AccuracyNovel_std': '0.0000', 'AccuracyNovel_cnf': '0.0000'}
    305. 2022-08-25 02:18:38,813 - bf3s.algorithms.algorithm - INFO   - Training epoch [  4 /  26]
    306. 2022-08-25 02:18:38,813 - bf3s.algorithms.algorithm - INFO   - ==> Set to feature_extractor optimizer lr = 0.1000000000
    307. 2022-08-25 02:18:38,813 - bf3s.algorithms.algorithm - INFO   - ==> Set to classifier optimizer lr = 0.1000000000
    308. 2022-08-25 02:18:38,813 - bf3s.algorithms.algorithm - INFO   - ==> Set to classifier_aux optimizer lr = 0.1000000000
    309. 2022-08-25 02:18:38,813 - bf3s.algorithms.algorithm - INFO   - Training: WRNd28w10CosineClassifierRotAugRotSelfsupervision
    310. 2022-08-25 02:18:38,814 - bf3s.algorithms.algorithm - INFO   - ==> Dataset: MiniImageNet_train [1000 batches]
    311.  20%|███████████████▉                                                                | 199/1000 [19:25<1:18:24,  5.87s/it]2022-08-25 02:38:09,951 - bf3s.algorithms.algorithm - INFO   - ==> Iteration [  4][ 200 / 1000]: {'loss_cls': '4.1589', 'loss_rot': '0.8806', 'loss_total': '5.0395', 'Accuracy': '1.6406', 'AccuracyRot': '50.6211'}
    312.  40%|████████████████████████████████▋                                                 | 399/1000 [38:59<58:47,  5.87s/it]2022-08-25 02:57:44,382 - bf3s.algorithms.algorithm - INFO   - ==> Iteration [  4][ 400 / 1000]: {'loss_cls': '4.1589', 'loss_rot': '0.8802', 'loss_total': '5.0391', 'Accuracy': '1.6211', 'AccuracyRot': '50.7100'}
    313.  60%|█████████████████████████████████████████████████                                 | 599/1000 [58:34<39:14,  5.87s/it]2022-08-25 03:17:18,876 - bf3s.algorithms.algorithm - INFO   - ==> Iteration [  4][ 600 / 1000]: {'loss_cls': '4.1589', 'loss_rot': '0.8802', 'loss_total': '5.0391', 'Accuracy': '1.5547', 'AccuracyRot': '50.6224'}
    314.  80%|███████████████████████████████████████████████████████████████▉                | 799/1000 [1:18:08<19:39,  5.87s/it]2022-08-25 03:36:53,327 - bf3s.algorithms.algorithm - INFO   - ==> Iteration [  4][ 800 / 1000]: {'loss_cls': '4.1589', 'loss_rot': '0.8800', 'loss_total': '5.0389', 'Accuracy': '1.6055', 'AccuracyRot': '50.5508'}
    315. 100%|███████████████████████████████████████████████████████████████████████████████▉| 999/1000 [1:37:43<00:05,  5.87s/it]2022-08-25 03:56:27,810 - bf3s.algorithms.algorithm - INFO   - ==> Iteration [  4][1000 / 1000]: {'loss_cls': '4.1589', 'loss_rot': '0.8800', 'loss_total': '5.0389', 'Accuracy': '1.5609', 'AccuracyRot': '50.4820'}
    316. 100%|███████████████████████████████████████████████████████████████████████████████| 1000/1000 [1:37:48<00:00,  5.87s/it]
    317. 2022-08-25 03:56:27,811 - bf3s.algorithms.algorithm - INFO   - ==> Training stats: {'loss_cls': '4.1589', 'loss_rot': '0.8800', 'loss_total': '5.0389', 'Accuracy': '1.5609', 'AccuracyRot': '50.4820'}
    318. 2022-08-25 03:56:31,390 - bf3s.algorithms.algorithm - INFO   - Evaluating: WRNd28w10CosineClassifierRotAugRotSelfsupervision
    319. 2022-08-25 03:56:31,390 - bf3s.algorithms.algorithm - INFO   - ==> Dataset: MiniImageNet_val [500 batches]
    320. 100%|███████████████████████████████████████████████████████████████████████████████████| 500/500 [08:43<00:00,  1.05s/it]
    321. 2022-08-25 04:05:15,021 - bf3s.algorithms.algorithm - INFO   - ==> Results: {'loss': 'nan', 'AccuracyBoth': '0.8107', 'AccuracyBase': '1.6213', 'AccuracyNovel': '20.0000', 'AccuracyNovel_std': '0.0000', 'AccuracyNovel_cnf': '0.0000'}
    322. NaN or Inf found in input tensor.
    323. 2022-08-25 04:05:15,022 - bf3s.algorithms.algorithm - INFO   - ==> Evaluation stats: {'loss': 'nan', 'AccuracyBoth': '0.8107', 'AccuracyBase': '1.6213', 'AccuracyNovel': '20.0000', 'AccuracyNovel_std': '0.0000', 'AccuracyNovel_cnf': '0.0000'}
    324. 2022-08-25 04:05:15,023 - bf3s.algorithms.algorithm - INFO   - ==> Best results w.r.t. AccuracyNovel metric: epoch: 1 - {'loss': '4.2341', 'AccuracyBoth': '0.8107', 'AccuracyBase': '1.6213', 'AccuracyNovel': '20.0000', 'AccuracyNovel_std': '0.0000', 'AccuracyNovel_cnf': '0.0000'}
    325. 2022-08-25 04:05:15,023 - bf3s.algorithms.algorithm - INFO   - Training epoch [  5 /  26]
    326. 2022-08-25 04:05:15,023 - bf3s.algorithms.algorithm - INFO   - ==> Set to feature_extractor optimizer lr = 0.1000000000
    327. 2022-08-25 04:05:15,023 - bf3s.algorithms.algorithm - INFO   - ==> Set to classifier optimizer lr = 0.1000000000
    328. 2022-08-25 04:05:15,023 - bf3s.algorithms.algorithm - INFO   - ==> Set to classifier_aux optimizer lr = 0.1000000000
    329. 2022-08-25 04:05:15,024 - bf3s.algorithms.algorithm - INFO   - Training: WRNd28w10CosineClassifierRotAugRotSelfsupervision
    330. 2022-08-25 04:05:15,024 - bf3s.algorithms.algorithm - INFO   - ==> Dataset: MiniImageNet_train [1000 batches]
    331.  20%|███████████████▉                                                                | 199/1000 [19:25<1:18:23,  5.87s/it]2022-08-25 04:24:46,183 - bf3s.algorithms.algorithm - INFO   - ==> Iteration [  5][ 200 / 1000]: {'loss_cls': '4.1589', 'loss_rot': '0.8810', 'loss_total': '5.0398', 'Accuracy': '1.4141', 'AccuracyRot': '50.5742'}
    332.  40%|████████████████████████████████▋                                                 | 399/1000 [38:59<58:50,  5.87s/it]2022-08-25 04:44:20,688 - bf3s.algorithms.algorithm - INFO   - ==> Iteration [  5][ 400 / 1000]: {'loss_cls': '4.1589', 'loss_rot': '0.8805', 'loss_total': '5.0393', 'Accuracy': '1.4414', 'AccuracyRot': '50.4531'}
    333.  60%|█████████████████████████████████████████████████                                 | 599/1000 [58:34<39:14,  5.87s/it]2022-08-25 05:03:55,117 - bf3s.algorithms.algorithm - INFO   - ==> Iteration [  5][ 600 / 1000]: {'loss_cls': '4.1589', 'loss_rot': '0.8801', 'loss_total': '5.0390', 'Accuracy': '1.4661', 'AccuracyRot': '50.4648'}
    334.  80%|███████████████████████████████████████████████████████████████▉                | 799/1000 [1:18:08<19:41,  5.88s/it]2022-08-25 05:23:29,643 - bf3s.algorithms.algorithm - INFO   - ==> Iteration [  5][ 800 / 1000]: {'loss_cls': '4.1589', 'loss_rot': '0.8800', 'loss_total': '5.0388', 'Accuracy': '1.5000', 'AccuracyRot': '50.4761'}
    335. 100%|███████████████████████████████████████████████████████████████████████████████▉| 999/1000 [1:37:43<00:05,  5.87s/it]2022-08-25 05:43:04,090 - bf3s.algorithms.algorithm - INFO   - ==> Iteration [  5][1000 / 1000]: {'loss_cls': '4.1589', 'loss_rot': '0.8799', 'loss_total': '5.0387', 'Accuracy': '1.4969', 'AccuracyRot': '50.5203'}
    336. 100%|███████████████████████████████████████████████████████████████████████████████| 1000/1000 [1:37:49<00:00,  5.87s/it]
    337. 2022-08-25 05:43:04,091 - bf3s.algorithms.algorithm - INFO   - ==> Training stats: {'loss_cls': '4.1589', 'loss_rot': '0.8799', 'loss_total': '5.0387', 'Accuracy': '1.4969', 'AccuracyRot': '50.5203'}
    338. 2022-08-25 05:43:07,710 - bf3s.algorithms.algorithm - INFO   - Evaluating: WRNd28w10CosineClassifierRotAugRotSelfsupervision
    339. 2022-08-25 05:43:07,710 - bf3s.algorithms.algorithm - INFO   - ==> Dataset: MiniImageNet_val [500 batches]
    340. 100%|███████████████████████████████████████████████████████████████████████████████████| 500/500 [08:46<00:00,  1.05s/it]
    341. 2022-08-25 05:51:53,727 - bf3s.algorithms.algorithm - INFO   - ==> Results: {'loss': 'nan', 'AccuracyBoth': '0.8107', 'AccuracyBase': '1.6213', 'AccuracyNovel': '20.0000', 'AccuracyNovel_std': '0.0000', 'AccuracyNovel_cnf': '0.0000'}
    342. NaN or Inf found in input tensor.
    343. 2022-08-25 05:51:53,728 - bf3s.algorithms.algorithm - INFO   - ==> Evaluation stats: {'loss': 'nan', 'AccuracyBoth': '0.8107', 'AccuracyBase': '1.6213', 'AccuracyNovel': '20.0000', 'AccuracyNovel_std': '0.0000', 'AccuracyNovel_cnf': '0.0000'}
    344. 2022-08-25 05:51:53,729 - bf3s.algorithms.algorithm - INFO   - ==> Best results w.r.t. AccuracyNovel metric: epoch: 1 - {'loss': '4.2341', 'AccuracyBoth': '0.8107', 'AccuracyBase': '1.6213', 'AccuracyNovel': '20.0000', 'AccuracyNovel_std': '0.0000', 'AccuracyNovel_cnf': '0.0000'}
    345. 2022-08-25 05:51:53,729 - bf3s.algorithms.algorithm - INFO   - Training epoch [  6 /  26]
    346. 2022-08-25 05:51:53,729 - bf3s.algorithms.algorithm - INFO   - ==> Set to feature_extractor optimizer lr = 0.1000000000
    347. 2022-08-25 05:51:53,729 - bf3s.algorithms.algorithm - INFO   - ==> Set to classifier optimizer lr = 0.1000000000
    348. 2022-08-25 05:51:53,729 - bf3s.algorithms.algorithm - INFO   - ==> Set to classifier_aux optimizer lr = 0.1000000000
    349. 2022-08-25 05:51:53,729 - bf3s.algorithms.algorithm - INFO   - Training: WRNd28w10CosineClassifierRotAugRotSelfsupervision
    350. 2022-08-25 05:51:53,730 - bf3s.algorithms.algorithm - INFO   - ==> Dataset: MiniImageNet_train [1000 batches]
    351.  20%|███████████████▉                                                                | 199/1000 [19:25<1:18:26,  5.88s/it]2022-08-25 06:11:24,909 - bf3s.algorithms.algorithm - INFO   - ==> Iteration [  6][ 200 / 1000]: {'loss_cls': '4.1589', 'loss_rot': '0.8808', 'loss_total': '5.0397', 'Accuracy': '1.6875', 'AccuracyRot': '50.5273'}
    352.  40%|████████████████████████████████▋                                                 | 399/1000 [38:59<58:48,  5.87s/it]2022-08-25 06:30:59,362 - bf3s.algorithms.algorithm - INFO   - ==> Iteration [  6][ 400 / 1000]: {'loss_cls': '4.1589', 'loss_rot': '0.8799', 'loss_total': '5.0388', 'Accuracy': '1.6445', 'AccuracyRot': '50.4277'}
    353.  60%|█████████████████████████████████████████████████                                 | 599/1000 [58:32<39:15,  5.87s/it]2022-08-25 06:50:32,443 - bf3s.algorithms.algorithm - INFO   - ==> Iteration [  6][ 600 / 1000]: {'loss_cls': '4.1589', 'loss_rot': '0.8795', 'loss_total': '5.0384', 'Accuracy': '1.5911', 'AccuracyRot': '50.5195'}
    354.  80%|███████████████████████████████████████████████████████████████▉                | 799/1000 [1:18:07<19:40,  5.87s/it]2022-08-25 07:10:06,814 - bf3s.algorithms.algorithm - INFO   - ==> Iteration [  6][ 800 / 1000]: {'loss_cls': '4.1589', 'loss_rot': '0.8794', 'loss_total': '5.0382', 'Accuracy': '1.5820', 'AccuracyRot': '50.5464'}
    355. 100%|███████████████████████████████████████████████████████████████████████████████▉| 999/1000 [1:37:41<00:05,  5.87s/it]2022-08-25 07:29:41,270 - bf3s.algorithms.algorithm - INFO   - ==> Iteration [  6][1000 / 1000]: {'loss_cls': '4.1589', 'loss_rot': '0.8792', 'loss_total': '5.0381', 'Accuracy': '1.5641', 'AccuracyRot': '50.5418'}
    356. 100%|███████████████████████████████████████████████████████████████████████████████| 1000/1000 [1:37:47<00:00,  5.87s/it]
    357. 2022-08-25 07:29:41,271 - bf3s.algorithms.algorithm - INFO   - ==> Training stats: {'loss_cls': '4.1589', 'loss_rot': '0.8792', 'loss_total': '5.0381', 'Accuracy': '1.5641', 'AccuracyRot': '50.5418'}
    358. 2022-08-25 07:29:44,885 - bf3s.algorithms.algorithm - INFO   - Evaluating: WRNd28w10CosineClassifierRotAugRotSelfsupervision
    359. 2022-08-25 07:29:44,885 - bf3s.algorithms.algorithm - INFO   - ==> Dataset: MiniImageNet_val [500 batches]
    360. 100%|███████████████████████████████████████████████████████████████████████████████████| 500/500 [08:45<00:00,  1.05s/it]
    361. 2022-08-25 07:38:30,437 - bf3s.algorithms.algorithm - INFO   - ==> Results: {'loss': 'nan', 'AccuracyBoth': '0.8107', 'AccuracyBase': '1.6213', 'AccuracyNovel': '20.0000', 'AccuracyNovel_std': '0.0000', 'AccuracyNovel_cnf': '0.0000'}
    362. NaN or Inf found in input tensor.
    363. 2022-08-25 07:38:30,438 - bf3s.algorithms.algorithm - INFO   - ==> Evaluation stats: {'loss': 'nan', 'AccuracyBoth': '0.8107', 'AccuracyBase': '1.6213', 'AccuracyNovel': '20.0000', 'AccuracyNovel_std': '0.0000', 'AccuracyNovel_cnf': '0.0000'}
    364. 2022-08-25 07:38:30,438 - bf3s.algorithms.algorithm - INFO   - ==> Best results w.r.t. AccuracyNovel metric: epoch: 1 - {'loss': '4.2341', 'AccuracyBoth': '0.8107', 'AccuracyBase': '1.6213', 'AccuracyNovel': '20.0000', 'AccuracyNovel_std': '0.0000', 'AccuracyNovel_cnf': '0.0000'}
    365. 2022-08-25 07:38:30,438 - bf3s.algorithms.algorithm - INFO   - Training epoch [  7 /  26]
    366. 2022-08-25 07:38:30,439 - bf3s.algorithms.algorithm - INFO   - ==> Set to feature_extractor optimizer lr = 0.1000000000
    367. 2022-08-25 07:38:30,439 - bf3s.algorithms.algorithm - INFO   - ==> Set to classifier optimizer lr = 0.1000000000
    368. 2022-08-25 07:38:30,439 - bf3s.algorithms.algorithm - INFO   - ==> Set to classifier_aux optimizer lr = 0.1000000000
    369. 2022-08-25 07:38:30,439 - bf3s.algorithms.algorithm - INFO   - Training: WRNd28w10CosineClassifierRotAugRotSelfsupervision
    370. 2022-08-25 07:38:30,440 - bf3s.algorithms.algorithm - INFO   - ==> Dataset: MiniImageNet_train [1000 batches]
    371.  20%|███████████████▉                                                                | 199/1000 [19:23<1:18:18,  5.87s/it]2022-08-25 07:58:00,266 - bf3s.algorithms.algorithm - INFO   - ==> Iteration [  7][ 200 / 1000]: {'loss_cls': '4.1589', 'loss_rot': '0.8805', 'loss_total': '5.0394', 'Accuracy': '1.6953', 'AccuracyRot': '50.3262'}
    372.  34%|███████████████████████████▌

     

    MiniImageNet: CC model with rotation prediction self-supervision and exploiting unlabeled images from tiered-MiniImageNet.

    To train and test the WRN-28-10 based CC few-shot model with rotation prediction self-supervision which also exploits (with the semi-supervised learning setting) unlabeled images from tiered-MiniImageNet, run (you can skip the training step if you have downloaded the pre-trained model):

    # Run from the BF3S directory
# Train the model.
$ python scripts/train_fewshot.py --config=miniImageNet/WRNd28w10CosineClassifierRotAugRotSelfsupervision_SemisupervisedTieredUnlabeled

# Test the model on the 1-shot setting.
$ python scripts/test_fewshot.py --config=miniImageNet/WRNd28w10CosineClassifierRotAugRotSelfsupervision_SemisupervisedTieredUnlabeled --num_novel=5 --num_train=1 --num_episodes=2000
# Expected 5-way classification accuracy: 64.03% with confidence interval +/- 0.46%

# Test the model on the 5-shot setting.
$ python scripts/test_fewshot.py --config=miniImageNet/WRNd28w10CosineClassifierRotAugRotSelfsupervision_SemisupervisedTieredUnlabeled --num_novel=5 --num_train=5 --num_episodes=2000
# Expected 5-way classification accuracy: 80.68% with confidence interval +/- 0.33%

    MiniImageNet: CC model with relative patch location prediction self-supervision.

    To train and test the WRN-28-10 based Cosine Classifier few-shot model with relative patch location prediction self-supervision run (you can skip the training step if you have downloaded the pre-trained model):

    # Run from the BF3S directory
# Train the model.
$ python scripts/train_fewshot.py --config=miniImageNet/WRNd28w10CosineClassifierLocSelfsupervision

# Test the model on the 1-shot setting.
$ python scripts/test_fewshot.py --config=miniImageNet/WRNd28w10CosineClassifierLocSelfsupervision --num_novel=5 --num_train=1 --num_episodes=2000
# Expected 5-way classification accuracy: 60.70% with confidence interval +/- 0.47%

# Test the model on the 5-shot setting.
$ python scripts/test_fewshot.py --config=miniImageNet/WRNd28w10CosineClassifierLocSelfsupervision --num_novel=5 --num_train=5 --num_episodes=2000
# Expected 5-way classification accuracy: 77.61% with confidence interval +/- 0.33%

    CIFAR-FS: CC model with rotation prediction self-supervision.

    To train and test the WRN-28-10 based Cosine Classifier few-shot model with rotation prediction self-supervision and rotation augmentations run (you can skip the training step if you have downloaded the pre-trained model):

    # Run from the BF3S directory
# Train the model.
$ python scripts/train_fewshot.py --config=cifar/WRNd28w10CosineClassifierRotAugRotSelfsupervision

# Test the model on the 1-shot setting.
$ python scripts/test_fewshot.py --config=cifar/WRNd28w10CosineClassifierRotAugRotSelfsupervision --num_novel=5 --num_train=1 --num_episodes=5000
# Expected 5-way classification accuracy: 75.38% with confidence interval +/- 0.31%

# Test the model on the 5-shot setting.
$ python scripts/test_fewshot.py --config=cifar/WRNd28w10CosineClassifierRotAugRotSelfsupervision --num_novel=5 --num_train=5 --num_episodes=5000
# Expected 5-way classification accuracy: 87.25% with confidence interval +/- 0.21%

    ImageNet-FS: CC model with rotation prediction self-supervision.

    Instructions for training and testing the CC few-shot model with rotation prediction self-supervision on the ImageNet based few-shot benchmark [1, 2].

    (1) To train the ResNet10 based Cosine Classifier few-shot model with rotation prediction self-supervision and rotation augmentations run:

    # Run from the BF3S directory
$ python scripts/train_fewshot.py --config=ImageNet/ResNet10CosineClassifierRotSelfsupervision --num_workers=8

    You can skip the above training step if you have downloaded the pre-trained model.

    (2) Extract and save the ResNet10 features (with the above model) from images of the ImageNet dataset:

    # Run from the BF3S directory
# Extract features from the validation image split of the Imagenet.
$ python scripts/save_features_imagenet.py --config=ImageNet/ResNet10CosineClassifierRotSelfsupervision --split='val'
# Extract features from the training image split of the Imagenet.
$ python scripts/save_features_imagenet.py with config=ImageNet/ResNet10CosineClassifierRotSelfsupervision --split='train'

    The features will be saved on ./datasets/feature_datasets/ImageNet/ResNet10CosineClassifierRotSelfsupervision.

    (4) Test the model:

    # Run from the BF3S directory
# Test the CC+Rot model on the 1-shot setting.
$ python scripts/test_fewshot_imagenet.py --config=ImageNet/ResNet10CosineClassifierRotSelfsupervision_eval --testset --num_train=1 --bias_novel=0.8
# ==> Top 5 Accuracies:      [Novel: 46.43 | Base: 93.52 | All 57.88]

# Test the CC+Rot model on the 2-shot setting.
$ python scripts/test_fewshot_imagenet.py --config=ImageNet/ResNet10CosineClassifierRotSelfsupervision_eval --testset --num_train=2 --bias_novel=0.75
# ==> Top 5 Accuracies:      [Novel: 57.80 | Base: 93.52 | All 64.76]

# Test the CC+Rot model on the 5-shot setting.
$ python scripts/test_fewshot_imagenet.py --config=ImageNet/ResNet10CosineClassifierRotSelfsupervision_eval --testset --num_train=5 --bias_novel=0.7
# ==> Top 5 Accuracies:      [Novel: 69.67 | Base: 93.52 | All 72.29]

# Test the CC+Rot model on the 10-shot setting.
$ python scripts/test_fewshot_imagenet.py --config=ImageNet/ResNet10CosineClassifierRotSelfsupervision_eval --testset --num_train=10 --bias_novel=0.65
# ==> Top 5 Accuracies:      [Novel: 74.64 | Base: 93.52 | All 75.63]

# Test the CC+Rot model on the 20-shot setting.
$ python scripts/test_fewshot_imagenet.py --config=ImageNet/ResNet10CosineClassifierRotSelfsupervision_eval --testset --num_train=20 --bias_novel=0.6
# ==> Top 5 Accuracies:      [Novel: 77.31 | Base: 93.52 | All 77.40]

    Note that here, to evaluate the model trained with the ./config/ImageNet/ResNet10CosineClassifierRotSelfsupervision.py config file, we used a different config file named ./config/ImageNet/ResNet10CosineClassifierRotSelfsupervision_eval.py. Also, the --bias_novel term specifies a multiplicative bias for the classification scores of the novel classes. Its purpose is to balance the classification scores of the base and novel classes (necessary since the classifiers for those two different classes are trained in different ways and different stages). It only affects the All classification accuracy metrics. The used bias values were tuned on the validation split.

  • 相关阅读:
    图片转换成base64格式的优缺点
    “互联网+”谋定现代农业-国稻种芯-万祥军：产业体系提升农业
    刘二大人 PyTorch深度学习实践 笔记 P3 梯度下降算法
    一种SIP和WebRTC互相通信的方案
    链表相关OJ及方法总结
    Nginx中关于虚拟主机的一点冷门知识
    注意 Laravel 清除缓存 php artisan cache:clear 的一个坑
    【广州华锐互动】VR历史古城复原：沉浸式体验古代建筑，感受千年风华！
    java 替换list中值的方法分享
    vm options、program arguments、environment property
  • 原文地址：https://blog.csdn.net/weixin_43135178/article/details/126512962