• mmpose系列 (一)：hrnet 基于mmpose 训练body+foot 23点关键点

    首先Git clone mmpose:  GitHub - open-mmlab/mmpose: OpenMMLab Pose Estimation Toolbox and Benchmark.

    然后根据github  安装依赖环境，这里不再赘述。

    背景

    官方的coco keypoints 是基于17点计算的，coco也给出了coco_whole_body的133个关键点

    目的是与openpose 2019的25点作对比，但是目前貌似没有人去复现openpose2019的，本人正在复现的过程的中，后续会给出。但是官方给出的caffemodel 我千方百计的去计算它的mAP 也不停的查阅相关论文。

    最后算出的mAP为52.3% 和官方给出的数据差异太大。仔细看看，论文中计算mAP的数据集并不是我测试的基于val2017 全量，而是东抽西抽的coco数据集的子集。

     并且看到一片文章在coco_whole_body中计算body的mAP为56.3%

     同时我也测了openpose在cocowholebody 中的表现:body为50.2 。

    精度很低，但是我需要body+foot的关键点。

    基于以上背景，改造23点。

    hrnet body+foot

    回到mmpose，hrnet 有在coco_whole_body 上进行训练，以及有它的配置文件：

    1. _base_ = [
    2.     '../../../../_base_/default_runtime.py',
    3.     '../../../../_base_/datasets/coco_wholebody.py'
    4. ]
    5. evaluation = dict(interval=10, metric='mAP', save_best='AP')
    6.  
    7. optimizer = dict(
    8.     type='Adam',
    9.     lr=5e-4,
    10. )
    11. optimizer_config = dict(grad_clip=None)
    12. # learning policy
    13. lr_config = dict(
    14.     policy='step',
    15.     warmup=None,
    16.     # warmup='linear',
    17.     # warmup_iters=500,
    18.     # warmup_ratio=0.001,
    19.     step=[170, 200])
    20. total_epochs = 210
    21. channel_cfg = dict(
    22.     num_output_channels=133,
    23.     dataset_joints=133,
    24.     dataset_channel=[
    25.         list(range(133)),
    26.     ],
    27.     inference_channel=list(range(133)))
    28.  
    29. # model settings
    30. model = dict(
    31.     type='TopDown',
    32.     pretrained='https://download.openmmlab.com/mmpose/'
    33.     'pretrain_models/hrnet_w48-8ef0771d.pth',
    34.     backbone=dict(
    35.         type='HRNet',
    36.         in_channels=3,
    37.         extra=dict(
    38.             stage1=dict(
    39.                 num_modules=1,
    40.                 num_branches=1,
    41.                 block='BOTTLENECK',
    42.                 num_blocks=(4, ),
    43.                 num_channels=(64, )),
    44.             stage2=dict(
    45.                 num_modules=1,
    46.                 num_branches=2,
    47.                 block='BASIC',
    48.                 num_blocks=(4, 4),
    49.                 num_channels=(48, 96)),
    50.             stage3=dict(
    51.                 num_modules=4,
    52.                 num_branches=3,
    53.                 block='BASIC',
    54.                 num_blocks=(4, 4, 4),
    55.                 num_channels=(48, 96, 192)),
    56.             stage4=dict(
    57.                 num_modules=3,
    58.                 num_branches=4,
    59.                 block='BASIC',
    60.                 num_blocks=(4, 4, 4, 4),
    61.                 num_channels=(48, 96, 192, 384))),
    62.     ),
    63.     keypoint_head=dict(
    64.         type='TopdownHeatmapSimpleHead',
    65.         in_channels=48,
    66.         out_channels=channel_cfg['num_output_channels'],
    67.         num_deconv_layers=0,
    68.         extra=dict(final_conv_kernel=1, ),
    69.         loss_keypoint=dict(type='JointsMSELoss', use_target_weight=True)),
    70.     train_cfg=dict(),
    71.     test_cfg=dict(
    72.         flip_test=True,
    73.         post_process='default',
    74.         shift_heatmap=True,
    75.         modulate_kernel=11))
    76.  
    77. data_cfg = dict(
    78.     image_size=[288, 384],
    79.     heatmap_size=[72, 96],
    80.     num_output_channels=channel_cfg['num_output_channels'],
    81.     num_joints=channel_cfg['dataset_joints'],
    82.     dataset_channel=channel_cfg['dataset_channel'],
    83.     inference_channel=channel_cfg['inference_channel'],
    84.     soft_nms=False,
    85.     nms_thr=1.0,
    86.     oks_thr=0.9,
    87.     vis_thr=0.2,
    88.     use_gt_bbox=False,
    89.     det_bbox_thr=0.0,
    90.     bbox_file='data/coco/person_detection_results/'
    91.     'COCO_val2017_detections_AP_H_56_person.json',
    92. )
    93.  
    94. train_pipeline = [
    95.     dict(type='LoadImageFromFile'),
    96.     dict(type='TopDownGetBboxCenterScale', padding=1.25),
    97.     dict(type='TopDownRandomShiftBboxCenter', shift_factor=0.16, prob=0.3),
    98.     dict(type='TopDownRandomFlip', flip_prob=0.5),
    99.     dict(
    100.         type='TopDownHalfBodyTransform',
    101.         num_joints_half_body=8,
    102.         prob_half_body=0.3),
    103.     dict(
    104.         type='TopDownGetRandomScaleRotation', rot_factor=40, scale_factor=0.5),
    105.     dict(type='TopDownAffine'),
    106.     dict(type='ToTensor'),
    107.     dict(
    108.         type='NormalizeTensor',
    109.         mean=[0.485, 0.456, 0.406],
    110.         std=[0.229, 0.224, 0.225]),
    111.     dict(type='TopDownGenerateTarget', sigma=3),
    112.     dict(
    113.         type='Collect',
    114.         keys=['img', 'target', 'target_weight'],
    115.         meta_keys=[
    116.             'image_file', 'joints_3d', 'joints_3d_visible', 'center', 'scale',
    117.             'rotation', 'bbox_score', 'flip_pairs'
    118.         ]),
    119. ]
    120.  
    121. val_pipeline = [
    122.     dict(type='LoadImageFromFile'),
    123.     dict(type='TopDownGetBboxCenterScale', padding=1.25),
    124.     dict(type='TopDownAffine'),
    125.     dict(type='ToTensor'),
    126.     dict(
    127.         type='NormalizeTensor',
    128.         mean=[0.485, 0.456, 0.406],
    129.         std=[0.229, 0.224, 0.225]),
    130.     dict(
    131.         type='Collect',
    132.         keys=['img'],
    133.         meta_keys=[
    134.             'image_file', 'center', 'scale', 'rotation', 'bbox_score',
    135.             'flip_pairs'
    136.         ]),
    137. ]
    138.  
    139. test_pipeline = val_pipeline
    140.  
    141. data_root = 'data/coco'
    142. data = dict(
    143.     samples_per_gpu=32,
    144.     workers_per_gpu=2,
    145.     val_dataloader=dict(samples_per_gpu=32),
    146.     test_dataloader=dict(samples_per_gpu=32),
    147.     train=dict(
    148.         type='TopDownCocoWholeBodyDataset',
    149.         ann_file=f'{data_root}/annotations/coco_wholebody_train_v1.0.json',
    150.         img_prefix=f'{data_root}/train2017/',
    151.         data_cfg=data_cfg,
    152.         pipeline=train_pipeline,
    153.         dataset_info={{_base_.dataset_info}}),
    154.     val=dict(
    155.         type='TopDownCocoWholeBodyDataset',
    156.         ann_file=f'{data_root}/annotations/coco_wholebody_val_v1.0.json',
    157.         img_prefix=f'{data_root}/val2017/',
    158.         data_cfg=data_cfg,
    159.         pipeline=val_pipeline,
    160.         dataset_info={{_base_.dataset_info}}),
    161.     test=dict(
    162.         type='TopDownCocoWholeBodyDataset',
    163.         ann_file=f'{data_root}/annotations/coco_wholebody_val_v1.0.json',
    164.         img_prefix=f'{data_root}/val2017/',
    165.         data_cfg=data_cfg,
    166.         pipeline=test_pipeline,
    167.         dataset_info={{_base_.dataset_info}}),
    168. )

    但他是基于133个关键点的，但是分为了 face body hand foot 几个部分，那么就简单了，将数据集的配置文件改为body+foot，并把hrnet 的配置文件也改为body+foot，同时计算mAP的方式也要更改为body+foot。在config目录下重新创建一个body23 文件夹，将config文件复制一份。

    一、查看数据集关键点的配置项：

    1. train_pipeline = [
    2.     dict(type='LoadImageFromFile'),
    3.     dict(type='TopDownGetBboxCenterScale', padding=1.25),
    4.     dict(type='TopDownRandomShiftBboxCenter', shift_factor=0.16, prob=0.3),
    5.     dict(type='TopDownRandomFlip', flip_prob=0.5),
    6.     dict(
    7.         type='TopDownHalfBodyTransform',
    8.         num_joints_half_body=8,
    9.         prob_half_body=0.3),
    10.     dict(
    11.         type='TopDownGetRandomScaleRotation', rot_factor=40, scale_factor=0.5),
    12.     dict(type='TopDownAffine'),
    13.     dict(type='ToTensor'),
    14.     dict(
    15.         type='NormalizeTensor',
    16.         mean=[0.485, 0.456, 0.406],
    17.         std=[0.229, 0.224, 0.225]),
    18.     dict(type='TopDownGenerateTarget', sigma=3),
    19.     dict(
    20.         type='Collect',
    21.         keys=['img', 'target', 'target_weight'],
    22.         meta_keys=[
    23.             'image_file', 'joints_3d', 'joints_3d_visible', 'center', 'scale',
    24.             'rotation', 'bbox_score', 'flip_pairs'
    25.         ]),
    26. ]
    27.  
    28. val_pipeline = [
    29.     dict(type='LoadImageFromFile'),
    30.     dict(type='TopDownGetBboxCenterScale', padding=1.25),
    31.     dict(type='TopDownAffine'),
    32.     dict(type='ToTensor'),
    33.     dict(
    34.         type='NormalizeTensor',
    35.         mean=[0.485, 0.456, 0.406],
    36.         std=[0.229, 0.224, 0.225]),
    37.     dict(
    38.         type='Collect',
    39.         keys=['img'],
    40.         meta_keys=[
    41.             'image_file', 'center', 'scale', 'rotation', 'bbox_score',
    42.             'flip_pairs'
    43.         ]),
    44. ]
    45.  
    46. test_pipeline = val_pipeline
    47. data_root = 'data/coco'
    48. data = dict(
    49.     samples_per_gpu=32,
    50.     workers_per_gpu=2,
    51.     val_dataloader=dict(samples_per_gpu=32),
    52.     test_dataloader=dict(samples_per_gpu=32),
    53.     train=dict(
    54.         type='TopDownCocoWholeBodyDataset',
    55.         ann_file=f'{data_root}/annotations/coco_wholebody_train_v1.0.json',
    56.         img_prefix=f'{data_root}/train2017/',
    57.         data_cfg=data_cfg,
    58.         pipeline=train_pipeline,
    59.         dataset_info={{_base_.dataset_info}}),
    60.     val=dict(
    61.         type='TopDownCocoWholeBodyDataset',
    62.         ann_file=f'{data_root}/annotations/coco_wholebody_val_v1.0.json',
    63.         img_prefix=f'{data_root}/val2017/',
    64.         data_cfg=data_cfg,
    65.         pipeline=val_pipeline,
    66.         dataset_info={{_base_.dataset_info}}),
    67.     test=dict(
    68.         type='TopDownCocoWholeBodyDataset',
    69.         ann_file=f'{data_root}/annotations/coco_wholebody_val_v1.0.json',
    70.         img_prefix=f'{data_root}/val2017/',
    71.         data_cfg=data_cfg,
    72.         pipeline=test_pipeline,
    73.         dataset_info={{_base_.dataset_info}}),
    74. )

    这部分是pipeline的配置不需要改动。查看dataload部分也是数据集最关键的部分，也就是代码的最上方 base项：

    1. _base_ = [
    2.     '../../../../_base_/default_runtime.py',
    3.     '../../../../_base_/datasets/coco_wholebody.py'
    4. ]

    找到coco_wholebody.py文件

    1. dataset_info = dict(
    2.     dataset_name='coco_wholebody',
    3.     paper_info=dict(
    4.         author='Jin, Sheng and Xu, Lumin and Xu, Jin and '
    5.         'Wang, Can and Liu, Wentao and '
    6.         'Qian, Chen and Ouyang, Wanli and Luo, Ping',
    7.         title='Whole-Body Human Pose Estimation in the Wild',
    8.         container='Proceedings of the European '
    9.         'Conference on Computer Vision (ECCV)',
    10.         year='2020',
    11.         homepage='https://github.com/jin-s13/COCO-WholeBody/',
    12.     ),
    13.     keypoint_info={
    14.         0:
    15.         dict(name='nose', id=0, color=[51, 153, 255], type='upper', swap=''),
    16.         1:
    17.         dict(
    18.             name='left_eye',
    19.             id=1,
    20.             color=[51, 153, 255],
    21.             type='upper',
    22.             swap='right_eye'),
    23.         2:
    24.         dict(
    25.             name='right_eye',
    26.             id=2,
    27.             color=[51, 153, 255],
    28.             type='upper',
    29.             swap='left_eye'),
    30.         3:
    31.         dict(
    32.             name='left_ear',
    33.             id=3,
    34.             color=[51, 153, 255],
    35.             type='upper',
    36.             swap='right_ear'),
    37.         4:
    38.         dict(
    39.             name='right_ear',
    40.             id=4,
    41.             color=[51, 153, 255],
    42.             type='upper',
    43.             swap='left_ear'),
    44.         5:
    45.         dict(
    46.             name='left_shoulder',
    47.             id=5,
    48.             color=[0, 255, 0],
    49.             type='upper',
    50.             swap='right_shoulder'),
    51.         6:
    52.         dict(
    53.             name='right_shoulder',
    54.             id=6,
    55.             color=[255, 128, 0],
    56.             type='upper',
    57.             swap='left_shoulder'),
    58.         7:
    59.         dict(
    60.             name='left_elbow',
    61.             id=7,
    62.             color=[0, 255, 0],
    63.             type='upper',
    64.             swap='right_elbow'),
    65.         8:
    66.         dict(
    67.             name='right_elbow',
    68.             id=8,
    69.             color=[255, 128, 0],
    70.             type='upper',
    71.             swap='left_elbow'),
    72.         9:
    73.         dict(
    74.             name='left_wrist',
    75.             id=9,
    76.             color=[0, 255, 0],
    77.             type='upper',
    78.             swap='right_wrist'),
    79.         10:
    80.         dict(
    81.             name='right_wrist',
    82.             id=10,
    83.             color=[255, 128, 0],
    84.             type='upper',
    85.             swap='left_wrist'),
    86.         11:
    87.         dict(
    88.             name='left_hip',
    89.             id=11,
    90.             color=[0, 255, 0],
    91.             type='lower',
    92.             swap='right_hip'),
    93.         12:
    94.         dict(
    95.             name='right_hip',
    96.             id=12,
    97.             color=[255, 128, 0],
    98.             type='lower',
    99.             swap='left_hip'),
    100.         13:
    101.         dict(
    102.             name='left_knee',
    103.             id=13,
    104.             color=[0, 255, 0],
    105.             type='lower',
    106.             swap='right_knee'),
    107.         14:
    108.         dict(
    109.             name='right_knee',
    110.             id=14,
    111.             color=[255, 128, 0],
    112.             type='lower',
    113.             swap='left_knee'),
    114.         15:
    115.         dict(
    116.             name='left_ankle',
    117.             id=15,
    118.             color=[0, 255, 0],
    119.             type='lower',
    120.             swap='right_ankle'),
    121.         16:
    122.         dict(
    123.             name='right_ankle',
    124.             id=16,
    125.             color=[255, 128, 0],
    126.             type='lower',
    127.             swap='left_ankle'),
    128.         17:
    129.         dict(
    130.             name='left_big_toe',
    131.             id=17,
    132.             color=[255, 128, 0],
    133.             type='lower',
    134.             swap='right_big_toe'),
    135.         18:
    136.         dict(
    137.             name='left_small_toe',
    138.             id=18,
    139.             color=[255, 128, 0],
    140.             type='lower',
    141.             swap='right_small_toe'),
    142.         19:
    143.         dict(
    144.             name='left_heel',
    145.             id=19,
    146.             color=[255, 128, 0],
    147.             type='lower',
    148.             swap='right_heel'),
    149.         20:
    150.         dict(
    151.             name='right_big_toe',
    152.             id=20,
    153.             color=[255, 128, 0],
    154.             type='lower',
    155.             swap='left_big_toe'),
    156.         21:
    157.         dict(
    158.             name='right_small_toe',
    159.             id=21,
    160.             color=[255, 128, 0],
    161.             type='lower',
    162.             swap='left_small_toe'),
    163.         22:
    164.         dict(
    165.             name='right_heel',
    166.             id=22,
    167.             color=[255, 128, 0],
    168.             type='lower',
    169.             swap='left_heel'),
    170.         23:
    171.         dict(
    172.             name='face-0',
    173.             id=23,
    174.             color=[255, 255, 255],
    175.             type='',
    176.             swap='face-16'),
    177.         24:
    178.         dict(
    179.             name='face-1',
    180.             id=24,
    181.             color=[255, 255, 255],
    182.             type='',
    183.             swap='face-15'),
    184.         25:
    185.         dict(
    186.             name='face-2',
    187.             id=25,
    188.             color=[255, 255, 255],
    189.             type='',
    190.             swap='face-14'),
    191.         26:
    192.         dict(
    193.             name='face-3',
    194.             id=26,
    195.             color=[255, 255, 255],
    196.             type='',
    197.             swap='face-13'),
    198.         27:
    199.         dict(
    200.             name='face-4',
    201.             id=27,
    202.             color=[255, 255, 255],
    203.             type='',
    204.             swap='face-12'),
    205.         28:
    206.         dict(
    207.             name='face-5',
    208.             id=28,
    209.             color=[255, 255, 255],
    210.             type='',
    211.             swap='face-11'),
    212.         29:
    213.         dict(
    214.             name='face-6',
    215.             id=29,
    216.             color=[255, 255, 255],
    217.             type='',
    218.             swap='face-10'),
    219.         30:
    220.         dict(
    221.             name='face-7',
    222.             id=30,
    223.             color=[255, 255, 255],
    224.             type='',
    225.             swap='face-9'),
    226.         31:
    227.         dict(name='face-8', id=31, color=[255, 255, 255], type='', swap=''),
    228.         32:
    229.         dict(
    230.             name='face-9',
    231.             id=32,
    232.             color=[255, 255, 255],
    233.             type='',
    234.             swap='face-7'),
    235.         33:
    236.         dict(
    237.             name='face-10',
    238.             id=33,
    239.             color=[255, 255, 255],
    240.             type='',
    241.             swap='face-6'),
    242.         34:
    243.         dict(
    244.             name='face-11',
    245.             id=34,
    246.             color=[255, 255, 255],
    247.             type='',
    248.             swap='face-5'),
    249.         35:
    250.         dict(
    251.             name='face-12',
    252.             id=35,
    253.             color=[255, 255, 255],
    254.             type='',
    255.             swap='face-4'),
    256.         36:
    257.         dict(
    258.             name='face-13',
    259.             id=36,
    260.             color=[255, 255, 255],
    261.             type='',
    262.             swap='face-3'),
    263.         37:
    264.         dict(
    265.             name='face-14',
    266.             id=37,
    267.             color=[255, 255, 255],
    268.             type='',
    269.             swap='face-2'),
    270.         38:
    271.         dict(
    272.             name='face-15',
    273.             id=38,
    274.             color=[255, 255, 255],
    275.             type='',
    276.             swap='face-1'),
    277.         39:
    278.         dict(
    279.             name='face-16',
    280.             id=39,
    281.             color=[255, 255, 255],
    282.             type='',
    283.             swap='face-0'),
    284.         40:
    285.         dict(
    286.             name='face-17',
    287.             id=40,
    288.             color=[255, 255, 255],
    289.             type='',
    290.             swap='face-26'),
    291.         41:
    292.         dict(
    293.             name='face-18',
    294.             id=41,
    295.             color=[255, 255, 255],
    296.             type='',
    297.             swap='face-25'),
    298.         42:
    299.         dict(
    300.             name='face-19',
    301.             id=42,
    302.             color=[255, 255, 255],
    303.             type='',
    304.             swap='face-24'),
    305.         43:
    306.         dict(
    307.             name='face-20',
    308.             id=43,
    309.             color=[255, 255, 255],
    310.             type='',
    311.             swap='face-23'),
    312.         44:
    313.         dict(
    314.             name='face-21',
    315.             id=44,
    316.             color=[255, 255, 255],
    317.             type='',
    318.             swap='face-22'),
    319.         45:
    320.         dict(
    321.             name='face-22',
    322.             id=45,
    323.             color=[255, 255, 255],
    324.             type='',
    325.             swap='face-21'),
    326.         46:
    327.         dict(
    328.             name='face-23',
    329.             id=46,
    330.             color=[255, 255, 255],
    331.             type='',
    332.             swap='face-20'),
    333.         47:
    334.         dict(
    335.             name='face-24',
    336.             id=47,
    337.             color=[255, 255, 255],
    338.             type='',
    339.             swap='face-19'),
    340.         48:
    341.         dict(
    342.             name='face-25',
    343.             id=48,
    344.             color=[255, 255, 255],
    345.             type='',
    346.             swap='face-18'),
    347.         49:
    348.         dict(
    349.             name='face-26',
    350.             id=49,
    351.             color=[255, 255, 255],
    352.             type='',
    353.             swap='face-17'),
    354.         50:
    355.         dict(name='face-27', id=50, color=[255, 255, 255], type='', swap=''),
    356.         51:
    357.         dict(name='face-28', id=51, color=[255, 255, 255], type='', swap=''),
    358.         52:
    359.         dict(name='face-29', id=52, color=[255, 255, 255], type='', swap=''),
    360.         53:
    361.         dict(name='face-30', id=53, color=[255, 255, 255], type='', swap=''),
    362.         54:
    363.         dict(
    364.             name='face-31',
    365.             id=54,
    366.             color=[255, 255, 255],
    367.             type='',
    368.             swap='face-35'),
    369.         55:
    370.         dict(
    371.             name='face-32',
    372.             id=55,
    373.             color=[255, 255, 255],
    374.             type='',
    375.             swap='face-34'),
    376.         56:
    377.         dict(name='face-33', id=56, color=[255, 255, 255], type='', swap=''),
    378.         57:
    379.         dict(
    380.             name='face-34',
    381.             id=57,
    382.             color=[255, 255, 255],
    383.             type='',
    384.             swap='face-32'),
    385.         58:
    386.         dict(
    387.             name='face-35',
    388.             id=58,
    389.             color=[255, 255, 255],
    390.             type='',
    391.             swap='face-31'),
    392.         59:
    393.         dict(
    394.             name='face-36',
    395.             id=59,
    396.             color=[255, 255, 255],
    397.             type='',
    398.             swap='face-45'),
    399.         60:
    400.         dict(
    401.             name='face-37',
    402.             id=60,
    403.             color=[255, 255, 255],
    404.             type='',
    405.             swap='face-44'),
    406.         61:
    407.         dict(
    408.             name='face-38',
    409.             id=61,
    410.             color=[255, 255, 255],
    411.             type='',
    412.             swap='face-43'),
    413.         62:
    414.         dict(
    415.             name='face-39',
    416.             id=62,
    417.             color=[255, 255, 255],
    418.             type='',
    419.             swap='face-42'),
    420.         63:
    421.         dict(
    422.             name='face-40',
    423.             id=63,
    424.             color=[255, 255, 255],
    425.             type='',
    426.             swap='face-47'),
    427.         64:
    428.         dict(
    429.             name='face-41',
    430.             id=64,
    431.             color=[255, 255, 255],
    432.             type='',
    433.             swap='face-46'),
    434.         65:
    435.         dict(
    436.             name='face-42',
    437.             id=65,
    438.             color=[255, 255, 255],
    439.             type='',
    440.             swap='face-39'),
    441.         66:
    442.         dict(
    443.             name='face-43',
    444.             id=66,
    445.             color=[255, 255, 255],
    446.             type='',
    447.             swap='face-38'),
    448.         67:
    449.         dict(
    450.             name='face-44',
    451.             id=67,
    452.             color=[255, 255, 255],
    453.             type='',
    454.             swap='face-37'),
    455.         68:
    456.         dict(
    457.             name='face-45',
    458.             id=68,
    459.             color=[255, 255, 255],
    460.             type='',
    461.             swap='face-36'),
    462.         69:
    463.         dict(
    464.             name='face-46',
    465.             id=69,
    466.             color=[255, 255, 255],
    467.             type='',
    468.             swap='face-41'),
    469.         70:
    470.         dict(
    471.             name='face-47',
    472.             id=70,
    473.             color=[255, 255, 255],
    474.             type='',
    475.             swap='face-40'),
    476.         71:
    477.         dict(
    478.             name='face-48',
    479.             id=71,
    480.             color=[255, 255, 255],
    481.             type='',
    482.             swap='face-54'),
    483.         72:
    484.         dict(
    485.             name='face-49',
    486.             id=72,
    487.             color=[255, 255, 255],
    488.             type='',
    489.             swap='face-53'),
    490.         73:
    491.         dict(
    492.             name='face-50',
    493.             id=73,
    494.             color=[255, 255, 255],
    495.             type='',
    496.             swap='face-52'),
    497.         74:
    498.         dict(name='face-51', id=74, color=[255, 255, 255], type='', swap=''),
    499.         75:
    500.         dict(
    501.             name='face-52',
    502.             id=75,
    503.             color=[255, 255, 255],
    504.             type='',
    505.             swap='face-50'),
    506.         76:
    507.         dict(
    508.             name='face-53',
    509.             id=76,
    510.             color=[255, 255, 255],
    511.             type='',
    512.             swap='face-49'),
    513.         77:
    514.         dict(
    515.             name='face-54',
    516.             id=77,
    517.             color=[255, 255, 255],
    518.             type='',
    519.             swap='face-48'),
    520.         78:
    521.         dict(
    522.             name='face-55',
    523.             id=78,
    524.             color=[255, 255, 255],
    525.             type='',
    526.             swap='face-59'),
    527.         79:
    528.         dict(
    529.             name='face-56',
    530.             id=79,
    531.             color=[255, 255, 255],
    532.             type='',
    533.             swap='face-58'),
    534.         80:
    535.         dict(name='face-57', id=80, color=[255, 255, 255], type='', swap=''),
    536.         81:
    537.         dict(
    538.             name='face-58',
    539.             id=81,
    540.             color=[255, 255, 255],
    541.             type='',
    542.             swap='face-56'),
    543.         82:
    544.         dict(
    545.             name='face-59',
    546.             id=82,
    547.             color=[255, 255, 255],
    548.             type='',
    549.             swap='face-55'),
    550.         83:
    551.         dict(
    552.             name='face-60',
    553.             id=83,
    554.             color=[255, 255, 255],
    555.             type='',
    556.             swap='face-64'),
    557.         84:
    558.         dict(
    559.             name='face-61',
    560.             id=84,
    561.             color=[255, 255, 255],
    562.             type='',
    563.             swap='face-63'),
    564.         85:
    565.         dict(name='face-62', id=85, color=[255, 255, 255], type='', swap=''),
    566.         86:
    567.         dict(
    568.             name='face-63',
    569.             id=86,
    570.             color=[255, 255, 255],
    571.             type='',
    572.             swap='face-61'),
    573.         87:
    574.         dict(
    575.             name='face-64',
    576.             id=87,
    577.             color=[255, 255, 255],
    578.             type='',
    579.             swap='face-60'),
    580.         88:
    581.         dict(
    582.             name='face-65',
    583.             id=88,
    584.             color=[255, 255, 255],
    585.             type='',
    586.             swap='face-67'),
    587.         89:
    588.         dict(name='face-66', id=89, color=[255, 255, 255], type='', swap=''),
    589.         90:
    590.         dict(
    591.             name='face-67',
    592.             id=90,
    593.             color=[255, 255, 255],
    594.             type='',
    595.             swap='face-65'),
    596.         91:
    597.         dict(
    598.             name='left_hand_root',
    599.             id=91,
    600.             color=[255, 255, 255],
    601.             type='',
    602.             swap='right_hand_root'),
    603.         92:
    604.         dict(
    605.             name='left_thumb1',
    606.             id=92,
    607.             color=[255, 128, 0],
    608.             type='',
    609.             swap='right_thumb1'),
    610.         93:
    611.         dict(
    612.             name='left_thumb2',
    613.             id=93,
    614.             color=[255, 128, 0],
    615.             type='',
    616.             swap='right_thumb2'),
    617.         94:
    618.         dict(
    619.             name='left_thumb3',
    620.             id=94,
    621.             color=[255, 128, 0],
    622.             type='',
    623.             swap='right_thumb3'),
    624.         95:
    625.         dict(
    626.             name='left_thumb4',
    627.             id=95,
    628.             color=[255, 128, 0],
    629.             type='',
    630.             swap='right_thumb4'),
    631.         96:
    632.         dict(
    633.             name='left_forefinger1',
    634.             id=96,
    635.             color=[255, 153, 255],
    636.             type='',
    637.             swap='right_forefinger1'),
    638.         97:
    639.         dict(
    640.             name='left_forefinger2',
    641.             id=97,
    642.             color=[255, 153, 255],
    643.             type='',
    644.             swap='right_forefinger2'),
    645.         98:
    646.         dict(
    647.             name='left_forefinger3',
    648.             id=98,
    649.             color=[255, 153, 255],
    650.             type='',
    651.             swap='right_forefinger3'),
    652.         99:
    653.         dict(
    654.             name='left_forefinger4',
    655.             id=99,
    656.             color=[255, 153, 255],
    657.             type='',
    658.             swap='right_forefinger4'),
    659.         100:
    660.         dict(
    661.             name='left_middle_finger1',
    662.             id=100,
    663.             color=[102, 178, 255],
    664.             type='',
    665.             swap='right_middle_finger1'),
    666.         101:
    667.         dict(
    668.             name='left_middle_finger2',
    669.             id=101,
    670.             color=[102, 178, 255],
    671.             type='',
    672.             swap='right_middle_finger2'),
    673.         102:
    674.         dict(
    675.             name='left_middle_finger3',
    676.             id=102,
    677.             color=[102, 178, 255],
    678.             type='',
    679.             swap='right_middle_finger3'),
    680.         103:
    681.         dict(
    682.             name='left_middle_finger4',
    683.             id=103,
    684.             color=[102, 178, 255],
    685.             type='',
    686.             swap='right_middle_finger4'),
    687.         104:
    688.         dict(
    689.             name='left_ring_finger1',
    690.             id=104,
    691.             color=[255, 51, 51],
    692.             type='',
    693.             swap='right_ring_finger1'),
    694.         105:
    695.         dict(
    696.             name='left_ring_finger2',
    697.             id=105,
    698.             color=[255, 51, 51],
    699.             type='',
    700.             swap='right_ring_finger2'),
    701.         106:
    702.         dict(
    703.             name='left_ring_finger3',
    704.             id=106,
    705.             color=[255, 51, 51],
    706.             type='',
    707.             swap='right_ring_finger3'),
    708.         107:
    709.         dict(
    710.             name='left_ring_finger4',
    711.             id=107,
    712.             color=[255, 51, 51],
    713.             type='',
    714.             swap='right_ring_finger4'),
    715.         108:
    716.         dict(
    717.             name='left_pinky_finger1',
    718.             id=108,
    719.             color=[0, 255, 0],
    720.             type='',
    721.             swap='right_pinky_finger1'),
    722.         109:
    723.         dict(
    724.             name='left_pinky_finger2',
    725.             id=109,
    726.             color=[0, 255, 0],
    727.             type='',
    728.             swap='right_pinky_finger2'),
    729.         110:
    730.         dict(
    731.             name='left_pinky_finger3',
    732.             id=110,
    733.             color=[0, 255, 0],
    734.             type='',
    735.             swap='right_pinky_finger3'),
    736.         111:
    737.         dict(
    738.             name='left_pinky_finger4',
    739.             id=111,
    740.             color=[0, 255, 0],
    741.             type='',
    742.             swap='right_pinky_finger4'),
    743.         112:
    744.         dict(
    745.             name='right_hand_root',
    746.             id=112,
    747.             color=[255, 255, 255],
    748.             type='',
    749.             swap='left_hand_root'),
    750.         113:
    751.         dict(
    752.             name='right_thumb1',
    753.             id=113,
    754.             color=[255, 128, 0],
    755.             type='',
    756.             swap='left_thumb1'),
    757.         114:
    758.         dict(
    759.             name='right_thumb2',
    760.             id=114,
    761.             color=[255, 128, 0],
    762.             type='',
    763.             swap='left_thumb2'),
    764.         115:
    765.         dict(
    766.             name='right_thumb3',
    767.             id=115,
    768.             color=[255, 128, 0],
    769.             type='',
    770.             swap='left_thumb3'),
    771.         116:
    772.         dict(
    773.             name='right_thumb4',
    774.             id=116,
    775.             color=[255, 128, 0],
    776.             type='',
    777.             swap='left_thumb4'),
    778.         117:
    779.         dict(
    780.             name='right_forefinger1',
    781.             id=117,
    782.             color=[255, 153, 255],
    783.             type='',
    784.             swap='left_forefinger1'),
    785.         118:
    786.         dict(
    787.             name='right_forefinger2',
    788.             id=118,
    789.             color=[255, 153, 255],
    790.             type='',
    791.             swap='left_forefinger2'),
    792.         119:
    793.         dict(
    794.             name='right_forefinger3',
    795.             id=119,
    796.             color=[255, 153, 255],
    797.             type='',
    798.             swap='left_forefinger3'),
    799.         120:
    800.         dict(
    801.             name='right_forefinger4',
    802.             id=120,
    803.             color=[255, 153, 255],
    804.             type='',
    805.             swap='left_forefinger4'),
    806.         121:
    807.         dict(
    808.             name='right_middle_finger1',
    809.             id=121,
    810.             color=[102, 178, 255],
    811.             type='',
    812.             swap='left_middle_finger1'),
    813.         122:
    814.         dict(
    815.             name='right_middle_finger2',
    816.             id=122,
    817.             color=[102, 178, 255],
    818.             type='',
    819.             swap='left_middle_finger2'),
    820.         123:
    821.         dict(
    822.             name='right_middle_finger3',
    823.             id=123,
    824.             color=[102, 178, 255],
    825.             type='',
    826.             swap='left_middle_finger3'),
    827.         124:
    828.         dict(
    829.             name='right_middle_finger4',
    830.             id=124,
    831.             color=[102, 178, 255],
    832.             type='',
    833.             swap='left_middle_finger4'),
    834.         125:
    835.         dict(
    836.             name='right_ring_finger1',
    837.             id=125,
    838.             color=[255, 51, 51],
    839.             type='',
    840.             swap='left_ring_finger1'),
    841.         126:
    842.         dict(
    843.             name='right_ring_finger2',
    844.             id=126,
    845.             color=[255, 51, 51],
    846.             type='',
    847.             swap='left_ring_finger2'),
    848.         127:
    849.         dict(
    850.             name='right_ring_finger3',
    851.             id=127,
    852.             color=[255, 51, 51],
    853.             type='',
    854.             swap='left_ring_finger3'),
    855.         128:
    856.         dict(
    857.             name='right_ring_finger4',
    858.             id=128,
    859.             color=[255, 51, 51],
    860.             type='',
    861.             swap='left_ring_finger4'),
    862.         129:
    863.         dict(
    864.             name='right_pinky_finger1',
    865.             id=129,
    866.             color=[0, 255, 0],
    867.             type='',
    868.             swap='left_pinky_finger1'),
    869.         130:
    870.         dict(
    871.             name='right_pinky_finger2',
    872.             id=130,
    873.             color=[0, 255, 0],
    874.             type='',
    875.             swap='left_pinky_finger2'),
    876.         131:
    877.         dict(
    878.             name='right_pinky_finger3',
    879.             id=131,
    880.             color=[0, 255, 0],
    881.             type='',
    882.             swap='left_pinky_finger3'),
    883.         132:
    884.         dict(
    885.             name='right_pinky_finger4',
    886.             id=132,
    887.             color=[0, 255, 0],
    888.             type='',
    889.             swap='left_pinky_finger4')
    890.     },
    891.     skeleton_info={
    892.         0:
    893.         dict(link=('left_ankle', 'left_knee'), id=0, color=[0, 255, 0]),
    894.         1:
    895.         dict(link=('left_knee', 'left_hip'), id=1, color=[0, 255, 0]),
    896.         2:
    897.         dict(link=('right_ankle', 'right_knee'), id=2, color=[255, 128, 0]),
    898.         3:
    899.         dict(link=('right_knee', 'right_hip'), id=3, color=[255, 128, 0]),
    900.         4:
    901.         dict(link=('left_hip', 'right_hip'), id=4, color=[51, 153, 255]),
    902.         5:
    903.         dict(link=('left_shoulder', 'left_hip'), id=5, color=[51, 153, 255]),
    904.         6:
    905.         dict(link=('right_shoulder', 'right_hip'), id=6, color=[51, 153, 255]),
    906.         7:
    907.         dict(
    908.             link=('left_shoulder', 'right_shoulder'),
    909.             id=7,
    910.             color=[51, 153, 255]),
    911.         8:
    912.         dict(link=('left_shoulder', 'left_elbow'), id=8, color=[0, 255, 0]),
    913.         9:
    914.         dict(
    915.             link=('right_shoulder', 'right_elbow'), id=9, color=[255, 128, 0]),
    916.         10:
    917.         dict(link=('left_elbow', 'left_wrist'), id=10, color=[0, 255, 0]),
    918.         11:
    919.         dict(link=('right_elbow', 'right_wrist'), id=11, color=[255, 128, 0]),
    920.         12:
    921.         dict(link=('left_eye', 'right_eye'), id=12, color=[51, 153, 255]),
    922.         13:
    923.         dict(link=('nose', 'left_eye'), id=13, color=[51, 153, 255]),
    924.         14:
    925.         dict(link=('nose', 'right_eye'), id=14, color=[51, 153, 255]),
    926.         15:
    927.         dict(link=('left_eye', 'left_ear'), id=15, color=[51, 153, 255]),
    928.         16:
    929.         dict(link=('right_eye', 'right_ear'), id=16, color=[51, 153, 255]),
    930.         17:
    931.         dict(link=('left_ear', 'left_shoulder'), id=17, color=[51, 153, 255]),
    932.         18:
    933.         dict(
    934.             link=('right_ear', 'right_shoulder'), id=18, color=[51, 153, 255]),
    935.         19:
    936.         dict(link=('left_ankle', 'left_big_toe'), id=19, color=[0, 255, 0]),
    937.         20:
    938.         dict(link=('left_ankle', 'left_small_toe'), id=20, color=[0, 255, 0]),
    939.         21:
    940.         dict(link=('left_ankle', 'left_heel'), id=21, color=[0, 255, 0]),
    941.         22:
    942.         dict(
    943.             link=('right_ankle', 'right_big_toe'), id=22, color=[255, 128, 0]),
    944.         23:
    945.         dict(
    946.             link=('right_ankle', 'right_small_toe'),
    947.             id=23,
    948.             color=[255, 128, 0]),
    949.         24:
    950.         dict(link=('right_ankle', 'right_heel'), id=24, color=[255, 128, 0]),
    951.         25:
    952.         dict(
    953.             link=('left_hand_root', 'left_thumb1'), id=25, color=[255, 128,
    954.                                                                   0]),
    955.         26:
    956.         dict(link=('left_thumb1', 'left_thumb2'), id=26, color=[255, 128, 0]),
    957.         27:
    958.         dict(link=('left_thumb2', 'left_thumb3'), id=27, color=[255, 128, 0]),
    959.         28:
    960.         dict(link=('left_thumb3', 'left_thumb4'), id=28, color=[255, 128, 0]),
    961.         29:
    962.         dict(
    963.             link=('left_hand_root', 'left_forefinger1'),
    964.             id=29,
    965.             color=[255, 153, 255]),
    966.         30:
    967.         dict(
    968.             link=('left_forefinger1', 'left_forefinger2'),
    969.             id=30,
    970.             color=[255, 153, 255]),
    971.         31:
    972.         dict(
    973.             link=('left_forefinger2', 'left_forefinger3'),
    974.             id=31,
    975.             color=[255, 153, 255]),
    976.         32:
    977.         dict(
    978.             link=('left_forefinger3', 'left_forefinger4'),
    979.             id=32,
    980.             color=[255, 153, 255]),
    981.         33:
    982.         dict(
    983.             link=('left_hand_root', 'left_middle_finger1'),
    984.             id=33,
    985.             color=[102, 178, 255]),
    986.         34:
    987.         dict(
    988.             link=('left_middle_finger1', 'left_middle_finger2'),
    989.             id=34,
    990.             color=[102, 178, 255]),
    991.         35:
    992.         dict(
    993.             link=('left_middle_finger2', 'left_middle_finger3'),
    994.             id=35,
    995.             color=[102, 178, 255]),
    996.         36:
    997.         dict(
    998.             link=('left_middle_finger3', 'left_middle_finger4'),
    999.             id=36,
    1000.             color=[102, 178, 255]),
    1001.         37:
    1002.         dict(
    1003.             link=('left_hand_root', 'left_ring_finger1'),
    1004.             id=37,
    1005.             color=[255, 51, 51]),
    1006.         38:
    1007.         dict(
    1008.             link=('left_ring_finger1', 'left_ring_finger2'),
    1009.             id=38,
    1010.             color=[255, 51, 51]),
    1011.         39:
    1012.         dict(
    1013.             link=('left_ring_finger2', 'left_ring_finger3'),
    1014.             id=39,
    1015.             color=[255, 51, 51]),
    1016.         40:
    1017.         dict(
    1018.             link=('left_ring_finger3', 'left_ring_finger4'),
    1019.             id=40,
    1020.             color=[255, 51, 51]),
    1021.         41:
    1022.         dict(
    1023.             link=('left_hand_root', 'left_pinky_finger1'),
    1024.             id=41,
    1025.             color=[0, 255, 0]),
    1026.         42:
    1027.         dict(
    1028.             link=('left_pinky_finger1', 'left_pinky_finger2'),
    1029.             id=42,
    1030.             color=[0, 255, 0]),
    1031.         43:
    1032.         dict(
    1033.             link=('left_pinky_finger2', 'left_pinky_finger3'),
    1034.             id=43,
    1035.             color=[0, 255, 0]),
    1036.         44:
    1037.         dict(
    1038.             link=('left_pinky_finger3', 'left_pinky_finger4'),
    1039.             id=44,
    1040.             color=[0, 255, 0]),
    1041.         45:
    1042.         dict(
    1043.             link=('right_hand_root', 'right_thumb1'),
    1044.             id=45,
    1045.             color=[255, 128, 0]),
    1046.         46:
    1047.         dict(
    1048.             link=('right_thumb1', 'right_thumb2'), id=46, color=[255, 128, 0]),
    1049.         47:
    1050.         dict(
    1051.             link=('right_thumb2', 'right_thumb3'), id=47, color=[255, 128, 0]),
    1052.         48:
    1053.         dict(
    1054.             link=('right_thumb3', 'right_thumb4'), id=48, color=[255, 128, 0]),
    1055.         49:
    1056.         dict(
    1057.             link=('right_hand_root', 'right_forefinger1'),
    1058.             id=49,
    1059.             color=[255, 153, 255]),
    1060.         50:
    1061.         dict(
    1062.             link=('right_forefinger1', 'right_forefinger2'),
    1063.             id=50,
    1064.             color=[255, 153, 255]),
    1065.         51:
    1066.         dict(
    1067.             link=('right_forefinger2', 'right_forefinger3'),
    1068.             id=51,
    1069.             color=[255, 153, 255]),
    1070.         52:
    1071.         dict(
    1072.             link=('right_forefinger3', 'right_forefinger4'),
    1073.             id=52,
    1074.             color=[255, 153, 255]),
    1075.         53:
    1076.         dict(
    1077.             link=('right_hand_root', 'right_middle_finger1'),
    1078.             id=53,
    1079.             color=[102, 178, 255]),
    1080.         54:
    1081.         dict(
    1082.             link=('right_middle_finger1', 'right_middle_finger2'),
    1083.             id=54,
    1084.             color=[102, 178, 255]),
    1085.         55:
    1086.         dict(
    1087.             link=('right_middle_finger2', 'right_middle_finger3'),
    1088.             id=55,
    1089.             color=[102, 178, 255]),
    1090.         56:
    1091.         dict(
    1092.             link=('right_middle_finger3', 'right_middle_finger4'),
    1093.             id=56,
    1094.             color=[102, 178, 255]),
    1095.         57:
    1096.         dict(
    1097.             link=('right_hand_root', 'right_ring_finger1'),
    1098.             id=57,
    1099.             color=[255, 51, 51]),
    1100.         58:
    1101.         dict(
    1102.             link=('right_ring_finger1', 'right_ring_finger2'),
    1103.             id=58,
    1104.             color=[255, 51, 51]),
    1105.         59:
    1106.         dict(
    1107.             link=('right_ring_finger2', 'right_ring_finger3'),
    1108.             id=59,
    1109.             color=[255, 51, 51]),
    1110.         60:
    1111.         dict(
    1112.             link=('right_ring_finger3', 'right_ring_finger4'),
    1113.             id=60,
    1114.             color=[255, 51, 51]),
    1115.         61:
    1116.         dict(
    1117.             link=('right_hand_root', 'right_pinky_finger1'),
    1118.             id=61,
    1119.             color=[0, 255, 0]),
    1120.         62:
    1121.         dict(
    1122.             link=('right_pinky_finger1', 'right_pinky_finger2'),
    1123.             id=62,
    1124.             color=[0, 255, 0]),
    1125.         63:
    1126.         dict(
    1127.             link=('right_pinky_finger2', 'right_pinky_finger3'),
    1128.             id=63,
    1129.             color=[0, 255, 0]),
    1130.         64:
    1131.         dict(
    1132.             link=('right_pinky_finger3', 'right_pinky_finger4'),
    1133.             id=64,
    1134.             color=[0, 255, 0])
    1135.     },
    1136.     joint_weights=[1.] * 133,
    1137.     # 'https://github.com/jin-s13/COCO-WholeBody/blob/master/'
    1138.     # 'evaluation/myeval_wholebody.py#L175'
    1139.     sigmas=[
    1140.         0.026, 0.025, 0.025, 0.035, 0.035, 0.079, 0.079, 0.072, 0.072, 0.062,
    1141.         0.062, 0.107, 0.107, 0.087, 0.087, 0.089, 0.089, 0.068, 0.066, 0.066,
    1142.         0.092, 0.094, 0.094, 0.042, 0.043, 0.044, 0.043, 0.040, 0.035, 0.031,
    1143.         0.025, 0.020, 0.023, 0.029, 0.032, 0.037, 0.038, 0.043, 0.041, 0.045,
    1144.         0.013, 0.012, 0.011, 0.011, 0.012, 0.012, 0.011, 0.011, 0.013, 0.015,
    1145.         0.009, 0.007, 0.007, 0.007, 0.012, 0.009, 0.008, 0.016, 0.010, 0.017,
    1146.         0.011, 0.009, 0.011, 0.009, 0.007, 0.013, 0.008, 0.011, 0.012, 0.010,
    1147.         0.034, 0.008, 0.008, 0.009, 0.008, 0.008, 0.007, 0.010, 0.008, 0.009,
    1148.         0.009, 0.009, 0.007, 0.007, 0.008, 0.011, 0.008, 0.008, 0.008, 0.01,
    1149.         0.008, 0.029, 0.022, 0.035, 0.037, 0.047, 0.026, 0.025, 0.024, 0.035,
    1150.         0.018, 0.024, 0.022, 0.026, 0.017, 0.021, 0.021, 0.032, 0.02, 0.019,
    1151.         0.022, 0.031, 0.029, 0.022, 0.035, 0.037, 0.047, 0.026, 0.025, 0.024,
    1152.         0.035, 0.018, 0.024, 0.022, 0.026, 0.017, 0.021, 0.021, 0.032, 0.02,
    1153.         0.019, 0.022, 0.031
    1154.     ])

    复制一份命名为coco_body23.py

    对他进行更改，我只需要body+foot，其他的对应的都不要，修改后为：这里提醒一句

    simga值为每个关键点各自的数据集标准差，COCO上是对同⼀个目标的5000次不同标注产生的标准差。其值越大，说明在整个数据集中对这个点的标注一致性越差; 值越小，说明整个数据集中对这个点的标注一致性越好。

    1. dataset_info = dict(
    2.     dataset_name='coco_body25',
    3.     paper_info=dict(
    4.         author='Jin, Sheng and Xu, Lumin and Xu, Jin and '
    5.         'Wang, Can and Liu, Wentao and '
    6.         'Qian, Chen and Ouyang, Wanli and Luo, Ping',
    7.         title='Whole-Body Human Pose Estimation in the Wild',
    8.         container='Proceedings of the European '
    9.         'Conference on Computer Vision (ECCV)',
    10.         year='2020',
    11.         homepage='https://github.com/jin-s13/COCO-WholeBody/',
    12.     ),
    13.     keypoint_info={
    14.         0:
    15.         dict(name='nose', id=0, color=[51, 153, 255], type='upper', swap=''),
    16.         1:
    17.         dict(
    18.             name='left_eye',
    19.             id=1,
    20.             color=[51, 153, 255],
    21.             type='upper',
    22.             swap='right_eye'),
    23.         2:
    24.         dict(
    25.             name='right_eye',
    26.             id=2,
    27.             color=[51, 153, 255],
    28.             type='upper',
    29.             swap='left_eye'),
    30.         3:
    31.         dict(
    32.             name='left_ear',
    33.             id=3,
    34.             color=[51, 153, 255],
    35.             type='upper',
    36.             swap='right_ear'),
    37.         4:
    38.         dict(
    39.             name='right_ear',
    40.             id=4,
    41.             color=[51, 153, 255],
    42.             type='upper',
    43.             swap='left_ear'),
    44.         5:
    45.         dict(
    46.             name='left_shoulder',
    47.             id=5,
    48.             color=[0, 255, 0],
    49.             type='upper',
    50.             swap='right_shoulder'),
    51.         6:
    52.         dict(
    53.             name='right_shoulder',
    54.             id=6,
    55.             color=[255, 128, 0],
    56.             type='upper',
    57.             swap='left_shoulder'),
    58.         7:
    59.         dict(
    60.             name='left_elbow',
    61.             id=7,
    62.             color=[0, 255, 0],
    63.             type='upper',
    64.             swap='right_elbow'),
    65.         8:
    66.         dict(
    67.             name='right_elbow',
    68.             id=8,
    69.             color=[255, 128, 0],
    70.             type='upper',
    71.             swap='left_elbow'),
    72.         9:
    73.         dict(
    74.             name='left_wrist',
    75.             id=9,
    76.             color=[0, 255, 0],
    77.             type='upper',
    78.             swap='right_wrist'),
    79.         10:
    80.         dict(
    81.             name='right_wrist',
    82.             id=10,
    83.             color=[255, 128, 0],
    84.             type='upper',
    85.             swap='left_wrist'),
    86.         11:
    87.         dict(
    88.             name='left_hip',
    89.             id=11,
    90.             color=[0, 255, 0],
    91.             type='lower',
    92.             swap='right_hip'),
    93.         12:
    94.         dict(
    95.             name='right_hip',
    96.             id=12,
    97.             color=[255, 128, 0],
    98.             type='lower',
    99.             swap='left_hip'),
    100.         13:
    101.         dict(
    102.             name='left_knee',
    103.             id=13,
    104.             color=[0, 255, 0],
    105.             type='lower',
    106.             swap='right_knee'),
    107.         14:
    108.         dict(
    109.             name='right_knee',
    110.             id=14,
    111.             color=[255, 128, 0],
    112.             type='lower',
    113.             swap='left_knee'),
    114.         15:
    115.         dict(
    116.             name='left_ankle',
    117.             id=15,
    118.             color=[0, 255, 0],
    119.             type='lower',
    120.             swap='right_ankle'),
    121.         16:
    122.         dict(
    123.             name='right_ankle',
    124.             id=16,
    125.             color=[255, 128, 0],
    126.             type='lower',
    127.             swap='left_ankle'),
    128.         17:
    129.         dict(
    130.             name='left_big_toe',
    131.             id=17,
    132.             color=[255, 128, 0],
    133.             type='lower',
    134.             swap='right_big_toe'),
    135.         18:
    136.         dict(
    137.             name='left_small_toe',
    138.             id=18,
    139.             color=[255, 128, 0],
    140.             type='lower',
    141.             swap='right_small_toe'),
    142.         19:
    143.         dict(
    144.             name='left_heel',
    145.             id=19,
    146.             color=[255, 128, 0],
    147.             type='lower',
    148.             swap='right_heel'),
    149.         20:
    150.         dict(
    151.             name='right_big_toe',
    152.             id=20,
    153.             color=[255, 128, 0],
    154.             type='lower',
    155.             swap='left_big_toe'),
    156.         21:
    157.         dict(
    158.             name='right_small_toe',
    159.             id=21,
    160.             color=[255, 128, 0],
    161.             type='lower',
    162.             swap='left_small_toe'),
    163.         22:
    164.         dict(
    165.             name='right_heel',
    166.             id=22,
    167.             color=[255, 128, 0],
    168.             type='lower',
    169.             swap='left_heel')
    170.     },
    171.     skeleton_info={
    172.         0:
    173.         dict(link=('left_ankle', 'left_knee'), id=0, color=[0, 255, 0]),
    174.         1:
    175.         dict(link=('left_knee', 'left_hip'), id=1, color=[0, 255, 0]),
    176.         2:
    177.         dict(link=('right_ankle', 'right_knee'), id=2, color=[255, 128, 0]),
    178.         3:
    179.         dict(link=('right_knee', 'right_hip'), id=3, color=[255, 128, 0]),
    180.         4:
    181.         dict(link=('left_hip', 'right_hip'), id=4, color=[51, 153, 255]),
    182.         5:
    183.         dict(link=('left_shoulder', 'left_hip'), id=5, color=[51, 153, 255]),
    184.         6:
    185.         dict(link=('right_shoulder', 'right_hip'), id=6, color=[51, 153, 255]),
    186.         7:
    187.         dict(
    188.             link=('left_shoulder', 'right_shoulder'),
    189.             id=7,
    190.             color=[51, 153, 255]),
    191.         8:
    192.         dict(link=('left_shoulder', 'left_elbow'), id=8, color=[0, 255, 0]),
    193.         9:
    194.         dict(
    195.             link=('right_shoulder', 'right_elbow'), id=9, color=[255, 128, 0]),
    196.         10:
    197.         dict(link=('left_elbow', 'left_wrist'), id=10, color=[0, 255, 0]),
    198.         11:
    199.         dict(link=('right_elbow', 'right_wrist'), id=11, color=[255, 128, 0]),
    200.         12:
    201.         dict(link=('left_eye', 'right_eye'), id=12, color=[51, 153, 255]),
    202.         13:
    203.         dict(link=('nose', 'left_eye'), id=13, color=[51, 153, 255]),
    204.         14:
    205.         dict(link=('nose', 'right_eye'), id=14, color=[51, 153, 255]),
    206.         15:
    207.         dict(link=('left_eye', 'left_ear'), id=15, color=[51, 153, 255]),
    208.         16:
    209.         dict(link=('right_eye', 'right_ear'), id=16, color=[51, 153, 255]),
    210.         17:
    211.         dict(link=('left_ear', 'left_shoulder'), id=17, color=[51, 153, 255]),
    212.         18:
    213.         dict(
    214.             link=('right_ear', 'right_shoulder'), id=18, color=[51, 153, 255]),
    215.         19:
    216.         dict(link=('left_ankle', 'left_big_toe'), id=19, color=[0, 255, 0]),
    217.         20:
    218.         dict(link=('left_ankle', 'left_small_toe'), id=20, color=[0, 255, 0]),
    219.         21:
    220.         dict(link=('left_ankle', 'left_heel'), id=21, color=[0, 255, 0]),
    221.         22:
    222.         dict(
    223.             link=('right_ankle', 'right_big_toe'), id=22, color=[255, 128, 0]),
    224.         23:
    225.         dict(
    226.             link=('right_ankle', 'right_small_toe'),
    227.             id=23,
    228.             color=[255, 128, 0]),
    229.         24:
    230.         dict(link=('right_ankle', 'right_heel'), id=24, color=[255, 128, 0]),
    231.     },
    232.     joint_weights=[1.] * 23,
    233.     # 'https://github.com/jin-s13/COCO-WholeBody/blob/master/'
    234.     # 'evaluation/myeval_wholebody.py#L175'
    235.     sigmas=[
    236.         0.026, 0.025, 0.025, 0.035, 0.035, 0.079, 0.079, 0.072, 0.072, 0.062,
    237.         0.062, 0.107, 0.107, 0.087, 0.087, 0.089, 0.089, 0.068, 0.066, 0.066,
    238.         0.092, 0.094, 0.094
    239.     ])
    240.  
    241.

    二、dataset mAP

    因为mmpose有注册机制因此在这建议先不要setup.py 更改后再setup.py。在build下的mmpose没有build 直接在根目录下的mmpose文件夹中找到datasets/datasets/topdown 下新建个top-down-cocobody23-datasets.py

    1. # Copyright (c) OpenMMLab. All rights reserved.
    2. import os
    3. import warnings
    4.  
    5. import numpy as np
    6. from mmcv import Config
    7. from xtcocotools.cocoeval import COCOeval
    8.  
    9. from ...builder import DATASETS
    10. from .topdown_coco_dataset import TopDownCocoDataset
    11.  
    12.  
    13. @DATASETS.register_module()
    14. class TopDownCocoBody25Dataset(TopDownCocoDataset):
    15.     """CocoWholeBodyDataset dataset for top-down pose estimation.
    17.     "Whole-Body Human Pose Estimation in the Wild", ECCV'2020.
    18.     More details can be found in the `paper
    19.     `__ .
    21.     The dataset loads raw features and apply specified transforms
    22.     to return a dict containing the image tensors and other information.
    24.     COCO-WholeBody keypoint indexes::
    26.         0-16: 17 body keypoints,
    27.         17-22: 6 foot keypoints,
    29.         In total, we have 23 keypoints for body25 pose estimation.
    31.     Args:
    32.         ann_file (str): Path to the annotation file.
    33.         img_prefix (str): Path to a directory where images are held.
    34.             Default: None.
    35.         data_cfg (dict): config
    36.         pipeline (list[dict | callable]): A sequence of data transforms.
    37.         dataset_info (DatasetInfo): A class containing all dataset info.
    38.         test_mode (bool): Store True when building test or
    39.             validation dataset. Default: False.
    40.     """
    41.  
    42.     def __init__(self,
    43.                  ann_file,
    44.                  img_prefix,
    45.                  data_cfg,
    46.                  pipeline,
    47.                  dataset_info=None,
    48.                  test_mode=False):
    49.  
    50.         if dataset_info is None:
    51.             warnings.warn(
    52.                 'dataset_info is missing. '
    53.                 'Check https://github.com/open-mmlab/mmpose/pull/663 '
    54.                 'for details.', DeprecationWarning)
    55.             cfg = Config.fromfile('configs/_base_/datasets/coco_body25.py')
    56.             dataset_info = cfg._cfg_dict['dataset_info']
    57.  
    58.         super(TopDownCocoDataset, self).__init__(
    59.             ann_file,
    60.             img_prefix,
    61.             data_cfg,
    62.             pipeline,
    63.             dataset_info=dataset_info,
    64.             test_mode=test_mode)
    65.  
    66.         self.use_gt_bbox = data_cfg['use_gt_bbox']
    67.         self.bbox_file = data_cfg['bbox_file']
    68.         self.det_bbox_thr = data_cfg.get('det_bbox_thr', 0.0)
    69.         self.use_nms = data_cfg.get('use_nms', True)
    70.         self.soft_nms = data_cfg['soft_nms']
    71.         self.nms_thr = data_cfg['nms_thr']
    72.         self.oks_thr = data_cfg['oks_thr']
    73.         self.vis_thr = data_cfg['vis_thr']
    74.  
    75.         self.body_num = 17
    76.         self.foot_num = 6
    77.  
    78.  
    79.         self.db = self._get_db()
    80.  
    81.         print(f'=> num_images: {self.num_images}')
    82.         print(f'=> load {len(self.db)} samples')
    83.  
    84.     def _load_coco_keypoint_annotation_kernel(self, img_id):
    85.         """load annotation from COCOAPI.
    87.         Note:
    88.             bbox:[x1, y1, w, h]
    89.         Args:
    90.             img_id: coco image id
    91.         Returns:
    92.             dict: db entry
    93.         """
    94.         img_ann = self.coco.loadImgs(img_id)[0]
    95.         width = img_ann['width']
    96.         height = img_ann['height']
    97.         num_joints = self.ann_info['num_joints']
    98.  
    99.         ann_ids = self.coco.getAnnIds(imgIds=img_id, iscrowd=False)
    100.         objs = self.coco.loadAnns(ann_ids)
    101.  
    102.         # sanitize bboxes
    103.         valid_objs = []
    104.         for obj in objs:
    105.             if 'bbox' not in obj:
    106.                 continue
    107.             x, y, w, h = obj['bbox']
    108.             x1 = max(0, x)
    109.             y1 = max(0, y)
    110.             x2 = min(width - 1, x1 + max(0, w))
    111.             y2 = min(height - 1, y1 + max(0, h))
    112.             if ('area' not in obj or obj['area'] > 0) and x2 > x1 and y2 > y1:
    113.                 obj['clean_bbox'] = [x1, y1, x2 - x1, y2 - y1]
    114.                 valid_objs.append(obj)
    115.         objs = valid_objs
    116.  
    117.         rec = []
    118.         bbox_id = 0
    119.         for obj in objs:
    120.             if 'keypoints' not in obj:
    121.                 continue
    122.             if max(obj['keypoints']) == 0:
    123.                 continue
    124.             joints_3d = np.zeros((num_joints, 3), dtype=np.float32)
    125.             joints_3d_visible = np.zeros((num_joints, 3), dtype=np.float32)
    126.  
    127.             keypoints = np.array(obj['keypoints'] + obj['foot_kpts']).reshape(-1, 3)
    128.             joints_3d[:, :2] = keypoints[:, :2]
    129.             joints_3d_visible[:, :2] = np.minimum(1, keypoints[:, 2:3] > 0)
    130.  
    131.             image_file = os.path.join(self.img_prefix, self.id2name[img_id])
    132.             rec.append({
    133.                 'image_file': image_file,
    134.                 'bbox': obj['clean_bbox'][:4],
    135.                 'rotation': 0,
    136.                 'joints_3d': joints_3d,
    137.                 'joints_3d_visible': joints_3d_visible,
    138.                 'dataset': self.dataset_name,
    139.                 'bbox_score': 1,
    140.                 'bbox_id': bbox_id
    141.             })
    142.             bbox_id = bbox_id + 1
    143.  
    144.         return rec
    145.  
    146.     def _coco_keypoint_results_one_category_kernel(self, data_pack):
    147.         """Get coco keypoint results."""
    148.         cat_id = data_pack['cat_id']
    149.         keypoints = data_pack['keypoints']
    150.         cat_results = []
    151.  
    152.         for img_kpts in keypoints:
    153.             if len(img_kpts) == 0:
    154.                 continue
    155.  
    156.             _key_points = np.array(
    157.                 [img_kpt['keypoints'] for img_kpt in img_kpts])
    158.             key_points = _key_points.reshape(-1,
    159.                                              self.ann_info['num_joints'] * 3)
    160.  
    161.             cuts = np.cumsum([
    162.                 0, self.body_num, self.foot_num
    163.             ]) * 3
    164.  
    165.             result = [{
    166.                 'image_id': img_kpt['image_id'],
    167.                 'category_id': cat_id,
    168.                 'keypoints': key_point[cuts[0]:cuts[1]].tolist(),
    169.                 'foot_kpts': key_point[cuts[1]:cuts[2]].tolist(),
    170.                 'score': float(img_kpt['score']),
    171.                 'center': img_kpt['center'].tolist(),
    172.                 'scale': img_kpt['scale'].tolist()
    173.             } for img_kpt, key_point in zip(img_kpts, key_points)]
    174.  
    175.             cat_results.extend(result)
    176.  
    177.         return cat_results
    178.  
    179.     def _do_python_keypoint_eval(self, res_file):
    180.         """Keypoint evaluation using COCOAPI."""
    181.         coco_det = self.coco.loadRes(res_file)
    182.  
    183.         cuts = np.cumsum([
    184.             0, self.body_num, self.foot_num
    185.         ])
    186.  
    187.         coco_eval = COCOeval(
    188.             self.coco,
    189.             coco_det,
    190.             'keypoints_body',
    191.             self.sigmas[cuts[0]:cuts[1]],
    192.             use_area=True)
    193.         coco_eval.params.useSegm = None
    194.         coco_eval.evaluate()
    195.         coco_eval.accumulate()
    196.         coco_eval.summarize()
    197.  
    198.         coco_eval = COCOeval(
    199.             self.coco,
    200.             coco_det,
    201.             'keypoints_foot',
    202.             self.sigmas[cuts[1]:cuts[2]],
    203.             use_area=True)
    204.         coco_eval.params.useSegm = None
    205.         coco_eval.evaluate()
    206.         coco_eval.accumulate()
    207.         coco_eval.summarize()
    208.  
    209.         stats_names = [
    210.             'AP', 'AP .5', 'AP .75', 'AP (M)', 'AP (L)', 'AR', 'AR .5',
    211.             'AR .75', 'AR (M)', 'AR (L)'
    212.         ]
    213.  
    214.         info_str = list(zip(stats_names, coco_eval.stats))
    215.  
    216.         return info_str

    在几层的__init__.py 中导入此包。就如下图

     然后通过mmpose 官方给的训练代码进行训练。

    训练了120个epoch:

     和论文在body上的效果差不多了

     我整合了body+foot的整个mAP而不是body mAP+foot mAP如下：

    下一篇mmpose（2）：复现知乎大佬mmpose中的shufflenetv2+deeppose的方法

  • 相关阅读:
    mac安装python虚拟环境
    Leetcode 剑指 Offer II 045. 找树左下角的值
    sqlmap获取目标
    Windbg可以看到Visual Studio中看不到的有效函数调用堆栈
    Linux 中安装MySQL
    C++期末课设—图书管理系统
    SB树，看这一篇就够了
    Redis线程模型
    巨控GRM530无线远从模块的实际使用时间，速度快
    JS-项目实战-鼠标悬浮设置字体颜色以及控制键盘输入
  • 原文地址：https://blog.csdn.net/qq_38284951/article/details/126345462