• YOLO V8训练自己的数据集并测试

    目录

    1 YOLOV8部署

    2 标注软件labelme安装

    3 将labelme转化为YOLOV8支持的数据格式

    4 开始训练

    5 利用训练结果进行测试

    1 YOLOV8部署

            我的一篇博客已经提到，这里不再赘述：

    YOLO V8语义分割模型部署-CSDN博客YOLO V8语义分割模型部署https://blog.csdn.net/qq_41694024/article/details/133983334

    2 标注软件labelme安装

            我们采用一键安装的方式：

     pip install labelme

            这个对python版本有要求，python版本最好是3.8。如果不是那就再茶ungjiange虚拟环境吧。

            直接命令行输入labelme启动，可能会出现这个问题：

    1. (yo) liuhongwei@liuhongwei-Legion-Y9000P-IRX8H:~/Desktop/ultralytics/examples/YOLOv8-CPP-Inference/build$ labelme
    2. [INFO   ] __init__:get_config:70 - Loading config file from: /home/liuhongwei/.labelmerc
    3. QObject::moveToThread: Current thread (0x564cffaed340) is not the object's thread (0x564d01248510).
    4. Cannot move to target thread (0x564cffaed340)
    6. qt.qpa.plugin: Could not load the Qt platform plugin "xcb" in "/home/liuhongwei/anaconda3/envs/yo/lib/python3.8/site-packages/cv2/qt/plugins" even though it was found.
    7. This application failed to start because no Qt platform plugin could be initialized. Reinstalling the application may fix this problem.
    9. Available platform plugins are: xcb, eglfs, linuxfb, minimal, minimalegl, offscreen, vnc, wayland-egl, wayland, wayland-xcomposite-egl, wayland-xcomposite-glx, webgl.

            这是QT版本太高了.....，降低QT版本：

    pip install pyqt5==5.14.0

            再次启动labelme：

            可以用了。

            label使用就是打开一个含有图像的文件夹就可以，使用很简单，上过本科的一看界面就会，给大家展示一张标注结果：

    3 将labelme转化为YOLOV8支持的数据格式

            先装依赖：

    pip install pycocotools

            用我的脚本将labelme形式的标注文件转化为yolo格式的文件：

    1. '''
    2. Created on Oct 23, 2023
    3. @author: Liuhongwei
    4. '''
    5. import os
    6. import sys
    7. import argparse
    8. import shutil
    9. import math
    10. from collections import OrderedDict
    11.  
    12. import json
    13. import cv2
    14. import PIL.Image
    15.   
    16. from sklearn.model_selection import train_test_split
    17. from labelme import utils
    18.  
    19.  
    20. class Labelme2YOLO(object):
    21.     
    22.     def __init__(self, json_dir):
    23.         self._json_dir = json_dir
    24.         
    25.         self._label_id_map = self._get_label_id_map(self._json_dir)
    26.         
    27.     def _make_train_val_dir(self):
    28.         self._label_dir_path = os.path.join(self._json_dir, 
    29.                                             'YOLODataset/labels/')
    30.         self._image_dir_path = os.path.join(self._json_dir, 
    31.                                             'YOLODataset/images/')
    32.         
    33.         for yolo_path in (os.path.join(self._label_dir_path + 'train/'), 
    34.                           os.path.join(self._label_dir_path + 'val/'),
    35.                           os.path.join(self._image_dir_path + 'train/'), 
    36.                           os.path.join(self._image_dir_path + 'val/')):
    37.             if os.path.exists(yolo_path):
    38.                 shutil.rmtree(yolo_path)
    39.             
    40.             os.makedirs(yolo_path)    
    41.                 
    42.     def _get_label_id_map(self, json_dir):
    43.         label_set = set()
    44.     
    45.         for file_name in os.listdir(json_dir):
    46.             if file_name.endswith('json'):
    47.                 json_path = os.path.join(json_dir, file_name)
    48.                 data = json.load(open(json_path))
    49.                 for shape in data['shapes']:
    50.                     label_set.add(shape['label'])
    51.         
    52.         return OrderedDict([(label, label_id) \
    53.                             for label_id, label in enumerate(label_set)])
    54.     
    55.     def _train_test_split(self, folders, json_names, val_size):
    56.         if len(folders) > 0 and 'train' in folders and 'val' in folders:
    57.             train_folder = os.path.join(self._json_dir, 'train/')
    58.             train_json_names = [train_sample_name + '.json' \
    59.                                 for train_sample_name in os.listdir(train_folder) \
    60.                                 if os.path.isdir(os.path.join(train_folder, train_sample_name))]
    61.             
    62.             val_folder = os.path.join(self._json_dir, 'val/')
    63.             val_json_names = [val_sample_name + '.json' \
    64.                               for val_sample_name in os.listdir(val_folder) \
    65.                               if os.path.isdir(os.path.join(val_folder, val_sample_name))]
    66.             
    67.             return train_json_names, val_json_names
    68.         
    69.         train_idxs, val_idxs = train_test_split(range(len(json_names)), 
    70.                                                 test_size=val_size)
    71.         train_json_names = [json_names[train_idx] for train_idx in train_idxs]
    72.         val_json_names = [json_names[val_idx] for val_idx in val_idxs]
    73.         
    74.         return train_json_names, val_json_names
    75.     
    76.     def convert(self, val_size):
    77.         json_names = [file_name for file_name in os.listdir(self._json_dir) \
    78.                       if os.path.isfile(os.path.join(self._json_dir, file_name)) and \
    79.                       file_name.endswith('.json')]
    80.         folders =  [file_name for file_name in os.listdir(self._json_dir) \
    81.                     if os.path.isdir(os.path.join(self._json_dir, file_name))]
    82.         train_json_names, val_json_names = self._train_test_split(folders, json_names, val_size)
    83.         
    84.         self._make_train_val_dir()
    85.     
    86.         # convert labelme object to yolo format object, and save them to files
    87.         # also get image from labelme json file and save them under images folder
    88.         for target_dir, json_names in zip(('train/', 'val/'), 
    89.                                           (train_json_names, val_json_names)):
    90.             for json_name in json_names:
    91.                 json_path = os.path.join(self._json_dir, json_name)
    92.                 json_data = json.load(open(json_path))
    93.                 
    94.                 print('Converting %s for %s ...' % (json_name, target_dir.replace('/', '')))
    95.                 
    96.                 img_path = self._save_yolo_image(json_data, 
    97.                                                  json_name, 
    98.                                                  self._image_dir_path, 
    99.                                                  target_dir)
    100.                     
    101.                 yolo_obj_list = self._get_yolo_object_list(json_data, img_path)
    102.                 self._save_yolo_label(json_name, 
    103.                                       self._label_dir_path, 
    104.                                       target_dir, 
    105.                                       yolo_obj_list)
    106.         
    107.         print('Generating dataset.yaml file ...')
    108.         self._save_dataset_yaml()
    109.                 
    110.     def convert_one(self, json_name):
    111.         json_path = os.path.join(self._json_dir, json_name)
    112.         json_data = json.load(open(json_path))
    113.         
    114.         print('Converting %s ...' % json_name)
    115.         
    116.         img_path = self._save_yolo_image(json_data, json_name, 
    117.                                          self._json_dir, '')
    118.         
    119.         yolo_obj_list = self._get_yolo_object_list(json_data, img_path)
    120.         self._save_yolo_label(json_name, self._json_dir, 
    121.                               '', yolo_obj_list)
    122.     
    123.     def _get_yolo_object_list(self, json_data, img_path):
    124.         yolo_obj_list = []
    125.         
    126.         img_h, img_w, _ = cv2.imread(img_path).shape
    127.         for shape in json_data['shapes']:
    128.             # labelme circle shape is different from others
    129.             # it only has 2 points, 1st is circle center, 2nd is drag end point
    130.             if shape['shape_type'] == 'circle':
    131.                 yolo_obj = self._get_circle_shape_yolo_object(shape, img_h, img_w)
    132.             else:
    133.                 yolo_obj = self._get_other_shape_yolo_object(shape, img_h, img_w)
    134.             
    135.             yolo_obj_list.append(yolo_obj)
    136.             
    137.         return yolo_obj_list
    138.     
    139.     def _get_circle_shape_yolo_object(self, shape, img_h, img_w):
    140.         obj_center_x, obj_center_y = shape['points'][0]
    141.         
    142.         radius = math.sqrt((obj_center_x - shape['points'][1][0]) ** 2 + 
    143.                            (obj_center_y - shape['points'][1][1]) ** 2)
    144.         obj_w = 2 * radius
    145.         obj_h = 2 * radius
    146.         
    147.         yolo_center_x= round(float(obj_center_x / img_w), 6)
    148.         yolo_center_y = round(float(obj_center_y / img_h), 6)
    149.         yolo_w = round(float(obj_w / img_w), 6)
    150.         yolo_h = round(float(obj_h / img_h), 6)
    151.             
    152.         label_id = self._label_id_map[shape['label']]
    153.         
    154.         return label_id, yolo_center_x, yolo_center_y, yolo_w, yolo_h
    155.     
    156.     def _get_other_shape_yolo_object(self, shape, img_h, img_w):
    157.         def __get_object_desc(obj_port_list):
    158.             __get_dist = lambda int_list: max(int_list) - min(int_list)
    159.             
    160.             x_lists = [port[0] for port in obj_port_list]        
    161.             y_lists = [port[1] for port in obj_port_list]
    162.             
    163.             return min(x_lists), __get_dist(x_lists), min(y_lists), __get_dist(y_lists)
    164.         
    165.         obj_x_min, obj_w, obj_y_min, obj_h = __get_object_desc(shape['points'])
    166.                     
    167.         yolo_center_x= round(float((obj_x_min + obj_w / 2.0) / img_w), 6)
    168.         yolo_center_y = round(float((obj_y_min + obj_h / 2.0) / img_h), 6)
    169.         yolo_w = round(float(obj_w / img_w), 6)
    170.         yolo_h = round(float(obj_h / img_h), 6)
    171.             
    172.         label_id = self._label_id_map[shape['label']]
    173.         
    174.         return label_id, yolo_center_x, yolo_center_y, yolo_w, yolo_h
    175.     
    176.     def _save_yolo_label(self, json_name, label_dir_path, target_dir, yolo_obj_list):
    177.         txt_path = os.path.join(label_dir_path, 
    178.                                 target_dir, 
    179.                                 json_name.replace('.json', '.txt'))
    180.  
    181.         with open(txt_path, 'w+') as f:
    182.             for yolo_obj_idx, yolo_obj in enumerate(yolo_obj_list):
    183.                 yolo_obj_line = '%s %s %s %s %s\n' % yolo_obj \
    184.                     if yolo_obj_idx + 1 != len(yolo_obj_list) else \
    185.                     '%s %s %s %s %s' % yolo_obj
    186.                 f.write(yolo_obj_line)
    187.                 
    188.     def _save_yolo_image(self, json_data, json_name, image_dir_path, target_dir):
    189.         img_name = json_name.replace('.json', '.png')
    190.         img_path = os.path.join(image_dir_path, target_dir,img_name)
    191.         
    192.         if not os.path.exists(img_path):
    193.             img = utils.img_b64_to_arr(json_data['imageData'])
    194.             PIL.Image.fromarray(img).save(img_path)
    195.         
    196.         return img_path
    197.     
    198.     def _save_dataset_yaml(self):
    199.         yaml_path = os.path.join(self._json_dir, 'YOLODataset/', 'dataset.yaml')
    200.         
    201.         with open(yaml_path, 'w+') as yaml_file:
    202.             yaml_file.write('train: %s\n' % \
    203.                             os.path.join(self._image_dir_path, 'train/'))
    204.             yaml_file.write('val: %s\n\n' % \
    205.                             os.path.join(self._image_dir_path, 'val/'))
    206.             yaml_file.write('nc: %i\n\n' % len(self._label_id_map))
    207.             
    208.             names_str = ''
    209.             for label, _ in self._label_id_map.items():
    210.                 names_str += "'%s', " % label
    211.             names_str = names_str.rstrip(', ')
    212.             yaml_file.write('names: [%s]' % names_str)
    213.     
    214.  
    215. if __name__ == '__main__':
    216.     parser = argparse.ArgumentParser()
    217.     parser.add_argument('--json_dir',type=str,
    218.                         help='Please input the path of the labelme json files.')
    219.     parser.add_argument('--val_size',type=float, nargs='?', default=None,
    220.                         help='Please input the validation dataset size, for example 0.1 ')
    221.     parser.add_argument('--json_name',type=str, nargs='?', default=None,
    222.                         help='If you put json name, it would convert only one json file to YOLO.')
    223.     args = parser.parse_args(sys.argv[1:])
    224.          
    225.     convertor = Labelme2YOLO(args.json_dir)
    226.     if args.json_name is None:
    227.         convertor.convert(val_size=args.val_size)
    228.     else:
    229.         convertor.convert_one(args.json_name)

            将标注文件全部放在labelmeannotation文件夹内：

            启动脚本，参数1为这个文件夹，参数2为val占数据集的比例，比如是0.1。命令如下：

     python 222.py --json_dir labelmeannotation/ --val_size 0.1

            开始转换ing：

            转化完毕：

            转化好的文件在labelmeannotation文件夹内。

            我们把这个文件拷贝到yolo的主目录下：

            新建一个文件default_copy.yaml，里面有几个部分注意修改。

    1. # Ultralytics YOLO 🚀, AGPL-3.0 license
    2. # Default training settings and hyperparameters for medium-augmentation COCO training
    3.  
    4. task: detect  # (str) YOLO task, i.e. detect, segment, classify, pose
    5. mode: train  # (str) YOLO mode, i.e. train, val, predict, export, track, benchmark
    6.  
    7. # Train settings -------------------------------------------------------------------------------------------------------
    8. model:  # (str, optional) path to model file, i.e. yolov8n.pt, yolov8n.yaml
    9. data: /home/liuhongwei/Desktop/ultralytics/YOLODataset/dataset.yaml # (str, optional) path to data file, i.e. coco128.yaml
    10. epochs: 300  # (int) number of epochs to train for
    11. patience: 50  # (int) epochs to wait for no observable improvement for early stopping of training
    12. batch: -1  # (int) number of images per batch (-1 for AutoBatch)
    13. imgsz: 640  # (int | list) input images size as int for train and val modes, or list[w,h] for predict and export modes
    14. save: True  # (bool) save train checkpoints and predict results
    15. save_period: -1 # (int) Save checkpoint every x epochs (disabled if < 1)
    16. cache: False  # (bool) True/ram, disk or False. Use cache for data loading
    17. device: 0  # (int | str | list, optional) device to run on, i.e. cuda device=0 or device=0,1,2,3 or device=cpu
    18. workers: 16  # (int) number of worker threads for data loading (per RANK if DDP)
    19. project:  # (str, optional) project name
    20. name: /home/liuhongwei/Desktop/ultralytics/666/ # (str, optional) experiment name, results saved to 'project/name' directory
    21. exist_ok: False  # (bool) whether to overwrite existing experiment
    22. pretrained: True  # (bool | str) whether to use a pretrained model (bool) or a model to load weights from (str)
    23. optimizer: auto  # (str) optimizer to use, choices=[SGD, Adam, Adamax, AdamW, NAdam, RAdam, RMSProp, auto]
    24. verbose: True  # (bool) whether to print verbose output
    25. seed: 0  # (int) random seed for reproducibility
    26. deterministic: True  # (bool) whether to enable deterministic mode
    27. single_cls: False  # (bool) train multi-class data as single-class
    28. rect: False  # (bool) rectangular training if mode='train' or rectangular validation if mode='val'
    29. cos_lr: False  # (bool) use cosine learning rate scheduler
    30. close_mosaic: 10  # (int) disable mosaic augmentation for final epochs (0 to disable)
    31. resume: False  # (bool) resume training from last checkpoint
    32. amp: True  # (bool) Automatic Mixed Precision (AMP) training, choices=[True, False], True runs AMP check
    33. fraction: 1.0  # (float) dataset fraction to train on (default is 1.0, all images in train set)
    34. profile: False  # (bool) profile ONNX and TensorRT speeds during training for loggers
    35. freeze: None  # (int | list, optional) freeze first n layers, or freeze list of layer indices during training
    36. # Segmentation
    37. overlap_mask: True  # (bool) masks should overlap during training (segment train only)
    38. mask_ratio: 4  # (int) mask downsample ratio (segment train only)
    39. # Classification
    40. dropout: 0.0  # (float) use dropout regularization (classify train only)
    41.  
    42. # Val/Test settings ----------------------------------------------------------------------------------------------------
    43. val: True  # (bool) validate/test during training
    44. split: val  # (str) dataset split to use for validation, i.e. 'val', 'test' or 'train'
    45. save_json: False  # (bool) save results to JSON file
    46. save_hybrid: False  # (bool) save hybrid version of labels (labels + additional predictions)
    47. conf:  # (float, optional) object confidence threshold for detection (default 0.25 predict, 0.001 val)
    48. iou: 0.7  # (float) intersection over union (IoU) threshold for NMS
    49. max_det: 300  # (int) maximum number of detections per image
    50. half: False  # (bool) use half precision (FP16)
    51. dnn: False  # (bool) use OpenCV DNN for ONNX inference
    52. plots: True  # (bool) save plots during train/val
    53.  
    54. # Prediction settings --------------------------------------------------------------------------------------------------
    55. source:  # (str, optional) source directory for images or videos
    56. show: False  # (bool) show results if possible
    57. save_txt: False  # (bool) save results as .txt file
    58. save_conf: False  # (bool) save results with confidence scores
    59. save_crop: False  # (bool) save cropped images with results
    60. show_labels: True  # (bool) show object labels in plots
    61. show_conf: True  # (bool) show object confidence scores in plots
    62. vid_stride: 1  # (int) video frame-rate stride
    63. stream_buffer: False  # (bool) buffer all streaming frames (True) or return the most recent frame (False)
    64. line_width:   # (int, optional) line width of the bounding boxes, auto if missing
    65. visualize: False  # (bool) visualize model features
    66. augment: False  # (bool) apply image augmentation to prediction sources
    67. agnostic_nms: False  # (bool) class-agnostic NMS
    68. classes:  # (int | list[int], optional) filter results by class, i.e. classes=0, or classes=[0,2,3]
    69. retina_masks: False  # (bool) use high-resolution segmentation masks
    70. boxes: True  # (bool) Show boxes in segmentation predictions
    71.  
    72. # Export settings ------------------------------------------------------------------------------------------------------
    73. format: torchscript  # (str) format to export to, choices at https://docs.ultralytics.com/modes/export/#export-formats
    74. keras: False  # (bool) use Kera=s
    75. optimize: False  # (bool) TorchScript: optimize for mobile
    76. int8: False  # (bool) CoreML/TF INT8 quantization
    77. dynamic: False  # (bool) ONNX/TF/TensorRT: dynamic axes
    78. simplify: False  # (bool) ONNX: simplify model
    79. opset:  # (int, optional) ONNX: opset version
    80. workspace: 4  # (int) TensorRT: workspace size (GB)
    81. nms: False  # (bool) CoreML: add NMS
    82.  
    83. # Hyperparameters ------------------------------------------------------------------------------------------------------
    84. lr0: 0.01  # (float) initial learning rate (i.e. SGD=1E-2, Adam=1E-3)
    85. lrf: 0.01  # (float) final learning rate (lr0 * lrf)
    86. momentum: 0.937  # (float) SGD momentum/Adam beta1
    87. weight_decay: 0.0005  # (float) optimizer weight decay 5e-4
    88. warmup_epochs: 3.0  # (float) warmup epochs (fractions ok)
    89. warmup_momentum: 0.8  # (float) warmup initial momentum
    90. warmup_bias_lr: 0.1  # (float) warmup initial bias lr
    91. box: 7.5  # (float) box loss gain
    92. cls: 0.5  # (float) cls loss gain (scale with pixels)
    93. dfl: 1.5  # (float) dfl loss gain
    94. pose: 12.0  # (float) pose loss gain
    95. kobj: 1.0  # (float) keypoint obj loss gain
    96. label_smoothing: 0.0  # (float) label smoothing (fraction)
    97. nbs: 64  # (int) nominal batch size
    98. hsv_h: 0.015  # (float) image HSV-Hue augmentation (fraction)
    99. hsv_s: 0.7  # (float) image HSV-Saturation augmentation (fraction)
    100. hsv_v: 0.4  # (float) image HSV-Value augmentation (fraction)
    101. degrees: 0.0  # (float) image rotation (+/- deg)
    102. translate: 0.1  # (float) image translation (+/- fraction)
    103. scale: 0.5  # (float) image scale (+/- gain)
    104. shear: 0.0  # (float) image shear (+/- deg)
    105. perspective: 0.0  # (float) image perspective (+/- fraction), range 0-0.001
    106. flipud: 0.0  # (float) image flip up-down (probability)
    107. fliplr: 0.5  # (float) image flip left-right (probability)
    108. mosaic: 1.0  # (float) image mosaic (probability)
    109. mixup: 0.0  # (float) image mixup (probability)
    110. copy_paste: 0.0  # (float) segment copy-paste (probability)
    111.  
    112. # Custom config.yaml ---------------------------------------------------------------------------------------------------
    113. cfg:  # (str, optional) for overriding defaults.yaml
    114.  
    115. # Tracker settings ------------------------------------------------------------------------------------------------------
    116. tracker: botsort.yaml  # (str) tracker type, choices=[botsort.yaml, bytetrack.yaml]

            1.data：改成我们生成文件的dataset.yaml文件路径 其他就用我的就行了

    4 开始训练

            将YOLODataset文件夹移动到ultralytics文件夹（YOLO主目录），修改/home/liuhongwei/Desktop/ultralytics/YOLODataset/datasetsyaml文件，改成绝对路径。

    1. train: /home/liuhongwei/Desktop/ultralytics/YOLODataset/images/train/
    2. val: /home/liuhongwei/Desktop/ultralytics/YOLODataset/images/val/
    3.  
    4. nc: 6
    5.  
    6. names: ['ZD', 'XALYS', 'ZZS', 'BH', 'SQL', 'HPYS']

           打开我们的yolo环境输入开始训练

    yolo cfg=/home/liuhongwei/Desktop/ultralytics/default_copy.yaml

            开始训练！这里训练300个epoch！GPU一定显存一定要大！！！mAP一直提高，yolo内部损失函数计算做的很好。

            训练完毕！

            得到我们的训练结果：

    5 利用训练结果进行测试

            YOLOV8为我们封装好了测试的功能包:

            直接调用我的代码即可：

    1. from ultralytics import YOLO
    2.  
    3.  
    4.  
    5. # 预测
    6. model = YOLO("/home/liuhongwei/Desktop/ultralytics/YOLODataset/runs/detect/train2/weights/best.pt")
    7. source = "/home/liuhongwei/Desktop/ultralytics/test_images"
    8. results = model.predict(source,save=True,show_conf=True)  # predict on an image
    9. print('ok')

  • 相关阅读:
    模拟电路设计（34）---脉宽调制型开关电路
    矿井水深度除总氮有什么好工艺？硝酸根离子交换柱
    JsonPath 数据快速查找和提取工具
    大数据相关积累
    C#压缩图片
    Golang context 原理分析
    目标检测算法——YOLOv5/YOLOv7改进之结合​PP-LCNet（轻量级CPU网络）
    配置elasticsearch用windows account（AD）登录
    react中jest配置，解决node_modules报错esm无法解析的问题
    HTML(23)——垂直对齐方式
  • 原文地址：https://blog.csdn.net/qq_41694024/article/details/133985854