Add YOLOv5 tutorials to docs.ultralytics.com (#1657)
Co-authored-by: ayush chaurasia <ayush.chaurarsia@gmail.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: Sergiu Waxmann <47978446+sergiuwaxmann@users.noreply.github.com>single_channel
Glenn Jocher
2 years ago
committed by
GitHub
parent
ec10002a4a
commit
ccb6419835
|
Before Width: | Height: | Size: 5.3 KiB After Width: | Height: | Size: 9.4 KiB |
@ -1,74 +1,45 @@
|
|||||||
<div align="center">
|
<div align="center">
|
||||||
<a href="https://github.com/ultralytics/ultralytics" target="_blank">
|
<p>
|
||||||
|
<a href="https://github.com/ultralytics/ultralytics" target="_blank">
|
||||||
<img width="1024" src="https://raw.githubusercontent.com/ultralytics/assets/main/yolov8/banner-yolov8.png"></a>
|
<img width="1024" src="https://raw.githubusercontent.com/ultralytics/assets/main/yolov8/banner-yolov8.png"></a>
|
||||||
<br>
|
</p>
|
||||||
<a href="https://github.com/ultralytics/ultralytics/actions/workflows/ci.yaml"><img src="https://github.com/ultralytics/ultralytics/actions/workflows/ci.yaml/badge.svg" alt="Ultralytics CI"></a>
|
<a href="https://github.com/ultralytics/ultralytics/actions/workflows/ci.yaml"><img src="https://github.com/ultralytics/ultralytics/actions/workflows/ci.yaml/badge.svg" alt="Ultralytics CI"></a>
|
||||||
<a href="https://zenodo.org/badge/latestdoi/264818686"><img src="https://zenodo.org/badge/264818686.svg" alt="YOLOv8 Citation"></a>
|
<a href="https://zenodo.org/badge/latestdoi/264818686"><img src="https://zenodo.org/badge/264818686.svg" alt="YOLOv8 Citation"></a>
|
||||||
<a href="https://hub.docker.com/r/ultralytics/ultralytics"><img src="https://img.shields.io/docker/pulls/ultralytics/ultralytics?logo=docker" alt="Docker Pulls"></a>
|
<a href="https://hub.docker.com/r/ultralytics/ultralytics"><img src="https://img.shields.io/docker/pulls/ultralytics/ultralytics?logo=docker" alt="Docker Pulls"></a>
|
||||||
<br>
|
<br>
|
||||||
<a href="https://console.paperspace.com/github/ultralytics/ultralytics"><img src="https://assets.paperspace.io/img/gradient-badge.svg" alt="Run on Gradient"/></a>
|
<a href="https://console.paperspace.com/github/ultralytics/ultralytics"><img src="https://assets.paperspace.io/img/gradient-badge.svg" alt="Run on Gradient"/></a>
|
||||||
<a href="https://colab.research.google.com/github/ultralytics/ultralytics/blob/main/examples/tutorial.ipynb"><img src="https://colab.research.google.com/assets/colab-badge.svg" alt="Open In Colab"></a>
|
<a href="https://colab.research.google.com/github/ultralytics/ultralytics/blob/main/examples/tutorial.ipynb"><img src="https://colab.research.google.com/assets/colab-badge.svg" alt="Open In Colab"></a>
|
||||||
<a href="https://www.kaggle.com/ultralytics/yolov8"><img src="https://kaggle.com/static/images/open-in-kaggle.svg" alt="Open In Kaggle"></a>
|
<a href="https://www.kaggle.com/ultralytics/yolov8"><img src="https://kaggle.com/static/images/open-in-kaggle.svg" alt="Open In Kaggle"></a>
|
||||||
<br>
|
|
||||||
</div>
|
</div>
|
||||||
|
|
||||||
Welcome to the Ultralytics YOLOv8 documentation landing
|
Introducing [Ultralytics](https://ultralytics.com) [YOLOv8](https://github.com/ultralytics/ultralytics), the latest version of the acclaimed real-time object detection and image segmentation model. YOLOv8 is built on cutting-edge advancements in deep learning and computer vision, offering unparalleled performance in terms of speed and accuracy. Its streamlined design makes it suitable for various applications and easily adaptable to different hardware platforms, from edge devices to cloud APIs.
|
||||||
page! [Ultralytics YOLOv8](https://github.com/ultralytics/ultralytics) is the latest version of the YOLO (You Only Look
|
|
||||||
Once) object detection and image segmentation model developed by [Ultralytics](https://ultralytics.com). This page
|
|
||||||
serves as the starting point for exploring the various resources available to help you get started with YOLOv8 and
|
|
||||||
understand its features and capabilities.
|
|
||||||
|
|
||||||
The YOLOv8 model is designed to be fast, accurate, and easy to use, making it an excellent choice for a wide range of
|
Explore the YOLOv8 Docs, a comprehensive resource designed to help you understand and utilize its features and capabilities. Whether you are a seasoned machine learning practitioner or new to the field, this hub aims to maximize YOLOv8's potential in your projects
|
||||||
object detection and image segmentation tasks. It can be trained on large datasets and is capable of running on a
|
|
||||||
variety of hardware platforms, from CPUs to GPUs.
|
|
||||||
|
|
||||||
Whether you are a seasoned machine learning practitioner or new to the field, we hope that the resources on this page
|
## Where to Start
|
||||||
will help you get the most out of YOLOv8. For any bugs and feature requests please
|
|
||||||
visit [GitHub Issues](https://github.com/ultralytics/ultralytics/issues). For professional support
|
|
||||||
please [Contact Us](https://ultralytics.com/contact).
|
|
||||||
|
|
||||||
## A Brief History of YOLO
|
- **Install** `ultralytics` with pip and get up and running in minutes [:material-clock-fast: Get Started](quickstart.md){ .md-button }
|
||||||
|
- **Predict** new images and videos with YOLOv8 [:octicons-image-16: Predict on Images](modes/predict.md){ .md-button }
|
||||||
|
- **Train** a new YOLOv8 model on your own custom dataset [:fontawesome-solid-brain: Train a Model](modes/train.md){ .md-button }
|
||||||
|
- **Explore** YOLOv8 tasks like segment, classify, pose and track [:material-magnify-expand: Explore Tasks](tasks/index.md){ .md-button }
|
||||||
|
|
||||||
YOLO (You Only Look Once) is a popular object detection and image segmentation model developed by Joseph Redmon and Ali
|
## YOLO: A Brief History
|
||||||
Farhadi at the University of Washington. The first version of YOLO was released in 2015 and quickly gained popularity
|
|
||||||
due to its high speed and accuracy.
|
|
||||||
|
|
||||||
YOLOv2 was released in 2016 and improved upon the original model by incorporating batch normalization, anchor boxes, and
|
[YOLO](https://arxiv.org/abs/1506.02640) (You Only Look Once), a popular object detection and image segmentation model, was developed by Joseph Redmon and Ali Farhadi at the University of Washington. Launched in 2015, YOLO quickly gained popularity for its high speed and accuracy.
|
||||||
dimension clusters. YOLOv3 was released in 2018 and further improved the model's performance by using a more efficient
|
|
||||||
backbone network, adding a feature pyramid, and making use of focal loss.
|
|
||||||
|
|
||||||
In 2020, YOLOv4 was released which introduced a number of innovations such as the use of Mosaic data augmentation, a new
|
- [YOLOv2](https://arxiv.org/abs/1612.08242), released in 2016, improved the original model by incorporating batch normalization, anchor boxes, and dimension clusters.
|
||||||
anchor-free detection head, and a new loss function.
|
- [YOLOv3](https://pjreddie.com/media/files/papers/YOLOv3.pdf), launched in 2018, further enhanced the model's performance using a more efficient backbone network, multiple anchors and spatial pyramid pooling.
|
||||||
|
- [YOLOv4](https://arxiv.org/abs/2004.10934) was released in 2020, introducing innovations like Mosaic data augmentation, a new anchor-free detection head, and a new loss function.
|
||||||
|
- [YOLOv5](https://github.com/ultralytics/yolov5) further improved the model's performance and added new features such as hyperparameter optimization, integrated experiment tracking and automatic export to popular export formats.
|
||||||
|
- [YOLOv6](https://github.com/meituan/YOLOv6) was open-sourced by Meituan in 2022 and is in use in many of the company's autonomous delivery robots.
|
||||||
|
- [YOLOv7](https://github.com/WongKinYiu/yolov7) added additional tasks such as pose estimation on the COCO keypoints dataset.
|
||||||
|
|
||||||
In 2021, Ultralytics released [YOLOv5](https://github.com/ultralytics/yolov5), which further improved the model's
|
Since its launch YOLO has been employed in various applications, including autonomous vehicles, security and surveillance, and medical imaging, and has won several competitions like the COCO Object Detection Challenge and the DOTA Object Detection Challenge.
|
||||||
performance and added new features such as support for panoptic segmentation and object tracking.
|
|
||||||
|
|
||||||
YOLO has been widely used in a variety of applications, including autonomous vehicles, security and surveillance, and
|
|
||||||
medical imaging. It has also been used to win several competitions, such as the COCO Object Detection Challenge and the
|
|
||||||
DOTA Object Detection Challenge.
|
|
||||||
|
|
||||||
For more information about the history and development of YOLO, you can refer to the following references:
|
|
||||||
|
|
||||||
- Redmon, J., & Farhadi, A. (2015). You only look once: Unified, real-time object detection. In Proceedings of the IEEE
|
|
||||||
conference on computer vision and pattern recognition (pp. 779-788).
|
|
||||||
- Redmon, J., & Farhadi, A. (2016). YOLO9000: Better, faster, stronger. In Proceedings
|
|
||||||
|
|
||||||
## Ultralytics YOLOv8
|
## Ultralytics YOLOv8
|
||||||
|
|
||||||
[Ultralytics YOLOv8](https://github.com/ultralytics/ultralytics) is the latest version of the YOLO object detection and
|
[Ultralytics YOLOv8](https://github.com/ultralytics/ultralytics) is the latest version of the YOLO object detection and image segmentation model. As a cutting-edge, state-of-the-art (SOTA) model, YOLOv8 builds on the success of previous versions, introducing new features and improvements for enhanced performance, flexibility, and efficiency.
|
||||||
image segmentation model developed by Ultralytics. YOLOv8 is a cutting-edge, state-of-the-art (SOTA) model that builds
|
|
||||||
upon the success of previous YOLO versions and introduces new features and improvements to further boost performance and
|
|
||||||
flexibility.
|
|
||||||
|
|
||||||
One key feature of YOLOv8 is its extensibility. It is designed as a framework that supports all previous versions of
|
|
||||||
YOLO, making it easy to switch between different versions and compare their performance. This makes YOLOv8 an ideal
|
|
||||||
choice for users who want to take advantage of the latest YOLO technology while still being able to use their existing
|
|
||||||
YOLO models.
|
|
||||||
|
|
||||||
In addition to its extensibility, YOLOv8 includes a number of other innovations that make it an appealing choice for a
|
YOLOv8 is designed with a strong focus on speed, size, and accuracy, making it a compelling choice for various vision AI tasks. It outperforms previous versions by incorporating innovations like a new backbone network, a new anchor-free split head, and new loss functions. These improvements enable YOLOv8 to deliver superior results, while maintaining a compact size and exceptional speed.
|
||||||
wide range of object detection and image segmentation tasks. These include a new backbone network, a new anchor-free
|
|
||||||
detection head, and a new loss function. YOLOv8 is also highly efficient and can be run on a variety of hardware
|
|
||||||
platforms, from CPUs to GPUs.
|
|
||||||
|
|
||||||
Overall, YOLOv8 is a powerful and flexible tool for object detection and image segmentation that offers the best of both
|
Additionally, YOLOv8 supports a full range of vision AI tasks, including [detection](tasks/detect.md), [segmentation](tasks/segment.md), [pose estimation](tasks/keypoints.md), [tracking](modes/track.md), and [classification](tasks/classify.md). This versatility allows users to leverage YOLOv8's capabilities across diverse applications and domains.
|
||||||
worlds: the latest SOTA technology and the ability to use and compare all previous YOLO versions.
|
|
||||||
|
|||||||
@ -1,14 +1,33 @@
|
|||||||
|
/* Table format like GitHub ----------------------------------------------------------------------------------------- */
|
||||||
th, td {
|
th, td {
|
||||||
border: 0.5px solid var(--md-typeset-table-color);
|
border: 1px solid var(--md-typeset-table-color);
|
||||||
border-spacing: 0px;
|
border-spacing: 0;
|
||||||
border-bottom: none;
|
border-bottom: none;
|
||||||
border-left: none;
|
border-left: none;
|
||||||
border-top: none;
|
border-top: none;
|
||||||
}
|
}
|
||||||
|
|
||||||
.md-typeset__table {
|
.md-typeset__table {
|
||||||
min-width: 100%;
|
line-height: 1;
|
||||||
line-height: 1;
|
|
||||||
}
|
}
|
||||||
.md-typeset table:not([class]) {
|
|
||||||
display: table;
|
.md-typeset__table table:not([class]) {
|
||||||
|
font-size: .74rem;
|
||||||
|
border-right: none;
|
||||||
}
|
}
|
||||||
|
|
||||||
|
.md-typeset__table table:not([class]) td,
|
||||||
|
.md-typeset__table table:not([class]) th {
|
||||||
|
padding: 9px;
|
||||||
|
}
|
||||||
|
|
||||||
|
/* light mode alternating table bg colors */
|
||||||
|
.md-typeset__table tr:nth-child(2n) {
|
||||||
|
background-color: #f8f8f8;
|
||||||
|
}
|
||||||
|
|
||||||
|
/* dark mode alternating table bg colors */
|
||||||
|
[data-md-color-scheme="slate"] .md-typeset__table tr:nth-child(2n) {
|
||||||
|
background-color: hsla(var(--md-hue),25%,25%,1)
|
||||||
|
}
|
||||||
|
/* Table format like GitHub ----------------------------------------------------------------------------------------- */
|
||||||
|
|||||||
@ -0,0 +1,209 @@
|
|||||||
|
## 1. Model Structure
|
||||||
|
|
||||||
|
YOLOv5 (v6.0/6.1) consists of:
|
||||||
|
- **Backbone**: `New CSP-Darknet53`
|
||||||
|
- **Neck**: `SPPF`, `New CSP-PAN`
|
||||||
|
- **Head**: `YOLOv3 Head`
|
||||||
|
|
||||||
|
Model structure (`yolov5l.yaml`):
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
|
Some minor changes compared to previous versions:
|
||||||
|
|
||||||
|
1. Replace the `Focus` structure with `6x6 Conv2d`(more efficient, refer #4825)
|
||||||
|
2. Replace the `SPP` structure with `SPPF`(more than double the speed)
|
||||||
|
|
||||||
|
<details markdown>
|
||||||
|
<summary>test code</summary>
|
||||||
|
|
||||||
|
```python
|
||||||
|
import time
|
||||||
|
import torch
|
||||||
|
import torch.nn as nn
|
||||||
|
|
||||||
|
|
||||||
|
class SPP(nn.Module):
|
||||||
|
def __init__(self):
|
||||||
|
super().__init__()
|
||||||
|
self.maxpool1 = nn.MaxPool2d(5, 1, padding=2)
|
||||||
|
self.maxpool2 = nn.MaxPool2d(9, 1, padding=4)
|
||||||
|
self.maxpool3 = nn.MaxPool2d(13, 1, padding=6)
|
||||||
|
|
||||||
|
def forward(self, x):
|
||||||
|
o1 = self.maxpool1(x)
|
||||||
|
o2 = self.maxpool2(x)
|
||||||
|
o3 = self.maxpool3(x)
|
||||||
|
return torch.cat([x, o1, o2, o3], dim=1)
|
||||||
|
|
||||||
|
|
||||||
|
class SPPF(nn.Module):
|
||||||
|
def __init__(self):
|
||||||
|
super().__init__()
|
||||||
|
self.maxpool = nn.MaxPool2d(5, 1, padding=2)
|
||||||
|
|
||||||
|
def forward(self, x):
|
||||||
|
o1 = self.maxpool(x)
|
||||||
|
o2 = self.maxpool(o1)
|
||||||
|
o3 = self.maxpool(o2)
|
||||||
|
return torch.cat([x, o1, o2, o3], dim=1)
|
||||||
|
|
||||||
|
|
||||||
|
def main():
|
||||||
|
input_tensor = torch.rand(8, 32, 16, 16)
|
||||||
|
spp = SPP()
|
||||||
|
sppf = SPPF()
|
||||||
|
output1 = spp(input_tensor)
|
||||||
|
output2 = sppf(input_tensor)
|
||||||
|
|
||||||
|
print(torch.equal(output1, output2))
|
||||||
|
|
||||||
|
t_start = time.time()
|
||||||
|
for _ in range(100):
|
||||||
|
spp(input_tensor)
|
||||||
|
print(f"spp time: {time.time() - t_start}")
|
||||||
|
|
||||||
|
t_start = time.time()
|
||||||
|
for _ in range(100):
|
||||||
|
sppf(input_tensor)
|
||||||
|
print(f"sppf time: {time.time() - t_start}")
|
||||||
|
|
||||||
|
|
||||||
|
if __name__ == '__main__':
|
||||||
|
main()
|
||||||
|
```
|
||||||
|
|
||||||
|
result:
|
||||||
|
```
|
||||||
|
True
|
||||||
|
spp time: 0.5373051166534424
|
||||||
|
sppf time: 0.20780706405639648
|
||||||
|
```
|
||||||
|
|
||||||
|
</details>
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
|
## 2. Data Augmentation
|
||||||
|
|
||||||
|
- Mosaic
|
||||||
|
<img src="https://user-images.githubusercontent.com/31005897/159109235-c7aad8f2-1d4f-41f9-8d5f-b2fde6f2885e.png#pic_center" width=80%>
|
||||||
|
|
||||||
|
- Copy paste
|
||||||
|
<img src="https://user-images.githubusercontent.com/31005897/159116277-91b45033-6bec-4f82-afc4-41138866628e.png#pic_center" width=80%>
|
||||||
|
|
||||||
|
- Random affine(Rotation, Scale, Translation and Shear)
|
||||||
|
<img src="https://user-images.githubusercontent.com/31005897/159109326-45cd5acb-14fa-43e7-9235-0f21b0021c7d.png#pic_center" width=80%>
|
||||||
|
|
||||||
|
- MixUp
|
||||||
|
<img src="https://user-images.githubusercontent.com/31005897/159109361-3b24333b-f481-478b-ae00-df7838f0b5cd.png#pic_center" width=80%>
|
||||||
|
|
||||||
|
- Albumentations
|
||||||
|
- Augment HSV(Hue, Saturation, Value)
|
||||||
|
<img src="https://user-images.githubusercontent.com/31005897/159109407-83d100ba-1aba-4f4b-aa03-4f048f815981.png#pic_center" width=80%>
|
||||||
|
|
||||||
|
- Random horizontal flip
|
||||||
|
<img src="https://user-images.githubusercontent.com/31005897/159109429-0d44619a-a76a-49eb-bfc0-6709860c043e.png#pic_center" width=80%>
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
|
## 3. Training Strategies
|
||||||
|
|
||||||
|
- Multi-scale training(0.5~1.5x)
|
||||||
|
- AutoAnchor(For training custom data)
|
||||||
|
- Warmup and Cosine LR scheduler
|
||||||
|
- EMA(Exponential Moving Average)
|
||||||
|
- Mixed precision
|
||||||
|
- Evolve hyper-parameters
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
|
## 4. Others
|
||||||
|
|
||||||
|
### 4.1 Compute Losses
|
||||||
|
|
||||||
|
The YOLOv5 loss consists of three parts:
|
||||||
|
|
||||||
|
- Classes loss(BCE loss)
|
||||||
|
- Objectness loss(BCE loss)
|
||||||
|
- Location loss(CIoU loss)
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
|
### 4.2 Balance Losses
|
||||||
|
The objectness losses of the three prediction layers(`P3`, `P4`, `P5`) are weighted differently. The balance weights are `[4.0, 1.0, 0.4]` respectively.
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
|
### 4.3 Eliminate Grid Sensitivity
|
||||||
|
In YOLOv2 and YOLOv3, the formula for calculating the predicted target information is:
|
||||||
|
|
||||||
|
+c_x)
|
||||||
|
+c_y)
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
|
<img src="https://user-images.githubusercontent.com/31005897/158508027-8bf63c28-8290-467b-8a3e-4ad09235001a.png#pic_center" width=40%>
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
|
In YOLOv5, the formula is:
|
||||||
|
|
||||||
|
-0.5)+c_x)
|
||||||
|
-0.5)+c_y)
|
||||||
|
)^2)
|
||||||
|
)^2)
|
||||||
|
|
||||||
|
Compare the center point offset before and after scaling. The center point offset range is adjusted from (0, 1) to (-0.5, 1.5).
|
||||||
|
Therefore, offset can easily get 0 or 1.
|
||||||
|
|
||||||
|
<img src="https://user-images.githubusercontent.com/31005897/158508052-c24bc5e8-05c1-4154-ac97-2e1ec71f582e.png#pic_center" width=40%>
|
||||||
|
|
||||||
|
Compare the height and width scaling ratio(relative to anchor) before and after adjustment. The original yolo/darknet box equations have a serious flaw. Width and Height are completely unbounded as they are simply out=exp(in), which is dangerous, as it can lead to runaway gradients, instabilities, NaN losses and ultimately a complete loss of training. [refer this issue](https://github.com/ultralytics/yolov5/issues/471#issuecomment-662009779)
|
||||||
|
|
||||||
|
<img src="https://user-images.githubusercontent.com/31005897/158508089-5ac0c7a3-6358-44b7-863e-a6e45babb842.png#pic_center" width=40%>
|
||||||
|
|
||||||
|
|
||||||
|
### 4.4 Build Targets
|
||||||
|
Match positive samples:
|
||||||
|
|
||||||
|
- Calculate the aspect ratio of GT and Anchor Templates
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
|
)
|
||||||
|
|
||||||
|
)
|
||||||
|
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
|
<img src="https://user-images.githubusercontent.com/31005897/158508119-fbb2e483-7b8c-4975-8e1f-f510d367f8ff.png#pic_center" width=70%>
|
||||||
|
|
||||||
|
- Assign the successfully matched Anchor Templates to the corresponding cells
|
||||||
|
|
||||||
|
<img src="https://user-images.githubusercontent.com/31005897/158508771-b6e7cab4-8de6-47f9-9abf-cdf14c275dfe.png#pic_center" width=70%>
|
||||||
|
|
||||||
|
- Because the center point offset range is adjusted from (0, 1) to (-0.5, 1.5). GT Box can be assigned to more anchors.
|
||||||
|
|
||||||
|
<img src="https://user-images.githubusercontent.com/31005897/158508139-9db4e8c2-cf96-47e0-bc80-35d11512f296.png#pic_center" width=70%>
|
||||||
|
|
||||||
|
## Environments
|
||||||
|
|
||||||
|
YOLOv5 may be run in any of the following up-to-date verified environments (with all dependencies including [CUDA](https://developer.nvidia.com/cuda)/[CUDNN](https://developer.nvidia.com/cudnn), [Python](https://www.python.org/) and [PyTorch](https://pytorch.org/) preinstalled):
|
||||||
|
|
||||||
|
- **Notebooks** with free GPU: <a href="https://bit.ly/yolov5-paperspace-notebook"><img src="https://assets.paperspace.io/img/gradient-badge.svg" alt="Run on Gradient"></a> <a href="https://colab.research.google.com/github/ultralytics/yolov5/blob/master/tutorial.ipynb"><img src="https://colab.research.google.com/assets/colab-badge.svg" alt="Open In Colab"></a> <a href="https://www.kaggle.com/ultralytics/yolov5"><img src="https://kaggle.com/static/images/open-in-kaggle.svg" alt="Open In Kaggle"></a>
|
||||||
|
- **Google Cloud** Deep Learning VM. See [GCP Quickstart Guide](https://github.com/ultralytics/yolov5/wiki/GCP-Quickstart)
|
||||||
|
- **Amazon** Deep Learning AMI. See [AWS Quickstart Guide](https://github.com/ultralytics/yolov5/wiki/AWS-Quickstart)
|
||||||
|
- **Docker Image**. See [Docker Quickstart Guide](https://github.com/ultralytics/yolov5/wiki/Docker-Quickstart) <a href="https://hub.docker.com/r/ultralytics/yolov5"><img src="https://img.shields.io/docker/pulls/ultralytics/yolov5?logo=docker" alt="Docker Pulls"></a>
|
||||||
|
|
||||||
|
|
||||||
|
## Status
|
||||||
|
|
||||||
|
<a href="https://github.com/ultralytics/yolov5/actions/workflows/ci-testing.yml"><img src="https://github.com/ultralytics/yolov5/actions/workflows/ci-testing.yml/badge.svg" alt="YOLOv5 CI"></a>
|
||||||
|
|
||||||
|
If this badge is green, all [YOLOv5 GitHub Actions](https://github.com/ultralytics/yolov5/actions) Continuous Integration (CI) tests are currently passing. CI tests verify correct operation of YOLOv5 [training](https://github.com/ultralytics/yolov5/blob/master/train.py), [validation](https://github.com/ultralytics/yolov5/blob/master/val.py), [inference](https://github.com/ultralytics/yolov5/blob/master/detect.py), [export](https://github.com/ultralytics/yolov5/blob/master/export.py) and [benchmarks](https://github.com/ultralytics/yolov5/blob/master/benchmarks.py) on macOS, Windows, and Ubuntu every 24 hours and on every commit.
|
||||||
@ -0,0 +1,237 @@
|
|||||||
|
# ClearML Integration
|
||||||
|
|
||||||
|
<img align="center" src="https://github.com/thepycoder/clearml_screenshots/raw/main/logos_dark.png#gh-light-mode-only" alt="Clear|ML"><img align="center" src="https://github.com/thepycoder/clearml_screenshots/raw/main/logos_light.png#gh-dark-mode-only" alt="Clear|ML">
|
||||||
|
|
||||||
|
## About ClearML
|
||||||
|
|
||||||
|
[ClearML](https://cutt.ly/yolov5-tutorial-clearml) is an [open-source](https://github.com/allegroai/clearml) toolbox designed to save you time ⏱️.
|
||||||
|
|
||||||
|
🔨 Track every YOLOv5 training run in the <b>experiment manager</b>
|
||||||
|
|
||||||
|
🔧 Version and easily access your custom training data with the integrated ClearML <b>Data Versioning Tool</b>
|
||||||
|
|
||||||
|
🔦 <b>Remotely train and monitor</b> your YOLOv5 training runs using ClearML Agent
|
||||||
|
|
||||||
|
🔬 Get the very best mAP using ClearML <b>Hyperparameter Optimization</b>
|
||||||
|
|
||||||
|
🔭 Turn your newly trained <b>YOLOv5 model into an API</b> with just a few commands using ClearML Serving
|
||||||
|
|
||||||
|
<br />
|
||||||
|
And so much more. It's up to you how many of these tools you want to use, you can stick to the experiment manager, or chain them all together into an impressive pipeline!
|
||||||
|
<br />
|
||||||
|
<br />
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
|
<br />
|
||||||
|
<br />
|
||||||
|
|
||||||
|
## 🦾 Setting Things Up
|
||||||
|
|
||||||
|
To keep track of your experiments and/or data, ClearML needs to communicate to a server. You have 2 options to get one:
|
||||||
|
|
||||||
|
Either sign up for free to the [ClearML Hosted Service](https://cutt.ly/yolov5-tutorial-clearml) or you can set up your own server, see [here](https://clear.ml/docs/latest/docs/deploying_clearml/clearml_server). Even the server is open-source, so even if you're dealing with sensitive data, you should be good to go!
|
||||||
|
|
||||||
|
1. Install the `clearml` python package:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
pip install clearml
|
||||||
|
```
|
||||||
|
|
||||||
|
2. Connect the ClearML SDK to the server by [creating credentials](https://app.clear.ml/settings/workspace-configuration) (go right top to Settings -> Workspace -> Create new credentials), then execute the command below and follow the instructions:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
clearml-init
|
||||||
|
```
|
||||||
|
|
||||||
|
That's it! You're done 😎
|
||||||
|
|
||||||
|
<br />
|
||||||
|
|
||||||
|
## 🚀 Training YOLOv5 With ClearML
|
||||||
|
|
||||||
|
To enable ClearML experiment tracking, simply install the ClearML pip package.
|
||||||
|
|
||||||
|
```bash
|
||||||
|
pip install clearml>=1.2.0
|
||||||
|
```
|
||||||
|
|
||||||
|
This will enable integration with the YOLOv5 training script. Every training run from now on, will be captured and stored by the ClearML experiment manager.
|
||||||
|
|
||||||
|
If you want to change the `project_name` or `task_name`, use the `--project` and `--name` arguments of the `train.py` script, by default the project will be called `YOLOv5` and the task `Training`.
|
||||||
|
PLEASE NOTE: ClearML uses `/` as a delimiter for subprojects, so be careful when using `/` in your project name!
|
||||||
|
|
||||||
|
```bash
|
||||||
|
python train.py --img 640 --batch 16 --epochs 3 --data coco128.yaml --weights yolov5s.pt --cache
|
||||||
|
```
|
||||||
|
|
||||||
|
or with custom project and task name:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
python train.py --project my_project --name my_training --img 640 --batch 16 --epochs 3 --data coco128.yaml --weights yolov5s.pt --cache
|
||||||
|
```
|
||||||
|
|
||||||
|
This will capture:
|
||||||
|
|
||||||
|
- Source code + uncommitted changes
|
||||||
|
- Installed packages
|
||||||
|
- (Hyper)parameters
|
||||||
|
- Model files (use `--save-period n` to save a checkpoint every n epochs)
|
||||||
|
- Console output
|
||||||
|
- Scalars (mAP_0.5, mAP_0.5:0.95, precision, recall, losses, learning rates, ...)
|
||||||
|
- General info such as machine details, runtime, creation date etc.
|
||||||
|
- All produced plots such as label correlogram and confusion matrix
|
||||||
|
- Images with bounding boxes per epoch
|
||||||
|
- Mosaic per epoch
|
||||||
|
- Validation images per epoch
|
||||||
|
- ...
|
||||||
|
|
||||||
|
That's a lot right? 🤯
|
||||||
|
Now, we can visualize all of this information in the ClearML UI to get an overview of our training progress. Add custom columns to the table view (such as e.g. mAP_0.5) so you can easily sort on the best performing model. Or select multiple experiments and directly compare them!
|
||||||
|
|
||||||
|
There even more we can do with all of this information, like hyperparameter optimization and remote execution, so keep reading if you want to see how that works!
|
||||||
|
|
||||||
|
<br />
|
||||||
|
|
||||||
|
## 🔗 Dataset Version Management
|
||||||
|
|
||||||
|
Versioning your data separately from your code is generally a good idea and makes it easy to acquire the latest version too. This repository supports supplying a dataset version ID, and it will make sure to get the data if it's not there yet. Next to that, this workflow also saves the used dataset ID as part of the task parameters, so you will always know for sure which data was used in which experiment!
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
|
### Prepare Your Dataset
|
||||||
|
|
||||||
|
The YOLOv5 repository supports a number of different datasets by using yaml files containing their information. By default datasets are downloaded to the `../datasets` folder in relation to the repository root folder. So if you downloaded the `coco128` dataset using the link in the yaml or with the scripts provided by yolov5, you get this folder structure:
|
||||||
|
|
||||||
|
```
|
||||||
|
..
|
||||||
|
|_ yolov5
|
||||||
|
|_ datasets
|
||||||
|
|_ coco128
|
||||||
|
|_ images
|
||||||
|
|_ labels
|
||||||
|
|_ LICENSE
|
||||||
|
|_ README.txt
|
||||||
|
```
|
||||||
|
|
||||||
|
But this can be any dataset you wish. Feel free to use your own, as long as you keep to this folder structure.
|
||||||
|
|
||||||
|
Next, ⚠️**copy the corresponding yaml file to the root of the dataset folder**⚠️. This yaml files contains the information ClearML will need to properly use the dataset. You can make this yourself too, of course, just follow the structure of the example yamls.
|
||||||
|
|
||||||
|
Basically we need the following keys: `path`, `train`, `test`, `val`, `nc`, `names`.
|
||||||
|
|
||||||
|
```
|
||||||
|
..
|
||||||
|
|_ yolov5
|
||||||
|
|_ datasets
|
||||||
|
|_ coco128
|
||||||
|
|_ images
|
||||||
|
|_ labels
|
||||||
|
|_ coco128.yaml # <---- HERE!
|
||||||
|
|_ LICENSE
|
||||||
|
|_ README.txt
|
||||||
|
```
|
||||||
|
|
||||||
|
### Upload Your Dataset
|
||||||
|
|
||||||
|
To get this dataset into ClearML as a versioned dataset, go to the dataset root folder and run the following command:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
cd coco128
|
||||||
|
clearml-data sync --project YOLOv5 --name coco128 --folder .
|
||||||
|
```
|
||||||
|
|
||||||
|
The command `clearml-data sync` is actually a shorthand command. You could also run these commands one after the other:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
# Optionally add --parent <parent_dataset_id> if you want to base
|
||||||
|
# this version on another dataset version, so no duplicate files are uploaded!
|
||||||
|
clearml-data create --name coco128 --project YOLOv5
|
||||||
|
clearml-data add --files .
|
||||||
|
clearml-data close
|
||||||
|
```
|
||||||
|
|
||||||
|
### Run Training Using A ClearML Dataset
|
||||||
|
|
||||||
|
Now that you have a ClearML dataset, you can very simply use it to train custom YOLOv5 🚀 models!
|
||||||
|
|
||||||
|
```bash
|
||||||
|
python train.py --img 640 --batch 16 --epochs 3 --data clearml://<your_dataset_id> --weights yolov5s.pt --cache
|
||||||
|
```
|
||||||
|
|
||||||
|
<br />
|
||||||
|
|
||||||
|
## 👀 Hyperparameter Optimization
|
||||||
|
|
||||||
|
Now that we have our experiments and data versioned, it's time to take a look at what we can build on top!
|
||||||
|
|
||||||
|
Using the code information, installed packages and environment details, the experiment itself is now **completely reproducible**. In fact, ClearML allows you to clone an experiment and even change its parameters. We can then just rerun it with these new parameters automatically, this is basically what HPO does!
|
||||||
|
|
||||||
|
To **run hyperparameter optimization locally**, we've included a pre-made script for you. Just make sure a training task has been run at least once, so it is in the ClearML experiment manager, we will essentially clone it and change its hyperparameters.
|
||||||
|
|
||||||
|
You'll need to fill in the ID of this `template task` in the script found at `utils/loggers/clearml/hpo.py` and then just run it :) You can change `task.execute_locally()` to `task.execute()` to put it in a ClearML queue and have a remote agent work on it instead.
|
||||||
|
|
||||||
|
```bash
|
||||||
|
# To use optuna, install it first, otherwise you can change the optimizer to just be RandomSearch
|
||||||
|
pip install optuna
|
||||||
|
python utils/loggers/clearml/hpo.py
|
||||||
|
```
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
|
## 🤯 Remote Execution (advanced)
|
||||||
|
|
||||||
|
Running HPO locally is really handy, but what if we want to run our experiments on a remote machine instead? Maybe you have access to a very powerful GPU machine on-site, or you have some budget to use cloud GPUs.
|
||||||
|
This is where the ClearML Agent comes into play. Check out what the agent can do here:
|
||||||
|
|
||||||
|
- [YouTube video](https://youtu.be/MX3BrXnaULs)
|
||||||
|
- [Documentation](https://clear.ml/docs/latest/docs/clearml_agent)
|
||||||
|
|
||||||
|
In short: every experiment tracked by the experiment manager contains enough information to reproduce it on a different machine (installed packages, uncommitted changes etc.). So a ClearML agent does just that: it listens to a queue for incoming tasks and when it finds one, it recreates the environment and runs it while still reporting scalars, plots etc. to the experiment manager.
|
||||||
|
|
||||||
|
You can turn any machine (a cloud VM, a local GPU machine, your own laptop ... ) into a ClearML agent by simply running:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
clearml-agent daemon --queue <queues_to_listen_to> [--docker]
|
||||||
|
```
|
||||||
|
|
||||||
|
### Cloning, Editing And Enqueuing
|
||||||
|
|
||||||
|
With our agent running, we can give it some work. Remember from the HPO section that we can clone a task and edit the hyperparameters? We can do that from the interface too!
|
||||||
|
|
||||||
|
🪄 Clone the experiment by right-clicking it
|
||||||
|
|
||||||
|
🎯 Edit the hyperparameters to what you wish them to be
|
||||||
|
|
||||||
|
⏳ Enqueue the task to any of the queues by right-clicking it
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
|
### Executing A Task Remotely
|
||||||
|
|
||||||
|
Now you can clone a task like we explained above, or simply mark your current script by adding `task.execute_remotely()` and on execution it will be put into a queue, for the agent to start working on!
|
||||||
|
|
||||||
|
To run the YOLOv5 training script remotely, all you have to do is add this line to the training.py script after the clearml logger has been instantiated:
|
||||||
|
|
||||||
|
```python
|
||||||
|
# ...
|
||||||
|
# Loggers
|
||||||
|
data_dict = None
|
||||||
|
if RANK in {-1, 0}:
|
||||||
|
loggers = Loggers(save_dir, weights, opt, hyp, LOGGER) # loggers instance
|
||||||
|
if loggers.clearml:
|
||||||
|
loggers.clearml.task.execute_remotely(queue="my_queue") # <------ ADD THIS LINE
|
||||||
|
# Data_dict is either None is user did not choose for ClearML dataset or is filled in by ClearML
|
||||||
|
data_dict = loggers.clearml.data_dict
|
||||||
|
# ...
|
||||||
|
```
|
||||||
|
|
||||||
|
When running the training script after this change, python will run the script up until that line, after which it will package the code and send it to the queue instead!
|
||||||
|
|
||||||
|
### Autoscaling workers
|
||||||
|
|
||||||
|
ClearML comes with autoscalers too! This tool will automatically spin up new remote machines in the cloud of your choice (AWS, GCP, Azure) and turn them into ClearML agents for you whenever there are experiments detected in the queue. Once the tasks are processed, the autoscaler will automatically shut down the remote machines, and you stop paying!
|
||||||
|
|
||||||
|
Check out the autoscalers getting started video below.
|
||||||
|
|
||||||
|
[](https://youtu.be/j4XVMAaUt3E)
|
||||||
@ -0,0 +1,258 @@
|
|||||||
|
<img src="https://cdn.comet.ml/img/notebook_logo.png">
|
||||||
|
|
||||||
|
# YOLOv5 with Comet
|
||||||
|
|
||||||
|
This guide will cover how to use YOLOv5 with [Comet](https://bit.ly/yolov5-readme-comet2)
|
||||||
|
|
||||||
|
# About Comet
|
||||||
|
|
||||||
|
Comet builds tools that help data scientists, engineers, and team leaders accelerate and optimize machine learning and deep learning models.
|
||||||
|
|
||||||
|
Track and visualize model metrics in real time, save your hyperparameters, datasets, and model checkpoints, and visualize your model predictions with [Comet Custom Panels](https://www.comet.com/docs/v2/guides/comet-dashboard/code-panels/about-panels/?utm_source=yolov5&utm_medium=partner&utm_campaign=partner_yolov5_2022&utm_content=github)!
|
||||||
|
Comet makes sure you never lose track of your work and makes it easy to share results and collaborate across teams of all sizes!
|
||||||
|
|
||||||
|
# Getting Started
|
||||||
|
|
||||||
|
## Install Comet
|
||||||
|
|
||||||
|
```shell
|
||||||
|
pip install comet_ml
|
||||||
|
```
|
||||||
|
|
||||||
|
## Configure Comet Credentials
|
||||||
|
|
||||||
|
There are two ways to configure Comet with YOLOv5.
|
||||||
|
|
||||||
|
You can either set your credentials through environment variables
|
||||||
|
|
||||||
|
**Environment Variables**
|
||||||
|
|
||||||
|
```shell
|
||||||
|
export COMET_API_KEY=<Your Comet API Key>
|
||||||
|
export COMET_PROJECT_NAME=<Your Comet Project Name> # This will default to 'yolov5'
|
||||||
|
```
|
||||||
|
|
||||||
|
Or create a `.comet.config` file in your working directory and set your credentials there.
|
||||||
|
|
||||||
|
**Comet Configuration File**
|
||||||
|
|
||||||
|
```
|
||||||
|
[comet]
|
||||||
|
api_key=<Your Comet API Key>
|
||||||
|
project_name=<Your Comet Project Name> # This will default to 'yolov5'
|
||||||
|
```
|
||||||
|
|
||||||
|
## Run the Training Script
|
||||||
|
|
||||||
|
```shell
|
||||||
|
# Train YOLOv5s on COCO128 for 5 epochs
|
||||||
|
python train.py --img 640 --batch 16 --epochs 5 --data coco128.yaml --weights yolov5s.pt
|
||||||
|
```
|
||||||
|
|
||||||
|
That's it! Comet will automatically log your hyperparameters, command line arguments, training and validation metrics. You can visualize and analyze your runs in the Comet UI
|
||||||
|
|
||||||
|
<img width="1920" alt="yolo-ui" src="https://user-images.githubusercontent.com/26833433/202851203-164e94e1-2238-46dd-91f8-de020e9d6b41.png">
|
||||||
|
|
||||||
|
# Try out an Example!
|
||||||
|
|
||||||
|
Check out an example of a [completed run here](https://www.comet.com/examples/comet-example-yolov5/a0e29e0e9b984e4a822db2a62d0cb357?experiment-tab=chart&showOutliers=true&smoothing=0&transformY=smoothing&xAxis=step&utm_source=yolov5&utm_medium=partner&utm_campaign=partner_yolov5_2022&utm_content=github)
|
||||||
|
|
||||||
|
Or better yet, try it out yourself in this Colab Notebook
|
||||||
|
|
||||||
|
[](https://colab.research.google.com/drive/1RG0WOQyxlDlo5Km8GogJpIEJlg_5lyYO?usp=sharing)
|
||||||
|
|
||||||
|
# Log automatically
|
||||||
|
|
||||||
|
By default, Comet will log the following items
|
||||||
|
|
||||||
|
## Metrics
|
||||||
|
|
||||||
|
- Box Loss, Object Loss, Classification Loss for the training and validation data
|
||||||
|
- mAP_0.5, mAP_0.5:0.95 metrics for the validation data.
|
||||||
|
- Precision and Recall for the validation data
|
||||||
|
|
||||||
|
## Parameters
|
||||||
|
|
||||||
|
- Model Hyperparameters
|
||||||
|
- All parameters passed through the command line options
|
||||||
|
|
||||||
|
## Visualizations
|
||||||
|
|
||||||
|
- Confusion Matrix of the model predictions on the validation data
|
||||||
|
- Plots for the PR and F1 curves across all classes
|
||||||
|
- Correlogram of the Class Labels
|
||||||
|
|
||||||
|
# Configure Comet Logging
|
||||||
|
|
||||||
|
Comet can be configured to log additional data either through command line flags passed to the training script
|
||||||
|
or through environment variables.
|
||||||
|
|
||||||
|
```shell
|
||||||
|
export COMET_MODE=online # Set whether to run Comet in 'online' or 'offline' mode. Defaults to online
|
||||||
|
export COMET_MODEL_NAME=<your model name> #Set the name for the saved model. Defaults to yolov5
|
||||||
|
export COMET_LOG_CONFUSION_MATRIX=false # Set to disable logging a Comet Confusion Matrix. Defaults to true
|
||||||
|
export COMET_MAX_IMAGE_UPLOADS=<number of allowed images to upload to Comet> # Controls how many total image predictions to log to Comet. Defaults to 100.
|
||||||
|
export COMET_LOG_PER_CLASS_METRICS=true # Set to log evaluation metrics for each detected class at the end of training. Defaults to false
|
||||||
|
export COMET_DEFAULT_CHECKPOINT_FILENAME=<your checkpoint filename> # Set this if you would like to resume training from a different checkpoint. Defaults to 'last.pt'
|
||||||
|
export COMET_LOG_BATCH_LEVEL_METRICS=true # Set this if you would like to log training metrics at the batch level. Defaults to false.
|
||||||
|
export COMET_LOG_PREDICTIONS=true # Set this to false to disable logging model predictions
|
||||||
|
```
|
||||||
|
|
||||||
|
## Logging Checkpoints with Comet
|
||||||
|
|
||||||
|
Logging Models to Comet is disabled by default. To enable it, pass the `save-period` argument to the training script. This will save the
|
||||||
|
logged checkpoints to Comet based on the interval value provided by `save-period`
|
||||||
|
|
||||||
|
```shell
|
||||||
|
python train.py \
|
||||||
|
--img 640 \
|
||||||
|
--batch 16 \
|
||||||
|
--epochs 5 \
|
||||||
|
--data coco128.yaml \
|
||||||
|
--weights yolov5s.pt \
|
||||||
|
--save-period 1
|
||||||
|
```
|
||||||
|
|
||||||
|
## Logging Model Predictions
|
||||||
|
|
||||||
|
By default, model predictions (images, ground truth labels and bounding boxes) will be logged to Comet.
|
||||||
|
|
||||||
|
You can control the frequency of logged predictions and the associated images by passing the `bbox_interval` command line argument. Predictions can be visualized using Comet's Object Detection Custom Panel. This frequency corresponds to every Nth batch of data per epoch. In the example below, we are logging every 2nd batch of data for each epoch.
|
||||||
|
|
||||||
|
**Note:** The YOLOv5 validation dataloader will default to a batch size of 32, so you will have to set the logging frequency accordingly.
|
||||||
|
|
||||||
|
Here is an [example project using the Panel](https://www.comet.com/examples/comet-example-yolov5?shareable=YcwMiJaZSXfcEXpGOHDD12vA1&utm_source=yolov5&utm_medium=partner&utm_campaign=partner_yolov5_2022&utm_content=github)
|
||||||
|
|
||||||
|
```shell
|
||||||
|
python train.py \
|
||||||
|
--img 640 \
|
||||||
|
--batch 16 \
|
||||||
|
--epochs 5 \
|
||||||
|
--data coco128.yaml \
|
||||||
|
--weights yolov5s.pt \
|
||||||
|
--bbox_interval 2
|
||||||
|
```
|
||||||
|
|
||||||
|
### Controlling the number of Prediction Images logged to Comet
|
||||||
|
|
||||||
|
When logging predictions from YOLOv5, Comet will log the images associated with each set of predictions. By default a maximum of 100 validation images are logged. You can increase or decrease this number using the `COMET_MAX_IMAGE_UPLOADS` environment variable.
|
||||||
|
|
||||||
|
```shell
|
||||||
|
env COMET_MAX_IMAGE_UPLOADS=200 python train.py \
|
||||||
|
--img 640 \
|
||||||
|
--batch 16 \
|
||||||
|
--epochs 5 \
|
||||||
|
--data coco128.yaml \
|
||||||
|
--weights yolov5s.pt \
|
||||||
|
--bbox_interval 1
|
||||||
|
```
|
||||||
|
|
||||||
|
### Logging Class Level Metrics
|
||||||
|
|
||||||
|
Use the `COMET_LOG_PER_CLASS_METRICS` environment variable to log mAP, precision, recall, f1 for each class.
|
||||||
|
|
||||||
|
```shell
|
||||||
|
env COMET_LOG_PER_CLASS_METRICS=true python train.py \
|
||||||
|
--img 640 \
|
||||||
|
--batch 16 \
|
||||||
|
--epochs 5 \
|
||||||
|
--data coco128.yaml \
|
||||||
|
--weights yolov5s.pt
|
||||||
|
```
|
||||||
|
|
||||||
|
## Uploading a Dataset to Comet Artifacts
|
||||||
|
|
||||||
|
If you would like to store your data using [Comet Artifacts](https://www.comet.com/docs/v2/guides/data-management/using-artifacts/#learn-more?utm_source=yolov5&utm_medium=partner&utm_campaign=partner_yolov5_2022&utm_content=github), you can do so using the `upload_dataset` flag.
|
||||||
|
|
||||||
|
The dataset be organized in the way described in the [YOLOv5 documentation](train_custom_data.md). The dataset config `yaml` file must follow the same format as that of the `coco128.yaml` file.
|
||||||
|
|
||||||
|
```shell
|
||||||
|
python train.py \
|
||||||
|
--img 640 \
|
||||||
|
--batch 16 \
|
||||||
|
--epochs 5 \
|
||||||
|
--data coco128.yaml \
|
||||||
|
--weights yolov5s.pt \
|
||||||
|
--upload_dataset
|
||||||
|
```
|
||||||
|
|
||||||
|
You can find the uploaded dataset in the Artifacts tab in your Comet Workspace
|
||||||
|
<img width="1073" alt="artifact-1" src="https://user-images.githubusercontent.com/7529846/186929193-162718bf-ec7b-4eb9-8c3b-86b3763ef8ea.png">
|
||||||
|
|
||||||
|
You can preview the data directly in the Comet UI.
|
||||||
|
<img width="1082" alt="artifact-2" src="https://user-images.githubusercontent.com/7529846/186929215-432c36a9-c109-4eb0-944b-84c2786590d6.png">
|
||||||
|
|
||||||
|
Artifacts are versioned and also support adding metadata about the dataset. Comet will automatically log the metadata from your dataset `yaml` file
|
||||||
|
<img width="963" alt="artifact-3" src="https://user-images.githubusercontent.com/7529846/186929256-9d44d6eb-1a19-42de-889a-bcbca3018f2e.png">
|
||||||
|
|
||||||
|
### Using a saved Artifact
|
||||||
|
|
||||||
|
If you would like to use a dataset from Comet Artifacts, set the `path` variable in your dataset `yaml` file to point to the following Artifact resource URL.
|
||||||
|
|
||||||
|
```
|
||||||
|
# contents of artifact.yaml file
|
||||||
|
path: "comet://<workspace name>/<artifact name>:<artifact version or alias>"
|
||||||
|
```
|
||||||
|
|
||||||
|
Then pass this file to your training script in the following way
|
||||||
|
|
||||||
|
```shell
|
||||||
|
python train.py \
|
||||||
|
--img 640 \
|
||||||
|
--batch 16 \
|
||||||
|
--epochs 5 \
|
||||||
|
--data artifact.yaml \
|
||||||
|
--weights yolov5s.pt
|
||||||
|
```
|
||||||
|
|
||||||
|
Artifacts also allow you to track the lineage of data as it flows through your Experimentation workflow. Here you can see a graph that shows you all the experiments that have used your uploaded dataset.
|
||||||
|
<img width="1391" alt="artifact-4" src="https://user-images.githubusercontent.com/7529846/186929264-4c4014fa-fe51-4f3c-a5c5-f6d24649b1b4.png">
|
||||||
|
|
||||||
|
## Resuming a Training Run
|
||||||
|
|
||||||
|
If your training run is interrupted for any reason, e.g. disrupted internet connection, you can resume the run using the `resume` flag and the Comet Run Path.
|
||||||
|
|
||||||
|
The Run Path has the following format `comet://<your workspace name>/<your project name>/<experiment id>`.
|
||||||
|
|
||||||
|
This will restore the run to its state before the interruption, which includes restoring the model from a checkpoint, restoring all hyperparameters and training arguments and downloading Comet dataset Artifacts if they were used in the original run. The resumed run will continue logging to the existing Experiment in the Comet UI
|
||||||
|
|
||||||
|
```shell
|
||||||
|
python train.py \
|
||||||
|
--resume "comet://<your run path>"
|
||||||
|
```
|
||||||
|
|
||||||
|
## Hyperparameter Search with the Comet Optimizer
|
||||||
|
|
||||||
|
YOLOv5 is also integrated with Comet's Optimizer, making is simple to visualize hyperparameter sweeps in the Comet UI.
|
||||||
|
|
||||||
|
### Configuring an Optimizer Sweep
|
||||||
|
|
||||||
|
To configure the Comet Optimizer, you will have to create a JSON file with the information about the sweep. An example file has been provided in `utils/loggers/comet/optimizer_config.json`
|
||||||
|
|
||||||
|
```shell
|
||||||
|
python utils/loggers/comet/hpo.py \
|
||||||
|
--comet_optimizer_config "utils/loggers/comet/optimizer_config.json"
|
||||||
|
```
|
||||||
|
|
||||||
|
The `hpo.py` script accepts the same arguments as `train.py`. If you wish to pass additional arguments to your sweep simply add them after
|
||||||
|
the script.
|
||||||
|
|
||||||
|
```shell
|
||||||
|
python utils/loggers/comet/hpo.py \
|
||||||
|
--comet_optimizer_config "utils/loggers/comet/optimizer_config.json" \
|
||||||
|
--save-period 1 \
|
||||||
|
--bbox_interval 1
|
||||||
|
```
|
||||||
|
|
||||||
|
### Running a Sweep in Parallel
|
||||||
|
|
||||||
|
```shell
|
||||||
|
comet optimizer -j <set number of workers> utils/loggers/comet/hpo.py \
|
||||||
|
utils/loggers/comet/optimizer_config.json"
|
||||||
|
```
|
||||||
|
|
||||||
|
### Visualizing Results
|
||||||
|
|
||||||
|
Comet provides a number of ways to visualize the results of your sweep. Take a look at a [project with a completed sweep here](https://www.comet.com/examples/comet-example-yolov5/view/PrlArHGuuhDTKC1UuBmTtOSXD/panels?utm_source=yolov5&utm_medium=partner&utm_campaign=partner_yolov5_2022&utm_content=github)
|
||||||
|
|
||||||
|
<img width="1626" alt="hyperparameter-yolo" src="https://user-images.githubusercontent.com/7529846/186914869-7dc1de14-583f-4323-967b-c9a66a29e495.png">
|
||||||
@ -0,0 +1,137 @@
|
|||||||
|
📚 This guide explains how to use YOLOv5 🚀 **model ensembling** during testing and inference for improved mAP and Recall.
|
||||||
|
UPDATED 25 September 2022.
|
||||||
|
|
||||||
|
From [https://en.wikipedia.org/wiki/Ensemble_learning](https://en.wikipedia.org/wiki/Ensemble_learning):
|
||||||
|
> Ensemble modeling is a process where multiple diverse models are created to predict an outcome, either by using many different modeling algorithms or using different training data sets. The ensemble model then aggregates the prediction of each base model and results in once final prediction for the unseen data. The motivation for using ensemble models is to reduce the generalization error of the prediction. As long as the base models are diverse and independent, the prediction error of the model decreases when the ensemble approach is used. The approach seeks the wisdom of crowds in making a prediction. Even though the ensemble model has multiple base models within the model, it acts and performs as a single model.
|
||||||
|
|
||||||
|
|
||||||
|
## Before You Start
|
||||||
|
|
||||||
|
Clone repo and install [requirements.txt](https://github.com/ultralytics/yolov5/blob/master/requirements.txt) in a [**Python>=3.7.0**](https://www.python.org/) environment, including [**PyTorch>=1.7**](https://pytorch.org/get-started/locally/). [Models](https://github.com/ultralytics/yolov5/tree/master/models) and [datasets](https://github.com/ultralytics/yolov5/tree/master/data) download automatically from the latest YOLOv5 [release](https://github.com/ultralytics/yolov5/releases).
|
||||||
|
|
||||||
|
```bash
|
||||||
|
git clone https://github.com/ultralytics/yolov5 # clone
|
||||||
|
cd yolov5
|
||||||
|
pip install -r requirements.txt # install
|
||||||
|
```
|
||||||
|
|
||||||
|
## Test Normally
|
||||||
|
|
||||||
|
Before ensembling we want to establish the baseline performance of a single model. This command tests YOLOv5x on COCO val2017 at image size 640 pixels. `yolov5x.pt` is the largest and most accurate model available. Other options are `yolov5s.pt`, `yolov5m.pt` and `yolov5l.pt`, or you own checkpoint from training a custom dataset `./weights/best.pt`. For details on all available models please see our README [table](https://github.com/ultralytics/yolov5#pretrained-checkpoints).
|
||||||
|
```bash
|
||||||
|
python val.py --weights yolov5x.pt --data coco.yaml --img 640 --half
|
||||||
|
```
|
||||||
|
|
||||||
|
Output:
|
||||||
|
```shell
|
||||||
|
val: data=./data/coco.yaml, weights=['yolov5x.pt'], batch_size=32, imgsz=640, conf_thres=0.001, iou_thres=0.65, task=val, device=, single_cls=False, augment=False, verbose=False, save_txt=False, save_hybrid=False, save_conf=False, save_json=True, project=runs/val, name=exp, exist_ok=False, half=True
|
||||||
|
YOLOv5 🚀 v5.0-267-g6a3ee7c torch 1.9.0+cu102 CUDA:0 (Tesla P100-PCIE-16GB, 16280.875MB)
|
||||||
|
|
||||||
|
Fusing layers...
|
||||||
|
Model Summary: 476 layers, 87730285 parameters, 0 gradients
|
||||||
|
|
||||||
|
val: Scanning '../datasets/coco/val2017' images and labels...4952 found, 48 missing, 0 empty, 0 corrupted: 100% 5000/5000 [00:01<00:00, 2846.03it/s]
|
||||||
|
val: New cache created: ../datasets/coco/val2017.cache
|
||||||
|
Class Images Labels P R mAP@.5 mAP@.5:.95: 100% 157/157 [02:30<00:00, 1.05it/s]
|
||||||
|
all 5000 36335 0.746 0.626 0.68 0.49
|
||||||
|
Speed: 0.1ms pre-process, 22.4ms inference, 1.4ms NMS per image at shape (32, 3, 640, 640) # <--- baseline speed
|
||||||
|
|
||||||
|
Evaluating pycocotools mAP... saving runs/val/exp/yolov5x_predictions.json...
|
||||||
|
...
|
||||||
|
Average Precision (AP) @[ IoU=0.50:0.95 | area= all | maxDets=100 ] = 0.504 # <--- baseline mAP
|
||||||
|
Average Precision (AP) @[ IoU=0.50 | area= all | maxDets=100 ] = 0.688
|
||||||
|
Average Precision (AP) @[ IoU=0.75 | area= all | maxDets=100 ] = 0.546
|
||||||
|
Average Precision (AP) @[ IoU=0.50:0.95 | area= small | maxDets=100 ] = 0.351
|
||||||
|
Average Precision (AP) @[ IoU=0.50:0.95 | area=medium | maxDets=100 ] = 0.551
|
||||||
|
Average Precision (AP) @[ IoU=0.50:0.95 | area= large | maxDets=100 ] = 0.644
|
||||||
|
Average Recall (AR) @[ IoU=0.50:0.95 | area= all | maxDets= 1 ] = 0.382
|
||||||
|
Average Recall (AR) @[ IoU=0.50:0.95 | area= all | maxDets= 10 ] = 0.628
|
||||||
|
Average Recall (AR) @[ IoU=0.50:0.95 | area= all | maxDets=100 ] = 0.681 # <--- baseline mAR
|
||||||
|
Average Recall (AR) @[ IoU=0.50:0.95 | area= small | maxDets=100 ] = 0.524
|
||||||
|
Average Recall (AR) @[ IoU=0.50:0.95 | area=medium | maxDets=100 ] = 0.735
|
||||||
|
Average Recall (AR) @[ IoU=0.50:0.95 | area= large | maxDets=100 ] = 0.826
|
||||||
|
```
|
||||||
|
|
||||||
|
## Ensemble Test
|
||||||
|
|
||||||
|
Multiple pretrained models may be ensembled together at test and inference time by simply appending extra models to the `--weights` argument in any existing val.py or detect.py command. This example tests an ensemble of 2 models together:
|
||||||
|
- YOLOv5x
|
||||||
|
- YOLOv5l6
|
||||||
|
|
||||||
|
```bash
|
||||||
|
python val.py --weights yolov5x.pt yolov5l6.pt --data coco.yaml --img 640 --half
|
||||||
|
```
|
||||||
|
|
||||||
|
Output:
|
||||||
|
```shell
|
||||||
|
val: data=./data/coco.yaml, weights=['yolov5x.pt', 'yolov5l6.pt'], batch_size=32, imgsz=640, conf_thres=0.001, iou_thres=0.6, task=val, device=, single_cls=False, augment=False, verbose=False, save_txt=False, save_hybrid=False, save_conf=False, save_json=True, project=runs/val, name=exp, exist_ok=False, half=True
|
||||||
|
YOLOv5 🚀 v5.0-267-g6a3ee7c torch 1.9.0+cu102 CUDA:0 (Tesla P100-PCIE-16GB, 16280.875MB)
|
||||||
|
|
||||||
|
Fusing layers...
|
||||||
|
Model Summary: 476 layers, 87730285 parameters, 0 gradients # Model 1
|
||||||
|
Fusing layers...
|
||||||
|
Model Summary: 501 layers, 77218620 parameters, 0 gradients # Model 2
|
||||||
|
Ensemble created with ['yolov5x.pt', 'yolov5l6.pt'] # Ensemble notice
|
||||||
|
|
||||||
|
val: Scanning '../datasets/coco/val2017.cache' images and labels... 4952 found, 48 missing, 0 empty, 0 corrupted: 100% 5000/5000 [00:00<00:00, 49695545.02it/s]
|
||||||
|
Class Images Labels P R mAP@.5 mAP@.5:.95: 100% 157/157 [03:58<00:00, 1.52s/it]
|
||||||
|
all 5000 36335 0.747 0.637 0.692 0.502
|
||||||
|
Speed: 0.1ms pre-process, 39.5ms inference, 2.0ms NMS per image at shape (32, 3, 640, 640) # <--- ensemble speed
|
||||||
|
|
||||||
|
Evaluating pycocotools mAP... saving runs/val/exp3/yolov5x_predictions.json...
|
||||||
|
...
|
||||||
|
Average Precision (AP) @[ IoU=0.50:0.95 | area= all | maxDets=100 ] = 0.515 # <--- ensemble mAP
|
||||||
|
Average Precision (AP) @[ IoU=0.50 | area= all | maxDets=100 ] = 0.699
|
||||||
|
Average Precision (AP) @[ IoU=0.75 | area= all | maxDets=100 ] = 0.557
|
||||||
|
Average Precision (AP) @[ IoU=0.50:0.95 | area= small | maxDets=100 ] = 0.356
|
||||||
|
Average Precision (AP) @[ IoU=0.50:0.95 | area=medium | maxDets=100 ] = 0.563
|
||||||
|
Average Precision (AP) @[ IoU=0.50:0.95 | area= large | maxDets=100 ] = 0.668
|
||||||
|
Average Recall (AR) @[ IoU=0.50:0.95 | area= all | maxDets= 1 ] = 0.387
|
||||||
|
Average Recall (AR) @[ IoU=0.50:0.95 | area= all | maxDets= 10 ] = 0.638
|
||||||
|
Average Recall (AR) @[ IoU=0.50:0.95 | area= all | maxDets=100 ] = 0.689 # <--- ensemble mAR
|
||||||
|
Average Recall (AR) @[ IoU=0.50:0.95 | area= small | maxDets=100 ] = 0.526
|
||||||
|
Average Recall (AR) @[ IoU=0.50:0.95 | area=medium | maxDets=100 ] = 0.743
|
||||||
|
Average Recall (AR) @[ IoU=0.50:0.95 | area= large | maxDets=100 ] = 0.844
|
||||||
|
```
|
||||||
|
|
||||||
|
## Ensemble Inference
|
||||||
|
|
||||||
|
Append extra models to the `--weights` argument to run ensemble inference:
|
||||||
|
```bash
|
||||||
|
python detect.py --weights yolov5x.pt yolov5l6.pt --img 640 --source data/images
|
||||||
|
```
|
||||||
|
|
||||||
|
Output:
|
||||||
|
```bash
|
||||||
|
detect: weights=['yolov5x.pt', 'yolov5l6.pt'], source=data/images, imgsz=640, conf_thres=0.25, iou_thres=0.45, max_det=1000, device=, view_img=False, save_txt=False, save_conf=False, save_crop=False, nosave=False, classes=None, agnostic_nms=False, augment=False, update=False, project=runs/detect, name=exp, exist_ok=False, line_thickness=3, hide_labels=False, hide_conf=False, half=False
|
||||||
|
YOLOv5 🚀 v5.0-267-g6a3ee7c torch 1.9.0+cu102 CUDA:0 (Tesla P100-PCIE-16GB, 16280.875MB)
|
||||||
|
|
||||||
|
Fusing layers...
|
||||||
|
Model Summary: 476 layers, 87730285 parameters, 0 gradients
|
||||||
|
Fusing layers...
|
||||||
|
Model Summary: 501 layers, 77218620 parameters, 0 gradients
|
||||||
|
Ensemble created with ['yolov5x.pt', 'yolov5l6.pt']
|
||||||
|
|
||||||
|
image 1/2 /content/yolov5/data/images/bus.jpg: 640x512 4 persons, 1 bus, 1 tie, Done. (0.063s)
|
||||||
|
image 2/2 /content/yolov5/data/images/zidane.jpg: 384x640 3 persons, 2 ties, Done. (0.056s)
|
||||||
|
Results saved to runs/detect/exp2
|
||||||
|
Done. (0.223s)
|
||||||
|
```
|
||||||
|
<img src="https://user-images.githubusercontent.com/26833433/124489091-ea4f9a00-ddb0-11eb-8ef1-d6f335c97f6f.jpg" width="500">
|
||||||
|
|
||||||
|
|
||||||
|
## Environments
|
||||||
|
|
||||||
|
YOLOv5 may be run in any of the following up-to-date verified environments (with all dependencies including [CUDA](https://developer.nvidia.com/cuda)/[CUDNN](https://developer.nvidia.com/cudnn), [Python](https://www.python.org/) and [PyTorch](https://pytorch.org/) preinstalled):
|
||||||
|
|
||||||
|
- **Notebooks** with free GPU: <a href="https://bit.ly/yolov5-paperspace-notebook"><img src="https://assets.paperspace.io/img/gradient-badge.svg" alt="Run on Gradient"></a> <a href="https://colab.research.google.com/github/ultralytics/yolov5/blob/master/tutorial.ipynb"><img src="https://colab.research.google.com/assets/colab-badge.svg" alt="Open In Colab"></a> <a href="https://www.kaggle.com/ultralytics/yolov5"><img src="https://kaggle.com/static/images/open-in-kaggle.svg" alt="Open In Kaggle"></a>
|
||||||
|
- **Google Cloud** Deep Learning VM. See [GCP Quickstart Guide](https://github.com/ultralytics/yolov5/wiki/GCP-Quickstart)
|
||||||
|
- **Amazon** Deep Learning AMI. See [AWS Quickstart Guide](https://github.com/ultralytics/yolov5/wiki/AWS-Quickstart)
|
||||||
|
- **Docker Image**. See [Docker Quickstart Guide](https://github.com/ultralytics/yolov5/wiki/Docker-Quickstart) <a href="https://hub.docker.com/r/ultralytics/yolov5"><img src="https://img.shields.io/docker/pulls/ultralytics/yolov5?logo=docker" alt="Docker Pulls"></a>
|
||||||
|
|
||||||
|
|
||||||
|
## Status
|
||||||
|
|
||||||
|
<a href="https://github.com/ultralytics/yolov5/actions/workflows/ci-testing.yml"><img src="https://github.com/ultralytics/yolov5/actions/workflows/ci-testing.yml/badge.svg" alt="YOLOv5 CI"></a>
|
||||||
|
|
||||||
|
If this badge is green, all [YOLOv5 GitHub Actions](https://github.com/ultralytics/yolov5/actions) Continuous Integration (CI) tests are currently passing. CI tests verify correct operation of YOLOv5 [training](https://github.com/ultralytics/yolov5/blob/master/train.py), [validation](https://github.com/ultralytics/yolov5/blob/master/val.py), [inference](https://github.com/ultralytics/yolov5/blob/master/detect.py), [export](https://github.com/ultralytics/yolov5/blob/master/export.py) and [benchmarks](https://github.com/ultralytics/yolov5/blob/master/benchmarks.py) on macOS, Windows, and Ubuntu every 24 hours and on every commit.
|
||||||
@ -0,0 +1,236 @@
|
|||||||
|
# TFLite, ONNX, CoreML, TensorRT Export
|
||||||
|
|
||||||
|
📚 This guide explains how to export a trained YOLOv5 🚀 model from PyTorch to ONNX and TorchScript formats.
|
||||||
|
UPDATED 8 December 2022.
|
||||||
|
|
||||||
|
## Before You Start
|
||||||
|
|
||||||
|
Clone repo and install [requirements.txt](https://github.com/ultralytics/yolov5/blob/master/requirements.txt) in a [**Python>=3.7.0**](https://www.python.org/) environment, including [**PyTorch>=1.7**](https://pytorch.org/get-started/locally/). [Models](https://github.com/ultralytics/yolov5/tree/master/models) and [datasets](https://github.com/ultralytics/yolov5/tree/master/data) download automatically from the latest YOLOv5 [release](https://github.com/ultralytics/yolov5/releases).
|
||||||
|
|
||||||
|
```bash
|
||||||
|
git clone https://github.com/ultralytics/yolov5 # clone
|
||||||
|
cd yolov5
|
||||||
|
pip install -r requirements.txt # install
|
||||||
|
```
|
||||||
|
|
||||||
|
For [TensorRT](https://developer.nvidia.com/tensorrt) export example (requires GPU) see our Colab [notebook](https://colab.research.google.com/github/ultralytics/yolov5/blob/master/tutorial.ipynb#scrollTo=VTRwsvA9u7ln&line=2&uniqifier=1) appendix section. <a href="https://colab.research.google.com/github/ultralytics/yolov5/blob/master/tutorial.ipynb"><img src="https://colab.research.google.com/assets/colab-badge.svg" alt="Open In Colab"></a>
|
||||||
|
|
||||||
|
## Formats
|
||||||
|
|
||||||
|
YOLOv5 inference is officially supported in 11 formats:
|
||||||
|
|
||||||
|
💡 ProTip: Export to ONNX or OpenVINO for up to 3x CPU speedup. See [CPU Benchmarks](https://github.com/ultralytics/yolov5/pull/6613).
|
||||||
|
💡 ProTip: Export to TensorRT for up to 5x GPU speedup. See [GPU Benchmarks](https://github.com/ultralytics/yolov5/pull/6963).
|
||||||
|
|
||||||
|
| Format | `export.py --include` | Model |
|
||||||
|
|:---------------------------------------------------------------------------|:----------------------|:--------------------------|
|
||||||
|
| [PyTorch](https://pytorch.org/) | - | `yolov5s.pt` |
|
||||||
|
| [TorchScript](https://pytorch.org/docs/stable/jit.html) | `torchscript` | `yolov5s.torchscript` |
|
||||||
|
| [ONNX](https://onnx.ai/) | `onnx` | `yolov5s.onnx` |
|
||||||
|
| [OpenVINO](https://docs.openvino.ai/latest/index.html) | `openvino` | `yolov5s_openvino_model/` |
|
||||||
|
| [TensorRT](https://developer.nvidia.com/tensorrt) | `engine` | `yolov5s.engine` |
|
||||||
|
| [CoreML](https://github.com/apple/coremltools) | `coreml` | `yolov5s.mlmodel` |
|
||||||
|
| [TensorFlow SavedModel](https://www.tensorflow.org/guide/saved_model) | `saved_model` | `yolov5s_saved_model/` |
|
||||||
|
| [TensorFlow GraphDef](https://www.tensorflow.org/api_docs/python/tf/Graph) | `pb` | `yolov5s.pb` |
|
||||||
|
| [TensorFlow Lite](https://www.tensorflow.org/lite) | `tflite` | `yolov5s.tflite` |
|
||||||
|
| [TensorFlow Edge TPU](https://coral.ai/docs/edgetpu/models-intro/) | `edgetpu` | `yolov5s_edgetpu.tflite` |
|
||||||
|
| [TensorFlow.js](https://www.tensorflow.org/js) | `tfjs` | `yolov5s_web_model/` |
|
||||||
|
| [PaddlePaddle](https://github.com/PaddlePaddle) | `paddle` | `yolov5s_paddle_model/` |
|
||||||
|
|
||||||
|
|
||||||
|
## Benchmarks
|
||||||
|
|
||||||
|
Benchmarks below run on a Colab Pro with the YOLOv5 tutorial notebook <a href="https://colab.research.google.com/github/ultralytics/yolov5/blob/master/tutorial.ipynb"><img src="https://colab.research.google.com/assets/colab-badge.svg" alt="Open In Colab"></a>. To reproduce:
|
||||||
|
```bash
|
||||||
|
python benchmarks.py --weights yolov5s.pt --imgsz 640 --device 0
|
||||||
|
```
|
||||||
|
|
||||||
|
### Colab Pro V100 GPU
|
||||||
|
|
||||||
|
```
|
||||||
|
benchmarks: weights=/content/yolov5/yolov5s.pt, imgsz=640, batch_size=1, data=/content/yolov5/data/coco128.yaml, device=0, half=False, test=False
|
||||||
|
Checking setup...
|
||||||
|
YOLOv5 🚀 v6.1-135-g7926afc torch 1.10.0+cu111 CUDA:0 (Tesla V100-SXM2-16GB, 16160MiB)
|
||||||
|
Setup complete ✅ (8 CPUs, 51.0 GB RAM, 46.7/166.8 GB disk)
|
||||||
|
|
||||||
|
Benchmarks complete (458.07s)
|
||||||
|
Format mAP@0.5:0.95 Inference time (ms)
|
||||||
|
0 PyTorch 0.4623 10.19
|
||||||
|
1 TorchScript 0.4623 6.85
|
||||||
|
2 ONNX 0.4623 14.63
|
||||||
|
3 OpenVINO NaN NaN
|
||||||
|
4 TensorRT 0.4617 1.89
|
||||||
|
5 CoreML NaN NaN
|
||||||
|
6 TensorFlow SavedModel 0.4623 21.28
|
||||||
|
7 TensorFlow GraphDef 0.4623 21.22
|
||||||
|
8 TensorFlow Lite NaN NaN
|
||||||
|
9 TensorFlow Edge TPU NaN NaN
|
||||||
|
10 TensorFlow.js NaN NaN
|
||||||
|
```
|
||||||
|
|
||||||
|
### Colab Pro CPU
|
||||||
|
|
||||||
|
```
|
||||||
|
benchmarks: weights=/content/yolov5/yolov5s.pt, imgsz=640, batch_size=1, data=/content/yolov5/data/coco128.yaml, device=cpu, half=False, test=False
|
||||||
|
Checking setup...
|
||||||
|
YOLOv5 🚀 v6.1-135-g7926afc torch 1.10.0+cu111 CPU
|
||||||
|
Setup complete ✅ (8 CPUs, 51.0 GB RAM, 41.5/166.8 GB disk)
|
||||||
|
|
||||||
|
Benchmarks complete (241.20s)
|
||||||
|
Format mAP@0.5:0.95 Inference time (ms)
|
||||||
|
0 PyTorch 0.4623 127.61
|
||||||
|
1 TorchScript 0.4623 131.23
|
||||||
|
2 ONNX 0.4623 69.34
|
||||||
|
3 OpenVINO 0.4623 66.52
|
||||||
|
4 TensorRT NaN NaN
|
||||||
|
5 CoreML NaN NaN
|
||||||
|
6 TensorFlow SavedModel 0.4623 123.79
|
||||||
|
7 TensorFlow GraphDef 0.4623 121.57
|
||||||
|
8 TensorFlow Lite 0.4623 316.61
|
||||||
|
9 TensorFlow Edge TPU NaN NaN
|
||||||
|
10 TensorFlow.js NaN NaN
|
||||||
|
```
|
||||||
|
|
||||||
|
## Export a Trained YOLOv5 Model
|
||||||
|
|
||||||
|
This command exports a pretrained YOLOv5s model to TorchScript and ONNX formats. `yolov5s.pt` is the 'small' model, the second-smallest model available. Other options are `yolov5n.pt`, `yolov5m.pt`, `yolov5l.pt` and `yolov5x.pt`, along with their P6 counterparts i.e. `yolov5s6.pt` or you own custom training checkpoint i.e. `runs/exp/weights/best.pt`. For details on all available models please see our README [table](https://github.com/ultralytics/yolov5#pretrained-checkpoints).
|
||||||
|
```bash
|
||||||
|
python export.py --weights yolov5s.pt --include torchscript onnx
|
||||||
|
```
|
||||||
|
|
||||||
|
💡 ProTip: Add `--half` to export models at FP16 half precision for smaller file sizes
|
||||||
|
|
||||||
|
Output:
|
||||||
|
```bash
|
||||||
|
export: data=data/coco128.yaml, weights=['yolov5s.pt'], imgsz=[640, 640], batch_size=1, device=cpu, half=False, inplace=False, train=False, keras=False, optimize=False, int8=False, dynamic=False, simplify=False, opset=12, verbose=False, workspace=4, nms=False, agnostic_nms=False, topk_per_class=100, topk_all=100, iou_thres=0.45, conf_thres=0.25, include=['torchscript', 'onnx']
|
||||||
|
YOLOv5 🚀 v6.2-104-ge3e5122 Python-3.7.13 torch-1.12.1+cu113 CPU
|
||||||
|
|
||||||
|
Downloading https://github.com/ultralytics/yolov5/releases/download/v6.2/yolov5s.pt to yolov5s.pt...
|
||||||
|
100% 14.1M/14.1M [00:00<00:00, 274MB/s]
|
||||||
|
|
||||||
|
Fusing layers...
|
||||||
|
YOLOv5s summary: 213 layers, 7225885 parameters, 0 gradients
|
||||||
|
|
||||||
|
PyTorch: starting from yolov5s.pt with output shape (1, 25200, 85) (14.1 MB)
|
||||||
|
|
||||||
|
TorchScript: starting export with torch 1.12.1+cu113...
|
||||||
|
TorchScript: export success ✅ 1.7s, saved as yolov5s.torchscript (28.1 MB)
|
||||||
|
|
||||||
|
ONNX: starting export with onnx 1.12.0...
|
||||||
|
ONNX: export success ✅ 2.3s, saved as yolov5s.onnx (28.0 MB)
|
||||||
|
|
||||||
|
Export complete (5.5s)
|
||||||
|
Results saved to /content/yolov5
|
||||||
|
Detect: python detect.py --weights yolov5s.onnx
|
||||||
|
Validate: python val.py --weights yolov5s.onnx
|
||||||
|
PyTorch Hub: model = torch.hub.load('ultralytics/yolov5', 'custom', 'yolov5s.onnx')
|
||||||
|
Visualize: https://netron.app/
|
||||||
|
```
|
||||||
|
|
||||||
|
The 3 exported models will be saved alongside the original PyTorch model:
|
||||||
|
<p align="center"><img width="700" src="https://user-images.githubusercontent.com/26833433/122827190-57a8f880-d2e4-11eb-860e-dbb7f9fc57fb.png"></p>
|
||||||
|
|
||||||
|
[Netron Viewer](https://github.com/lutzroeder/netron) is recommended for visualizing exported models:
|
||||||
|
<p align="center"><img width="850" src="https://user-images.githubusercontent.com/26833433/191003260-f94011a7-5b2e-4fe3-93c1-e1a935e0a728.png"></p>
|
||||||
|
|
||||||
|
|
||||||
|
## Exported Model Usage Examples
|
||||||
|
|
||||||
|
`detect.py` runs inference on exported models:
|
||||||
|
```bash
|
||||||
|
python detect.py --weights yolov5s.pt # PyTorch
|
||||||
|
yolov5s.torchscript # TorchScript
|
||||||
|
yolov5s.onnx # ONNX Runtime or OpenCV DNN with --dnn
|
||||||
|
yolov5s_openvino_model # OpenVINO
|
||||||
|
yolov5s.engine # TensorRT
|
||||||
|
yolov5s.mlmodel # CoreML (macOS only)
|
||||||
|
yolov5s_saved_model # TensorFlow SavedModel
|
||||||
|
yolov5s.pb # TensorFlow GraphDef
|
||||||
|
yolov5s.tflite # TensorFlow Lite
|
||||||
|
yolov5s_edgetpu.tflite # TensorFlow Edge TPU
|
||||||
|
yolov5s_paddle_model # PaddlePaddle
|
||||||
|
```
|
||||||
|
|
||||||
|
`val.py` runs validation on exported models:
|
||||||
|
```bash
|
||||||
|
python val.py --weights yolov5s.pt # PyTorch
|
||||||
|
yolov5s.torchscript # TorchScript
|
||||||
|
yolov5s.onnx # ONNX Runtime or OpenCV DNN with --dnn
|
||||||
|
yolov5s_openvino_model # OpenVINO
|
||||||
|
yolov5s.engine # TensorRT
|
||||||
|
yolov5s.mlmodel # CoreML (macOS Only)
|
||||||
|
yolov5s_saved_model # TensorFlow SavedModel
|
||||||
|
yolov5s.pb # TensorFlow GraphDef
|
||||||
|
yolov5s.tflite # TensorFlow Lite
|
||||||
|
yolov5s_edgetpu.tflite # TensorFlow Edge TPU
|
||||||
|
yolov5s_paddle_model # PaddlePaddle
|
||||||
|
```
|
||||||
|
|
||||||
|
Use PyTorch Hub with exported YOLOv5 models:
|
||||||
|
``` python
|
||||||
|
import torch
|
||||||
|
|
||||||
|
# Model
|
||||||
|
model = torch.hub.load('ultralytics/yolov5', 'custom', 'yolov5s.pt')
|
||||||
|
'yolov5s.torchscript ') # TorchScript
|
||||||
|
'yolov5s.onnx') # ONNX Runtime
|
||||||
|
'yolov5s_openvino_model') # OpenVINO
|
||||||
|
'yolov5s.engine') # TensorRT
|
||||||
|
'yolov5s.mlmodel') # CoreML (macOS Only)
|
||||||
|
'yolov5s_saved_model') # TensorFlow SavedModel
|
||||||
|
'yolov5s.pb') # TensorFlow GraphDef
|
||||||
|
'yolov5s.tflite') # TensorFlow Lite
|
||||||
|
'yolov5s_edgetpu.tflite') # TensorFlow Edge TPU
|
||||||
|
'yolov5s_paddle_model') # PaddlePaddle
|
||||||
|
|
||||||
|
# Images
|
||||||
|
img = 'https://ultralytics.com/images/zidane.jpg' # or file, Path, PIL, OpenCV, numpy, list
|
||||||
|
|
||||||
|
# Inference
|
||||||
|
results = model(img)
|
||||||
|
|
||||||
|
# Results
|
||||||
|
results.print() # or .show(), .save(), .crop(), .pandas(), etc.
|
||||||
|
```
|
||||||
|
|
||||||
|
## OpenCV DNN inference
|
||||||
|
|
||||||
|
OpenCV inference with ONNX models:
|
||||||
|
```bash
|
||||||
|
python export.py --weights yolov5s.pt --include onnx
|
||||||
|
|
||||||
|
python detect.py --weights yolov5s.onnx --dnn # detect
|
||||||
|
python val.py --weights yolov5s.onnx --dnn # validate
|
||||||
|
```
|
||||||
|
|
||||||
|
## C++ Inference
|
||||||
|
|
||||||
|
YOLOv5 OpenCV DNN C++ inference on exported ONNX model examples:
|
||||||
|
|
||||||
|
- [https://github.com/Hexmagic/ONNX-yolov5/blob/master/src/test.cpp](https://github.com/Hexmagic/ONNX-yolov5/blob/master/src/test.cpp)
|
||||||
|
- [https://github.com/doleron/yolov5-opencv-cpp-python](https://github.com/doleron/yolov5-opencv-cpp-python)
|
||||||
|
|
||||||
|
YOLOv5 OpenVINO C++ inference examples:
|
||||||
|
|
||||||
|
- [https://github.com/dacquaviva/yolov5-openvino-cpp-python](https://github.com/dacquaviva/yolov5-openvino-cpp-python)
|
||||||
|
- [https://github.com/UNeedCryDear/yolov5-seg-opencv-dnn-cpp](https://github.com/UNeedCryDear/yolov5-seg-opencv-dnn-cpp)
|
||||||
|
|
||||||
|
## TensorFlow.js Web Browser Inference
|
||||||
|
|
||||||
|
- [https://aukerul-shuvo.github.io/YOLOv5_TensorFlow-JS/](https://aukerul-shuvo.github.io/YOLOv5_TensorFlow-JS/)
|
||||||
|
|
||||||
|
## Environments
|
||||||
|
|
||||||
|
YOLOv5 may be run in any of the following up-to-date verified environments (with all dependencies including [CUDA](https://developer.nvidia.com/cuda)/[CUDNN](https://developer.nvidia.com/cudnn), [Python](https://www.python.org/) and [PyTorch](https://pytorch.org/) preinstalled):
|
||||||
|
|
||||||
|
- **Notebooks** with free GPU: <a href="https://bit.ly/yolov5-paperspace-notebook"><img src="https://assets.paperspace.io/img/gradient-badge.svg" alt="Run on Gradient"></a> <a href="https://colab.research.google.com/github/ultralytics/yolov5/blob/master/tutorial.ipynb"><img src="https://colab.research.google.com/assets/colab-badge.svg" alt="Open In Colab"></a> <a href="https://www.kaggle.com/ultralytics/yolov5"><img src="https://kaggle.com/static/images/open-in-kaggle.svg" alt="Open In Kaggle"></a>
|
||||||
|
- **Google Cloud** Deep Learning VM. See [GCP Quickstart Guide](https://github.com/ultralytics/yolov5/wiki/GCP-Quickstart)
|
||||||
|
- **Amazon** Deep Learning AMI. See [AWS Quickstart Guide](https://github.com/ultralytics/yolov5/wiki/AWS-Quickstart)
|
||||||
|
- **Docker Image**. See [Docker Quickstart Guide](https://github.com/ultralytics/yolov5/wiki/Docker-Quickstart) <a href="https://hub.docker.com/r/ultralytics/yolov5"><img src="https://img.shields.io/docker/pulls/ultralytics/yolov5?logo=docker" alt="Docker Pulls"></a>
|
||||||
|
|
||||||
|
|
||||||
|
## Status
|
||||||
|
|
||||||
|
<a href="https://github.com/ultralytics/yolov5/actions/workflows/ci-testing.yml"><img src="https://github.com/ultralytics/yolov5/actions/workflows/ci-testing.yml/badge.svg" alt="YOLOv5 CI"></a>
|
||||||
|
|
||||||
|
If this badge is green, all [YOLOv5 GitHub Actions](https://github.com/ultralytics/yolov5/actions) Continuous Integration (CI) tests are currently passing. CI tests verify correct operation of YOLOv5 [training](https://github.com/ultralytics/yolov5/blob/master/train.py), [validation](https://github.com/ultralytics/yolov5/blob/master/val.py), [inference](https://github.com/ultralytics/yolov5/blob/master/detect.py), [export](https://github.com/ultralytics/yolov5/blob/master/export.py) and [benchmarks](https://github.com/ultralytics/yolov5/blob/master/benchmarks.py) on macOS, Windows, and Ubuntu every 24 hours and on every commit.
|
||||||
@ -0,0 +1,161 @@
|
|||||||
|
📚 This guide explains **hyperparameter evolution** for YOLOv5 🚀. Hyperparameter evolution is a method of [Hyperparameter Optimization](https://en.wikipedia.org/wiki/Hyperparameter_optimization) using a [Genetic Algorithm](https://en.wikipedia.org/wiki/Genetic_algorithm) (GA) for optimization. UPDATED 25 September 2022.
|
||||||
|
|
||||||
|
Hyperparameters in ML control various aspects of training, and finding optimal values for them can be a challenge. Traditional methods like grid searches can quickly become intractable due to 1) the high dimensional search space 2) unknown correlations among the dimensions, and 3) expensive nature of evaluating the fitness at each point, making GA a suitable candidate for hyperparameter searches.
|
||||||
|
|
||||||
|
|
||||||
|
## Before You Start
|
||||||
|
|
||||||
|
Clone repo and install [requirements.txt](https://github.com/ultralytics/yolov5/blob/master/requirements.txt) in a [**Python>=3.7.0**](https://www.python.org/) environment, including [**PyTorch>=1.7**](https://pytorch.org/get-started/locally/). [Models](https://github.com/ultralytics/yolov5/tree/master/models) and [datasets](https://github.com/ultralytics/yolov5/tree/master/data) download automatically from the latest YOLOv5 [release](https://github.com/ultralytics/yolov5/releases).
|
||||||
|
|
||||||
|
```bash
|
||||||
|
git clone https://github.com/ultralytics/yolov5 # clone
|
||||||
|
cd yolov5
|
||||||
|
pip install -r requirements.txt # install
|
||||||
|
```
|
||||||
|
|
||||||
|
|
||||||
|
## 1. Initialize Hyperparameters
|
||||||
|
|
||||||
|
YOLOv5 has about 30 hyperparameters used for various training settings. These are defined in `*.yaml` files in the `/data/hyps` directory. Better initial guesses will produce better final results, so it is important to initialize these values properly before evolving. If in doubt, simply use the default values, which are optimized for YOLOv5 COCO training from scratch.
|
||||||
|
|
||||||
|
```yaml
|
||||||
|
# YOLOv5 🚀 by Ultralytics, GPL-3.0 license
|
||||||
|
# Hyperparameters for low-augmentation COCO training from scratch
|
||||||
|
# python train.py --batch 64 --cfg yolov5n6.yaml --weights '' --data coco.yaml --img 640 --epochs 300 --linear
|
||||||
|
# See tutorials for hyperparameter evolution https://github.com/ultralytics/yolov5#tutorials
|
||||||
|
|
||||||
|
lr0: 0.01 # initial learning rate (SGD=1E-2, Adam=1E-3)
|
||||||
|
lrf: 0.01 # final OneCycleLR learning rate (lr0 * lrf)
|
||||||
|
momentum: 0.937 # SGD momentum/Adam beta1
|
||||||
|
weight_decay: 0.0005 # optimizer weight decay 5e-4
|
||||||
|
warmup_epochs: 3.0 # warmup epochs (fractions ok)
|
||||||
|
warmup_momentum: 0.8 # warmup initial momentum
|
||||||
|
warmup_bias_lr: 0.1 # warmup initial bias lr
|
||||||
|
box: 0.05 # box loss gain
|
||||||
|
cls: 0.5 # cls loss gain
|
||||||
|
cls_pw: 1.0 # cls BCELoss positive_weight
|
||||||
|
obj: 1.0 # obj loss gain (scale with pixels)
|
||||||
|
obj_pw: 1.0 # obj BCELoss positive_weight
|
||||||
|
iou_t: 0.20 # IoU training threshold
|
||||||
|
anchor_t: 4.0 # anchor-multiple threshold
|
||||||
|
# anchors: 3 # anchors per output layer (0 to ignore)
|
||||||
|
fl_gamma: 0.0 # focal loss gamma (efficientDet default gamma=1.5)
|
||||||
|
hsv_h: 0.015 # image HSV-Hue augmentation (fraction)
|
||||||
|
hsv_s: 0.7 # image HSV-Saturation augmentation (fraction)
|
||||||
|
hsv_v: 0.4 # image HSV-Value augmentation (fraction)
|
||||||
|
degrees: 0.0 # image rotation (+/- deg)
|
||||||
|
translate: 0.1 # image translation (+/- fraction)
|
||||||
|
scale: 0.5 # image scale (+/- gain)
|
||||||
|
shear: 0.0 # image shear (+/- deg)
|
||||||
|
perspective: 0.0 # image perspective (+/- fraction), range 0-0.001
|
||||||
|
flipud: 0.0 # image flip up-down (probability)
|
||||||
|
fliplr: 0.5 # image flip left-right (probability)
|
||||||
|
mosaic: 1.0 # image mosaic (probability)
|
||||||
|
mixup: 0.0 # image mixup (probability)
|
||||||
|
copy_paste: 0.0 # segment copy-paste (probability)
|
||||||
|
```
|
||||||
|
|
||||||
|
## 2. Define Fitness
|
||||||
|
|
||||||
|
Fitness is the value we seek to maximize. In YOLOv5 we define a default fitness function as a weighted combination of metrics: `mAP@0.5` contributes 10% of the weight and `mAP@0.5:0.95` contributes the remaining 90%, with [Precision `P` and Recall `R`](https://en.wikipedia.org/wiki/Precision_and_recall) absent. You may adjust these as you see fit or use the default fitness definition in utils/metrics.py (recommended).
|
||||||
|
|
||||||
|
```python
|
||||||
|
def fitness(x):
|
||||||
|
# Model fitness as a weighted combination of metrics
|
||||||
|
w = [0.0, 0.0, 0.1, 0.9] # weights for [P, R, mAP@0.5, mAP@0.5:0.95]
|
||||||
|
return (x[:, :4] * w).sum(1)
|
||||||
|
```
|
||||||
|
|
||||||
|
## 3. Evolve
|
||||||
|
|
||||||
|
Evolution is performed about a base scenario which we seek to improve upon. The base scenario in this example is finetuning COCO128 for 10 epochs using pretrained YOLOv5s. The base scenario training command is:
|
||||||
|
```bash
|
||||||
|
python train.py --epochs 10 --data coco128.yaml --weights yolov5s.pt --cache
|
||||||
|
```
|
||||||
|
To evolve hyperparameters **specific to this scenario**, starting from our initial values defined in **Section 1.**, and maximizing the fitness defined in **Section 2.**, append `--evolve`:
|
||||||
|
```bash
|
||||||
|
# Single-GPU
|
||||||
|
python train.py --epochs 10 --data coco128.yaml --weights yolov5s.pt --cache --evolve
|
||||||
|
|
||||||
|
# Multi-GPU
|
||||||
|
for i in 0 1 2 3 4 5 6 7; do
|
||||||
|
sleep $(expr 30 \* $i) && # 30-second delay (optional)
|
||||||
|
echo 'Starting GPU '$i'...' &&
|
||||||
|
nohup python train.py --epochs 10 --data coco128.yaml --weights yolov5s.pt --cache --device $i --evolve > evolve_gpu_$i.log &
|
||||||
|
done
|
||||||
|
|
||||||
|
# Multi-GPU bash-while (not recommended)
|
||||||
|
for i in 0 1 2 3 4 5 6 7; do
|
||||||
|
sleep $(expr 30 \* $i) && # 30-second delay (optional)
|
||||||
|
echo 'Starting GPU '$i'...' &&
|
||||||
|
"$(while true; do nohup python train.py... --device $i --evolve 1 > evolve_gpu_$i.log; done)" &
|
||||||
|
done
|
||||||
|
```
|
||||||
|
|
||||||
|
The default evolution settings will run the base scenario 300 times, i.e. for 300 generations. You can modify generations via the `--evolve` argument, i.e. `python train.py --evolve 1000`.
|
||||||
|
https://github.com/ultralytics/yolov5/blob/6a3ee7cf03efb17fbffde0e68b1a854e80fe3213/train.py#L608
|
||||||
|
|
||||||
|
The main genetic operators are **crossover** and **mutation**. In this work mutation is used, with an 80% probability and a 0.04 variance to create new offspring based on a combination of the best parents from all previous generations. Results are logged to `runs/evolve/exp/evolve.csv`, and the highest fitness offspring is saved every generation as `runs/evolve/hyp_evolved.yaml`:
|
||||||
|
```yaml
|
||||||
|
# YOLOv5 Hyperparameter Evolution Results
|
||||||
|
# Best generation: 287
|
||||||
|
# Last generation: 300
|
||||||
|
# metrics/precision, metrics/recall, metrics/mAP_0.5, metrics/mAP_0.5:0.95, val/box_loss, val/obj_loss, val/cls_loss
|
||||||
|
# 0.54634, 0.55625, 0.58201, 0.33665, 0.056451, 0.042892, 0.013441
|
||||||
|
|
||||||
|
lr0: 0.01 # initial learning rate (SGD=1E-2, Adam=1E-3)
|
||||||
|
lrf: 0.2 # final OneCycleLR learning rate (lr0 * lrf)
|
||||||
|
momentum: 0.937 # SGD momentum/Adam beta1
|
||||||
|
weight_decay: 0.0005 # optimizer weight decay 5e-4
|
||||||
|
warmup_epochs: 3.0 # warmup epochs (fractions ok)
|
||||||
|
warmup_momentum: 0.8 # warmup initial momentum
|
||||||
|
warmup_bias_lr: 0.1 # warmup initial bias lr
|
||||||
|
box: 0.05 # box loss gain
|
||||||
|
cls: 0.5 # cls loss gain
|
||||||
|
cls_pw: 1.0 # cls BCELoss positive_weight
|
||||||
|
obj: 1.0 # obj loss gain (scale with pixels)
|
||||||
|
obj_pw: 1.0 # obj BCELoss positive_weight
|
||||||
|
iou_t: 0.20 # IoU training threshold
|
||||||
|
anchor_t: 4.0 # anchor-multiple threshold
|
||||||
|
# anchors: 3 # anchors per output layer (0 to ignore)
|
||||||
|
fl_gamma: 0.0 # focal loss gamma (efficientDet default gamma=1.5)
|
||||||
|
hsv_h: 0.015 # image HSV-Hue augmentation (fraction)
|
||||||
|
hsv_s: 0.7 # image HSV-Saturation augmentation (fraction)
|
||||||
|
hsv_v: 0.4 # image HSV-Value augmentation (fraction)
|
||||||
|
degrees: 0.0 # image rotation (+/- deg)
|
||||||
|
translate: 0.1 # image translation (+/- fraction)
|
||||||
|
scale: 0.5 # image scale (+/- gain)
|
||||||
|
shear: 0.0 # image shear (+/- deg)
|
||||||
|
perspective: 0.0 # image perspective (+/- fraction), range 0-0.001
|
||||||
|
flipud: 0.0 # image flip up-down (probability)
|
||||||
|
fliplr: 0.5 # image flip left-right (probability)
|
||||||
|
mosaic: 1.0 # image mosaic (probability)
|
||||||
|
mixup: 0.0 # image mixup (probability)
|
||||||
|
copy_paste: 0.0 # segment copy-paste (probability)
|
||||||
|
```
|
||||||
|
|
||||||
|
We recommend a minimum of 300 generations of evolution for best results. Note that **evolution is generally expensive and time-consuming**, as the base scenario is trained hundreds of times, possibly requiring hundreds or thousands of GPU hours.
|
||||||
|
|
||||||
|
|
||||||
|
## 4. Visualize
|
||||||
|
|
||||||
|
`evolve.csv` is plotted as `evolve.png` by `utils.plots.plot_evolve()` after evolution finishes with one subplot per hyperparameter showing fitness (y-axis) vs hyperparameter values (x-axis). Yellow indicates higher concentrations. Vertical distributions indicate that a parameter has been disabled and does not mutate. This is user selectable in the `meta` dictionary in train.py, and is useful for fixing parameters and preventing them from evolving.
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
|
## Environments
|
||||||
|
|
||||||
|
YOLOv5 may be run in any of the following up-to-date verified environments (with all dependencies including [CUDA](https://developer.nvidia.com/cuda)/[CUDNN](https://developer.nvidia.com/cudnn), [Python](https://www.python.org/) and [PyTorch](https://pytorch.org/) preinstalled):
|
||||||
|
|
||||||
|
- **Notebooks** with free GPU: <a href="https://bit.ly/yolov5-paperspace-notebook"><img src="https://assets.paperspace.io/img/gradient-badge.svg" alt="Run on Gradient"></a> <a href="https://colab.research.google.com/github/ultralytics/yolov5/blob/master/tutorial.ipynb"><img src="https://colab.research.google.com/assets/colab-badge.svg" alt="Open In Colab"></a> <a href="https://www.kaggle.com/ultralytics/yolov5"><img src="https://kaggle.com/static/images/open-in-kaggle.svg" alt="Open In Kaggle"></a>
|
||||||
|
- **Google Cloud** Deep Learning VM. See [GCP Quickstart Guide](https://github.com/ultralytics/yolov5/wiki/GCP-Quickstart)
|
||||||
|
- **Amazon** Deep Learning AMI. See [AWS Quickstart Guide](https://github.com/ultralytics/yolov5/wiki/AWS-Quickstart)
|
||||||
|
- **Docker Image**. See [Docker Quickstart Guide](https://github.com/ultralytics/yolov5/wiki/Docker-Quickstart) <a href="https://hub.docker.com/r/ultralytics/yolov5"><img src="https://img.shields.io/docker/pulls/ultralytics/yolov5?logo=docker" alt="Docker Pulls"></a>
|
||||||
|
|
||||||
|
|
||||||
|
## Status
|
||||||
|
|
||||||
|
<a href="https://github.com/ultralytics/yolov5/actions/workflows/ci-testing.yml"><img src="https://github.com/ultralytics/yolov5/actions/workflows/ci-testing.yml/badge.svg" alt="YOLOv5 CI"></a>
|
||||||
|
|
||||||
|
If this badge is green, all [YOLOv5 GitHub Actions](https://github.com/ultralytics/yolov5/actions) Continuous Integration (CI) tests are currently passing. CI tests verify correct operation of YOLOv5 [training](https://github.com/ultralytics/yolov5/blob/master/train.py), [validation](https://github.com/ultralytics/yolov5/blob/master/val.py), [inference](https://github.com/ultralytics/yolov5/blob/master/detect.py), [export](https://github.com/ultralytics/yolov5/blob/master/export.py) and [benchmarks](https://github.com/ultralytics/yolov5/blob/master/benchmarks.py) on macOS, Windows, and Ubuntu every 24 hours and on every commit.
|
||||||
@ -0,0 +1,87 @@
|
|||||||
|
# YOLOv5 Docs
|
||||||
|
|
||||||
|
<div align="center">
|
||||||
|
<p>
|
||||||
|
<a href="https://ultralytics.com/yolov5" target="_blank">
|
||||||
|
<img width="100%" src="https://raw.githubusercontent.com/ultralytics/assets/main/yolov5/v70/splash.png"></a>
|
||||||
|
</p>
|
||||||
|
|
||||||
|
<a href="https://github.com/ultralytics/yolov5/actions/workflows/ci-testing.yml"><img src="https://github.com/ultralytics/yolov5/actions/workflows/ci-testing.yml/badge.svg" alt="YOLOv5 CI"></a>
|
||||||
|
<a href="https://zenodo.org/badge/latestdoi/264818686"><img src="https://zenodo.org/badge/264818686.svg" alt="YOLOv5 Citation"></a>
|
||||||
|
<a href="https://hub.docker.com/r/ultralytics/yolov5"><img src="https://img.shields.io/docker/pulls/ultralytics/yolov5?logo=docker" alt="Docker Pulls"></a>
|
||||||
|
<br>
|
||||||
|
<a href="https://bit.ly/yolov5-paperspace-notebook"><img src="https://assets.paperspace.io/img/gradient-badge.svg" alt="Run on Gradient"></a>
|
||||||
|
<a href="https://colab.research.google.com/github/ultralytics/yolov5/blob/master/tutorial.ipynb"><img src="https://colab.research.google.com/assets/colab-badge.svg" alt="Open In Colab"></a>
|
||||||
|
<a href="https://www.kaggle.com/ultralytics/yolov5"><img src="https://kaggle.com/static/images/open-in-kaggle.svg" alt="Open In Kaggle"></a>
|
||||||
|
<br>
|
||||||
|
<br>
|
||||||
|
|
||||||
|
Welcome to the Ultralytics YOLOv5 🚀 Docs! YOLOv5, or You Only Look Once version 5, is an Ultralytics object detection model designed to deliver fast and accurate real-time results.
|
||||||
|
<br><br>
|
||||||
|
This powerful deep learning framework is built on the PyTorch platform and has gained immense popularity due to its ease of use, high performance, and versatility. In this documentation, we will guide you through the installation process, explain the model's architecture, showcase various use-cases, and provide detailed tutorials to help you harness the full potential of YOLOv5 for your computer vision projects. Let's dive in!
|
||||||
|
|
||||||
|
</div>
|
||||||
|
|
||||||
|
## Tutorials
|
||||||
|
|
||||||
|
* [Train Custom Data](train_custom_data.md) 🚀 RECOMMENDED
|
||||||
|
* [Tips for Best Training Results](tips_for_best_training_results.md) ☘️
|
||||||
|
* [Multi-GPU Training](multi_gpu_training.md)
|
||||||
|
* [PyTorch Hub](pytorch_hub.md) 🌟 NEW
|
||||||
|
* [TFLite, ONNX, CoreML, TensorRT Export](export.md) 🚀
|
||||||
|
* [NVIDIA Jetson platform Deployment](jetson_nano.md) 🌟 NEW
|
||||||
|
* [Test-Time Augmentation (TTA)](tta.md)
|
||||||
|
* [Model Ensembling](ensemble.md)
|
||||||
|
* [Model Pruning/Sparsity](pruning_sparsity.md)
|
||||||
|
* [Hyperparameter Evolution](hyp_evolution.md)
|
||||||
|
* [Transfer Learning with Frozen Layers](transfer_learn_frozen.md)
|
||||||
|
* [Architecture Summary](architecture.md) 🌟 NEW
|
||||||
|
* [Roboflow for Datasets, Labeling, and Active Learning](roboflow.md)
|
||||||
|
* [ClearML Logging](clearml.md) 🌟 NEW
|
||||||
|
* [YOLOv5 with Neural Magic's Deepsparse](neural_magic.md) 🌟 NEW
|
||||||
|
* [Comet Logging](comet.md) 🌟 NEW
|
||||||
|
|
||||||
|
## Environments
|
||||||
|
|
||||||
|
YOLOv5 may be run in any of the following up-to-date verified environments (with all dependencies
|
||||||
|
including [CUDA](https://developer.nvidia.com/cuda)/[CUDNN](https://developer.nvidia.com/cudnn), [Python](https://www.python.org/)
|
||||||
|
and [PyTorch](https://pytorch.org/) preinstalled):
|
||||||
|
|
||||||
|
- **Notebooks** with free
|
||||||
|
GPU: <a href="https://bit.ly/yolov5-paperspace-notebook"><img src="https://assets.paperspace.io/img/gradient-badge.svg" alt="Run on Gradient"></a> <a href="https://colab.research.google.com/github/ultralytics/yolov5/blob/master/tutorial.ipynb"><img src="https://colab.research.google.com/assets/colab-badge.svg" alt="Open In Colab"></a> <a href="https://www.kaggle.com/ultralytics/yolov5"><img src="https://kaggle.com/static/images/open-in-kaggle.svg" alt="Open In Kaggle"></a>
|
||||||
|
- **Google Cloud** Deep Learning VM.
|
||||||
|
See [GCP Quickstart Guide](https://github.com/ultralytics/yolov5/wiki/GCP-Quickstart)
|
||||||
|
- **Amazon** Deep Learning AMI. See [AWS Quickstart Guide](https://github.com/ultralytics/yolov5/wiki/AWS-Quickstart)
|
||||||
|
- **Docker Image**.
|
||||||
|
See [Docker Quickstart Guide](https://github.com/ultralytics/yolov5/wiki/Docker-Quickstart) <a href="https://hub.docker.com/r/ultralytics/yolov5"><img src="https://img.shields.io/docker/pulls/ultralytics/yolov5?logo=docker" alt="Docker Pulls"></a>
|
||||||
|
|
||||||
|
## Status
|
||||||
|
|
||||||
|
<a href="https://github.com/ultralytics/yolov5/actions/workflows/ci-testing.yml"><img src="https://github.com/ultralytics/yolov5/actions/workflows/ci-testing.yml/badge.svg" alt="YOLOv5 CI"></a>
|
||||||
|
|
||||||
|
If this badge is green, all [YOLOv5 GitHub Actions](https://github.com/ultralytics/yolov5/actions) Continuous
|
||||||
|
Integration (CI) tests are currently passing. CI tests verify correct operation of
|
||||||
|
YOLOv5 [training](https://github.com/ultralytics/yolov5/blob/master/train.py), [validation](https://github.com/ultralytics/yolov5/blob/master/val.py), [inference](https://github.com/ultralytics/yolov5/blob/master/detect.py), [export](https://github.com/ultralytics/yolov5/blob/master/export.py)
|
||||||
|
and [benchmarks](https://github.com/ultralytics/yolov5/blob/master/benchmarks.py) on macOS, Windows, and Ubuntu every 24
|
||||||
|
hours and on every commit.
|
||||||
|
|
||||||
|
<br>
|
||||||
|
<div align="center">
|
||||||
|
<a href="https://github.com/ultralytics" style="text-decoration:none;">
|
||||||
|
<img src="https://github.com/ultralytics/assets/raw/main/social/logo-social-github.png" width="3%" alt="" /></a>
|
||||||
|
<img src="https://github.com/ultralytics/assets/raw/main/social/logo-transparent.png" width="3%" alt="" />
|
||||||
|
<a href="https://www.linkedin.com/company/ultralytics" style="text-decoration:none;">
|
||||||
|
<img src="https://github.com/ultralytics/assets/raw/main/social/logo-social-linkedin.png" width="3%" alt="" /></a>
|
||||||
|
<img src="https://github.com/ultralytics/assets/raw/main/social/logo-transparent.png" width="3%" alt="" />
|
||||||
|
<a href="https://twitter.com/ultralytics" style="text-decoration:none;">
|
||||||
|
<img src="https://github.com/ultralytics/assets/raw/main/social/logo-social-twitter.png" width="3%" alt="" /></a>
|
||||||
|
<img src="https://github.com/ultralytics/assets/raw/main/social/logo-transparent.png" width="3%" alt="" />
|
||||||
|
<a href="https://youtube.com/ultralytics" style="text-decoration:none;">
|
||||||
|
<img src="https://github.com/ultralytics/assets/raw/main/social/logo-social-youtube.png" width="3%" alt="" /></a>
|
||||||
|
<img src="https://github.com/ultralytics/assets/raw/main/social/logo-transparent.png" width="3%" alt="" />
|
||||||
|
<a href="https://www.tiktok.com/@ultralytics" style="text-decoration:none;">
|
||||||
|
<img src="https://github.com/ultralytics/assets/raw/main/social/logo-social-tiktok.png" width="3%" alt="" /></a>
|
||||||
|
<img src="https://github.com/ultralytics/assets/raw/main/social/logo-transparent.png" width="3%" alt="" />
|
||||||
|
<a href="https://www.instagram.com/ultralytics/" style="text-decoration:none;">
|
||||||
|
<img src="https://github.com/ultralytics/assets/raw/main/social/logo-social-instagram.png" width="3%" alt="" /></a>
|
||||||
|
</div>
|
||||||
@ -0,0 +1,316 @@
|
|||||||
|
# Deploy on NVIDIA Jetson using TensorRT and DeepStream SDK
|
||||||
|
|
||||||
|
📚 This guide explains how to deploy a trained model into NVIDIA Jetson Platform and perform inference using TensorRT and DeepStream SDK. Here we use TensorRT to maximize the inference performance on the Jetson platform.
|
||||||
|
UPDATED 18 November 2022.
|
||||||
|
|
||||||
|
## Hardware Verification
|
||||||
|
|
||||||
|
We have tested and verified this guide on the following Jetson devices
|
||||||
|
|
||||||
|
- [Seeed reComputer J1010 built with Jetson Nano module](https://www.seeedstudio.com/Jetson-10-1-A0-p-5336.html)
|
||||||
|
- [Seeed reComputer J2021 built with Jetson Xavier NX module](https://www.seeedstudio.com/reComputer-J2021-p-5438.html)
|
||||||
|
|
||||||
|
## Before You Start
|
||||||
|
|
||||||
|
Make sure you have properly installed **JetPack SDK** with all the **SDK Components** and **DeepStream SDK** on the Jetson device as this includes CUDA, TensorRT and DeepStream SDK which are needed for this guide.
|
||||||
|
|
||||||
|
JetPack SDK provides a full development environment for hardware-accelerated AI-at-the-edge development. All Jetson modules and developer kits are supported by JetPack SDK.
|
||||||
|
|
||||||
|
There are two major installation methods including,
|
||||||
|
|
||||||
|
1. SD Card Image Method
|
||||||
|
2. NVIDIA SDK Manager Method
|
||||||
|
|
||||||
|
You can find a very detailed installation guide from NVIDIA [official website](https://developer.nvidia.com/jetpack-sdk-461). You can also find guides corresponding to the above-mentioned [reComputer J1010](https://wiki.seeedstudio.com/reComputer_J1010_J101_Flash_Jetpack) and [reComputer J2021](https://wiki.seeedstudio.com/reComputer_J2021_J202_Flash_Jetpack).
|
||||||
|
|
||||||
|
|
||||||
|
## Install Necessary Packages
|
||||||
|
|
||||||
|
- **Step 1.** Access the terminal of Jetson device, install pip and upgrade it
|
||||||
|
|
||||||
|
```sh
|
||||||
|
sudo apt update
|
||||||
|
sudo apt install -y python3-pip
|
||||||
|
pip3 install --upgrade pip
|
||||||
|
```
|
||||||
|
|
||||||
|
- **Step 2.** Clone the following repo
|
||||||
|
|
||||||
|
```sh
|
||||||
|
git clone https://github.com/ultralytics/yolov5
|
||||||
|
```
|
||||||
|
|
||||||
|
- **Step 3.** Open **requirements.txt**
|
||||||
|
|
||||||
|
```sh
|
||||||
|
cd yolov5
|
||||||
|
vi requirements.txt
|
||||||
|
```
|
||||||
|
|
||||||
|
- **Step 5.** Edit the following lines. Here you need to press **i** first to enter editing mode. Press **ESC**, then type **:wq** to save and quit
|
||||||
|
|
||||||
|
```sh
|
||||||
|
# torch>=1.7.0
|
||||||
|
# torchvision>=0.8.1
|
||||||
|
```
|
||||||
|
|
||||||
|
**Note:** torch and torchvision are excluded for now because they will be installed later.
|
||||||
|
|
||||||
|
- **Step 6.** install the below dependency
|
||||||
|
|
||||||
|
```sh
|
||||||
|
sudo apt install -y libfreetype6-dev
|
||||||
|
```
|
||||||
|
|
||||||
|
- **Step 7.** Install the necessary packages
|
||||||
|
|
||||||
|
```sh
|
||||||
|
pip3 install -r requirements.txt
|
||||||
|
```
|
||||||
|
|
||||||
|
## Install PyTorch and Torchvision
|
||||||
|
|
||||||
|
We cannot install PyTorch and Torchvision from pip because they are not compatible to run on Jetson platform which is based on **ARM aarch64 architecture**. Therefore, we need to manually install pre-built PyTorch pip wheel and compile/ install Torchvision from source.
|
||||||
|
|
||||||
|
Visit [this page](https://forums.developer.nvidia.com/t/pytorch-for-jetson) to access all the PyTorch and Torchvision links.
|
||||||
|
|
||||||
|
Here are some of the versions supported by JetPack 4.6 and above.
|
||||||
|
|
||||||
|
**PyTorch v1.10.0**
|
||||||
|
|
||||||
|
Supported by JetPack 4.4 (L4T R32.4.3) / JetPack 4.4.1 (L4T R32.4.4) / JetPack 4.5 (L4T R32.5.0) / JetPack 4.5.1 (L4T R32.5.1) / JetPack 4.6 (L4T R32.6.1) with Python 3.6
|
||||||
|
|
||||||
|
**file_name:** torch-1.10.0-cp36-cp36m-linux_aarch64.whl
|
||||||
|
**URL:** [https://nvidia.box.com/shared/static/fjtbno0vpo676a25cgvuqc1wty0fkkg6.whl](https://nvidia.box.com/shared/static/fjtbno0vpo676a25cgvuqc1wty0fkkg6.whl)
|
||||||
|
|
||||||
|
**PyTorch v1.12.0**
|
||||||
|
|
||||||
|
Supported by JetPack 5.0 (L4T R34.1.0) / JetPack 5.0.1 (L4T R34.1.1) / JetPack 5.0.2 (L4T R35.1.0) with Python 3.8
|
||||||
|
|
||||||
|
**file_name:** torch-1.12.0a0+2c916ef.nv22.3-cp38-cp38-linux_aarch64.whl
|
||||||
|
**URL:** [https://developer.download.nvidia.com/compute/redist/jp/v50/pytorch/torch-1.12.0a0+2c916ef.nv22.3-cp38-cp38-linux_aarch64.whl](https://developer.download.nvidia.com/compute/redist/jp/v50/pytorch/torch-1.12.0a0+2c916ef.nv22.3-cp38-cp38-linux_aarch64.whl)
|
||||||
|
|
||||||
|
- **Step 1.** Install torch according to your JetPack version in the following format
|
||||||
|
|
||||||
|
```sh
|
||||||
|
wget <URL> -O <file_name>
|
||||||
|
pip3 install <file_name>
|
||||||
|
```
|
||||||
|
|
||||||
|
For example, here we are running **JP4.6.1**, and therefore we choose **PyTorch v1.10.0**
|
||||||
|
|
||||||
|
```sh
|
||||||
|
cd ~
|
||||||
|
sudo apt-get install -y libopenblas-base libopenmpi-dev
|
||||||
|
wget https://nvidia.box.com/shared/static/fjtbno0vpo676a25cgvuqc1wty0fkkg6.whl -O torch-1.10.0-cp36-cp36m-linux_aarch64.whl
|
||||||
|
pip3 install torch-1.10.0-cp36-cp36m-linux_aarch64.whl
|
||||||
|
```
|
||||||
|
|
||||||
|
- **Step 2.** Install torchvision depending on the version of PyTorch that you have installed. For example, we chose **PyTorch v1.10.0**, which means, we need to choose **Torchvision v0.11.1**
|
||||||
|
|
||||||
|
```sh
|
||||||
|
sudo apt install -y libjpeg-dev zlib1g-dev
|
||||||
|
git clone --branch v0.11.1 https://github.com/pytorch/vision torchvision
|
||||||
|
cd torchvision
|
||||||
|
sudo python3 setup.py install
|
||||||
|
```
|
||||||
|
|
||||||
|
Here a list of the corresponding torchvision version that you need to install according to the PyTorch version:
|
||||||
|
|
||||||
|
- PyTorch v1.10 - torchvision v0.11.1
|
||||||
|
- PyTorch v1.12 - torchvision v0.13.0
|
||||||
|
|
||||||
|
## DeepStream Configuration for YOLOv5
|
||||||
|
|
||||||
|
- **Step 1.** Clone the following repo
|
||||||
|
|
||||||
|
```sh
|
||||||
|
cd ~
|
||||||
|
git clone https://github.com/marcoslucianops/DeepStream-Yolo
|
||||||
|
```
|
||||||
|
|
||||||
|
- **Step 2.** Copy **gen_wts_yoloV5.py** from **DeepStream-Yolo/utils** into **yolov5** directory
|
||||||
|
|
||||||
|
```sh
|
||||||
|
cp DeepStream-Yolo/utils/gen_wts_yoloV5.py yolov5
|
||||||
|
```
|
||||||
|
|
||||||
|
- **Step 3.** Inside the yolov5 repo, download **pt file** from YOLOv5 releases (example for YOLOv5s 6.1)
|
||||||
|
|
||||||
|
```sh
|
||||||
|
cd yolov5
|
||||||
|
wget https://github.com/ultralytics/yolov5/releases/download/v6.1/yolov5s.pt
|
||||||
|
```
|
||||||
|
|
||||||
|
- **Step 4.** Generate the **cfg** and **wts** files
|
||||||
|
|
||||||
|
```sh
|
||||||
|
python3 gen_wts_yoloV5.py -w yolov5s.pt
|
||||||
|
```
|
||||||
|
|
||||||
|
**Note**: To change the inference size (default: 640)
|
||||||
|
|
||||||
|
```sh
|
||||||
|
-s SIZE
|
||||||
|
--size SIZE
|
||||||
|
-s HEIGHT WIDTH
|
||||||
|
--size HEIGHT WIDTH
|
||||||
|
|
||||||
|
Example for 1280:
|
||||||
|
|
||||||
|
-s 1280
|
||||||
|
or
|
||||||
|
-s 1280 1280
|
||||||
|
```
|
||||||
|
|
||||||
|
- **Step 5.** Copy the generated **cfg** and **wts** files into the **DeepStream-Yolo** folder
|
||||||
|
|
||||||
|
```sh
|
||||||
|
cp yolov5s.cfg ~/DeepStream-Yolo
|
||||||
|
cp yolov5s.wts ~/DeepStream-Yolo
|
||||||
|
```
|
||||||
|
|
||||||
|
- **Step 6.** Open the **DeepStream-Yolo** folder and compile the library
|
||||||
|
|
||||||
|
```sh
|
||||||
|
cd ~/DeepStream-Yolo
|
||||||
|
CUDA_VER=11.4 make -C nvdsinfer_custom_impl_Yolo # for DeepStream 6.1
|
||||||
|
CUDA_VER=10.2 make -C nvdsinfer_custom_impl_Yolo # for DeepStream 6.0.1 / 6.0
|
||||||
|
```
|
||||||
|
|
||||||
|
- **Step 7.** Edit the **config_infer_primary_yoloV5.txt** file according to your model
|
||||||
|
|
||||||
|
```sh
|
||||||
|
[property]
|
||||||
|
...
|
||||||
|
custom-network-config=yolov5s.cfg
|
||||||
|
model-file=yolov5s.wts
|
||||||
|
...
|
||||||
|
```
|
||||||
|
|
||||||
|
- **Step 8.** Edit the **deepstream_app_config** file
|
||||||
|
|
||||||
|
```sh
|
||||||
|
...
|
||||||
|
[primary-gie]
|
||||||
|
...
|
||||||
|
config-file=config_infer_primary_yoloV5.txt
|
||||||
|
```
|
||||||
|
|
||||||
|
- **Step 9.** Change the video source in **deepstream_app_config** file. Here a default video file is loaded as you can see below
|
||||||
|
|
||||||
|
```sh
|
||||||
|
...
|
||||||
|
[source0]
|
||||||
|
...
|
||||||
|
uri=file:///opt/nvidia/deepstream/deepstream/samples/streams/sample_1080p_h264.mp4
|
||||||
|
```
|
||||||
|
|
||||||
|
## Run the Inference
|
||||||
|
|
||||||
|
```sh
|
||||||
|
deepstream-app -c deepstream_app_config.txt
|
||||||
|
```
|
||||||
|
|
||||||
|
<div align=center><img width=1000 src="https://files.seeedstudio.com/wiki/YOLOV5/FP32-yolov5s.gif"/></div>
|
||||||
|
|
||||||
|
The above result is running on **Jetson Xavier NX** with **FP32** and **YOLOv5s 640x640**. We can see that the **FPS** is around **30**.
|
||||||
|
|
||||||
|
## INT8 Calibration
|
||||||
|
|
||||||
|
If you want to use INT8 precision for inference, you need to follow the steps below
|
||||||
|
|
||||||
|
- **Step 1.** Install OpenCV
|
||||||
|
|
||||||
|
```sh
|
||||||
|
sudo apt-get install libopencv-dev
|
||||||
|
```
|
||||||
|
|
||||||
|
- **Step 2.** Compile/recompile the **nvdsinfer_custom_impl_Yolo** library with OpenCV support
|
||||||
|
|
||||||
|
```sh
|
||||||
|
cd ~/DeepStream-Yolo
|
||||||
|
CUDA_VER=11.4 OPENCV=1 make -C nvdsinfer_custom_impl_Yolo # for DeepStream 6.1
|
||||||
|
CUDA_VER=10.2 OPENCV=1 make -C nvdsinfer_custom_impl_Yolo # for DeepStream 6.0.1 / 6.0
|
||||||
|
```
|
||||||
|
|
||||||
|
- **Step 3.** For COCO dataset, download the [val2017](https://drive.google.com/file/d/1gbvfn7mcsGDRZ_luJwtITL-ru2kK99aK/view?usp=sharing), extract, and move to **DeepStream-Yolo** folder
|
||||||
|
|
||||||
|
- **Step 4.** Make a new directory for calibration images
|
||||||
|
|
||||||
|
```sh
|
||||||
|
mkdir calibration
|
||||||
|
```
|
||||||
|
|
||||||
|
- **Step 5.** Run the following to select 1000 random images from COCO dataset to run calibration
|
||||||
|
|
||||||
|
```sh
|
||||||
|
for jpg in $(ls -1 val2017/*.jpg | sort -R | head -1000); do \
|
||||||
|
cp ${jpg} calibration/; \
|
||||||
|
done
|
||||||
|
```
|
||||||
|
|
||||||
|
**Note:** NVIDIA recommends at least 500 images to get a good accuracy. On this example, 1000 images are chosen to get better accuracy (more images = more accuracy). Higher INT8_CALIB_BATCH_SIZE values will result in more accuracy and faster calibration speed. Set it according to you GPU memory. You can set it from **head -1000**. For example, for 2000 images, **head -2000**. This process can take a long time.
|
||||||
|
|
||||||
|
- **Step 6.** Create the **calibration.txt** file with all selected images
|
||||||
|
|
||||||
|
```sh
|
||||||
|
realpath calibration/*jpg > calibration.txt
|
||||||
|
```
|
||||||
|
|
||||||
|
- **Step 7.** Set environment variables
|
||||||
|
|
||||||
|
```sh
|
||||||
|
export INT8_CALIB_IMG_PATH=calibration.txt
|
||||||
|
export INT8_CALIB_BATCH_SIZE=1
|
||||||
|
```
|
||||||
|
|
||||||
|
- **Step 8.** Update the **config_infer_primary_yoloV5.txt** file
|
||||||
|
|
||||||
|
From
|
||||||
|
|
||||||
|
```sh
|
||||||
|
...
|
||||||
|
model-engine-file=model_b1_gpu0_fp32.engine
|
||||||
|
#int8-calib-file=calib.table
|
||||||
|
...
|
||||||
|
network-mode=0
|
||||||
|
...
|
||||||
|
```
|
||||||
|
|
||||||
|
To
|
||||||
|
|
||||||
|
```sh
|
||||||
|
...
|
||||||
|
model-engine-file=model_b1_gpu0_int8.engine
|
||||||
|
int8-calib-file=calib.table
|
||||||
|
...
|
||||||
|
network-mode=1
|
||||||
|
...
|
||||||
|
```
|
||||||
|
|
||||||
|
- **Step 9.** Run the inference
|
||||||
|
|
||||||
|
```sh
|
||||||
|
deepstream-app -c deepstream_app_config.txt
|
||||||
|
```
|
||||||
|
|
||||||
|
<div align=center><img width=1000 src="https://files.seeedstudio.com/wiki/YOLOV5/INT8-yolov5s.gif"/></div>
|
||||||
|
|
||||||
|
The above result is running on **Jetson Xavier NX** with **INT8** and **YOLOv5s 640x640**. We can see that the **FPS** is around **60**.
|
||||||
|
|
||||||
|
## Benchmark results
|
||||||
|
|
||||||
|
The following table summarizes how different models perform on **Jetson Xavier NX**.
|
||||||
|
|
||||||
|
| Model Name | Precision | Inference Size | Inference Time (ms) | FPS |
|
||||||
|
|------------|-----------|----------------|---------------------|-----|
|
||||||
|
| YOLOv5s | FP32 | 320x320 | 16.66 | 60 |
|
||||||
|
| | FP32 | 640x640 | 33.33 | 30 |
|
||||||
|
| | INT8 | 640x640 | 16.66 | 60 |
|
||||||
|
| YOLOv5n | FP32 | 640x640 | 16.66 | 60 |
|
||||||
|
|
||||||
|
|
||||||
|
### Additional
|
||||||
|
|
||||||
|
This tutorial is written by our friends at seeed @lakshanthad and Elaine
|
||||||
@ -0,0 +1,186 @@
|
|||||||
|
📚 This guide explains how to properly use **multiple** GPUs to train a dataset with YOLOv5 🚀 on single or multiple machine(s).
|
||||||
|
UPDATED 25 December 2022.
|
||||||
|
|
||||||
|
## Before You Start
|
||||||
|
|
||||||
|
Clone repo and install [requirements.txt](https://github.com/ultralytics/yolov5/blob/master/requirements.txt) in a [**Python>=3.7.0**](https://www.python.org/) environment, including [**PyTorch>=1.7**](https://pytorch.org/get-started/locally/). [Models](https://github.com/ultralytics/yolov5/tree/master/models) and [datasets](https://github.com/ultralytics/yolov5/tree/master/data) download automatically from the latest YOLOv5 [release](https://github.com/ultralytics/yolov5/releases).
|
||||||
|
|
||||||
|
```bash
|
||||||
|
git clone https://github.com/ultralytics/yolov5 # clone
|
||||||
|
cd yolov5
|
||||||
|
pip install -r requirements.txt # install
|
||||||
|
```
|
||||||
|
|
||||||
|
💡 ProTip! **Docker Image** is recommended for all Multi-GPU trainings. See [Docker Quickstart Guide](https://github.com/ultralytics/yolov5/wiki/Docker-Quickstart) <a href="https://hub.docker.com/r/ultralytics/yolov5"><img src="https://img.shields.io/docker/pulls/ultralytics/yolov5?logo=docker" alt="Docker Pulls"></a>
|
||||||
|
|
||||||
|
💡 ProTip! `torch.distributed.run` replaces `torch.distributed.launch` in **PyTorch>=1.9**. See [docs](https://pytorch.org/docs/stable/distributed.html) for details.
|
||||||
|
|
||||||
|
## Training
|
||||||
|
|
||||||
|
Select a pretrained model to start training from. Here we select [YOLOv5s](https://github.com/ultralytics/yolov5/blob/master/models/yolov5s.yaml), the smallest and fastest model available. See our README [table](https://github.com/ultralytics/yolov5#pretrained-checkpoints) for a full comparison of all models. We will train this model with Multi-GPU on the [COCO](https://github.com/ultralytics/yolov5/blob/master/data/scripts/get_coco.sh) dataset.
|
||||||
|
|
||||||
|
<p align="center"><img width="700" alt="YOLOv5 Models" src="https://github.com/ultralytics/yolov5/releases/download/v1.0/model_comparison.png"></p>
|
||||||
|
|
||||||
|
|
||||||
|
### Single GPU
|
||||||
|
|
||||||
|
```bash
|
||||||
|
python train.py --batch 64 --data coco.yaml --weights yolov5s.pt --device 0
|
||||||
|
```
|
||||||
|
|
||||||
|
### Multi-GPU [DataParallel](https://pytorch.org/docs/stable/nn.html#torch.nn.DataParallel) Mode (⚠️ not recommended)
|
||||||
|
|
||||||
|
You can increase the `device` to use Multiple GPUs in DataParallel mode.
|
||||||
|
```bash
|
||||||
|
python train.py --batch 64 --data coco.yaml --weights yolov5s.pt --device 0,1
|
||||||
|
```
|
||||||
|
|
||||||
|
This method is slow and barely speeds up training compared to using just 1 GPU.
|
||||||
|
|
||||||
|
### Multi-GPU [DistributedDataParallel](https://pytorch.org/docs/stable/nn.html#torch.nn.parallel.DistributedDataParallel) Mode (✅ recommended)
|
||||||
|
|
||||||
|
You will have to pass `python -m torch.distributed.run --nproc_per_node`, followed by the usual arguments.
|
||||||
|
|
||||||
|
```bash
|
||||||
|
python -m torch.distributed.run --nproc_per_node 2 train.py --batch 64 --data coco.yaml --weights yolov5s.pt --device 0,1
|
||||||
|
```
|
||||||
|
|
||||||
|
`--nproc_per_node` specifies how many GPUs you would like to use. In the example above, it is 2.
|
||||||
|
`--batch ` is the total batch-size. It will be divided evenly to each GPU. In the example above, it is 64/2=32 per GPU.
|
||||||
|
|
||||||
|
The code above will use GPUs `0... (N-1)`.
|
||||||
|
|
||||||
|
<details markdown>
|
||||||
|
<summary>Use specific GPUs (click to expand)</summary>
|
||||||
|
|
||||||
|
You can do so by simply passing `--device` followed by your specific GPUs. For example, in the code below, we will use GPUs `2,3`.
|
||||||
|
|
||||||
|
```bash
|
||||||
|
python -m torch.distributed.run --nproc_per_node 2 train.py --batch 64 --data coco.yaml --cfg yolov5s.yaml --weights '' --device 2,3
|
||||||
|
```
|
||||||
|
|
||||||
|
</details>
|
||||||
|
|
||||||
|
<details markdown>
|
||||||
|
<summary>Use SyncBatchNorm (click to expand)</summary>
|
||||||
|
|
||||||
|
[SyncBatchNorm](https://pytorch.org/docs/master/generated/torch.nn.SyncBatchNorm.html) could increase accuracy for multiple gpu training, however, it will slow down training by a significant factor. It is **only** available for Multiple GPU DistributedDataParallel training.
|
||||||
|
|
||||||
|
It is best used when the batch-size on **each** GPU is small (<= 8).
|
||||||
|
|
||||||
|
To use SyncBatchNorm, simple pass `--sync-bn` to the command like below,
|
||||||
|
|
||||||
|
```bash
|
||||||
|
python -m torch.distributed.run --nproc_per_node 2 train.py --batch 64 --data coco.yaml --cfg yolov5s.yaml --weights '' --sync-bn
|
||||||
|
```
|
||||||
|
</details>
|
||||||
|
|
||||||
|
<details markdown>
|
||||||
|
<summary>Use Multiple machines (click to expand)</summary>
|
||||||
|
|
||||||
|
This is **only** available for Multiple GPU DistributedDataParallel training.
|
||||||
|
|
||||||
|
Before we continue, make sure the files on all machines are the same, dataset, codebase, etc. Afterwards, make sure the machines can communicate to each other.
|
||||||
|
|
||||||
|
You will have to choose a master machine(the machine that the others will talk to). Note down its address(`master_addr`) and choose a port(`master_port`). I will use `master_addr = 192.168.1.1` and `master_port = 1234` for the example below.
|
||||||
|
|
||||||
|
To use it, you can do as the following,
|
||||||
|
|
||||||
|
```bash
|
||||||
|
# On master machine 0
|
||||||
|
python -m torch.distributed.run --nproc_per_node G --nnodes N --node_rank 0 --master_addr "192.168.1.1" --master_port 1234 train.py --batch 64 --data coco.yaml --cfg yolov5s.yaml --weights ''
|
||||||
|
```
|
||||||
|
```bash
|
||||||
|
# On machine R
|
||||||
|
python -m torch.distributed.run --nproc_per_node G --nnodes N --node_rank R --master_addr "192.168.1.1" --master_port 1234 train.py --batch 64 --data coco.yaml --cfg yolov5s.yaml --weights ''
|
||||||
|
```
|
||||||
|
where `G` is number of GPU per machine, `N` is the number of machines, and `R` is the machine number from `0...(N-1)`.
|
||||||
|
Let's say I have two machines with two GPUs each, it would be `G = 2` , `N = 2`, and `R = 1` for the above.
|
||||||
|
|
||||||
|
Training will not start until <b>all </b> `N` machines are connected. Output will only be shown on master machine!
|
||||||
|
|
||||||
|
</details>
|
||||||
|
|
||||||
|
|
||||||
|
### Notes
|
||||||
|
|
||||||
|
- Windows support is untested, Linux is recommended.
|
||||||
|
- `--batch ` must be a multiple of the number of GPUs.
|
||||||
|
- GPU 0 will take slightly more memory than the other GPUs as it maintains EMA and is responsible for checkpointing etc.
|
||||||
|
- If you get `RuntimeError: Address already in use`, it could be because you are running multiple trainings at a time. To fix this, simply use a different port number by adding `--master_port` like below,
|
||||||
|
|
||||||
|
```bash
|
||||||
|
python -m torch.distributed.run --master_port 1234 --nproc_per_node 2 ...
|
||||||
|
```
|
||||||
|
|
||||||
|
## Results
|
||||||
|
|
||||||
|
DDP profiling results on an [AWS EC2 P4d instance](https://github.com/ultralytics/yolov5/wiki/AWS-Quickstart) with 8x A100 SXM4-40GB for YOLOv5l for 1 COCO epoch.
|
||||||
|
|
||||||
|
<details markdown>
|
||||||
|
<summary>Profiling code</summary>
|
||||||
|
|
||||||
|
```bash
|
||||||
|
# prepare
|
||||||
|
t=ultralytics/yolov5:latest && sudo docker pull $t && sudo docker run -it --ipc=host --gpus all -v "$(pwd)"/coco:/usr/src/coco $t
|
||||||
|
pip3 install torch==1.9.0+cu111 torchvision==0.10.0+cu111 -f https://download.pytorch.org/whl/torch_stable.html
|
||||||
|
cd .. && rm -rf app && git clone https://github.com/ultralytics/yolov5 -b master app && cd app
|
||||||
|
cp data/coco.yaml data/coco_profile.yaml
|
||||||
|
|
||||||
|
# profile
|
||||||
|
python train.py --batch-size 16 --data coco_profile.yaml --weights yolov5l.pt --epochs 1 --device 0
|
||||||
|
python -m torch.distributed.run --nproc_per_node 2 train.py --batch-size 32 --data coco_profile.yaml --weights yolov5l.pt --epochs 1 --device 0,1
|
||||||
|
python -m torch.distributed.run --nproc_per_node 4 train.py --batch-size 64 --data coco_profile.yaml --weights yolov5l.pt --epochs 1 --device 0,1,2,3
|
||||||
|
python -m torch.distributed.run --nproc_per_node 8 train.py --batch-size 128 --data coco_profile.yaml --weights yolov5l.pt --epochs 1 --device 0,1,2,3,4,5,6,7
|
||||||
|
```
|
||||||
|
|
||||||
|
</details>
|
||||||
|
|
||||||
|
| GPUs<br>A100 | batch-size | CUDA_mem<br><sup>device0 (G) | COCO<br><sup>train | COCO<br><sup>val |
|
||||||
|
|--------------|------------|------------------------------|--------------------|------------------|
|
||||||
|
| 1x | 16 | 26GB | 20:39 | 0:55 |
|
||||||
|
| 2x | 32 | 26GB | 11:43 | 0:57 |
|
||||||
|
| 4x | 64 | 26GB | 5:57 | 0:55 |
|
||||||
|
| 8x | 128 | 26GB | 3:09 | 0:57 |
|
||||||
|
|
||||||
|
## FAQ
|
||||||
|
|
||||||
|
If an error occurs, please read the checklist below first! (It could save your time)
|
||||||
|
|
||||||
|
<details markdown>
|
||||||
|
<summary>Checklist (click to expand) </summary>
|
||||||
|
|
||||||
|
<ul>
|
||||||
|
<li>Have you properly read this post? </li>
|
||||||
|
<li>Have you tried to reclone the codebase? The code changes <b>daily</b>.</li>
|
||||||
|
<li>Have you tried to search for your error? Someone may have already encountered it in this repo or in another and have the solution. </li>
|
||||||
|
<li>Have you installed all the requirements listed on top (including the correct Python and Pytorch versions)? </li>
|
||||||
|
<li>Have you tried in other environments listed in the "Environments" section below? </li>
|
||||||
|
<li>Have you tried with another dataset like coco128 or coco2017? It will make it easier to find the root cause. </li>
|
||||||
|
</ul>
|
||||||
|
|
||||||
|
If you went through all the above, feel free to raise an Issue by giving as much detail as possible following the template.
|
||||||
|
|
||||||
|
</details>
|
||||||
|
|
||||||
|
|
||||||
|
## Environments
|
||||||
|
|
||||||
|
YOLOv5 may be run in any of the following up-to-date verified environments (with all dependencies including [CUDA](https://developer.nvidia.com/cuda)/[CUDNN](https://developer.nvidia.com/cudnn), [Python](https://www.python.org/) and [PyTorch](https://pytorch.org/) preinstalled):
|
||||||
|
|
||||||
|
- **Notebooks** with free GPU: <a href="https://bit.ly/yolov5-paperspace-notebook"><img src="https://assets.paperspace.io/img/gradient-badge.svg" alt="Run on Gradient"></a> <a href="https://colab.research.google.com/github/ultralytics/yolov5/blob/master/tutorial.ipynb"><img src="https://colab.research.google.com/assets/colab-badge.svg" alt="Open In Colab"></a> <a href="https://www.kaggle.com/ultralytics/yolov5"><img src="https://kaggle.com/static/images/open-in-kaggle.svg" alt="Open In Kaggle"></a>
|
||||||
|
- **Google Cloud** Deep Learning VM. See [GCP Quickstart Guide](https://github.com/ultralytics/yolov5/wiki/GCP-Quickstart)
|
||||||
|
- **Amazon** Deep Learning AMI. See [AWS Quickstart Guide](https://github.com/ultralytics/yolov5/wiki/AWS-Quickstart)
|
||||||
|
- **Docker Image**. See [Docker Quickstart Guide](https://github.com/ultralytics/yolov5/wiki/Docker-Quickstart) <a href="https://hub.docker.com/r/ultralytics/yolov5"><img src="https://img.shields.io/docker/pulls/ultralytics/yolov5?logo=docker" alt="Docker Pulls"></a>
|
||||||
|
|
||||||
|
|
||||||
|
## Status
|
||||||
|
|
||||||
|
<a href="https://github.com/ultralytics/yolov5/actions/workflows/ci-testing.yml"><img src="https://github.com/ultralytics/yolov5/actions/workflows/ci-testing.yml/badge.svg" alt="YOLOv5 CI"></a>
|
||||||
|
|
||||||
|
If this badge is green, all [YOLOv5 GitHub Actions](https://github.com/ultralytics/yolov5/actions) Continuous Integration (CI) tests are currently passing. CI tests verify correct operation of YOLOv5 [training](https://github.com/ultralytics/yolov5/blob/master/train.py), [validation](https://github.com/ultralytics/yolov5/blob/master/val.py), [inference](https://github.com/ultralytics/yolov5/blob/master/detect.py), [export](https://github.com/ultralytics/yolov5/blob/master/export.py) and [benchmarks](https://github.com/ultralytics/yolov5/blob/master/benchmarks.py) on macOS, Windows, and Ubuntu every 24 hours and on every commit.
|
||||||
|
|
||||||
|
|
||||||
|
## Credits
|
||||||
|
|
||||||
|
I would like to thank @MagicFrogSJTU, who did all the heavy lifting, and @glenn-jocher for guiding us along the way.
|
||||||
@ -0,0 +1,260 @@
|
|||||||
|
<!--
|
||||||
|
Copyright (c) 2021 - present / Neuralmagic, Inc. All Rights Reserved.
|
||||||
|
|
||||||
|
Licensed under the Apache License, Version 2.0 (the "License");
|
||||||
|
you may not use this file except in compliance with the License.
|
||||||
|
You may obtain a copy of the License at
|
||||||
|
|
||||||
|
http://www.apache.org/licenses/LICENSE-2.0
|
||||||
|
|
||||||
|
Unless required by applicable law or agreed to in writing,
|
||||||
|
software distributed under the License is distributed on an "AS IS" BASIS,
|
||||||
|
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||||
|
See the License for the specific language governing permissions and
|
||||||
|
limitations under the License.
|
||||||
|
-->
|
||||||
|
|
||||||
|
Welcome to software-delivered AI.
|
||||||
|
|
||||||
|
This guide explains how to deploy YOLOv5 with Neural Magic's DeepSparse.
|
||||||
|
|
||||||
|
DeepSparse is an inference runtime with exceptional performance on CPUs. For instance, compared to the ONNX Runtime baseline, DeepSparse offers a 5.8x speed-up for YOLOv5s, running on the same machine!
|
||||||
|
|
||||||
|
<p align="center">
|
||||||
|
<img width="60%" src="https://github.com/neuralmagic/deepsparse/raw/main/examples/ultralytics-yolo/ultralytics-readmes/performance-chart-5.8x.png">
|
||||||
|
</p>
|
||||||
|
|
||||||
|
For the first time, your deep learning workloads can meet the performance demands of production without the complexity and costs of hardware accelerators.
|
||||||
|
Put simply, DeepSparse gives you the performance of GPUs and the simplicity of software:
|
||||||
|
- **Flexible Deployments**: Run consistently across cloud, data center, and edge with any hardware provider from Intel to AMD to ARM
|
||||||
|
- **Infinite Scalability**: Scale vertically to 100s of cores, out with standard Kubernetes, or fully-abstracted with Serverless
|
||||||
|
- **Easy Integration**: Clean APIs for integrating your model into an application and monitoring it in production
|
||||||
|
|
||||||
|
**[Start your 90 day Free Trial](https://neuralmagic.com/deepsparse-free-trial/?utm_campaign=free_trial&utm_source=ultralytics_github).**
|
||||||
|
|
||||||
|
### How Does DeepSparse Achieve GPU-Class Performance?
|
||||||
|
|
||||||
|
DeepSparse takes advantage of model sparsity to gain its performance speedup.
|
||||||
|
|
||||||
|
Sparsification through pruning and quantization is a broadly studied technique, allowing order-of-magnitude reductions in the size and compute needed to
|
||||||
|
execute a network, while maintaining high accuracy. DeepSparse is sparsity-aware, meaning it skips the zeroed out parameters, shrinking amount of compute
|
||||||
|
in a forward pass. Since the sparse computation is now memory bound, DeepSparse executes the network depth-wise, breaking the problem into Tensor Columns,
|
||||||
|
vertical stripes of computation that fit in cache.
|
||||||
|
|
||||||
|
<p align="center">
|
||||||
|
<img width="60%" src="https://github.com/neuralmagic/deepsparse/raw/main/examples/ultralytics-yolo/ultralytics-readmes/tensor-columns.png">
|
||||||
|
</p>
|
||||||
|
|
||||||
|
Sparse networks with compressed computation, executed depth-wise in cache, allows DeepSparse to deliver GPU-class performance on CPUs!
|
||||||
|
|
||||||
|
### How Do I Create A Sparse Version of YOLOv5 Trained on My Data?
|
||||||
|
|
||||||
|
Neural Magic's open-source model repository, SparseZoo, contains pre-sparsified checkpoints of each YOLOv5 model. Using SparseML, which is integrated with Ultralytics, you can fine-tune a sparse checkpoint onto your data with a single CLI command.
|
||||||
|
|
||||||
|
[Checkout Neural Magic's YOLOv5 documentation for more details](https://docs.neuralmagic.com/use-cases/object-detection/sparsifying).
|
||||||
|
|
||||||
|
## DeepSparse Usage
|
||||||
|
|
||||||
|
We will walk through an example benchmarking and deploying a sparse version of YOLOv5s with DeepSparse.
|
||||||
|
|
||||||
|
### Install DeepSparse
|
||||||
|
|
||||||
|
Run the following to install DeepSparse. We recommend you use a virtual environment with Python.
|
||||||
|
|
||||||
|
```bash
|
||||||
|
pip install deepsparse[server,yolo,onnxruntime]
|
||||||
|
```
|
||||||
|
|
||||||
|
### Collect an ONNX File
|
||||||
|
|
||||||
|
DeepSparse accepts a model in the ONNX format, passed either as:
|
||||||
|
- A SparseZoo stub which identifies an ONNX file in the SparseZoo
|
||||||
|
- A local path to an ONNX model in a filesystem
|
||||||
|
|
||||||
|
The examples below use the standard dense and pruned-quantized YOLOv5s checkpoints, identified by the following SparseZoo stubs:
|
||||||
|
```bash
|
||||||
|
zoo:cv/detection/yolov5-s/pytorch/ultralytics/coco/base-none
|
||||||
|
zoo:cv/detection/yolov5-s/pytorch/ultralytics/coco/pruned65_quant-none
|
||||||
|
```
|
||||||
|
|
||||||
|
### Deploy a Model
|
||||||
|
|
||||||
|
DeepSparse offers convenient APIs for integrating your model into an application.
|
||||||
|
|
||||||
|
To try the deployment examples below, pull down a sample image and save it as `basilica.jpg` with the following:
|
||||||
|
```bash
|
||||||
|
wget -O basilica.jpg https://raw.githubusercontent.com/neuralmagic/deepsparse/main/src/deepsparse/yolo/sample_images/basilica.jpg
|
||||||
|
```
|
||||||
|
|
||||||
|
#### Python API
|
||||||
|
|
||||||
|
`Pipelines` wrap pre-processing and output post-processing around the runtime, providing a clean interface for adding DeepSparse to an application.
|
||||||
|
The DeepSparse-Ultralytics integration includes an out-of-the-box `Pipeline` that accepts raw images and outputs the bounding boxes.
|
||||||
|
|
||||||
|
Create a `Pipeline` and run inference:
|
||||||
|
|
||||||
|
```python
|
||||||
|
from deepsparse import Pipeline
|
||||||
|
|
||||||
|
# list of images in local filesystem
|
||||||
|
images = ["basilica.jpg"]
|
||||||
|
|
||||||
|
# create Pipeline
|
||||||
|
model_stub = "zoo:cv/detection/yolov5-s/pytorch/ultralytics/coco/pruned65_quant-none"
|
||||||
|
yolo_pipeline = Pipeline.create(
|
||||||
|
task="yolo",
|
||||||
|
model_path=model_stub,
|
||||||
|
)
|
||||||
|
|
||||||
|
# run inference on images, receive bounding boxes + classes
|
||||||
|
pipeline_outputs = yolo_pipeline(images=images, iou_thres=0.6, conf_thres=0.001)
|
||||||
|
print(pipeline_outputs)
|
||||||
|
```
|
||||||
|
|
||||||
|
If you are running in the cloud, you may get an error that open-cv cannot find `libGL.so.1`. Running the following on Ubuntu installs it:
|
||||||
|
|
||||||
|
```
|
||||||
|
apt-get install libgl1-mesa-glx
|
||||||
|
```
|
||||||
|
|
||||||
|
#### HTTP Server
|
||||||
|
|
||||||
|
DeepSparse Server runs on top of the popular FastAPI web framework and Uvicorn web server. With just a single CLI command, you can easily setup a model
|
||||||
|
service endpoint with DeepSparse. The Server supports any Pipeline from DeepSparse, including object detection with YOLOv5, enabling you to send raw
|
||||||
|
images to the endpoint and receive the bounding boxes.
|
||||||
|
|
||||||
|
Spin up the Server with the pruned-quantized YOLOv5s:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
deepsparse.server \
|
||||||
|
--task yolo \
|
||||||
|
--model_path zoo:cv/detection/yolov5-s/pytorch/ultralytics/coco/pruned65_quant-none
|
||||||
|
```
|
||||||
|
|
||||||
|
An example request, using Python's `requests` package:
|
||||||
|
```python
|
||||||
|
import requests, json
|
||||||
|
|
||||||
|
# list of images for inference (local files on client side)
|
||||||
|
path = ['basilica.jpg']
|
||||||
|
files = [('request', open(img, 'rb')) for img in path]
|
||||||
|
|
||||||
|
# send request over HTTP to /predict/from_files endpoint
|
||||||
|
url = 'http://0.0.0.0:5543/predict/from_files'
|
||||||
|
resp = requests.post(url=url, files=files)
|
||||||
|
|
||||||
|
# response is returned in JSON
|
||||||
|
annotations = json.loads(resp.text) # dictionary of annotation results
|
||||||
|
bounding_boxes = annotations["boxes"]
|
||||||
|
labels = annotations["labels"]
|
||||||
|
```
|
||||||
|
|
||||||
|
#### Annotate CLI
|
||||||
|
You can also use the annotate command to have the engine save an annotated photo on disk. Try --source 0 to annotate your live webcam feed!
|
||||||
|
```bash
|
||||||
|
deepsparse.object_detection.annotate --model_filepath zoo:cv/detection/yolov5-s/pytorch/ultralytics/coco/pruned65_quant-none --source basilica.jpg
|
||||||
|
```
|
||||||
|
|
||||||
|
Running the above command will create an `annotation-results` folder and save the annotated image inside.
|
||||||
|
|
||||||
|
<p align = "center">
|
||||||
|
<img src="https://github.com/neuralmagic/deepsparse/raw/d31f02596ebff2ec62761d0bc9ca14c4663e8858/src/deepsparse/yolo/sample_images/basilica-annotated.jpg" alt="annotated" width="60%"/>
|
||||||
|
</p>
|
||||||
|
|
||||||
|
## Benchmarking Performance
|
||||||
|
|
||||||
|
We will compare DeepSparse's throughput to ONNX Runtime's throughput on YOLOv5s, using DeepSparse's benchmarking script.
|
||||||
|
|
||||||
|
The benchmarks were run on an AWS `c6i.8xlarge` instance (16 cores).
|
||||||
|
|
||||||
|
### Batch 32 Performance Comparison
|
||||||
|
|
||||||
|
#### ONNX Runtime Baseline
|
||||||
|
|
||||||
|
At batch 32, ONNX Runtime achieves 42 images/sec with the standard dense YOLOv5s:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
deepsparse.benchmark zoo:cv/detection/yolov5-s/pytorch/ultralytics/coco/base-none -s sync -b 32 -nstreams 1 -e onnxruntime
|
||||||
|
|
||||||
|
> Original Model Path: zoo:cv/detection/yolov5-s/pytorch/ultralytics/coco/base-none
|
||||||
|
> Batch Size: 32
|
||||||
|
> Scenario: sync
|
||||||
|
> Throughput (items/sec): 41.9025
|
||||||
|
```
|
||||||
|
|
||||||
|
#### DeepSparse Dense Performance
|
||||||
|
|
||||||
|
While DeepSparse offers its best performance with optimized sparse models, it also performs well with the standard dense YOLOv5s.
|
||||||
|
|
||||||
|
At batch 32, DeepSparse achieves 70 images/sec with the standard dense YOLOv5s, a **1.7x performance improvement over ORT**!
|
||||||
|
|
||||||
|
```bash
|
||||||
|
deepsparse.benchmark zoo:cv/detection/yolov5-s/pytorch/ultralytics/coco/base-none -s sync -b 32 -nstreams 1
|
||||||
|
|
||||||
|
> Original Model Path: zoo:cv/detection/yolov5-s/pytorch/ultralytics/coco/base-none
|
||||||
|
> Batch Size: 32
|
||||||
|
> Scenario: sync
|
||||||
|
> Throughput (items/sec): 69.5546
|
||||||
|
```
|
||||||
|
#### DeepSparse Sparse Performance
|
||||||
|
|
||||||
|
When sparsity is applied to the model, DeepSparse's performance gains over ONNX Runtime is even stronger.
|
||||||
|
|
||||||
|
At batch 32, DeepSparse achieves 241 images/sec with the pruned-quantized YOLOv5s, a **5.8x performance improvement over ORT**!
|
||||||
|
|
||||||
|
```bash
|
||||||
|
deepsparse.benchmark zoo:cv/detection/yolov5-s/pytorch/ultralytics/coco/pruned65_quant-none -s sync -b 32 -nstreams 1
|
||||||
|
|
||||||
|
> Original Model Path: zoo:cv/detection/yolov5-s/pytorch/ultralytics/coco/pruned65_quant-none
|
||||||
|
> Batch Size: 32
|
||||||
|
> Scenario: sync
|
||||||
|
> Throughput (items/sec): 241.2452
|
||||||
|
```
|
||||||
|
|
||||||
|
### Batch 1 Performance Comparison
|
||||||
|
|
||||||
|
DeepSparse is also able to gain a speed-up over ONNX Runtime for the latency-sensitive, batch 1 scenario.
|
||||||
|
|
||||||
|
#### ONNX Runtime Baseline
|
||||||
|
At batch 1, ONNX Runtime achieves 48 images/sec with the standard, dense YOLOv5s.
|
||||||
|
|
||||||
|
```bash
|
||||||
|
deepsparse.benchmark zoo:cv/detection/yolov5-s/pytorch/ultralytics/coco/base-none -s sync -b 1 -nstreams 1 -e onnxruntime
|
||||||
|
|
||||||
|
> Original Model Path: zoo:cv/detection/yolov5-s/pytorch/ultralytics/coco/base-none
|
||||||
|
> Batch Size: 1
|
||||||
|
> Scenario: sync
|
||||||
|
> Throughput (items/sec): 48.0921
|
||||||
|
```
|
||||||
|
|
||||||
|
#### DeepSparse Sparse Performance
|
||||||
|
|
||||||
|
At batch 1, DeepSparse achieves 135 items/sec with a pruned-quantized YOLOv5s, **a 2.8x performance gain over ONNX Runtime!**
|
||||||
|
|
||||||
|
```bash
|
||||||
|
deepsparse.benchmark zoo:cv/detection/yolov5-s/pytorch/ultralytics/coco/pruned65_quant-none -s sync -b 1 -nstreams 1
|
||||||
|
|
||||||
|
> Original Model Path: zoo:cv/detection/yolov5-s/pytorch/ultralytics/coco/pruned65_quant-none
|
||||||
|
> Batch Size: 1
|
||||||
|
> Scenario: sync
|
||||||
|
> Throughput (items/sec): 134.9468
|
||||||
|
```
|
||||||
|
|
||||||
|
Since `c6i.8xlarge` instances have VNNI instructions, DeepSparse's throughput can be pushed further if weights are pruned in blocks of 4.
|
||||||
|
|
||||||
|
At batch 1, DeepSparse achieves 180 items/sec with a 4-block pruned-quantized YOLOv5s, a **3.7x performance gain over ONNX Runtime!**
|
||||||
|
|
||||||
|
```bash
|
||||||
|
deepsparse.benchmark zoo:cv/detection/yolov5-s/pytorch/ultralytics/coco/pruned35_quant-none-vnni -s sync -b 1 -nstreams 1
|
||||||
|
|
||||||
|
> Original Model Path: zoo:cv/detection/yolov5-s/pytorch/ultralytics/coco/pruned35_quant-none-vnni
|
||||||
|
> Batch Size: 1
|
||||||
|
> Scenario: sync
|
||||||
|
> Throughput (items/sec): 179.7375
|
||||||
|
```
|
||||||
|
|
||||||
|
## Get Started With DeepSparse
|
||||||
|
|
||||||
|
**Research or Testing?** DeepSparse Community is free for research and testing. Get started with our [Documentation](https://docs.neuralmagic.com/).
|
||||||
|
|
||||||
|
**Want to Try DeepSparse Enterprise?** [Start your 90 day free trial](https://neuralmagic.com/deepsparse-free-trial/?utm_campaign=free_trial&utm_source=ultralytics_github).
|
||||||
@ -0,0 +1,103 @@
|
|||||||
|
📚 This guide explains how to apply **pruning** to YOLOv5 🚀 models.
|
||||||
|
UPDATED 25 September 2022.
|
||||||
|
|
||||||
|
## Before You Start
|
||||||
|
|
||||||
|
Clone repo and install [requirements.txt](https://github.com/ultralytics/yolov5/blob/master/requirements.txt) in a [**Python>=3.7.0**](https://www.python.org/) environment, including [**PyTorch>=1.7**](https://pytorch.org/get-started/locally/). [Models](https://github.com/ultralytics/yolov5/tree/master/models) and [datasets](https://github.com/ultralytics/yolov5/tree/master/data) download automatically from the latest YOLOv5 [release](https://github.com/ultralytics/yolov5/releases).
|
||||||
|
|
||||||
|
```bash
|
||||||
|
git clone https://github.com/ultralytics/yolov5 # clone
|
||||||
|
cd yolov5
|
||||||
|
pip install -r requirements.txt # install
|
||||||
|
```
|
||||||
|
|
||||||
|
## Test Normally
|
||||||
|
|
||||||
|
Before pruning we want to establish a baseline performance to compare to. This command tests YOLOv5x on COCO val2017 at image size 640 pixels. `yolov5x.pt` is the largest and most accurate model available. Other options are `yolov5s.pt`, `yolov5m.pt` and `yolov5l.pt`, or you own checkpoint from training a custom dataset `./weights/best.pt`. For details on all available models please see our README [table](https://github.com/ultralytics/yolov5#pretrained-checkpoints).
|
||||||
|
```bash
|
||||||
|
python val.py --weights yolov5x.pt --data coco.yaml --img 640 --half
|
||||||
|
```
|
||||||
|
|
||||||
|
Output:
|
||||||
|
```shell
|
||||||
|
val: data=/content/yolov5/data/coco.yaml, weights=['yolov5x.pt'], batch_size=32, imgsz=640, conf_thres=0.001, iou_thres=0.65, task=val, device=, workers=8, single_cls=False, augment=False, verbose=False, save_txt=False, save_hybrid=False, save_conf=False, save_json=True, project=runs/val, name=exp, exist_ok=False, half=True, dnn=False
|
||||||
|
YOLOv5 🚀 v6.0-224-g4c40933 torch 1.10.0+cu111 CUDA:0 (Tesla V100-SXM2-16GB, 16160MiB)
|
||||||
|
|
||||||
|
Fusing layers...
|
||||||
|
Model Summary: 444 layers, 86705005 parameters, 0 gradients
|
||||||
|
val: Scanning '/content/datasets/coco/val2017.cache' images and labels... 4952 found, 48 missing, 0 empty, 0 corrupt: 100% 5000/5000 [00:00<?, ?it/s]
|
||||||
|
Class Images Labels P R mAP@.5 mAP@.5:.95: 100% 157/157 [01:12<00:00, 2.16it/s]
|
||||||
|
all 5000 36335 0.732 0.628 0.683 0.496
|
||||||
|
Speed: 0.1ms pre-process, 5.2ms inference, 1.7ms NMS per image at shape (32, 3, 640, 640) # <--- base speed
|
||||||
|
|
||||||
|
Evaluating pycocotools mAP... saving runs/val/exp2/yolov5x_predictions.json...
|
||||||
|
...
|
||||||
|
Average Precision (AP) @[ IoU=0.50:0.95 | area= all | maxDets=100 ] = 0.507 # <--- base mAP
|
||||||
|
Average Precision (AP) @[ IoU=0.50 | area= all | maxDets=100 ] = 0.689
|
||||||
|
Average Precision (AP) @[ IoU=0.75 | area= all | maxDets=100 ] = 0.552
|
||||||
|
Average Precision (AP) @[ IoU=0.50:0.95 | area= small | maxDets=100 ] = 0.345
|
||||||
|
Average Precision (AP) @[ IoU=0.50:0.95 | area=medium | maxDets=100 ] = 0.559
|
||||||
|
Average Precision (AP) @[ IoU=0.50:0.95 | area= large | maxDets=100 ] = 0.652
|
||||||
|
Average Recall (AR) @[ IoU=0.50:0.95 | area= all | maxDets= 1 ] = 0.381
|
||||||
|
Average Recall (AR) @[ IoU=0.50:0.95 | area= all | maxDets= 10 ] = 0.630
|
||||||
|
Average Recall (AR) @[ IoU=0.50:0.95 | area= all | maxDets=100 ] = 0.682
|
||||||
|
Average Recall (AR) @[ IoU=0.50:0.95 | area= small | maxDets=100 ] = 0.526
|
||||||
|
Average Recall (AR) @[ IoU=0.50:0.95 | area=medium | maxDets=100 ] = 0.731
|
||||||
|
Average Recall (AR) @[ IoU=0.50:0.95 | area= large | maxDets=100 ] = 0.829
|
||||||
|
Results saved to runs/val/exp
|
||||||
|
```
|
||||||
|
|
||||||
|
## Test YOLOv5x on COCO (0.30 sparsity)
|
||||||
|
|
||||||
|
We repeat the above test with a pruned model by using the `torch_utils.prune()` command. We update `val.py` to prune YOLOv5x to 0.3 sparsity:
|
||||||
|
|
||||||
|
<img width="894" alt="Screenshot 2022-02-02 at 22 54 18" src="https://user-images.githubusercontent.com/26833433/152243799-b0ac2777-b1a8-47b1-801a-2e4c93c06ead.png">
|
||||||
|
|
||||||
|
30% pruned output:
|
||||||
|
```bash
|
||||||
|
val: data=/content/yolov5/data/coco.yaml, weights=['yolov5x.pt'], batch_size=32, imgsz=640, conf_thres=0.001, iou_thres=0.65, task=val, device=, workers=8, single_cls=False, augment=False, verbose=False, save_txt=False, save_hybrid=False, save_conf=False, save_json=True, project=runs/val, name=exp, exist_ok=False, half=True, dnn=False
|
||||||
|
YOLOv5 🚀 v6.0-224-g4c40933 torch 1.10.0+cu111 CUDA:0 (Tesla V100-SXM2-16GB, 16160MiB)
|
||||||
|
|
||||||
|
Fusing layers...
|
||||||
|
Model Summary: 444 layers, 86705005 parameters, 0 gradients
|
||||||
|
Pruning model... 0.3 global sparsity
|
||||||
|
val: Scanning '/content/datasets/coco/val2017.cache' images and labels... 4952 found, 48 missing, 0 empty, 0 corrupt: 100% 5000/5000 [00:00<?, ?it/s]
|
||||||
|
Class Images Labels P R mAP@.5 mAP@.5:.95: 100% 157/157 [01:11<00:00, 2.19it/s]
|
||||||
|
all 5000 36335 0.724 0.614 0.671 0.478
|
||||||
|
Speed: 0.1ms pre-process, 5.2ms inference, 1.7ms NMS per image at shape (32, 3, 640, 640) # <--- prune mAP
|
||||||
|
|
||||||
|
Evaluating pycocotools mAP... saving runs/val/exp3/yolov5x_predictions.json...
|
||||||
|
...
|
||||||
|
Average Precision (AP) @[ IoU=0.50:0.95 | area= all | maxDets=100 ] = 0.489 # <--- prune mAP
|
||||||
|
Average Precision (AP) @[ IoU=0.50 | area= all | maxDets=100 ] = 0.677
|
||||||
|
Average Precision (AP) @[ IoU=0.75 | area= all | maxDets=100 ] = 0.537
|
||||||
|
Average Precision (AP) @[ IoU=0.50:0.95 | area= small | maxDets=100 ] = 0.334
|
||||||
|
Average Precision (AP) @[ IoU=0.50:0.95 | area=medium | maxDets=100 ] = 0.542
|
||||||
|
Average Precision (AP) @[ IoU=0.50:0.95 | area= large | maxDets=100 ] = 0.635
|
||||||
|
Average Recall (AR) @[ IoU=0.50:0.95 | area= all | maxDets= 1 ] = 0.370
|
||||||
|
Average Recall (AR) @[ IoU=0.50:0.95 | area= all | maxDets= 10 ] = 0.612
|
||||||
|
Average Recall (AR) @[ IoU=0.50:0.95 | area= all | maxDets=100 ] = 0.664
|
||||||
|
Average Recall (AR) @[ IoU=0.50:0.95 | area= small | maxDets=100 ] = 0.496
|
||||||
|
Average Recall (AR) @[ IoU=0.50:0.95 | area=medium | maxDets=100 ] = 0.722
|
||||||
|
Average Recall (AR) @[ IoU=0.50:0.95 | area= large | maxDets=100 ] = 0.803
|
||||||
|
Results saved to runs/val/exp3
|
||||||
|
```
|
||||||
|
|
||||||
|
In the results we can observe that we have achieved a **sparsity of 30%** in our model after pruning, which means that 30% of the model's weight parameters in `nn.Conv2d` layers are equal to 0. **Inference time is essentially unchanged**, while the model's **AP and AR scores a slightly reduced**.
|
||||||
|
|
||||||
|
|
||||||
|
## Environments
|
||||||
|
|
||||||
|
YOLOv5 may be run in any of the following up-to-date verified environments (with all dependencies including [CUDA](https://developer.nvidia.com/cuda)/[CUDNN](https://developer.nvidia.com/cudnn), [Python](https://www.python.org/) and [PyTorch](https://pytorch.org/) preinstalled):
|
||||||
|
|
||||||
|
- **Notebooks** with free GPU: <a href="https://bit.ly/yolov5-paperspace-notebook"><img src="https://assets.paperspace.io/img/gradient-badge.svg" alt="Run on Gradient"></a> <a href="https://colab.research.google.com/github/ultralytics/yolov5/blob/master/tutorial.ipynb"><img src="https://colab.research.google.com/assets/colab-badge.svg" alt="Open In Colab"></a> <a href="https://www.kaggle.com/ultralytics/yolov5"><img src="https://kaggle.com/static/images/open-in-kaggle.svg" alt="Open In Kaggle"></a>
|
||||||
|
- **Google Cloud** Deep Learning VM. See [GCP Quickstart Guide](https://github.com/ultralytics/yolov5/wiki/GCP-Quickstart)
|
||||||
|
- **Amazon** Deep Learning AMI. See [AWS Quickstart Guide](https://github.com/ultralytics/yolov5/wiki/AWS-Quickstart)
|
||||||
|
- **Docker Image**. See [Docker Quickstart Guide](https://github.com/ultralytics/yolov5/wiki/Docker-Quickstart) <a href="https://hub.docker.com/r/ultralytics/yolov5"><img src="https://img.shields.io/docker/pulls/ultralytics/yolov5?logo=docker" alt="Docker Pulls"></a>
|
||||||
|
|
||||||
|
|
||||||
|
## Status
|
||||||
|
|
||||||
|
<a href="https://github.com/ultralytics/yolov5/actions/workflows/ci-testing.yml"><img src="https://github.com/ultralytics/yolov5/actions/workflows/ci-testing.yml/badge.svg" alt="YOLOv5 CI"></a>
|
||||||
|
|
||||||
|
If this badge is green, all [YOLOv5 GitHub Actions](https://github.com/ultralytics/yolov5/actions) Continuous Integration (CI) tests are currently passing. CI tests verify correct operation of YOLOv5 [training](https://github.com/ultralytics/yolov5/blob/master/train.py), [validation](https://github.com/ultralytics/yolov5/blob/master/val.py), [inference](https://github.com/ultralytics/yolov5/blob/master/detect.py), [export](https://github.com/ultralytics/yolov5/blob/master/export.py) and [benchmarks](https://github.com/ultralytics/yolov5/blob/master/benchmarks.py) on macOS, Windows, and Ubuntu every 24 hours and on every commit.
|
||||||
@ -0,0 +1,290 @@
|
|||||||
|
📚 This guide explains how to load YOLOv5 🚀 from PyTorch Hub at [https://pytorch.org/hub/ultralytics_yolov5](https://pytorch.org/hub/ultralytics_yolov5).
|
||||||
|
UPDATED 26 March 2023.
|
||||||
|
|
||||||
|
## Before You Start
|
||||||
|
|
||||||
|
Install [requirements.txt](https://github.com/ultralytics/yolov5/blob/master/requirements.txt) in a [**Python>=3.7.0**](https://www.python.org/) environment, including [**PyTorch>=1.7**](https://pytorch.org/get-started/locally/). [Models](https://github.com/ultralytics/yolov5/tree/master/models) and [datasets](https://github.com/ultralytics/yolov5/tree/master/data) download automatically from the latest YOLOv5 [release](https://github.com/ultralytics/yolov5/releases).
|
||||||
|
|
||||||
|
```bash
|
||||||
|
pip install -r https://raw.githubusercontent.com/ultralytics/yolov5/master/requirements.txt
|
||||||
|
```
|
||||||
|
|
||||||
|
💡 ProTip: Cloning [https://github.com/ultralytics/yolov5](https://github.com/ultralytics/yolov5) is **not** required 😃
|
||||||
|
|
||||||
|
## Load YOLOv5 with PyTorch Hub
|
||||||
|
|
||||||
|
### Simple Example
|
||||||
|
|
||||||
|
This example loads a pretrained YOLOv5s model from PyTorch Hub as `model` and passes an image for inference. `'yolov5s'` is the lightest and fastest YOLOv5 model. For details on all available models please see the [README](https://github.com/ultralytics/yolov5#pretrained-checkpoints).
|
||||||
|
```python
|
||||||
|
import torch
|
||||||
|
|
||||||
|
# Model
|
||||||
|
model = torch.hub.load('ultralytics/yolov5', 'yolov5s')
|
||||||
|
|
||||||
|
# Image
|
||||||
|
im = 'https://ultralytics.com/images/zidane.jpg'
|
||||||
|
|
||||||
|
# Inference
|
||||||
|
results = model(im)
|
||||||
|
|
||||||
|
results.pandas().xyxy[0]
|
||||||
|
# xmin ymin xmax ymax confidence class name
|
||||||
|
# 0 749.50 43.50 1148.0 704.5 0.874023 0 person
|
||||||
|
# 1 433.50 433.50 517.5 714.5 0.687988 27 tie
|
||||||
|
# 2 114.75 195.75 1095.0 708.0 0.624512 0 person
|
||||||
|
# 3 986.00 304.00 1028.0 420.0 0.286865 27 tie
|
||||||
|
```
|
||||||
|
|
||||||
|
|
||||||
|
### Detailed Example
|
||||||
|
|
||||||
|
This example shows **batched inference** with **PIL** and **OpenCV** image sources. `results` can be **printed** to console, **saved** to `runs/hub`, **showed** to screen on supported environments, and returned as **tensors** or **pandas** dataframes.
|
||||||
|
```python
|
||||||
|
import cv2
|
||||||
|
import torch
|
||||||
|
from PIL import Image
|
||||||
|
|
||||||
|
# Model
|
||||||
|
model = torch.hub.load('ultralytics/yolov5', 'yolov5s')
|
||||||
|
|
||||||
|
# Images
|
||||||
|
for f in 'zidane.jpg', 'bus.jpg':
|
||||||
|
torch.hub.download_url_to_file('https://ultralytics.com/images/' + f, f) # download 2 images
|
||||||
|
im1 = Image.open('zidane.jpg') # PIL image
|
||||||
|
im2 = cv2.imread('bus.jpg')[..., ::-1] # OpenCV image (BGR to RGB)
|
||||||
|
|
||||||
|
# Inference
|
||||||
|
results = model([im1, im2], size=640) # batch of images
|
||||||
|
|
||||||
|
# Results
|
||||||
|
results.print()
|
||||||
|
results.save() # or .show()
|
||||||
|
|
||||||
|
results.xyxy[0] # im1 predictions (tensor)
|
||||||
|
results.pandas().xyxy[0] # im1 predictions (pandas)
|
||||||
|
# xmin ymin xmax ymax confidence class name
|
||||||
|
# 0 749.50 43.50 1148.0 704.5 0.874023 0 person
|
||||||
|
# 1 433.50 433.50 517.5 714.5 0.687988 27 tie
|
||||||
|
# 2 114.75 195.75 1095.0 708.0 0.624512 0 person
|
||||||
|
# 3 986.00 304.00 1028.0 420.0 0.286865 27 tie
|
||||||
|
```
|
||||||
|
<img src="https://user-images.githubusercontent.com/26833433/124915064-62a49e00-dff1-11eb-86b3-a85b97061afb.jpg" width="500"> <img src="https://user-images.githubusercontent.com/26833433/124915055-60424400-dff1-11eb-9055-24585b375a29.jpg" width="300">
|
||||||
|
|
||||||
|
For all inference options see YOLOv5 `AutoShape()` forward [method](https://github.com/ultralytics/yolov5/blob/30e4c4f09297b67afedf8b2bcd851833ddc9dead/models/common.py#L243-L252).
|
||||||
|
|
||||||
|
### Inference Settings
|
||||||
|
YOLOv5 models contain various inference attributes such as **confidence threshold**, **IoU threshold**, etc. which can be set by:
|
||||||
|
```python
|
||||||
|
model.conf = 0.25 # NMS confidence threshold
|
||||||
|
iou = 0.45 # NMS IoU threshold
|
||||||
|
agnostic = False # NMS class-agnostic
|
||||||
|
multi_label = False # NMS multiple labels per box
|
||||||
|
classes = None # (optional list) filter by class, i.e. = [0, 15, 16] for COCO persons, cats and dogs
|
||||||
|
max_det = 1000 # maximum number of detections per image
|
||||||
|
amp = False # Automatic Mixed Precision (AMP) inference
|
||||||
|
|
||||||
|
results = model(im, size=320) # custom inference size
|
||||||
|
```
|
||||||
|
|
||||||
|
|
||||||
|
### Device
|
||||||
|
Models can be transferred to any device after creation:
|
||||||
|
```python
|
||||||
|
model.cpu() # CPU
|
||||||
|
model.cuda() # GPU
|
||||||
|
model.to(device) # i.e. device=torch.device(0)
|
||||||
|
```
|
||||||
|
|
||||||
|
Models can also be created directly on any `device`:
|
||||||
|
```python
|
||||||
|
model = torch.hub.load('ultralytics/yolov5', 'yolov5s', device='cpu') # load on CPU
|
||||||
|
```
|
||||||
|
|
||||||
|
💡 ProTip: Input images are automatically transferred to the correct model device before inference.
|
||||||
|
|
||||||
|
### Silence Outputs
|
||||||
|
Models can be loaded silently with `_verbose=False`:
|
||||||
|
```python
|
||||||
|
model = torch.hub.load('ultralytics/yolov5', 'yolov5s', _verbose=False) # load silently
|
||||||
|
```
|
||||||
|
|
||||||
|
### Input Channels
|
||||||
|
To load a pretrained YOLOv5s model with 4 input channels rather than the default 3:
|
||||||
|
```python
|
||||||
|
model = torch.hub.load('ultralytics/yolov5', 'yolov5s', channels=4)
|
||||||
|
```
|
||||||
|
In this case the model will be composed of pretrained weights **except for** the very first input layer, which is no longer the same shape as the pretrained input layer. The input layer will remain initialized by random weights.
|
||||||
|
|
||||||
|
### Number of Classes
|
||||||
|
To load a pretrained YOLOv5s model with 10 output classes rather than the default 80:
|
||||||
|
```python
|
||||||
|
model = torch.hub.load('ultralytics/yolov5', 'yolov5s', classes=10)
|
||||||
|
```
|
||||||
|
In this case the model will be composed of pretrained weights **except for** the output layers, which are no longer the same shape as the pretrained output layers. The output layers will remain initialized by random weights.
|
||||||
|
|
||||||
|
### Force Reload
|
||||||
|
If you run into problems with the above steps, setting `force_reload=True` may help by discarding the existing cache and force a fresh download of the latest YOLOv5 version from PyTorch Hub.
|
||||||
|
```python
|
||||||
|
model = torch.hub.load('ultralytics/yolov5', 'yolov5s', force_reload=True) # force reload
|
||||||
|
```
|
||||||
|
|
||||||
|
### Screenshot Inference
|
||||||
|
To run inference on your desktop screen:
|
||||||
|
```python
|
||||||
|
import torch
|
||||||
|
from PIL import ImageGrab
|
||||||
|
|
||||||
|
# Model
|
||||||
|
model = torch.hub.load('ultralytics/yolov5', 'yolov5s')
|
||||||
|
|
||||||
|
# Image
|
||||||
|
im = ImageGrab.grab() # take a screenshot
|
||||||
|
|
||||||
|
# Inference
|
||||||
|
results = model(im)
|
||||||
|
```
|
||||||
|
|
||||||
|
### Multi-GPU Inference
|
||||||
|
|
||||||
|
YOLOv5 models can be loaded to multiple GPUs in parallel with threaded inference:
|
||||||
|
|
||||||
|
```python
|
||||||
|
import torch
|
||||||
|
import threading
|
||||||
|
|
||||||
|
def run(model, im):
|
||||||
|
results = model(im)
|
||||||
|
results.save()
|
||||||
|
|
||||||
|
# Models
|
||||||
|
model0 = torch.hub.load('ultralytics/yolov5', 'yolov5s', device=0)
|
||||||
|
model1 = torch.hub.load('ultralytics/yolov5', 'yolov5s', device=1)
|
||||||
|
|
||||||
|
# Inference
|
||||||
|
threading.Thread(target=run, args=[model0, 'https://ultralytics.com/images/zidane.jpg'], daemon=True).start()
|
||||||
|
threading.Thread(target=run, args=[model1, 'https://ultralytics.com/images/bus.jpg'], daemon=True).start()
|
||||||
|
```
|
||||||
|
|
||||||
|
### Training
|
||||||
|
To load a YOLOv5 model for training rather than inference, set `autoshape=False`. To load a model with randomly initialized weights (to train from scratch) use `pretrained=False`. You must provide your own training script in this case. Alternatively see our YOLOv5 [Train Custom Data Tutorial](https://github.com/ultralytics/yolov5/wiki/Train-Custom-Data) for model training.
|
||||||
|
```python
|
||||||
|
model = torch.hub.load('ultralytics/yolov5', 'yolov5s', autoshape=False) # load pretrained
|
||||||
|
model = torch.hub.load('ultralytics/yolov5', 'yolov5s', autoshape=False, pretrained=False) # load scratch
|
||||||
|
```
|
||||||
|
|
||||||
|
### Base64 Results
|
||||||
|
For use with API services. See https://github.com/ultralytics/yolov5/pull/2291 and [Flask REST API](https://github.com/ultralytics/yolov5/tree/master/utils/flask_rest_api) example for details.
|
||||||
|
```python
|
||||||
|
results = model(im) # inference
|
||||||
|
|
||||||
|
results.ims # array of original images (as np array) passed to model for inference
|
||||||
|
results.render() # updates results.ims with boxes and labels
|
||||||
|
for im in results.ims:
|
||||||
|
buffered = BytesIO()
|
||||||
|
im_base64 = Image.fromarray(im)
|
||||||
|
im_base64.save(buffered, format="JPEG")
|
||||||
|
print(base64.b64encode(buffered.getvalue()).decode('utf-8')) # base64 encoded image with results
|
||||||
|
```
|
||||||
|
|
||||||
|
### Cropped Results
|
||||||
|
Results can be returned and saved as detection crops:
|
||||||
|
```python
|
||||||
|
results = model(im) # inference
|
||||||
|
crops = results.crop(save=True) # cropped detections dictionary
|
||||||
|
```
|
||||||
|
|
||||||
|
### Pandas Results
|
||||||
|
Results can be returned as [Pandas DataFrames](https://pandas.pydata.org/):
|
||||||
|
```python
|
||||||
|
results = model(im) # inference
|
||||||
|
results.pandas().xyxy[0] # Pandas DataFrame
|
||||||
|
```
|
||||||
|
<details markdown>
|
||||||
|
<summary>Pandas Output (click to expand)</summary>
|
||||||
|
|
||||||
|
```python
|
||||||
|
print(results.pandas().xyxy[0])
|
||||||
|
# xmin ymin xmax ymax confidence class name
|
||||||
|
# 0 749.50 43.50 1148.0 704.5 0.874023 0 person
|
||||||
|
# 1 433.50 433.50 517.5 714.5 0.687988 27 tie
|
||||||
|
# 2 114.75 195.75 1095.0 708.0 0.624512 0 person
|
||||||
|
# 3 986.00 304.00 1028.0 420.0 0.286865 27 tie
|
||||||
|
```
|
||||||
|
</details>
|
||||||
|
|
||||||
|
### Sorted Results
|
||||||
|
Results can be sorted by column, i.e. to sort license plate digit detection left-to-right (x-axis):
|
||||||
|
```python
|
||||||
|
results = model(im) # inference
|
||||||
|
results.pandas().xyxy[0].sort_values('xmin') # sorted left-right
|
||||||
|
```
|
||||||
|
|
||||||
|
### Box-Cropped Results
|
||||||
|
Results can be returned and saved as detection crops:
|
||||||
|
```python
|
||||||
|
results = model(im) # inference
|
||||||
|
crops = results.crop(save=True) # cropped detections dictionary
|
||||||
|
```
|
||||||
|
|
||||||
|
### JSON Results
|
||||||
|
Results can be returned in JSON format once converted to `.pandas()` dataframes using the `.to_json()` method. The JSON format can be modified using the `orient` argument. See pandas `.to_json()` [documentation](https://pandas.pydata.org/docs/reference/api/pandas.DataFrame.to_json.html) for details.
|
||||||
|
```python
|
||||||
|
results = model(ims) # inference
|
||||||
|
results.pandas().xyxy[0].to_json(orient="records") # JSON img1 predictions
|
||||||
|
```
|
||||||
|
|
||||||
|
<details markdown>
|
||||||
|
<summary>JSON Output (click to expand)</summary>
|
||||||
|
|
||||||
|
```json
|
||||||
|
[
|
||||||
|
{"xmin":749.5,"ymin":43.5,"xmax":1148.0,"ymax":704.5,"confidence":0.8740234375,"class":0,"name":"person"},
|
||||||
|
{"xmin":433.5,"ymin":433.5,"xmax":517.5,"ymax":714.5,"confidence":0.6879882812,"class":27,"name":"tie"},
|
||||||
|
{"xmin":115.25,"ymin":195.75,"xmax":1096.0,"ymax":708.0,"confidence":0.6254882812,"class":0,"name":"person"},
|
||||||
|
{"xmin":986.0,"ymin":304.0,"xmax":1028.0,"ymax":420.0,"confidence":0.2873535156,"class":27,"name":"tie"}
|
||||||
|
]
|
||||||
|
```
|
||||||
|
|
||||||
|
</details>
|
||||||
|
|
||||||
|
## Custom Models
|
||||||
|
This example loads a custom 20-class [VOC](https://github.com/ultralytics/yolov5/blob/master/data/VOC.yaml)-trained YOLOv5s model `'best.pt'` with PyTorch Hub.
|
||||||
|
```python
|
||||||
|
model = torch.hub.load('ultralytics/yolov5', 'custom', path='path/to/best.pt') # local model
|
||||||
|
model = torch.hub.load('path/to/yolov5', 'custom', path='path/to/best.pt', source='local') # local repo
|
||||||
|
```
|
||||||
|
|
||||||
|
## TensorRT, ONNX and OpenVINO Models
|
||||||
|
|
||||||
|
PyTorch Hub supports inference on most YOLOv5 export formats, including custom trained models. See [TFLite, ONNX, CoreML, TensorRT Export tutorial](https://github.com/ultralytics/yolov5/issues/251) for details on exporting models.
|
||||||
|
|
||||||
|
💡 ProTip: **TensorRT** may be up to 2-5X faster than PyTorch on [**GPU benchmarks**](https://github.com/ultralytics/yolov5/pull/6963)
|
||||||
|
💡 ProTip: **ONNX** and **OpenVINO** may be up to 2-3X faster than PyTorch on [**CPU benchmarks**](https://github.com/ultralytics/yolov5/pull/6613)
|
||||||
|
|
||||||
|
```python
|
||||||
|
model = torch.hub.load('ultralytics/yolov5', 'custom', path='yolov5s.pt') # PyTorch
|
||||||
|
'yolov5s.torchscript') # TorchScript
|
||||||
|
'yolov5s.onnx') # ONNX
|
||||||
|
'yolov5s_openvino_model/') # OpenVINO
|
||||||
|
'yolov5s.engine') # TensorRT
|
||||||
|
'yolov5s.mlmodel') # CoreML (macOS-only)
|
||||||
|
'yolov5s.tflite') # TFLite
|
||||||
|
'yolov5s_paddle_model/') # PaddlePaddle
|
||||||
|
```
|
||||||
|
|
||||||
|
## Environments
|
||||||
|
|
||||||
|
YOLOv5 may be run in any of the following up-to-date verified environments (with all dependencies including [CUDA](https://developer.nvidia.com/cuda)/[CUDNN](https://developer.nvidia.com/cudnn), [Python](https://www.python.org/) and [PyTorch](https://pytorch.org/) preinstalled):
|
||||||
|
|
||||||
|
- **Notebooks** with free GPU: <a href="https://bit.ly/yolov5-paperspace-notebook"><img src="https://assets.paperspace.io/img/gradient-badge.svg" alt="Run on Gradient"></a> <a href="https://colab.research.google.com/github/ultralytics/yolov5/blob/master/tutorial.ipynb"><img src="https://colab.research.google.com/assets/colab-badge.svg" alt="Open In Colab"></a> <a href="https://www.kaggle.com/ultralytics/yolov5"><img src="https://kaggle.com/static/images/open-in-kaggle.svg" alt="Open In Kaggle"></a>
|
||||||
|
- **Google Cloud** Deep Learning VM. See [GCP Quickstart Guide](https://github.com/ultralytics/yolov5/wiki/GCP-Quickstart)
|
||||||
|
- **Amazon** Deep Learning AMI. See [AWS Quickstart Guide](https://github.com/ultralytics/yolov5/wiki/AWS-Quickstart)
|
||||||
|
- **Docker Image**. See [Docker Quickstart Guide](https://github.com/ultralytics/yolov5/wiki/Docker-Quickstart) <a href="https://hub.docker.com/r/ultralytics/yolov5"><img src="https://img.shields.io/docker/pulls/ultralytics/yolov5?logo=docker" alt="Docker Pulls"></a>
|
||||||
|
|
||||||
|
|
||||||
|
## Status
|
||||||
|
|
||||||
|
<a href="https://github.com/ultralytics/yolov5/actions/workflows/ci-testing.yml"><img src="https://github.com/ultralytics/yolov5/actions/workflows/ci-testing.yml/badge.svg" alt="YOLOv5 CI"></a>
|
||||||
|
|
||||||
|
If this badge is green, all [YOLOv5 GitHub Actions](https://github.com/ultralytics/yolov5/actions) Continuous Integration (CI) tests are currently passing. CI tests verify correct operation of YOLOv5 [training](https://github.com/ultralytics/yolov5/blob/master/train.py), [validation](https://github.com/ultralytics/yolov5/blob/master/val.py), [inference](https://github.com/ultralytics/yolov5/blob/master/detect.py), [export](https://github.com/ultralytics/yolov5/blob/master/export.py) and [benchmarks](https://github.com/ultralytics/yolov5/blob/master/benchmarks.py) on macOS, Windows, and Ubuntu every 24 hours and on every commit.
|
||||||
@ -0,0 +1,37 @@
|
|||||||
|
# Roboflow Datasets
|
||||||
|
|
||||||
|
You can now use Roboflow to organize, label, prepare, version, and host your datasets for training YOLOv5 🚀 models. Roboflow is free to use with YOLOv5 if you make your workspace public.
|
||||||
|
UPDATED 30 September 2021.
|
||||||
|
|
||||||
|
## Upload
|
||||||
|
You can upload your data to Roboflow via [web UI](https://docs.roboflow.com/adding-data), [rest API](https://docs.roboflow.com/adding-data/upload-api), or [python](https://docs.roboflow.com/python).
|
||||||
|
|
||||||
|
## Labeling
|
||||||
|
After uploading data to Roboflow, you can label your data and review previous labels.
|
||||||
|
|
||||||
|
[](https://roboflow.com/annotate)
|
||||||
|
|
||||||
|
## Versioning
|
||||||
|
You can make versions of your dataset with different preprocessing and offline augmentation options. YOLOv5 does online augmentations natively, so be intentional when layering Roboflow's offline augs on top.
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
|
## Exporting Data
|
||||||
|
You can download your data in YOLOv5 format to quickly begin training.
|
||||||
|
|
||||||
|
```
|
||||||
|
from roboflow import Roboflow
|
||||||
|
rf = Roboflow(api_key="YOUR API KEY HERE")
|
||||||
|
project = rf.workspace().project("YOUR PROJECT")
|
||||||
|
dataset = project.version("YOUR VERSION").download("yolov5")
|
||||||
|
```
|
||||||
|
|
||||||
|
## Custom Training
|
||||||
|
We have released a custom training tutorial demonstrating all of the above capabilities. You can access the code here:
|
||||||
|
|
||||||
|
[](https://colab.research.google.com/github/roboflow-ai/yolov5-custom-training-tutorial/blob/main/yolov5-custom-training.ipynb)
|
||||||
|
|
||||||
|
## Active Learning
|
||||||
|
The real world is messy and your model will invariably encounter situations your dataset didn't anticipate. Using [active learning](https://blog.roboflow.com/what-is-active-learning/) is an important strategy to iteratively improve your dataset and model. With the Roboflow and YOLOv5 integration, you can quickly make improvements on your model deployments by using a battle tested machine learning pipeline.
|
||||||
|
|
||||||
|
<p align=""><a href="https://roboflow.com/?ref=ultralytics"><img width="1000" src="https://uploads-ssl.webflow.com/5f6bc60e665f54545a1e52a5/615627e5824c9c6195abfda9_computer-vision-cycle.png"/></a></p>
|
||||||
@ -0,0 +1,59 @@
|
|||||||
|
📚 This guide explains how to produce the best mAP and training results with YOLOv5 🚀.
|
||||||
|
UPDATED 25 May 2022.
|
||||||
|
|
||||||
|
Most of the time good results can be obtained with no changes to the models or training settings, **provided your dataset is sufficiently large and well labelled**. If at first you don't get good results, there are steps you might be able to take to improve, but we always recommend users **first train with all default settings** before considering any changes. This helps establish a performance baseline and spot areas for improvement.
|
||||||
|
|
||||||
|
If you have questions about your training results **we recommend you provide the maximum amount of information possible** if you expect a helpful response, including results plots (train losses, val losses, P, R, mAP), PR curve, confusion matrix, training mosaics, test results and dataset statistics images such as labels.png. All of these are located in your `project/name` directory, typically `yolov5/runs/train/exp`.
|
||||||
|
|
||||||
|
We've put together a full guide for users looking to get the best results on their YOLOv5 trainings below.
|
||||||
|
|
||||||
|
## Dataset
|
||||||
|
|
||||||
|
- **Images per class.** ≥ 1500 images per class recommended
|
||||||
|
- **Instances per class.** ≥ 10000 instances (labeled objects) per class recommended
|
||||||
|
- **Image variety.** Must be representative of deployed environment. For real-world use cases we recommend images from different times of day, different seasons, different weather, different lighting, different angles, different sources (scraped online, collected locally, different cameras) etc.
|
||||||
|
- **Label consistency.** All instances of all classes in all images must be labelled. Partial labelling will not work.
|
||||||
|
- **Label accuracy.** Labels must closely enclose each object. No space should exist between an object and it's bounding box. No objects should be missing a label.
|
||||||
|
- **Label verification.** View `train_batch*.jpg` on train start to verify your labels appear correct, i.e. see [example](https://github.com/ultralytics/yolov5/wiki/Train-Custom-Data#local-logging) mosaic.
|
||||||
|
- **Background images.** Background images are images with no objects that are added to a dataset to reduce False Positives (FP). We recommend about 0-10% background images to help reduce FPs (COCO has 1000 background images for reference, 1% of the total). No labels are required for background images.
|
||||||
|
|
||||||
|
<a href="https://arxiv.org/abs/1405.0312"><img width="800" src="https://user-images.githubusercontent.com/26833433/109398377-82b0ac00-78f1-11eb-9c76-cc7820669d0d.png" alt="COCO Analysis"></a>
|
||||||
|
|
||||||
|
|
||||||
|
## Model Selection
|
||||||
|
|
||||||
|
Larger models like YOLOv5x and [YOLOv5x6](https://github.com/ultralytics/yolov5/releases/tag/v5.0) will produce better results in nearly all cases, but have more parameters, require more CUDA memory to train, and are slower to run. For **mobile** deployments we recommend YOLOv5s/m, for **cloud** deployments we recommend YOLOv5l/x. See our README [table](https://github.com/ultralytics/yolov5#pretrained-checkpoints) for a full comparison of all models.
|
||||||
|
|
||||||
|
<p align="center"><img width="700" alt="YOLOv5 Models" src="https://github.com/ultralytics/yolov5/releases/download/v1.0/model_comparison.png"></p>
|
||||||
|
|
||||||
|
- **Start from Pretrained weights.** Recommended for small to medium-sized datasets (i.e. [VOC](https://github.com/ultralytics/yolov5/blob/master/data/VOC.yaml), [VisDrone](https://github.com/ultralytics/yolov5/blob/master/data/VisDrone.yaml), [GlobalWheat](https://github.com/ultralytics/yolov5/blob/master/data/GlobalWheat2020.yaml)). Pass the name of the model to the `--weights` argument. Models download automatically from the [latest YOLOv5 release](https://github.com/ultralytics/yolov5/releases).
|
||||||
|
```shell
|
||||||
|
python train.py --data custom.yaml --weights yolov5s.pt
|
||||||
|
yolov5m.pt
|
||||||
|
yolov5l.pt
|
||||||
|
yolov5x.pt
|
||||||
|
custom_pretrained.pt
|
||||||
|
```
|
||||||
|
- **Start from Scratch.** Recommended for large datasets (i.e. [COCO](https://github.com/ultralytics/yolov5/blob/master/data/coco.yaml), [Objects365](https://github.com/ultralytics/yolov5/blob/master/data/Objects365.yaml), [OIv6](https://storage.googleapis.com/openimages/web/index.html)). Pass the model architecture yaml you are interested in, along with an empty `--weights ''` argument:
|
||||||
|
```bash
|
||||||
|
python train.py --data custom.yaml --weights '' --cfg yolov5s.yaml
|
||||||
|
yolov5m.yaml
|
||||||
|
yolov5l.yaml
|
||||||
|
yolov5x.yaml
|
||||||
|
```
|
||||||
|
|
||||||
|
|
||||||
|
## Training Settings
|
||||||
|
|
||||||
|
Before modifying anything, **first train with default settings to establish a performance baseline**. A full list of train.py settings can be found in the [train.py](https://github.com/ultralytics/yolov5/blob/master/train.py) argparser.
|
||||||
|
|
||||||
|
- **Epochs.** Start with 300 epochs. If this overfits early then you can reduce epochs. If overfitting does not occur after 300 epochs, train longer, i.e. 600, 1200 etc epochs.
|
||||||
|
- **Image size.** COCO trains at native resolution of `--img 640`, though due to the high amount of small objects in the dataset it can benefit from training at higher resolutions such as `--img 1280`. If there are many small objects then custom datasets will benefit from training at native or higher resolution. Best inference results are obtained at the same `--img` as the training was run at, i.e. if you train at `--img 1280` you should also test and detect at `--img 1280`.
|
||||||
|
- **Batch size.** Use the largest `--batch-size` that your hardware allows for. Small batch sizes produce poor batchnorm statistics and should be avoided.
|
||||||
|
- **Hyperparameters.** Default hyperparameters are in [hyp.scratch-low.yaml](https://github.com/ultralytics/yolov5/blob/master/data/hyps/hyp.scratch-low.yaml). We recommend you train with default hyperparameters first before thinking of modifying any. In general, increasing augmentation hyperparameters will reduce and delay overfitting, allowing for longer trainings and higher final mAP. Reduction in loss component gain hyperparameters like `hyp['obj']` will help reduce overfitting in those specific loss components. For an automated method of optimizing these hyperparameters, see our [Hyperparameter Evolution Tutorial](https://github.com/ultralytics/yolov5/issues/607).
|
||||||
|
|
||||||
|
## Further Reading
|
||||||
|
|
||||||
|
If you'd like to know more, a good place to start is Karpathy's 'Recipe for Training Neural Networks', which has great ideas for training that apply broadly across all ML domains: [http://karpathy.github.io/2019/04/25/recipe/](http://karpathy.github.io/2019/04/25/recipe/)
|
||||||
|
|
||||||
|
Good luck 🍀 and let us know if you have any other questions!
|
||||||
@ -0,0 +1,230 @@
|
|||||||
|
📚 This guide explains how to train your own **custom dataset** with [YOLOv5](https://github.com/ultralytics/yolov5) 🚀.
|
||||||
|
UPDATED 26 March 2023.
|
||||||
|
|
||||||
|
## Before You Start
|
||||||
|
|
||||||
|
Clone repo and install [requirements.txt](https://github.com/ultralytics/yolov5/blob/master/requirements.txt) in a [**Python>=3.7.0**](https://www.python.org/) environment, including [**PyTorch>=1.7**](https://pytorch.org/get-started/locally/). [Models](https://github.com/ultralytics/yolov5/tree/master/models) and [datasets](https://github.com/ultralytics/yolov5/tree/master/data) download automatically from the latest YOLOv5 [release](https://github.com/ultralytics/yolov5/releases).
|
||||||
|
|
||||||
|
```bash
|
||||||
|
git clone https://github.com/ultralytics/yolov5 # clone
|
||||||
|
cd yolov5
|
||||||
|
pip install -r requirements.txt # install
|
||||||
|
```
|
||||||
|
|
||||||
|
## Train On Custom Data
|
||||||
|
|
||||||
|
<a href="https://bit.ly/ultralytics_hub" target="_blank">
|
||||||
|
<img width="100%" src="https://github.com/ultralytics/assets/raw/main/im/integrations-loop.png"></a>
|
||||||
|
<br>
|
||||||
|
<br>
|
||||||
|
|
||||||
|
Creating a custom model to detect your objects is an iterative process of collecting and organizing images, labeling your objects of interest, training a model, deploying it into the wild to make predictions, and then using that deployed model to collect examples of edge cases to repeat and improve.
|
||||||
|
|
||||||
|
### 1. Create Dataset
|
||||||
|
|
||||||
|
YOLOv5 models must be trained on labelled data in order to learn classes of objects in that data. There are two options for creating your dataset before you start training:
|
||||||
|
|
||||||
|
<details markdown>
|
||||||
|
<summary>Use <a href="https://roboflow.com/?ref=ultralytics">Roboflow</a> to manage your dataset in YOLO format</summary>
|
||||||
|
|
||||||
|
### 1.1 Collect Images
|
||||||
|
|
||||||
|
Your model will learn by example. Training on images similar to the ones it will see in the wild is of the utmost importance. Ideally, you will collect a wide variety of images from the same configuration (camera, angle, lighting, etc.) as you will ultimately deploy your project.
|
||||||
|
|
||||||
|
If this is not possible, you can start from [a public dataset](https://universe.roboflow.com/?ref=ultralytics) to train your initial model and then [sample images from the wild during inference](https://blog.roboflow.com/computer-vision-active-learning-tips/?ref=ultralytics) to improve your dataset and model iteratively.
|
||||||
|
|
||||||
|
### 1.2 Create Labels
|
||||||
|
|
||||||
|
Once you have collected images, you will need to annotate the objects of interest to create a ground truth for your model to learn from.
|
||||||
|
|
||||||
|
<p align="center"><a href="https://app.roboflow.com/?model=yolov5&ref=ultralytics" title="Create a Free Roboflow Account"><img width="450" src="https://uploads-ssl.webflow.com/5f6bc60e665f54545a1e52a5/6152a275ad4b4ac20cd2e21a_roboflow-annotate.gif" /></a></p>
|
||||||
|
|
||||||
|
[Roboflow Annotate](https://roboflow.com/annotate?ref=ultralytics) is a simple
|
||||||
|
web-based tool for managing and labeling your images with your team and exporting
|
||||||
|
them in [YOLOv5's annotation format](https://roboflow.com/formats/yolov5-pytorch-txt?ref=ultralytics).
|
||||||
|
|
||||||
|
### 1.3 Prepare Dataset for YOLOv5
|
||||||
|
|
||||||
|
Whether you [label your images with Roboflow](https://roboflow.com/annotate?ref=ultralytics) or not, you can use it to convert your dataset into YOLO format, create a YOLOv5 YAML configuration file, and host it for importing into your training script.
|
||||||
|
|
||||||
|
[Create a free Roboflow account](https://app.roboflow.com/?model=yolov5&ref=ultralytics)
|
||||||
|
and upload your dataset to a `Public` workspace, label any unannotated images,
|
||||||
|
then generate and export a version of your dataset in `YOLOv5 Pytorch` format.
|
||||||
|
|
||||||
|
Note: YOLOv5 does online augmentation during training, so we do not recommend
|
||||||
|
applying any augmentation steps in Roboflow for training with YOLOv5. But we
|
||||||
|
recommend applying the following preprocessing steps:
|
||||||
|
|
||||||
|
<p align="center"><img width="450" src="https://uploads-ssl.webflow.com/5f6bc60e665f54545a1e52a5/6152a273477fccf42a0fd3d6_roboflow-preprocessing.png" title="Recommended Preprocessing Steps" /></p>
|
||||||
|
|
||||||
|
* **Auto-Orient** - to strip EXIF orientation from your images.
|
||||||
|
* **Resize (Stretch)** - to the square input size of your model (640x640 is the YOLOv5 default).
|
||||||
|
|
||||||
|
Generating a version will give you a point in time snapshot of your dataset so
|
||||||
|
you can always go back and compare your future model training runs against it,
|
||||||
|
even if you add more images or change its configuration later.
|
||||||
|
|
||||||
|
<p align="center"><img width="450" src="https://uploads-ssl.webflow.com/5f6bc60e665f54545a1e52a5/6152a2733fd1da943619934e_roboflow-export.png" title="Export in YOLOv5 Format" /></p>
|
||||||
|
|
||||||
|
Export in `YOLOv5 Pytorch` format, then copy the snippet into your training
|
||||||
|
script or notebook to download your dataset.
|
||||||
|
|
||||||
|
<p align="center"><img width="450" src="https://uploads-ssl.webflow.com/5f6bc60e665f54545a1e52a5/6152a273a92e4f5cb72594df_roboflow-snippet.png" title="Roboflow dataset download snippet" /></p>
|
||||||
|
|
||||||
|
Now continue with `2. Select a Model`.
|
||||||
|
</details>
|
||||||
|
|
||||||
|
<details open markdown>
|
||||||
|
<summary>Or manually prepare your dataset</summary>
|
||||||
|
|
||||||
|
### 1.1 Create dataset.yaml
|
||||||
|
|
||||||
|
[COCO128](https://www.kaggle.com/ultralytics/coco128) is an example small tutorial dataset composed of the first 128 images in [COCO](http://cocodataset.org/#home) train2017. These same 128 images are used for both training and validation to verify our training pipeline is capable of overfitting. [data/coco128.yaml](https://github.com/ultralytics/yolov5/blob/master/data/coco128.yaml), shown below, is the dataset config file that defines 1) the dataset root directory `path` and relative paths to `train` / `val` / `test` image directories (or *.txt files with image paths) and 2) a class `names` dictionary:
|
||||||
|
```yaml
|
||||||
|
# Train/val/test sets as 1) dir: path/to/imgs, 2) file: path/to/imgs.txt, or 3) list: [path/to/imgs1, path/to/imgs2, ..]
|
||||||
|
path: ../datasets/coco128 # dataset root dir
|
||||||
|
train: images/train2017 # train images (relative to 'path') 128 images
|
||||||
|
val: images/train2017 # val images (relative to 'path') 128 images
|
||||||
|
test: # test images (optional)
|
||||||
|
|
||||||
|
# Classes (80 COCO classes)
|
||||||
|
names:
|
||||||
|
0: person
|
||||||
|
1: bicycle
|
||||||
|
2: car
|
||||||
|
...
|
||||||
|
77: teddy bear
|
||||||
|
78: hair drier
|
||||||
|
79: toothbrush
|
||||||
|
|
||||||
|
```
|
||||||
|
|
||||||
|
|
||||||
|
### 1.2 Create Labels
|
||||||
|
|
||||||
|
After using a tool like [Roboflow Annotate](https://roboflow.com/annotate?ref=ultralytics) to label your images, export your labels to **YOLO format**, with one `*.txt` file per image (if no objects in image, no `*.txt` file is required). The `*.txt` file specifications are:
|
||||||
|
|
||||||
|
- One row per object
|
||||||
|
- Each row is `class x_center y_center width height` format.
|
||||||
|
- Box coordinates must be in **normalized xywh** format (from 0 - 1). If your boxes are in pixels, divide `x_center` and `width` by image width, and `y_center` and `height` by image height.
|
||||||
|
- Class numbers are zero-indexed (start from 0).
|
||||||
|
|
||||||
|
<p align="center"><img width="750" src="https://user-images.githubusercontent.com/26833433/91506361-c7965000-e886-11ea-8291-c72b98c25eec.jpg"></p>
|
||||||
|
|
||||||
|
The label file corresponding to the above image contains 2 persons (class `0`) and a tie (class `27`):
|
||||||
|
|
||||||
|
<p align="center"><img width="428" src="https://user-images.githubusercontent.com/26833433/112467037-d2568c00-8d66-11eb-8796-55402ac0d62f.png"></p>
|
||||||
|
|
||||||
|
|
||||||
|
### 1.3 Organize Directories
|
||||||
|
|
||||||
|
Organize your train and val images and labels according to the example below. YOLOv5 assumes `/coco128` is inside a `/datasets` directory **next to** the `/yolov5` directory. **YOLOv5 locates labels automatically for each image** by replacing the last instance of `/images/` in each image path with `/labels/`. For example:
|
||||||
|
```bash
|
||||||
|
../datasets/coco128/images/im0.jpg # image
|
||||||
|
../datasets/coco128/labels/im0.txt # label
|
||||||
|
```
|
||||||
|
|
||||||
|
<p align="center"><img width="700" src="https://user-images.githubusercontent.com/26833433/134436012-65111ad1-9541-4853-81a6-f19a3468b75f.png"></p>
|
||||||
|
</details>
|
||||||
|
|
||||||
|
|
||||||
|
### 2. Select a Model
|
||||||
|
|
||||||
|
Select a pretrained model to start training from. Here we select [YOLOv5s](https://github.com/ultralytics/yolov5/blob/master/models/yolov5s.yaml), the second-smallest and fastest model available. See our README [table](https://github.com/ultralytics/yolov5#pretrained-checkpoints) for a full comparison of all models.
|
||||||
|
|
||||||
|
<p align="center"><img width="800" alt="YOLOv5 Models" src="https://github.com/ultralytics/yolov5/releases/download/v1.0/model_comparison.png"></p>
|
||||||
|
|
||||||
|
### 3. Train
|
||||||
|
|
||||||
|
Train a YOLOv5s model on COCO128 by specifying dataset, batch-size, image size and either pretrained `--weights yolov5s.pt` (recommended), or randomly initialized `--weights '' --cfg yolov5s.yaml` (not recommended). Pretrained weights are auto-downloaded from the [latest YOLOv5 release](https://github.com/ultralytics/yolov5/releases).
|
||||||
|
|
||||||
|
```bash
|
||||||
|
python train.py --img 640 --epochs 3 --data coco128.yaml --weights yolov5s.pt
|
||||||
|
```
|
||||||
|
!!! tip "Tip"
|
||||||
|
|
||||||
|
💡 Add `--cache ram` or `--cache disk` to speed up training (requires significant RAM/disk resources).
|
||||||
|
|
||||||
|
!!! tip "Tip"
|
||||||
|
|
||||||
|
💡 Always train from a local dataset. Mounted or network drives like Google Drive will be very slow.
|
||||||
|
|
||||||
|
All training results are saved to `runs/train/` with incrementing run directories, i.e. `runs/train/exp2`, `runs/train/exp3` etc. For more details see the Training section of our tutorial notebook. <a href="https://colab.research.google.com/github/ultralytics/yolov5/blob/master/tutorial.ipynb"><img src="https://colab.research.google.com/assets/colab-badge.svg" alt="Open In Colab"></a> <a href="https://www.kaggle.com/ultralytics/yolov5"><img src="https://kaggle.com/static/images/open-in-kaggle.svg" alt="Open In Kaggle"></a>
|
||||||
|
|
||||||
|
### 4. Visualize
|
||||||
|
|
||||||
|
#### Comet Logging and Visualization 🌟 NEW
|
||||||
|
|
||||||
|
[Comet](https://bit.ly/yolov5-readme-comet) is now fully integrated with YOLOv5. Track and visualize model metrics in real time, save your hyperparameters, datasets, and model checkpoints, and visualize your model predictions with [Comet Custom Panels](https://bit.ly/yolov5-colab-comet-panels)! Comet makes sure you never lose track of your work and makes it easy to share results and collaborate across teams of all sizes!
|
||||||
|
|
||||||
|
Getting started is easy:
|
||||||
|
```shell
|
||||||
|
pip install comet_ml # 1. install
|
||||||
|
export COMET_API_KEY=<Your API Key> # 2. paste API key
|
||||||
|
python train.py --img 640 --epochs 3 --data coco128.yaml --weights yolov5s.pt # 3. train
|
||||||
|
```
|
||||||
|
|
||||||
|
To learn more about all the supported Comet features for this integration, check out the [Comet Tutorial](https://github.com/ultralytics/yolov5/tree/master/utils/loggers/comet). If you'd like to learn more about Comet, head over to our [documentation](https://bit.ly/yolov5-colab-comet-docs). Get started by trying out the Comet Colab Notebook:
|
||||||
|
[](https://colab.research.google.com/drive/1RG0WOQyxlDlo5Km8GogJpIEJlg_5lyYO?usp=sharing)
|
||||||
|
|
||||||
|
<img width="1920" alt="yolo-ui" src="https://user-images.githubusercontent.com/26833433/202851203-164e94e1-2238-46dd-91f8-de020e9d6b41.png">
|
||||||
|
|
||||||
|
#### ClearML Logging and Automation 🌟 NEW
|
||||||
|
|
||||||
|
[ClearML](https://cutt.ly/yolov5-notebook-clearml) is completely integrated into YOLOv5 to track your experimentation, manage dataset versions and even remotely execute training runs. To enable ClearML:
|
||||||
|
|
||||||
|
- `pip install clearml`
|
||||||
|
- run `clearml-init` to connect to a ClearML server (**deploy your own open-source server [here](https://github.com/allegroai/clearml-server)**, or use our free hosted server [here](https://cutt.ly/yolov5-notebook-clearml))
|
||||||
|
|
||||||
|
You'll get all the great expected features from an experiment manager: live updates, model upload, experiment comparison etc. but ClearML also tracks uncommitted changes and installed packages for example. Thanks to that ClearML Tasks (which is what we call experiments) are also reproducible on different machines! With only 1 extra line, we can schedule a YOLOv5 training task on a queue to be executed by any number of ClearML Agents (workers).
|
||||||
|
|
||||||
|
You can use ClearML Data to version your dataset and then pass it to YOLOv5 simply using its unique ID. This will help you keep track of your data without adding extra hassle. Explore the [ClearML Tutorial](https://github.com/ultralytics/yolov5/tree/master/utils/loggers/clearml) for details!
|
||||||
|
|
||||||
|
<a href="https://cutt.ly/yolov5-notebook-clearml">
|
||||||
|
<img alt="ClearML Experiment Management UI" src="https://github.com/thepycoder/clearml_screenshots/raw/main/scalars.jpg" width="1280"/></a>
|
||||||
|
|
||||||
|
|
||||||
|
#### Local Logging
|
||||||
|
|
||||||
|
Training results are automatically logged with [Tensorboard](https://www.tensorflow.org/tensorboard) and [CSV](https://github.com/ultralytics/yolov5/pull/4148) loggers to `runs/train`, with a new experiment directory created for each new training as `runs/train/exp2`, `runs/train/exp3`, etc.
|
||||||
|
|
||||||
|
This directory contains train and val statistics, mosaics, labels, predictions and augmented mosaics, as well as metrics and charts including precision-recall (PR) curves and confusion matrices.
|
||||||
|
|
||||||
|
<img alt="Local logging results" src="https://github.com/ultralytics/yolov5/releases/download/v1.0/image-local_logging.jpg" width="1280"/>
|
||||||
|
|
||||||
|
Results file `results.csv` is updated after each epoch, and then plotted as `results.png` (below) after training completes. You can also plot any `results.csv` file manually:
|
||||||
|
|
||||||
|
```python
|
||||||
|
from utils.plots import plot_results
|
||||||
|
plot_results('path/to/results.csv') # plot 'results.csv' as 'results.png'
|
||||||
|
```
|
||||||
|
|
||||||
|
<p align="center"><img width="800" alt="results.png" src="https://github.com/ultralytics/yolov5/releases/download/v1.0/results.png"></p>
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
|
## Next Steps
|
||||||
|
|
||||||
|
Once your model is trained you can use your best checkpoint `best.pt` to:
|
||||||
|
* Run [CLI](https://github.com/ultralytics/yolov5#quick-start-examples) or [Python](https://github.com/ultralytics/yolov5/issues/36) inference on new images and videos
|
||||||
|
* [Validate](https://github.com/ultralytics/yolov5/blob/master/val.py) accuracy on train, val and test splits
|
||||||
|
* [Export](https://github.com/ultralytics/yolov5/issues/251) to TensorFlow, Keras, ONNX, TFlite, TF.js, CoreML and TensorRT formats
|
||||||
|
* [Evolve](https://github.com/ultralytics/yolov5/issues/607) hyperparameters to improve performance
|
||||||
|
* [Improve](https://docs.roboflow.com/adding-data/upload-api?ref=ultralytics) your model by sampling real-world images and adding them to your dataset
|
||||||
|
|
||||||
|
|
||||||
|
## Environments
|
||||||
|
|
||||||
|
YOLOv5 may be run in any of the following up-to-date verified environments (with all dependencies including [CUDA](https://developer.nvidia.com/cuda)/[CUDNN](https://developer.nvidia.com/cudnn), [Python](https://www.python.org/) and [PyTorch](https://pytorch.org/) preinstalled):
|
||||||
|
|
||||||
|
- **Notebooks** with free GPU: <a href="https://bit.ly/yolov5-paperspace-notebook"><img src="https://assets.paperspace.io/img/gradient-badge.svg" alt="Run on Gradient"></a> <a href="https://colab.research.google.com/github/ultralytics/yolov5/blob/master/tutorial.ipynb"><img src="https://colab.research.google.com/assets/colab-badge.svg" alt="Open In Colab"></a> <a href="https://www.kaggle.com/ultralytics/yolov5"><img src="https://kaggle.com/static/images/open-in-kaggle.svg" alt="Open In Kaggle"></a>
|
||||||
|
- **Google Cloud** Deep Learning VM. See [GCP Quickstart Guide](https://github.com/ultralytics/yolov5/wiki/GCP-Quickstart)
|
||||||
|
- **Amazon** Deep Learning AMI. See [AWS Quickstart Guide](https://github.com/ultralytics/yolov5/wiki/AWS-Quickstart)
|
||||||
|
- **Docker Image**. See [Docker Quickstart Guide](https://github.com/ultralytics/yolov5/wiki/Docker-Quickstart) <a href="https://hub.docker.com/r/ultralytics/yolov5"><img src="https://img.shields.io/docker/pulls/ultralytics/yolov5?logo=docker" alt="Docker Pulls"></a>
|
||||||
|
|
||||||
|
|
||||||
|
## Status
|
||||||
|
|
||||||
|
<a href="https://github.com/ultralytics/yolov5/actions/workflows/ci-testing.yml"><img src="https://github.com/ultralytics/yolov5/actions/workflows/ci-testing.yml/badge.svg" alt="YOLOv5 CI"></a>
|
||||||
|
|
||||||
|
If this badge is green, all [YOLOv5 GitHub Actions](https://github.com/ultralytics/yolov5/actions) Continuous Integration (CI) tests are currently passing. CI tests verify correct operation of YOLOv5 [training](https://github.com/ultralytics/yolov5/blob/master/train.py), [validation](https://github.com/ultralytics/yolov5/blob/master/val.py), [inference](https://github.com/ultralytics/yolov5/blob/master/detect.py), [export](https://github.com/ultralytics/yolov5/blob/master/export.py) and [benchmarks](https://github.com/ultralytics/yolov5/blob/master/benchmarks.py) on macOS, Windows, and Ubuntu every 24 hours and on every commit.
|
||||||
@ -0,0 +1,146 @@
|
|||||||
|
📚 This guide explains how to **freeze** YOLOv5 🚀 layers when **transfer learning**. Transfer learning is a useful way to quickly retrain a model on new data without having to retrain the entire network. Instead, part of the initial weights are frozen in place, and the rest of the weights are used to compute loss and are updated by the optimizer. This requires less resources than normal training and allows for faster training times, though it may also result in reductions to final trained accuracy.
|
||||||
|
UPDATED 25 September 2022.
|
||||||
|
|
||||||
|
|
||||||
|
## Before You Start
|
||||||
|
|
||||||
|
Clone repo and install [requirements.txt](https://github.com/ultralytics/yolov5/blob/master/requirements.txt) in a [**Python>=3.7.0**](https://www.python.org/) environment, including [**PyTorch>=1.7**](https://pytorch.org/get-started/locally/). [Models](https://github.com/ultralytics/yolov5/tree/master/models) and [datasets](https://github.com/ultralytics/yolov5/tree/master/data) download automatically from the latest YOLOv5 [release](https://github.com/ultralytics/yolov5/releases).
|
||||||
|
|
||||||
|
```bash
|
||||||
|
git clone https://github.com/ultralytics/yolov5 # clone
|
||||||
|
cd yolov5
|
||||||
|
pip install -r requirements.txt # install
|
||||||
|
```
|
||||||
|
|
||||||
|
## Freeze Backbone
|
||||||
|
|
||||||
|
All layers that match the train.py `freeze` list in train.py will be frozen by setting their gradients to zero before training starts.
|
||||||
|
```python
|
||||||
|
# Freeze
|
||||||
|
freeze = [f'model.{x}.' for x in range(freeze)] # layers to freeze
|
||||||
|
for k, v in model.named_parameters():
|
||||||
|
v.requires_grad = True # train all layers
|
||||||
|
if any(x in k for x in freeze):
|
||||||
|
print(f'freezing {k}')
|
||||||
|
v.requires_grad = False
|
||||||
|
```
|
||||||
|
|
||||||
|
To see a list of module names:
|
||||||
|
```python
|
||||||
|
for k, v in model.named_parameters():
|
||||||
|
print(k)
|
||||||
|
|
||||||
|
# Output
|
||||||
|
model.0.conv.conv.weight
|
||||||
|
model.0.conv.bn.weight
|
||||||
|
model.0.conv.bn.bias
|
||||||
|
model.1.conv.weight
|
||||||
|
model.1.bn.weight
|
||||||
|
model.1.bn.bias
|
||||||
|
model.2.cv1.conv.weight
|
||||||
|
model.2.cv1.bn.weight
|
||||||
|
...
|
||||||
|
model.23.m.0.cv2.bn.weight
|
||||||
|
model.23.m.0.cv2.bn.bias
|
||||||
|
model.24.m.0.weight
|
||||||
|
model.24.m.0.bias
|
||||||
|
model.24.m.1.weight
|
||||||
|
model.24.m.1.bias
|
||||||
|
model.24.m.2.weight
|
||||||
|
model.24.m.2.bias
|
||||||
|
```
|
||||||
|
|
||||||
|
Looking at the model architecture we can see that the model backbone is layers 0-9:
|
||||||
|
```yaml
|
||||||
|
# YOLOv5 backbone
|
||||||
|
backbone:
|
||||||
|
# [from, number, module, args]
|
||||||
|
[[-1, 1, Focus, [64, 3]], # 0-P1/2
|
||||||
|
[-1, 1, Conv, [128, 3, 2]], # 1-P2/4
|
||||||
|
[-1, 3, BottleneckCSP, [128]],
|
||||||
|
[-1, 1, Conv, [256, 3, 2]], # 3-P3/8
|
||||||
|
[-1, 9, BottleneckCSP, [256]],
|
||||||
|
[-1, 1, Conv, [512, 3, 2]], # 5-P4/16
|
||||||
|
[-1, 9, BottleneckCSP, [512]],
|
||||||
|
[-1, 1, Conv, [1024, 3, 2]], # 7-P5/32
|
||||||
|
[-1, 1, SPP, [1024, [5, 9, 13]]],
|
||||||
|
[-1, 3, BottleneckCSP, [1024, False]], # 9
|
||||||
|
]
|
||||||
|
|
||||||
|
# YOLOv5 head
|
||||||
|
head:
|
||||||
|
[[-1, 1, Conv, [512, 1, 1]],
|
||||||
|
[-1, 1, nn.Upsample, [None, 2, 'nearest']],
|
||||||
|
[[-1, 6], 1, Concat, [1]], # cat backbone P4
|
||||||
|
[-1, 3, BottleneckCSP, [512, False]], # 13
|
||||||
|
|
||||||
|
[-1, 1, Conv, [256, 1, 1]],
|
||||||
|
[-1, 1, nn.Upsample, [None, 2, 'nearest']],
|
||||||
|
[[-1, 4], 1, Concat, [1]], # cat backbone P3
|
||||||
|
[-1, 3, BottleneckCSP, [256, False]], # 17 (P3/8-small)
|
||||||
|
|
||||||
|
[-1, 1, Conv, [256, 3, 2]],
|
||||||
|
[[-1, 14], 1, Concat, [1]], # cat head P4
|
||||||
|
[-1, 3, BottleneckCSP, [512, False]], # 20 (P4/16-medium)
|
||||||
|
|
||||||
|
[-1, 1, Conv, [512, 3, 2]],
|
||||||
|
[[-1, 10], 1, Concat, [1]], # cat head P5
|
||||||
|
[-1, 3, BottleneckCSP, [1024, False]], # 23 (P5/32-large)
|
||||||
|
|
||||||
|
[[17, 20, 23], 1, Detect, [nc, anchors]], # Detect(P3, P4, P5)
|
||||||
|
]
|
||||||
|
```
|
||||||
|
|
||||||
|
so we can define the freeze list to contain all modules with 'model.0.' - 'model.9.' in their names:
|
||||||
|
```bash
|
||||||
|
python train.py --freeze 10
|
||||||
|
```
|
||||||
|
|
||||||
|
## Freeze All Layers
|
||||||
|
|
||||||
|
To freeze the full model except for the final output convolution layers in Detect(), we set freeze list to contain all modules with 'model.0.' - 'model.23.' in their names:
|
||||||
|
```bash
|
||||||
|
python train.py --freeze 24
|
||||||
|
```
|
||||||
|
|
||||||
|
## Results
|
||||||
|
|
||||||
|
We train YOLOv5m on VOC on both of the above scenarios, along with a default model (no freezing), starting from the official COCO pretrained `--weights yolov5m.pt`:
|
||||||
|
```python
|
||||||
|
train.py --batch 48 --weights yolov5m.pt --data voc.yaml --epochs 50 --cache --img 512 --hyp hyp.finetune.yaml
|
||||||
|
```
|
||||||
|
|
||||||
|
### Accuracy Comparison
|
||||||
|
|
||||||
|
The results show that freezing speeds up training, but reduces final accuracy slightly.
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
|
<img width="922" alt="Screenshot 2020-11-06 at 18 08 13" src="https://user-images.githubusercontent.com/26833433/98394485-22081580-205b-11eb-9e37-1f9869fe91d8.png">
|
||||||
|
|
||||||
|
### GPU Utilization Comparison
|
||||||
|
|
||||||
|
Interestingly, the more modules are frozen the less GPU memory is required to train, and the lower GPU utilization. This indicates that larger models, or models trained at larger --image-size may benefit from freezing in order to train faster.
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
|
## Environments
|
||||||
|
|
||||||
|
YOLOv5 may be run in any of the following up-to-date verified environments (with all dependencies including [CUDA](https://developer.nvidia.com/cuda)/[CUDNN](https://developer.nvidia.com/cudnn), [Python](https://www.python.org/) and [PyTorch](https://pytorch.org/) preinstalled):
|
||||||
|
|
||||||
|
- **Notebooks** with free GPU: <a href="https://bit.ly/yolov5-paperspace-notebook"><img src="https://assets.paperspace.io/img/gradient-badge.svg" alt="Run on Gradient"></a> <a href="https://colab.research.google.com/github/ultralytics/yolov5/blob/master/tutorial.ipynb"><img src="https://colab.research.google.com/assets/colab-badge.svg" alt="Open In Colab"></a> <a href="https://www.kaggle.com/ultralytics/yolov5"><img src="https://kaggle.com/static/images/open-in-kaggle.svg" alt="Open In Kaggle"></a>
|
||||||
|
- **Google Cloud** Deep Learning VM. See [GCP Quickstart Guide](https://github.com/ultralytics/yolov5/wiki/GCP-Quickstart)
|
||||||
|
- **Amazon** Deep Learning AMI. See [AWS Quickstart Guide](https://github.com/ultralytics/yolov5/wiki/AWS-Quickstart)
|
||||||
|
- **Docker Image**. See [Docker Quickstart Guide](https://github.com/ultralytics/yolov5/wiki/Docker-Quickstart) <a href="https://hub.docker.com/r/ultralytics/yolov5"><img src="https://img.shields.io/docker/pulls/ultralytics/yolov5?logo=docker" alt="Docker Pulls"></a>
|
||||||
|
|
||||||
|
|
||||||
|
## Status
|
||||||
|
|
||||||
|
<a href="https://github.com/ultralytics/yolov5/actions/workflows/ci-testing.yml"><img src="https://github.com/ultralytics/yolov5/actions/workflows/ci-testing.yml/badge.svg" alt="YOLOv5 CI"></a>
|
||||||
|
|
||||||
|
If this badge is green, all [YOLOv5 GitHub Actions](https://github.com/ultralytics/yolov5/actions) Continuous Integration (CI) tests are currently passing. CI tests verify correct operation of YOLOv5 [training](https://github.com/ultralytics/yolov5/blob/master/train.py), [validation](https://github.com/ultralytics/yolov5/blob/master/val.py), [inference](https://github.com/ultralytics/yolov5/blob/master/detect.py), [export](https://github.com/ultralytics/yolov5/blob/master/export.py) and [benchmarks](https://github.com/ultralytics/yolov5/blob/master/benchmarks.py) on macOS, Windows, and Ubuntu every 24 hours and on every commit.
|
||||||
@ -0,0 +1,154 @@
|
|||||||
|
# Test-Time Augmentation (TTA)
|
||||||
|
|
||||||
|
📚 This guide explains how to use Test Time Augmentation (TTA) during testing and inference for improved mAP and Recall with YOLOv5 🚀.
|
||||||
|
UPDATED 25 September 2022.
|
||||||
|
|
||||||
|
## Before You Start
|
||||||
|
|
||||||
|
Clone repo and install [requirements.txt](https://github.com/ultralytics/yolov5/blob/master/requirements.txt) in a [**Python>=3.7.0**](https://www.python.org/) environment, including [**PyTorch>=1.7**](https://pytorch.org/get-started/locally/). [Models](https://github.com/ultralytics/yolov5/tree/master/models) and [datasets](https://github.com/ultralytics/yolov5/tree/master/data) download automatically from the latest YOLOv5 [release](https://github.com/ultralytics/yolov5/releases).
|
||||||
|
|
||||||
|
```bash
|
||||||
|
git clone https://github.com/ultralytics/yolov5 # clone
|
||||||
|
cd yolov5
|
||||||
|
pip install -r requirements.txt # install
|
||||||
|
```
|
||||||
|
|
||||||
|
## Test Normally
|
||||||
|
|
||||||
|
Before trying TTA we want to establish a baseline performance to compare to. This command tests YOLOv5x on COCO val2017 at image size 640 pixels. `yolov5x.pt` is the largest and most accurate model available. Other options are `yolov5s.pt`, `yolov5m.pt` and `yolov5l.pt`, or you own checkpoint from training a custom dataset `./weights/best.pt`. For details on all available models please see our README [table](https://github.com/ultralytics/yolov5#pretrained-checkpoints).
|
||||||
|
```bash
|
||||||
|
python val.py --weights yolov5x.pt --data coco.yaml --img 640 --half
|
||||||
|
```
|
||||||
|
|
||||||
|
Output:
|
||||||
|
```shell
|
||||||
|
val: data=./data/coco.yaml, weights=['yolov5x.pt'], batch_size=32, imgsz=640, conf_thres=0.001, iou_thres=0.65, task=val, device=, single_cls=False, augment=False, verbose=False, save_txt=False, save_hybrid=False, save_conf=False, save_json=True, project=runs/val, name=exp, exist_ok=False, half=True
|
||||||
|
YOLOv5 🚀 v5.0-267-g6a3ee7c torch 1.9.0+cu102 CUDA:0 (Tesla P100-PCIE-16GB, 16280.875MB)
|
||||||
|
|
||||||
|
Fusing layers...
|
||||||
|
Model Summary: 476 layers, 87730285 parameters, 0 gradients
|
||||||
|
|
||||||
|
val: Scanning '../datasets/coco/val2017' images and labels...4952 found, 48 missing, 0 empty, 0 corrupted: 100% 5000/5000 [00:01<00:00, 2846.03it/s]
|
||||||
|
val: New cache created: ../datasets/coco/val2017.cache
|
||||||
|
Class Images Labels P R mAP@.5 mAP@.5:.95: 100% 157/157 [02:30<00:00, 1.05it/s]
|
||||||
|
all 5000 36335 0.746 0.626 0.68 0.49
|
||||||
|
Speed: 0.1ms pre-process, 22.4ms inference, 1.4ms NMS per image at shape (32, 3, 640, 640) # <--- baseline speed
|
||||||
|
|
||||||
|
Evaluating pycocotools mAP... saving runs/val/exp/yolov5x_predictions.json...
|
||||||
|
...
|
||||||
|
Average Precision (AP) @[ IoU=0.50:0.95 | area= all | maxDets=100 ] = 0.504 # <--- baseline mAP
|
||||||
|
Average Precision (AP) @[ IoU=0.50 | area= all | maxDets=100 ] = 0.688
|
||||||
|
Average Precision (AP) @[ IoU=0.75 | area= all | maxDets=100 ] = 0.546
|
||||||
|
Average Precision (AP) @[ IoU=0.50:0.95 | area= small | maxDets=100 ] = 0.351
|
||||||
|
Average Precision (AP) @[ IoU=0.50:0.95 | area=medium | maxDets=100 ] = 0.551
|
||||||
|
Average Precision (AP) @[ IoU=0.50:0.95 | area= large | maxDets=100 ] = 0.644
|
||||||
|
Average Recall (AR) @[ IoU=0.50:0.95 | area= all | maxDets= 1 ] = 0.382
|
||||||
|
Average Recall (AR) @[ IoU=0.50:0.95 | area= all | maxDets= 10 ] = 0.628
|
||||||
|
Average Recall (AR) @[ IoU=0.50:0.95 | area= all | maxDets=100 ] = 0.681 # <--- baseline mAR
|
||||||
|
Average Recall (AR) @[ IoU=0.50:0.95 | area= small | maxDets=100 ] = 0.524
|
||||||
|
Average Recall (AR) @[ IoU=0.50:0.95 | area=medium | maxDets=100 ] = 0.735
|
||||||
|
Average Recall (AR) @[ IoU=0.50:0.95 | area= large | maxDets=100 ] = 0.826
|
||||||
|
```
|
||||||
|
|
||||||
|
## Test with TTA
|
||||||
|
Append `--augment` to any existing `val.py` command to enable TTA, and increase the image size by about 30% for improved results. Note that inference with TTA enabled will typically take about 2-3X the time of normal inference as the images are being left-right flipped and processed at 3 different resolutions, with the outputs merged before NMS. Part of the speed decrease is simply due to larger image sizes (832 vs 640), while part is due to the actual TTA operations.
|
||||||
|
```bash
|
||||||
|
python val.py --weights yolov5x.pt --data coco.yaml --img 832 --augment --half
|
||||||
|
```
|
||||||
|
|
||||||
|
Output:
|
||||||
|
```shell
|
||||||
|
val: data=./data/coco.yaml, weights=['yolov5x.pt'], batch_size=32, imgsz=832, conf_thres=0.001, iou_thres=0.6, task=val, device=, single_cls=False, augment=True, verbose=False, save_txt=False, save_hybrid=False, save_conf=False, save_json=True, project=runs/val, name=exp, exist_ok=False, half=True
|
||||||
|
YOLOv5 🚀 v5.0-267-g6a3ee7c torch 1.9.0+cu102 CUDA:0 (Tesla P100-PCIE-16GB, 16280.875MB)
|
||||||
|
|
||||||
|
Fusing layers...
|
||||||
|
/usr/local/lib/python3.7/dist-packages/torch/nn/functional.py:718: UserWarning: Named tensors and all their associated APIs are an experimental feature and subject to change. Please do not use them for anything important until they are released as stable. (Triggered internally at /pytorch/c10/core/TensorImpl.h:1156.)
|
||||||
|
return torch.max_pool2d(input, kernel_size, stride, padding, dilation, ceil_mode)
|
||||||
|
Model Summary: 476 layers, 87730285 parameters, 0 gradients
|
||||||
|
val: Scanning '../datasets/coco/val2017' images and labels...4952 found, 48 missing, 0 empty, 0 corrupted: 100% 5000/5000 [00:01<00:00, 2885.61it/s]
|
||||||
|
val: New cache created: ../datasets/coco/val2017.cache
|
||||||
|
Class Images Labels P R mAP@.5 mAP@.5:.95: 100% 157/157 [07:29<00:00, 2.86s/it]
|
||||||
|
all 5000 36335 0.718 0.656 0.695 0.503
|
||||||
|
Speed: 0.2ms pre-process, 80.6ms inference, 2.7ms NMS per image at shape (32, 3, 832, 832) # <--- TTA speed
|
||||||
|
|
||||||
|
Evaluating pycocotools mAP... saving runs/val/exp2/yolov5x_predictions.json...
|
||||||
|
...
|
||||||
|
Average Precision (AP) @[ IoU=0.50:0.95 | area= all | maxDets=100 ] = 0.516 # <--- TTA mAP
|
||||||
|
Average Precision (AP) @[ IoU=0.50 | area= all | maxDets=100 ] = 0.701
|
||||||
|
Average Precision (AP) @[ IoU=0.75 | area= all | maxDets=100 ] = 0.562
|
||||||
|
Average Precision (AP) @[ IoU=0.50:0.95 | area= small | maxDets=100 ] = 0.361
|
||||||
|
Average Precision (AP) @[ IoU=0.50:0.95 | area=medium | maxDets=100 ] = 0.564
|
||||||
|
Average Precision (AP) @[ IoU=0.50:0.95 | area= large | maxDets=100 ] = 0.656
|
||||||
|
Average Recall (AR) @[ IoU=0.50:0.95 | area= all | maxDets= 1 ] = 0.388
|
||||||
|
Average Recall (AR) @[ IoU=0.50:0.95 | area= all | maxDets= 10 ] = 0.640
|
||||||
|
Average Recall (AR) @[ IoU=0.50:0.95 | area= all | maxDets=100 ] = 0.696 # <--- TTA mAR
|
||||||
|
Average Recall (AR) @[ IoU=0.50:0.95 | area= small | maxDets=100 ] = 0.553
|
||||||
|
Average Recall (AR) @[ IoU=0.50:0.95 | area=medium | maxDets=100 ] = 0.744
|
||||||
|
Average Recall (AR) @[ IoU=0.50:0.95 | area= large | maxDets=100 ] = 0.833
|
||||||
|
```
|
||||||
|
|
||||||
|
## Inference with TTA
|
||||||
|
|
||||||
|
`detect.py` TTA inference operates identically to `val.py` TTA: simply append `--augment` to any existing `detect.py` command:
|
||||||
|
```bash
|
||||||
|
python detect.py --weights yolov5s.pt --img 832 --source data/images --augment
|
||||||
|
```
|
||||||
|
|
||||||
|
Output:
|
||||||
|
```bash
|
||||||
|
detect: weights=['yolov5s.pt'], source=data/images, imgsz=832, conf_thres=0.25, iou_thres=0.45, max_det=1000, device=, view_img=False, save_txt=False, save_conf=False, save_crop=False, nosave=False, classes=None, agnostic_nms=False, augment=True, update=False, project=runs/detect, name=exp, exist_ok=False, line_thickness=3, hide_labels=False, hide_conf=False, half=False
|
||||||
|
YOLOv5 🚀 v5.0-267-g6a3ee7c torch 1.9.0+cu102 CUDA:0 (Tesla P100-PCIE-16GB, 16280.875MB)
|
||||||
|
|
||||||
|
Downloading https://github.com/ultralytics/yolov5/releases/download/v5.0/yolov5s.pt to yolov5s.pt...
|
||||||
|
100% 14.1M/14.1M [00:00<00:00, 81.9MB/s]
|
||||||
|
|
||||||
|
Fusing layers...
|
||||||
|
Model Summary: 224 layers, 7266973 parameters, 0 gradients
|
||||||
|
image 1/2 /content/yolov5/data/images/bus.jpg: 832x640 4 persons, 1 bus, 1 fire hydrant, Done. (0.029s)
|
||||||
|
image 2/2 /content/yolov5/data/images/zidane.jpg: 480x832 3 persons, 3 ties, Done. (0.024s)
|
||||||
|
Results saved to runs/detect/exp
|
||||||
|
Done. (0.156s)
|
||||||
|
```
|
||||||
|
|
||||||
|
<img src="https://user-images.githubusercontent.com/26833433/124491703-dbb6b200-ddb3-11eb-8b57-ed0d58d0d8b4.jpg" width="500">
|
||||||
|
|
||||||
|
|
||||||
|
### PyTorch Hub TTA
|
||||||
|
|
||||||
|
TTA is automatically integrated into all [YOLOv5 PyTorch Hub](https://pytorch.org/hub/ultralytics_yolov5) models, and can be accessed by passing `augment=True` at inference time.
|
||||||
|
```python
|
||||||
|
import torch
|
||||||
|
|
||||||
|
# Model
|
||||||
|
model = torch.hub.load('ultralytics/yolov5', 'yolov5s') # or yolov5m, yolov5x, custom
|
||||||
|
|
||||||
|
# Images
|
||||||
|
img = 'https://ultralytics.com/images/zidane.jpg' # or file, PIL, OpenCV, numpy, multiple
|
||||||
|
|
||||||
|
# Inference
|
||||||
|
results = model(img, augment=True) # <--- TTA inference
|
||||||
|
|
||||||
|
# Results
|
||||||
|
results.print() # or .show(), .save(), .crop(), .pandas(), etc.
|
||||||
|
```
|
||||||
|
|
||||||
|
### Customize
|
||||||
|
|
||||||
|
You can customize the TTA ops applied in the YOLOv5 `forward_augment()` method [here](https://github.com/ultralytics/yolov5/blob/8c6f9e15bfc0000d18b976a95b9d7c17d407ec91/models/yolo.py#L125-L137).
|
||||||
|
|
||||||
|
|
||||||
|
## Environments
|
||||||
|
|
||||||
|
YOLOv5 may be run in any of the following up-to-date verified environments (with all dependencies including [CUDA](https://developer.nvidia.com/cuda)/[CUDNN](https://developer.nvidia.com/cudnn), [Python](https://www.python.org/) and [PyTorch](https://pytorch.org/) preinstalled):
|
||||||
|
|
||||||
|
- **Notebooks** with free GPU: <a href="https://bit.ly/yolov5-paperspace-notebook"><img src="https://assets.paperspace.io/img/gradient-badge.svg" alt="Run on Gradient"></a> <a href="https://colab.research.google.com/github/ultralytics/yolov5/blob/master/tutorial.ipynb"><img src="https://colab.research.google.com/assets/colab-badge.svg" alt="Open In Colab"></a> <a href="https://www.kaggle.com/ultralytics/yolov5"><img src="https://kaggle.com/static/images/open-in-kaggle.svg" alt="Open In Kaggle"></a>
|
||||||
|
- **Google Cloud** Deep Learning VM. See [GCP Quickstart Guide](https://github.com/ultralytics/yolov5/wiki/GCP-Quickstart)
|
||||||
|
- **Amazon** Deep Learning AMI. See [AWS Quickstart Guide](https://github.com/ultralytics/yolov5/wiki/AWS-Quickstart)
|
||||||
|
- **Docker Image**. See [Docker Quickstart Guide](https://github.com/ultralytics/yolov5/wiki/Docker-Quickstart) <a href="https://hub.docker.com/r/ultralytics/yolov5"><img src="https://img.shields.io/docker/pulls/ultralytics/yolov5?logo=docker" alt="Docker Pulls"></a>
|
||||||
|
|
||||||
|
|
||||||
|
## Status
|
||||||
|
|
||||||
|
<a href="https://github.com/ultralytics/yolov5/actions/workflows/ci-testing.yml"><img src="https://github.com/ultralytics/yolov5/actions/workflows/ci-testing.yml/badge.svg" alt="YOLOv5 CI"></a>
|
||||||
|
|
||||||
|
If this badge is green, all [YOLOv5 GitHub Actions](https://github.com/ultralytics/yolov5/actions) Continuous Integration (CI) tests are currently passing. CI tests verify correct operation of YOLOv5 [training](https://github.com/ultralytics/yolov5/blob/master/train.py), [validation](https://github.com/ultralytics/yolov5/blob/master/val.py), [inference](https://github.com/ultralytics/yolov5/blob/master/detect.py), [export](https://github.com/ultralytics/yolov5/blob/master/export.py) and [benchmarks](https://github.com/ultralytics/yolov5/blob/master/benchmarks.py) on macOS, Windows, and Ubuntu every 24 hours and on every commit.
|
||||||
Loading…
Reference in new issue