Erklärung der Yolo-Loss-Funktion

15

Ich versuche die Yolo v2-Verlustfunktion zu verstehen:

\begin{aligned} λ_{c o o r d} \sum_{i = 0}^{S^{2}} \sum_{j = 0}^{B} 1_{i j}^{o b j} [(x_{i} - {\hat{x}}_{i})^{2} + (y_{i} - {\hat{y}}_{i})^{2}] \\ + λ_{c o o r d} \sum_{i = 0}^{S^{2}} \sum_{j = 0}^{B} 1_{i j}^{o b j} [(\sqrt{w_{i}} - \sqrt{{\hat{w}}_{i}})^{2} + (\sqrt{h_{i}} - \sqrt{{\hat{h}}_{i}})^{2}] \\ + \sum_{i = 0}^{S^{2}} \sum_{j = 0}^{B} 1_{i j}^{o b j} (C_{i} - {\hat{C}}_{i})^{2} + λ_{n o o b j} \sum_{i = 0}^{S^{2}} \sum_{j = 0}^{B} 1_{i j}^{n o o b j} (C_{i} - {\hat{C}}_{i})^{2} \\ + \sum_{i = 0}^{S^{2}} 1_{i}^{o b j} \sum_{c \in c l a s s e s} (p_{i} (c) - {\hat{p}}_{i} (c))^{2} \end{aligned}

$\begin{align} &\lambda_{coord} \sum_{i=0}^{S^2}\sum_{j=0}^B \mathbb{1}_{ij}^{obj}[(x_i-\hat{x}_i)^2 + (y_i-\hat{y}_i)^2 ] \\&+ \lambda_{coord} \sum_{i=0}^{S^2}\sum_{j=0}^B \mathbb{1}_{ij}^{obj}[(\sqrt{w_i}-\sqrt{\hat{w}_i})^2 +(\sqrt{h_i}-\sqrt{\hat{h}_i})^2 ]\\ &+ \sum_{i=0}^{S^2}\sum_{j=0}^B \mathbb{1}_{ij}^{obj}(C_i - \hat{C}_i)^2 + \lambda_{noobj}\sum_{i=0}^{S^2}\sum_{j=0}^B \mathbb{1}_{ij}^{noobj}(C_i - \hat{C}_i)^2 \\ &+ \sum_{i=0}^{S^2} \mathbb{1}_{i}^{obj}\sum_{c \in classes}(p_i(c) - \hat{p}_i(c))^2 \\ \end{align}$

Wenn jemand die Funktion detaillieren kann.

— Kamel BOUYACOUB
quelle

5

niemand kann dir ohne kontext helfen ... sag uns wenigstens, von welchem papier das ist.

— Bdeonovic

1

"Ich verstehe nicht" und "Detail die Funktion" sind zu breit. Bitte versuchen Sie, bestimmte Fragen zu identifizieren. Beachten Sie, dass es bereits zahlreiche Fragen zu Yolo gibt , von denen einige Ihnen möglicherweise zumindest einen Teil Ihrer Suche

— liefern

Ich würde meine Antwort hinzufügen, wenn Sie auf das hinweisen

— Aksakal

In diesem Blog hier gibt es eine detaillierte grafische Erklärung von yolo und yolov2. Es beantwortet die Frage nach der Verlustfunktion. Ich finde es sehr nützlich für Anfänger und fortgeschrittene Benutzer.

— MBoaretto

18

Erklärung der verschiedenen Begriffe:

Die 3 Konstanten sind nur Konstanten, um mehr als einen Aspekt der Verlustfunktion zu berücksichtigen. Im Artikel ist am höchsten, um im ersten Term die größere Bedeutung zu haben $\lambda$ $\lambda_{coord}$
Die Vorhersage von YOLO ist ein Vektor : bbox-Vorhersagen für jede Gitterzelle und Klassen-Vorhersage für jede Gitterzelle (wobei die Anzahl der Klassen ist). Die 5 Bbox Ausgänge der Box j der Zelle i sind Koordinaten des TTE Zentrum der Bbox , Höhe , Breite und ein Vertrauensindex $S*S*(B*5+C)$ $B$ $C$ $C$ $x_{ij}$ $y_{ij}$ $h_{ij}$ $w_{ij}$ $C_{ij}$
Ich stelle mir vor, dass die Werte mit einem Hut die echten sind, die vom Etikett gelesen wurden, und die Werte ohne Hut die vorhergesagten sind. Also , was ist der reale Wert von der Bezeichnung für das Konfidenzmaß für jeden bbox ? Dies ist der Schnittpunkt zwischen dem vorhergesagten Begrenzungsrahmen und dem vom Etikett. $\hat{C}_{ij}$
istwenn sich ein Objekt in Zelleundanderer Stelle $\mathbb{1}_{i}^{obj}$ $1$ $i$ $0$
"bedeutet, dass derte Bounding-Box-Prädiktor in Zellefür diese Vorhersage verantwortlich ist". Mit anderen Worten, es ist gleichwenn sich ein Objekt in Zelleund das Vertrauen derten Prädiktoren dieser Zelle unter allen Prädiktoren dieser Zelle am höchsten ist. ist fast gleich, außer dass es den Wert 1 gibt, wenn sich KEINE Objekte in Zelle $\mathbb{1}_{ij}^{obj}$ $j$ $i$ $1$ $i$ $j$ $\mathbb{1}_{ij}^{noobj}$ $i$

Beachten Sie, dass ich verwendet , um zwei Indizes und für jede bbox Prognosen, ist dies nicht der Fall , in dem Artikel ist , weil es immer ein Faktor ist oder es so nicht mehrdeutig Interpretation: das gewählte ist dasjenige, das der höchsten Vertrauensbewertung in dieser Zelle entspricht. $i$ $j$ $\mathbb{1}_{ij}^{obj}$ $\mathbb{1}_{ij}^{noobj}$ $j$

Allgemeinere Erklärung jedes Terms der Summe:

Dieser Begriff bestraft eine schlechte Lokalisierung des Zellzentrums
Dieser Begriff bestraft den Begrenzungsrahmen mit ungenauer Höhe und Breite. Die Quadratwurzel ist vorhanden, sodass Fehler in kleinen Begrenzungsrahmen strafender sind als Fehler in großen Begrenzungsrahmen.
Dieser Ausdruck versucht, die Vertrauensbewertung gleich der IOU zwischen dem Objekt und der Vorhersage zu machen, wenn es ein Objekt gibt
Versucht, eine Vertrauensbewertung nahe erzielen, wenn sich kein Objekt in der Zelle befindet $0$
Dies ist ein einfacher Klassifizierungsverlust (nicht im Artikel erklärt)

— user7573566
quelle

1

Soll der zweite Punkt sein B*(5+C)? Zumindest ist dies bei YOLO v3 der Fall.

— Sachinruk

@sachinruk Dies spiegelt Änderungen im Modell zwischen dem ursprünglichen YOLO und den Versionen 2 und 3 wider.

— David Refaeli

12

\begin{aligned} λ_{c o o r d} \sum_{i = 0}^{S^{2}} \sum_{j = 0}^{B} 1_{i j}^{o b j} [(x_{i} - {\hat{x}}_{i})^{2} + (y_{i} - {\hat{y}}_{i})^{2}] \\ + λ_{c o o r d} \sum_{i = 0}^{S^{2}} \sum_{j = 0}^{B} 1_{i j}^{o b j} [(\sqrt{w_{i}} - \sqrt{{\hat{w}}_{i}})^{2} + (\sqrt{h_{i}} - \sqrt{{\hat{h}}_{i}})^{2}] \\ + \sum_{i = 0}^{S^{2}} \sum_{j = 0}^{B} 1_{i j}^{o b j} (C_{i} - {\hat{C}}_{i})^{2} + λ_{n o o b j} \sum_{i = 0}^{S^{2}} \sum_{j = 0}^{B} 1_{i j}^{n o o b j} (C_{i} - {\hat{C}}_{i})^{2} \\ + \sum_{i = 0}^{S^{2}} 1_{i}^{o b j} \sum_{c \in c l a s s e s} (p_{i} (c) - {\hat{p}}_{i} (c))^{2} \end{aligned}

$\begin{align} &\lambda_{coord} \sum_{i=0}^{S^2}\sum_{j=0}^B \mathbb{1}_{ij}^{obj}[(x_i-\hat{x}_i)^2 + (y_i-\hat{y}_i)^2 ] \\&+ \lambda_{coord} \sum_{i=0}^{S^2}\sum_{j=0}^B \mathbb{1}_{ij}^{obj}[(\sqrt{w_i}-\sqrt{\hat{w}_i})^2 +(\sqrt{h_i}-\sqrt{\hat{h}_i})^2 ]\\ &+ \sum_{i=0}^{S^2}\sum_{j=0}^B \mathbb{1}_{ij}^{obj}(C_i - \hat{C}_i)^2 + \lambda_{noobj}\sum_{i=0}^{S^2}\sum_{j=0}^B \mathbb{1}_{ij}^{noobj}(C_i - \hat{C}_i)^2 \\ &+ \sum_{i=0}^{S^2} \mathbb{1}_{i}^{obj}\sum_{c \in classes}(p_i(c) - \hat{p}_i(c))^2 \\ \end{align}$

Doesn't the YOLOv2 Loss function looks scary? It's not actually! It is one of the boldest, smartest loss function around.

Let's first look at what the network actually predicts.

If we recap, YOLOv2 predicts detections on a 13x13 feature map, so in total, we have 169 maps/cells.

We have 5 anchor boxes. For each anchor box we need Objectness-Confidence Score (whether any object was found?), 4 Coordinates ( $t_x, t_y, t_w,$ and $t_h$ ) for the anchor box, and 20 top classes. This can crudely be seen as 20 coordinates, 5 confidence scores, and 100 class probabilities for all 5 anchor box predictions put together.

We have few things to worry about:

$x_i, y_i$ , which is the location of the centroid of the anchor box
$w_i, h_i$ , which is the width and height of the anchor box
$C_i$ , which is the Objectness, i.e. confidence score of whether there is an object or not, and
$p_i(c)$ , which is the classification loss.
We not only need to train the network to detect an object if there is an object in a cell, we also need to punish the network, it if predicts an object in a cell, when there wasn't any. How do we do this? We use a mask ( $𝟙_{i}^{obj}$ and $𝟙_{i}^{noobj}$ ) for each cell. If originally there was an object $𝟙_{i}^{obj}$ is 1 and other no-object cells are 0. $𝟙_{i}^{noobj}$ is just inverse of $𝟙_{i}^{obj}$ , where it is 1 if there was no object in the cell and 0 if there was.
We need to do this for all 169 cells, and
We need to do this 5 times (for each anchor box).

All losses are mean-squared errors, except classification loss, which uses cross-entropy function.

Now, let's break the code in the image.

We need to compute losses for each Anchor Box (5 in total)
- $\sum_{j=0}^B$ represents this part, where B = 4 (5 - 1, since the index starts from 0)
We need to do this for each of the 13x13 cells where S = 12 (since we start index from 0)
- $\sum_{i=0}^{S^2}$ represents this part.
$𝟙_{ij}^{obj}$ is 1 when there is an object in the cell $i$ , else 0.
$𝟙_{ij}^{noobj}$ is 1 when there is no object in the cell $i$ , else 0.
$𝟙_{i}^{obj}$ is 1 when there is a particular class is predicted, else 0.
λs are constants. λ is highest for coordinates in order to focus more on detection (remember, in YOLOv2, we first train it for recognition and then for detection, penalizing heavily for recognition is waste of time, rather we focus on getting best bounding boxes!)
We can also notice that $w_i, h_i$ are under square-root. This is done to penalize the smaller bounding boxes as we need better prediction on smaller objects than on bigger objects (author's call). Check out the table below and observe how the smaller values are punished more if we follow "square-root" method (look at the inflection point when we have 0.3 and 0.2 as the input values) (PS: I have kept the ratio of var1 and var2 same just for explanation):

var1 | var2 | (var1 - var2)^2 | (sqrtvar1 - sqrtvar2)^2

0.0300 | 0.020 | 9.99e-05 | 0.001

0.0330 | 0.022 | 0.00012 | 0.0011

0.0693 | 0.046 | 0.000533 | 0.00233

0.2148 | 0.143 | 0.00512 | 0.00723

0.3030 | 0.202 | 0.01 | 0.01

0.8808 | 0.587 | 0.0862 | 0.0296

4.4920 | 2.994 | 2.2421 | 0.1512

Not that scary, right!

Read HERE for further details.

— RShravan
quelle

1

Should i and j in \sum start from 1 instead of 0?

— webbertiger

1

Yes, that's correct webertiger, have updated the answer accordingly. Thanks!

— RShravan

Isnt

1_{i j}^{o b j}

$\mathbb{1}_{ij}^{obj}$ 1 when there is an object in cell i of bounding box j? and not for all j? how do we choose which j to set to one and the rest to zero. i.e. what is the correct scale/ anchor where it is turned on.

— sachinruk

1

I believe S should still be 13 but if the summation starts in 0 it should end in

S^{2} - 1

$S^2 -1$

— Julian

3

@RShravan, you say: "All losses are mean-squared errors, except classification loss, which uses cross-entropy function". Could you explain? In this equation, it looks as MSE also. Thanks in advance

— Julian

3

Your loss function is for YOLO v1 and not YOLO v2. I was also confused with the difference in the two loss functions and seems like many people are: https://groups.google.com/forum/#!topic/darknet/TJ4dN9R4iJk

YOLOv2 paper explains the difference in architecture from YOLOv1 as follows:

We remove the fully connected layers from YOLO(v1) and use anchor boxes to predict bounding boxes... When we move to anchor boxes we also decouple the class prediction mechanism from the spatial location and instead predict class and objectness for every anchorbox.

This means that the confidence probability $p_i(c)$ above should depend not only on $i$ and $c$ but also an anchor box index, say $j$ . Therefore, the loss needs to be different from above. Unfortunately, YOLOv2 paper does not explicitly state its loss function.

I try to make a guess on the loss function of YOLOv2 and discuss it here: https://fairyonice.github.io/Part_4_Object_Detection_with_Yolo_using_VOC_2012_data_loss.html

— FairyOnIce
quelle

1

Hier ist meine Studiennotiz

Verlustfunktion: Quadratischer Fehler

ein. Grund: Einfach zu optimieren b. Problem: (1) Passt nicht perfekt zu unserem Ziel, die durchschnittliche Präzision zu maximieren. (2) In jedem Bild enthalten viele Gitterzellen kein Objekt. Dadurch werden die Konfidenzwerte dieser Zellen in Richtung 0 verschoben, wodurch der Gradient von Zellen, die ein Objekt enthalten, häufig übersteuert wird. c. Lösung: Erhöhen Sie den Verlust durch Vorhersagen von Bounding-Box-Koordinaten und verringern Sie den Verlust durch Konfidenzvorhersagen von Boxen, die keine Objekte enthalten. Wir verwenden zwei Parameter
$λ_{c Ö Ö r d} = 5$ $\lambda_{coord} = 5$ und $\lambda_{noobj}$ = 0.5 d. Sum-squared error also equally weights errors in large boxes and small boxes
Only one bounding box should be responsible for each obejct. We assign one predictor to be responsible for predicting an object based on which prediction has the highest current IOU with the ground truth.

a. Loss from bound box coordinate (x, y) Note that the loss comes from one bounding box from one grid cell. Even if obj not in grid cell as ground truth.

{\begin{cases} λ_{c o o r d} \sum_{i = 0}^{S^{2}} [(x_{i} - {\hat{x}}_{i})^{2} + (y_{i} - \hat{y_{i}})^{2}] & responsible bounding box \\ 0 & other \end{cases}

$\begin{cases} \lambda_{coord} \sum^{S^2}_{i=0} [(x_i - \hat{x}_i)^2 + (y_i - \hat{y_i})^2] &\text{responsible bounding box} \\ 0 &\text{ other} \\ \end {cases}$

b. Loss from width w and height h. Note that the loss comes from one bounding box from one grid cell, even if the object is not in the grid cell as ground truth.

{\begin{cases} λ_{c o o r d} \sum_{i = 0}^{S^{2}} [(\sqrt{w_{i}} - \sqrt{{\hat{w}}_{i}})^{2} + (\sqrt{h_{i}} - \sqrt{{\hat{h}}_{i}})^{2}] & responsible bounding box \\ 0 & other \end{cases}

$\begin {cases} \lambda_{coord} \sum^{S^2}_{i=0} [(\sqrt{w_i} - \sqrt{\hat{w}_i})^2 + (\sqrt{h_i} - \sqrt{\hat{h}_i})^2] &\text{responsible bounding box} \\ 0 &\text{ other} \\ \end {cases}$

c. Loss from the confidence in each bound box. Not that the loss comes from one bounding box from one grid cel, even if the object is not in the grid cell as ground truth.

{\begin{cases} \sum_{i = 0}^{S^{2}} (C_{i} - {\hat{C}}_{i})^{2} & obj in grid cell and responsible bounding box \\ λ_{n o o b j} \sum_{i = 0}^{S^{2}} (C_{i} - {\hat{C}}_{i})^{2} & obj not in grid cell and responsible bounding box \\ 0 & other \end{cases}

$\begin {cases} \sum^{S^2}_{i=0}(C_i - \hat{C}_i)^2 &\text{obj in grid cell and responsible bounding box} \\ \lambda_{noobj} \sum^{S^2}_{i=0}(C_i - \hat{C}_i)^2 &\text{obj not in grid cell and responsible bounding box} \\ 0 &\text{other} \end {cases}$ d. Loss from the class probability of grid cell, only when object is in the grid cell as ground truth.

{\begin{cases} \sum_{i = 0}^{S^{2}} \sum_{c \in c l a s s e s} (p_{i} (c) - {\hat{p}}_{i} (c))^{2} & obj in grid cell \\ 0 & other \end{cases}

$\begin {cases} \sum^{S^2}_{i=0} \sum_{c \in classes} (p_i(c) - \hat{p}_i(c))^2 &\text{obj in grid cell}\\ 0 &\text{other} \\ \end {cases}$

Loss function only penalizes classification if obj is present in the grid cell. It also penalize bounding box coordinate if that box is responsible for the ground box (highest IOU)

— roy
quelle

Question about 'C', in the paper, confidence is the object-or-no object value outputted multiply by the IOU; is that just for test time or is that used for training cost function as well? I thought we just subtract C value from output and label (just like we did with grid values), but that is incorrect?

— moondra

0

The loss formula you wrote is of the original YOLO paper loss, not the v2, or v3 loss.

There are some major differences between versions. I suggest reading the papers, or checking the code implementations. Papers: v2, v3.

Some major differences I noticed:

Class probability is calculated per bounding box (hence output is now S∗S∗B*(5+C) instead of SS(B*5 + C))
Bounding box coordinates now have a different representation
In v3 they use 3 boxes across 3 different "scales"

You can try getting into the nitty-gritty details of the loss, either by looking at the python/keras implementation v2, v3 (look for the function yolo_loss) or directly at the c implementation v3 (look for delta_yolo_box, and delta_yolo_class).

— David Refaeli
quelle