1

Per page 7 of this MIT lecture notes, the original single-layer Perceptron uses 0-1 loss function.

Wikipedia uses

$${\displaystyle {\frac {1}{s}}\sum _{j=1}^{s}|d_{j}-y_{j}(t)|} \tag{1}$$

to denote the error.

Is the formula (1) the correct form of 0-1 loss function?

nbro
  • 39,006
  • 12
  • 98
  • 176
JJJohn
  • 211
  • 2
  • 9
  • There is no loss function for this particular case since this algorithm iteratively updates each weight separately. The wiki says this formula is used to stop training if it is less than a certain threshold. Take a look at [this](https://ai.stackexchange.com/questions/28235/where-does-the-so-called-loss-loss-function-fit-into-the-idea-of-a-percept/28283#28283) and [this](https://ai.stackexchange.com/questions/28395/an-explanation-involving-the-sign-activation-its-affect-on-the-loss-function-a/28396#28396) answers for more details. – Aray Karjauv Jun 26 '21 at 10:27
  • @ArayKarjauv Thank you. What does "this particular case" refer to? Both sources claim there **is** a loss function. – JJJohn Jun 26 '21 at 11:07
  • This (particular) iterative learning algorithm presented in the provided articles implies a non-differentiable loss function (usually `sign`) that does not quantify the predictions, but only says whether they were correct or not, so you cannot optimize it using gradient descent. Therefore, it is not a loss function in the usual sense. There is a loss function if your activation is differentiable. For instance, sigmoid. Then, the loss will be binary cross-entropy. – Aray Karjauv Jun 26 '21 at 11:59
  • I've created a chat room about perceptrons. Feel free to join it: https://chat.stackexchange.com/rooms/126900/perceptron-learning-algorithm – Aray Karjauv Jun 26 '21 at 20:03
  • @ArayKarjauv I don't think that a "loss function" necessarily needs to be differentiable, although this is probably just a matter of convention. Anyway, if you have an answer to this question, feel free to write it below. Comments are meant to be temporary and to provide additional info rather than full answers. – nbro Jun 29 '21 at 15:52

0 Answers0