5

In a comment to this question user nbro comments:

As a side note, "perceptrons" and "neural networks" may not be the same thing. People usually use the term perceptron to refer to a very simple neural network that has no hidden layer. Maybe you meant the term "multi-layer perceptron" (MLP).

As I understand it, a simple neural network with no hidden layer would simply be a linear model with a non-linearity put on top of it. That sounds exactly like a generalized linear model (GLM), with the non-linearity being the GLM's link function.

Is there a notable difference between (non-multi-layer) perceptrons and GLMs? Or is it simply another case of two equivalent methods having different names from different researchers?

R.M.
  • 298
  • 1
  • 6

1 Answers1

3

The perceptron uses the Heaviside step (or sign) function as the activation function (so you are not free to use any activation function), while a GLM is a generalization of linear regression, where the link function can be, for example, the logit (which leads to the logistic regression), identity function (which leads to linear regression), and so on. The sign function has the role in the perceptron as the sigmoid in logistic regression.

So, GLM models have a probabilistic interpretation (i.e. you assume that the response variable follows some distribution), while perceptrons do not, even though, for example, the perceptron and logistic regression can both be used for classification.

nbro
  • 39,006
  • 12
  • 98
  • 176