3

What is the simplest classification problem which cannot be solved by a perceptron (that is a single-layered feed-forward neural network, with no hidden layers and step activation function), but it can be solved by the same network if the activation function is swapped out to a differentiable activation function (e.g. sigmoid, tanh)?

In the first case, the training would be done with the perceptron training rule, in the second case with the delta rule.

Note that regression problems cannot be solved by perceptrons, so I'm interested in classification only.

nbro
  • 39,006
  • 12
  • 98
  • 176
yejaxey
  • 31
  • 2
  • 1
    Anything with a non-step output, perhaps? I thought that a perceptron with a different activation function is still a perceptron. – user253751 Mar 13 '20 at 10:42

1 Answers1

1

Anything that is not linearly separable cant be solved perceptrons, unless you use feature maps on data to map them to a higher dimension in which it is linearly separable.

As a simple, concrete example, perceptron cant learn the XOR function.

This page might help you further.

SpiderRico
  • 960
  • 8
  • 18
  • 1
    Someone had already provided an answer to this question in the past, but it was deleted by the author after I commented saying that they were not addressing this part "_but it can be solved by the same network if the activation function is swapped out to a differentiable activation function (e.g. sigmoid, tanh)?_". Maybe you should address this part too. – nbro Mar 22 '20 at 20:33
  • I am not saying changing the activation function changes or not the set of problems that the perceptron can solve. I am just asking you to explain why. – nbro Mar 22 '20 at 21:15