0

I am developing an ANN from scratch which classifies MNIST digits.

These are the curves I get using only one hidden layer composed of 100 neurons activated by ReLU function. The output's neurons are activated by the softmax function:

curves

Is it correct that training and validation loss are almost identical? Does it mean that my model perfectly fit the data?

tail
  • 147
  • 6
  • Usually it means you did something wrong. Double check everything. – lpounng Nov 03 '22 at 10:38
  • @lpounng What If I do not spot errors? – tail Nov 03 '22 at 10:42
  • If you have implemented from scratch, I would verify that your validation data is being held out during training. It can happen that the two track quite tightly (as these do) but in my experience it is somewhat unusual for them to reflect the same hills and valleys throughout. – David Hoelzer Nov 03 '22 at 11:07
  • @DavidHoelzer No bugs spotted... – tail Nov 03 '22 at 15:33
  • @tail try get someone else to look at it, or post more details here so we can help. – lpounng Nov 04 '22 at 01:15

1 Answers1

0

If your accuracy is 50% in train and val, would you say that the model fit the data perfectly?

One guess is that your val and train data are pretty similar. Have you split the data before training the model?

The loss seems quite high. What does your accuracy look like? Have you tried to train for more epochs?

Skobo Do
  • 51
  • 6
  • Accuracy on both data are around 95% at the end of the learning. Yes I have split the data. Yes I have tried to train for more epochs and the losses still seem to be identical. – tail Nov 03 '22 at 11:06
  • 50% accuracy is relative. On a 10 class problem, 50% is much better than on a 2 class problem. Also, loss is entirely relative and mostly meaningless without another loss for the same problem (and same loss function) to compare it to. Here we have only the validation loss for comparison. – David Hoelzer Nov 03 '22 at 11:06
  • @tail Have you tried kfold split? https://www.askpython.com/python/examples/k-fold-cross-validation – Skobo Do Nov 03 '22 at 11:09
  • @DavidHoelzer that's the point. It's relative. It does highly depend on the data chosen for training and validation. I can have easily 100% accuracy on a multiclass classification with arbitrary number of classes when choosing the validation data-set accordingly. – Skobo Do Nov 03 '22 at 11:13
  • @DavidHoelzer "50% accuracy is relative. On a 10 class problem, 50% is much better than on a 2 class problem." That not generally true. It depends on whether your data is balanced or not. For balanced data, it would be true. – Skobo Do Nov 03 '22 at 11:16