How to understand 'losses' in Spacy's custom NER training engine?

Question

From the tid-bits, I understand of neural networks (NN), the Loss function is the difference between predicted output and expected output of the NN. I am following this tutorial, the losses are included at line #81 in the nlp.update() function.

I am getting losses in the range 300-100. How to interpret them? What should be the ideal output of this losses variable? I went through Spacy's documentation, but nothing much is written there about losses. Also, please let me know the links to relevant theories to understand this in general.

Just to add to the accepted answer, the specific value of the loss is meaningless, you just want it to go down during training. — polm23, Jun 24 '21 at 06:09

Brian O'Donnell · Accepted Answer · 2021-01-08T15:01:59.910

A critical goal of training a neural network is to minimize the loss. Loss is not explained for spaCy because it is a general concept for machine learning and deep learning. Loss is not specific to spaCy and although there are some finer details I don't believe that is your inquiry.

In general, to understand loss functions, I recommend the following resources:

Loss and Loss Functions for Training Deep Learning Neural Networks by Jason Brownlee. Jason writes some great articles and tutorials and I highly suggest reading his other articles and books.
Introduction to Loss Functions
Loss Functions in Machine Learning for Beginners by John Kaller

If you like videos watch:

Loss Functions Explained by Siraj Raval. Siraj has many great videos on machine learning.

How to understand 'losses' in Spacy's custom NER training engine?

1 Answers1