1

I am working on a classification algorithm for brain rhythms. However, when I implemented the metrics for precision, accuracy, F1 score and recall. My results show that my algorithm has a high precision but a low recall.

I am not expert on this kind of metrics and analysis and I don't know if it makes sense to have a high precision but a low recall. What does it mean?

This is my reference and output models.

enter image description here

                "A": {
                    "FN": 5,
                    "FP": 0,
                    "Jaccard Index": 0.5454545454545454,
                    "TP": 6,
                    "f1-score": 0.7058823529411764,
                    "precision": 1.0,
                    "recall": 0.5454545454545454
                },
                "B": {
                    "FN": 34,
                    "FP": 5,
                    "Jaccard Index": 0.38095238095238093,
                    "TP": 24,
                    "f1-score": 0.5517241379310345,
                    "precision": 0.8275862068965517,
                    "recall": 0.41379310344827586
                },
                "C": {
                    "FN": 39,
                    "FP": 9,
                    "Jaccard Index": 0.36,
                    "TP": 27,
                    "f1-score": 0.5294117647058824,
                    "precision": 0.75,
                    "recall": 0.4090909090909091
                },
                "SNR": 28.121645860790924
nbro
  • 39,006
  • 12
  • 98
  • 176
GGChe
  • 23
  • 5

1 Answers1

3

Recall relates to your ability to detect the positive cases. Since you have low recall, you are missing many of those cases.

Precision relates to the credibility of a claim that a case is positive. Since you score high here, when the model flags a case as positive, you should believe it.

Combined, your model misses many of the positive cases. However, when it does flag a case as positive, the case is likely to be a positive case. Consider your model a skeptic: it is unlikely to believe a case to be positive, but when it does, it must have been because of overwhelming evidence that is worth believing.

If you know the story of the boy who cried wolf, your model does not cry wolf very often when there is no wolf, but it does miss instances of a wolf on the loose.

Dave
  • 538
  • 2
  • 9
  • Note, however, the [issues](https://stats.stackexchange.com/q/603663/247274) with the metrics you apply. – Dave Feb 03 '23 at 12:27
  • Wow, that's a really good explanation. Thanks for the answer, truly. I guess know my question should be focus on how I want my model to be. Since I am capturing seizures in real time, I might balance between recall and precision. However, I think it is my part know to rethink my algorithm to fit my goal scenario – GGChe Feb 04 '23 at 19:12
  • Thanks a lot for the answer – GGChe Feb 04 '23 at 19:12
  • @GGChe [This](https://stats.stackexchange.com/questions/604405/why-does-positive-predictive-value-depend-on-sensitivity) might be worth a read. Note that PPV and precision are synonymous. – Dave Feb 08 '23 at 03:31