1

The following table shows the precision and recall values I obtained for three object detection models.

enter image description here

The goal is to find the best object detection model for that particular data set.

I evaluate the first two models as the following.

  • Model 1 has a high recall and precision values. High precision relates to a low false-positive rate, and high recall relates to a low false-negative rate. High scores for both show that the model is returning accurate results.

  • Model 2 has high precision but low recall. This means it returns very few results, but most of its identified objects are correct.

How can I evaluate the third one?

nbro
  • 39,006
  • 12
  • 98
  • 176
Nilani Algiriyage
  • 155
  • 5

1 Answers1

1

The second model has the same precision, but worse recall, than model 1. Therefore we would rather have model 1 than model 2.

The third model has worse recall than model 1, and worse precision than model 1, therefore we would rather have model 1 than model 3.

Thus, model 1 is the best object detection model.

nbro
  • 39,006
  • 12
  • 98
  • 176
John Doucette
  • 9,147
  • 1
  • 17
  • 52
  • @ John Doucette: Oh! no this is not a home work problem. This is the analysis of my research output. I want to evaluate it in the similar manner as I did for the first and second. Thanks a lot. – Nilani Algiriyage Jan 13 '20 at 23:19
  • @NilaniAlgiriyage The way you analyzed the other two doesn't seem to make a lot of sense. You can draw comparisons (as in my answer), or you can use domain-specific valuations (e.g. this translates to a certain expected cost savings in operation), but concepts like "accurate", "high", and "low" don't have objective meanings in analysis. High precision in spam detection might be 0.9999. High precision in stock market prediction might only be 75%. – John Doucette Jan 13 '20 at 23:39
  • Thanks. My actual problem is this : https://ai.stackexchange.com/questions/17264/accuracy-scores-in-a-deep-learning-project – Nilani Algiriyage Jan 15 '20 at 01:18
  • @NilaniAlgiriyage I think without knowing what the best or worst possible performance is for problems of this kind, it's still not a good idea to make statements about "high" or "low" performance. What we can say definitively is that model 1 has high*er* precision and recall than the other two models, so it is clearly the best choice. Model 3 has higher recall than model 2, but lower precision than model 2. If you care a lot about recall for your application you would prefer 3 to 2. If you care more about precision, you would prefer 2 to 3. – John Doucette Jan 15 '20 at 02:59
  • Thanks a lot for helping me to think in a different way. – Nilani Algiriyage Jan 15 '20 at 08:29