0

BACKGROUND: I am trying to think of rational approaches to designing deep learning models for image classification. One thought is to quantify the complexity of image datasets and use that to inform model design. By the way, I know that rational model design is much more complex than just quantifying image complexity, but right now, I'm in the brainstorming phase.

In the below examples, I qualitatively describe the complexity of the images as a function of the number of channels, complexity of foreground object, complexity of the background, and the number of classes. Certainly, there can be many other factors, such as image dimensions or bytes/pixel. I think all images below are 28x28-pixels and 8-bits/pixel.

MNIST digits           --> greyscale, simple objects belonging to 10 classes and a white background
MNIST digits corrupted --> greyscale, same as above but with added noise
MNIST fashion          --> greyscale, more complex objects belonging to 10 classes and a white background
CIFAR-10               --> RGB, even more complex objects belonging to 10 classes and complex backgrounds
CIFAR-100              --> RGB, same as above but with 10X more classes

Based on the above one has a subjective sense that image datasets can be ordered as shown below based on complexity. It is reasonable to hypothesize then that the computer vison models should be progressively more complex as well (more neurons, more layers, more parameters, etc.).

MNIST digits > MNIST digits corrupted > MNIST fashion > CIFAR-10 > CIFAR-100

SPECIFIC QUESTION: Are there any existing quantitative measures of image complexity that capture these aspects of image datasets?

PRIOR RESEARCH: Various searches lead me to computational complexity and model complexity, which are not what I'm looking for.

Snehal Patel
  • 912
  • 1
  • 1
  • 25

1 Answers1

1

In the realm of computer vision and machine learning, "task" complexity is more important than image complexity. This is one of the major reasons that research has not formally described any specific measures for image complexity. Fundamentally image complexity is actually defined by task complexity. For example, classifying objects which look very similar to the background is more difficult than objects that distinctly stand out from the background. Additionally model complexity is something dependent on task complexity as well rather than image complexity. You can generally overcome challenges with image complexity by training strategies (say with large variety of samples) even with simple models.

For your specific question, I think the natural approach is to maybe define a basic task and then derive complexity measures that affect the quality of the task. For example, let's say classifying fish images (task), there the image quality metrics can be items like brightness, blurriness of image, amount of bluish tint in the image, etc.

Arun Aniyan
  • 441
  • 2
  • 8