0

I'm working on a computer vision application in Python to analyze images of ice cream cuttings to measure the amount of variegate(ie. fruit syrup or fudge) compared to the base ice cream. My approach is to crop the ice cream, generate a mask including only the cropped image (removing any additional background after cropping), transform the image, and finally count black pixels (variegate) compared to white pixels(ice cream base). The results can be seen in the first picture

I'm using the rembg package and remove function with the code seen below for my cropping:

from rembg import remove

image_crop = rembg.remove(image)

image_crop_arr = np.array(image_crop, dtype=np.uint8)

Input image, Cropped image, Mask, Transformed image1

The problem is that the remove function is including the ice cream container(see the picture above), and if the photo is not taken vertically a fair bit of the container is included. This throws off my calculations and color scale for transforming.

What is the best approach to remove this additional container from the cropped image?

  1. Train a custom CNN to generate masks (sounds like a lot of work)?
  2. Increase the edge factor to remove say 10 extra pixels from the cropped line?
  3. Modify the remove function and its U2 training?
  4. Are there other simple approaches I'm not considering

I tried drawing a rectangular mask around the center of the image to capture the ice cream and remove the container, but the ice cream is not rectangular, introducing an error.

I annotated 30 or so images with the idea of making a custom mask to train an image cropping algorithm to better select the shape of the ice cream cutting from the background and ice cream container, which is seen in the image below. Is this a good approach to take for training a custom model? If so how many more annotated photos do you think a CNN or other model would required? I can only buy and eat so much ice cream. :) Annotated image, red line is ice cream seagment

Thank you for any feedback!!!

nbro
  • 39,006
  • 12
  • 98
  • 176
RustyGoat
  • 1
  • 2

0 Answers0