0

Classification: binary

Model: CNN (ResNet50V2)

During our research we've had 91x109x91 images (3-dimensional). We've used 2D CNN to train and evaluate our images and make predictions on labelled cases, thus we had to convert 3D to 2D this way (slices (n*x), y, z, #color_channels):

maindata = maindata.reshape(n * 91, 109, 91, 3)

What we did here effectively is merge a number of images (n) with dimension x (in our case 91). So we basically trained our model on slices (2D images) of this 3D image. Thus far everything was good. We've received great results.

However, now we need to create prediction probabilities of n images (3D images) and not n x 91 slices (2D images). Can we take the prediction probabilities we've received (2D) and for every 91 prediction probabilities calculate the mean or is there a better way of interpreting 2D results to 3D images?

Both train (labelled) and test (unlabelled) data are shaped the same way, so the only part missing is interpretation to 3D.

Amadej Šenk
  • 1
  • 2
  • two questions: you say you have images of size 91x109x91 but then you say you reshaped them this way (n * 91, 109, 91, 3), introducing channels, so are your images normal 3d images or 4d? Also it's not clear to me why did you have to reshape to 2d? 2d convolutions don't need reshaping, it's called 2d only cause the kernel move along the x and y axis (2d) but the kernel can be 3d, actually that's the standard procedure when processing images. – Edoardo Guerriero Mar 20 '22 at 11:51
  • We were unable to find Conv3D in Tensorflow library (https://www.tensorflow.org/api_docs/python/tf/keras/applications/resnet). Answer to questions: 1. I have an array of 3D images (4 dimensions), I need to rescale the dimensions to 2D for each image in order to use any of the Conv2D they use (Tensorflow). 2. Keras models only support 4 parameters, that is (#images, x, y, #color channels), not the depth dimension(z axis) that we would require. – Amadej Šenk Mar 20 '22 at 12:05
  • Sorry but I'm still confused cause saying 3D images with 4 dimensions is a contradiction. You talk about depth, is that the extra 4th dimension you have along with the 3D RGB image? Or is the 4th dimension just the batch size? – Edoardo Guerriero Mar 20 '22 at 12:21
  • Moreover if i look at the documentation of [resnet50](https://www.tensorflow.org/api_docs/python/tf/keras/applications/resnet50/ResNet50) they explicitly write the allowed input dimension, either (224, 224, 3) or (3, 224, 224), which follow the standard resnet paper architecture. So again I don't see why is necessary to reshape your input images. Unless they are 4D, in which case I wouldn't advice to use models pretrained on 3D images anyway. – Edoardo Guerriero Mar 20 '22 at 12:22
  • just to be extra clear, what are the 91, 109, 91 numbers you mention? Are they 91=height, 109=width, 91=channels? or 91=height, 109=width, 91=extra dimension + 3=channels (RGB)? – Edoardo Guerriero Mar 20 '22 at 12:28
  • Sorry for late response, 91x109x91 are height, width, depth, in this order. We have 3D models (cubes - MRI scans of the brains) – Amadej Šenk Mar 20 '22 at 18:08

0 Answers0