3

I have a problem in which my input data may have a varying number of channels. Let me explain with an example.

Imagine we have a classification problem in which we wish to identify if certain species are present in wildlife photographs. This can be done via a neural network including maybe some convolutions. For the first layer of the network we could set up a convolutional layer with 3 input channels (one for R, G and B respectively) and this would probably work well enough.

Now imagine that someone comes along with some new data for us and this time they have not only taken regular RGB images but they have used an IR-camera as well. Great, but how do we treat this data, we have one more channel?! One could of course simply add an extra channel and re-train the network but that would mean that our old data (without IR-info) is useless and what if someone comes along with a UV-camera.....

My situation is similar but I will most definitely be dealing with varying numbers of channels and the range can be quite wide (from 5 channels all the way up to maybe 50). Is there a good way of dealing with a situation like this?

nbro
  • 39,006
  • 12
  • 98
  • 176
Ivar Eriksson
  • 131
  • 3
  • 1
    Meta Learning and learning with privileged information might be of help here. –  Mar 03 '21 at 11:54
  • 1
    A naive way will be to 0-pad all the training samples to fit the maximum number of channels. – razvanc92 Mar 03 '21 at 12:11
  • @DuttaA Do you care to expand on that? I've never heard of meta learning and learning with privileged information before. – Ivar Eriksson Mar 03 '21 at 12:44
  • 1
    I don't really want to because you have to adapt it to your case, but a short description would be. ML- Few shot learning i.e you find a suitable set of weights from which you can adapt to a new task very easily with a few examples. LUPI - Here a human expert gives us some prior information about each of the samples which you leverage for a better classification. In your case your prior training can be used as this suitable weight or as expert advice etc etc. In short you have to create an approach yourself. Not really readymade. –  Mar 03 '21 at 13:03
  • @DuttaA Okay, thanks for your help. – Ivar Eriksson Mar 03 '21 at 13:22
  • 1
    Given the enormous range of the number of channels per input, I would suggest treating each channel as a step in a time series. So create some kind of RNN with convolutions throughout that accepts a channel of the input. That way it will be able to deal with varying input sizes and still give valid predictions. – Recessive Mar 04 '21 at 02:32

0 Answers0