3

Autoencoders are used for unsupervised anomaly detection by first learning the features of the data set with mainly "normal" data points. Then new data can be considered anomalous if the new data has a large reconstruction error, i.e. it was hard to fit the features as in the normal data.

Even if the training is supervised by learning to reconstruct the same data, how is the reconstruction error computed for the new data?

nbro
  • 39,006
  • 12
  • 98
  • 176
Brian
  • 131
  • 2
  • Hi! Was my answer unclear or were you looking for something else and I answered something else? –  Jun 05 '19 at 06:09
  • 1
    Hi, thanks for the detailed answer, I appreciated it. It guided me towards the right direction. However, it was not directly answering my question. When you say, the AE expects specific features to be of a specific magnitude compared to other features, I feel like that is misunderstanding a bit. The AE itself does not reason about the magnitude. I think the point I was missing is that the recon. error is actually just a value for how different the output is to the (ideally identical) input. If the magnitude of features is out of order as you describe, the recon. error most probably increases. – Brian Jun 06 '19 at 07:11

1 Answers1

1

It is computed just like in training. You take an MSE or something along these lines between the input and the output. You set a threshold for it. If new data's reconstruction error is higher than your threshold, then it is anomalous otherwise it isn't.

Abhishek Verma
  • 858
  • 3
  • 6