Artificial Intelligence Stack Exchange
Artificial Intelligence Stack Exchange

Questions tagged [anomaly-detection]

For questions related to anomaly detection (or outlier detection) algorithms, which is the identification of rare items, events or observations which raise suspicions by differing significantly from the majority of the data. There are unsupervised, supervised and semi-supervised anomaly detection algorithms.

31 questions
8
votes
1 answer

Which unsupervised learning technique can be used for anomaly detection in a time series?

I've started working on anomaly detection in Python. My dataset is a time series one. The data is being collected by some sensors which record and collect data on semiconductor-making machines. My dataset looks like this: ContextID Time_ms…
some_programmer
  • 225
  • 1
  • 4
5
votes
1 answer

Find anomalies from records of categorical data

I have a data-set with $m$ observations and $p$ categorical variables (nominal), each variable $X_1, X_2,\dots, X_p$ has several different possible values. Ultimately, I am looking for a way to find anomalies i.e. to identify rows for which the…
bat
  • 51
  • 1
4
votes
1 answer

What is the difference between out of distribution detection and anomaly detection?

I'm currently reading the paper Likelihood Ratios for Out-of-Distribution Detection, and it seems that their problem is very similar to the problem of anomaly detection. More precisely, given a neural network trained on a dataset consisting of…
mhdadk
  • 233
  • 2
  • 7
4
votes
1 answer

Which unsupervised learning algorithm can be used for peaks detection?

So, I have a dataset that has around 1388 unique products and I have to do unsupervised learning on them in order to find anomalies (high/low peaks). The data below just represents one product. The ContextID is the product number, and the StepID…
some_programmer
  • 225
  • 1
  • 4
3
votes
0 answers

Defect Detection System using Deep Learning

What is the general approach to defect detection in deep learning? Would the approach be better if we try to learn the positive images (defects in images) as much as possible or we try to learn the negative images (images without blemishes) and try…
user1538798
  • 131
  • 1
3
votes
1 answer

How can auto-encoders compute the reconstruction error for the new data?

Autoencoders are used for unsupervised anomaly detection by first learning the features of the data set with mainly "normal" data points. Then new data can be considered anomalous if the new data has a large reconstruction error, i.e. it was hard to…
Brian
  • 131
  • 2
2
votes
1 answer

How to calculate a meaningful distance between multidimensional tensors

TLDR: given two tensors $t_1$ and $t_2$, both with shape $(c,h,w),$ how shall the distance between them be measured? More Info: I'm working on a project in which I'm trying to distinguish between an anomalous sample (specifically from MNIST) and a…
Hadar Sharvit
  • 371
  • 1
  • 12
2
votes
0 answers

Application of Blockchain in Fraud detection in stock market

I want to develop a fraud detection application in the stock market Using Blockchain technology, we have some pattern that defines the anomaly for use of supervised machine learning but there is one question remain: What is role of machine learning…
R1-
  • 137
  • 1
  • 9
2
votes
0 answers

How to perform unsupervised anomaly detection from log file with mostly textual data?

I have a log file of the format, Index, Date, Timestamp, Module, App, Context, Session, Verbosity level, Description The log file can be considered as a master log, which consists of individual logs from several modules constituting a distributed…
Kraken10
  • 21
  • 3
2
votes
2 answers

Which unsupervised anomaly detection algorithms are there?

I need to create model which will find suspicious entries or anomalies in a network, whose characteristics or features are the asset_id, user_id, IP accessed from and time_stamp. Which unsupervised anomaly detection algorithms or models should I use…
Abishek
  • 33
  • 6
1
vote
2 answers

How to detect outlier images?

Before I describe my challenge, I want to point out that I have searched extensively online for "outlier image detection", "anomaly images detection", etc., but all returned results are about finding anomalies or outliers within an image (e.g., a…
pookie
  • 121
  • 2
1
vote
0 answers

Pseudo Label Generation for Generative Cooperative Learning

I am trying to implement this paper for unsupervised video anomaly detection. The gist of the paper seems to be: Create a dataset for an unsupervised setting, by mixing up the train and anomalous videos (section 4) Divide each video into $p(=16)$…
satan 29
  • 141
  • 3
1
vote
1 answer

ML model to predict timeouts

I am new to ML and am trying to build a model to predict timeouts for a website. The website is being monitored once a minute and the data consists of a timestamp and the response time in seconds. E.g.: 1650539220000 6.234041.In this particular case…
gwolter
  • 11
  • 2
1
vote
0 answers

What is the best clustering method to detect anomalies for data with mostly categorical data?

I have a dataset with about 85 columns. Out of the 85 columns, 70+ are categorical. My goal is to identify the outliers in this dataset through clustering methods as I do not have a target column. What is the best way to approach this? Is it…
user13074756
  • 111
  • 1
1
vote
0 answers

Object Detection as a means of Anomaly Detection

Is it possible to train an Object Detector (e.g. SSD), to detect when something is not in the image. Imagine an assembly line that transports some objects. Each object needs to have 5 screws. If the Object Detector detects 4 screws, we know that one…
oezguensi
  • 205
  • 1
  • 8
1
2 3