Artificial Intelligence Stack Exchange
Artificial Intelligence Stack Exchange

Questions tagged [representation-learning]

For questions related to feature learning (also known as representation learning), which is a set of techniques that can learn the features associated with the raw data. It is similar to feature engineering, but, in the case of feature learning, the features are learned and not handcrafted.

For more details, see e.g. https://en.wikipedia.org/wiki/Feature_learning or Representation Learning: A Review and New Perspectives by Yoshua Bengio, Aaron Courville, and Pascal Vincent.

39 questions
95
votes
3 answers

What is self-supervised learning in machine learning?

What is self-supervised learning in machine learning? How is it different from supervised learning?
nbro
  • 39,006
  • 12
  • 98
  • 176
5
votes
2 answers

What is feature embedding in the context of convolutional neural networks?

What are feature embeddings in the context of convolutional neural networks? Is it related to bottleneck features or feature vectors?
Kaustubh
  • 51
  • 1
  • 4
5
votes
2 answers

What are examples of approaches to dimensionality reduction of feature vectors?

Given a pre-trained CNN model, I extract feature vector of images in reference and query dataset with several thousands of elements. I would like to apply some augmentation techniques to reduce the feature vector dimension to speed up cosine…
Farid Alijani
  • 299
  • 3
  • 10
5
votes
2 answers

How to understand the concept of self-supervised learning in AI?

I am new to self-supervised learning and it all seems a little magical at the moment. The only way I can get an intuitive understanding is to assume that, for real-world problems, features are still embedded at a per-object level. For example, to…
user3546025
  • 51
  • 2
4
votes
2 answers

What is the difference between representation and embedding?

As I searched about this two terms, I found they are somehow like each other, both try to create a vector from raw data as I understood. But, what is the difference of this two term?
aliiiiiiiiiiiiiiiiiiiii
  • 43
  • 3
4
votes
2 answers

Is there any proper literature on the types of features that different layers of a deep neural network learn?

Let's consider a deep convolutional network. It seems that there is some consensus on the following notions: 1. Shallow layers tend to recognise more low-level features such as edges and curves. 2. Deeper layers tend to recognise more high-level…
mesllo
  • 141
  • 2
4
votes
1 answer

Where do the feature extraction and representation learning differ?

Feature selection is a process of selecting a subset of features that contribute the most. Feature extraction allows getting new features that are not actually present in the given set of features. Representation learning is the process of learning…
hanugm
  • 3,571
  • 3
  • 18
  • 50
3
votes
1 answer

Why different images of the same person, under some restrictions, are in a 50 dimension manifold?

In this lecture (starting from 1:31:00) the professor says that the set of all images of a person lives in a low dimensional surface (compared the the set of all possible images). And he says that the dimension of that surface is 50 and that they…
Daviiid
  • 563
  • 3
  • 15
3
votes
2 answers

How do we know that the neurons of an artificial neural network start by learning small features?

I'd like to ask you how do we know that neural networks start by learning small, basic features or "parts" of the data and then use them to build up more complex features as we go through the layers. I've heard this a lot and seen it on videos like…
Daviiid
  • 563
  • 3
  • 15
3
votes
1 answer

How to generate labels for self-supervised training?

I've been reading a lot lately about self-supervised learning and I didn't understand very well how to generate the desired label for a given image. Let's say that I have an image classification task, and I have very little labeled data. How can I…
Vesko Vujovic
  • 161
  • 2
  • 8
3
votes
1 answer

Does self-supervised learning require auxiliary tasks?

Self-supervised learning algorithms provide labels automatically. But, it is not clear what else is required for an algorithm to fall under the category "self-supervised": Some say, self-supervised learning algorithms learn on a set of auxiliary…
Make42
  • 153
  • 5
2
votes
0 answers

Does this learning scenario have a name? If so, can someone point me to relevant literature?

I am faced with a problem which I bet was already solved before, but that I had never seen. Perhaps by discussing it abstractly, someone can point me to relevant literature. It goes like this: I have a dataset of images $I_j$ and numerical features…
Alek Fröhlich
  • 121
  • 3
2
votes
1 answer

Order of features learned by DNNs during training?

I'm looking for papers probing into the question of what features get learned when (or equivalently what subproblems get "solved" when) during the training process. For example, a paper showing that a Convnet trained on MNIST learns to distinguish 0…
jon_simon
  • 121
  • 4
2
votes
1 answer

Multi-objective training involving maximization of one loss function and minimization of another

I need my model to predict $s$ from my data $x$. Additionally, I need the model to not use signals in $x$ that are predictive of a separate target $a$. My approach is to transform $x$ into a representation $\Psi(x)$ such that it's good at predicting…
ChargeShivers
  • 121
  • 1
2
votes
1 answer

Does a bigger neural network learn "worse" representations than a small neural network when the amount of data isn't enough?

Assume we have a neural network and we want to train it on a classification problem. The hidden layers of the neural network are kind of feature representations of the input data. If the neural network is big and the amount of data isn't enough for…
realmarv
  • 33
  • 2
1
2 3