Questions tagged [contrastive-learning]
8 questions
5
votes
1 answer
What is the difference between the triplet loss and the contrastive loss?
What is the difference between the triplet loss and the contrastive loss?
They look same to me. I don't understand the nuances between the two. I have the following queries:
When to use what?
What are the use cases and advantages or disadvantages…
Exploring
- 223
- 6
- 16
2
votes
2 answers
Why does triplet loss allow to learn a ranking whereas contrastive loss only allows to learn similarity?
I am looking at this lecture, which states (link to exact time):
What the triplet loss allows us in contrast to the contrastive loss is
that we can learn a ranking. So it's not only about similarity, being
closer together or being further apart,…
Gulzar
- 729
- 1
- 8
- 23
1
vote
1 answer
Definition of negatives in NT-Xent loss
I'm trying to understand few details about NT-Xent loss defined in SimCLR paper(link). The loss is defined as
$$\mathcal{l}_{i,j} = -\log\frac{\exp(sim(z_i,z_j)/\tau)}{\sum_{k=1}^{2N}\mathbb{1}_{[k\neq i]} \exp(sim(z_i,z_k)/\tau)}$$
Where $z_i$ and…
James Arten
- 297
- 1
- 8
1
vote
1 answer
Embedding Quality of Transfer Learning model vs Contrastive learning model
I am working on Contrastive learning which is a technique to learn features based on the concept of learning from comparing two or more instances.
The downstream task is a classification problem.
Transfer Learning
Due to limited data, I tried to use…
Raj Rajeshwari Prasad
- 123
- 3
1
vote
1 answer
What is the difference between Mean Teacher and Knowledge Distillation?
I recently read two papers:
BYOL Bootstrap your own latent: A new approach to self-supervised Learning
DINO Emerging Properties in Self-Supervised Vision Transformers.
I am confused about the terms Mean Teacher in BYOL and Knowledge…
Đặng Huy Hoàng
- 11
- 1
1
vote
0 answers
How to use K-means clustering to visualise learnt features of a CNN model?
Recently, I was going through the paper Intriguing Properties of Contrastive Losses. In the paper (section 3.2), the authors try to determine how well the SimCLR framework has allowed the ResNet50 Model to learn good quality/generalised features…
VEDANT JOSHI
- 31
- 1
0
votes
0 answers
Why the positive example is included in the denominator of NT-Xent loss?
I have a little perplexity about the NT-Xent loss employed in self-supervised contrastive learning.
What we are essentially doing is maximizing the similarity of pairs of augmented images while minimzing the similarity with all the other instances…
James Arten
- 297
- 1
- 8
0
votes
0 answers
Explicit representation learning task benefits than simply considering last nerual network layer
Neural networks are ineherently representation learners, so one could simply extract the last layer embedding $\textbf{z} \in \mathbb{R}^d$ of a neural network model and consider it as a representation of raw input data. But in a supervised ML…
James Arten
- 297
- 1
- 8