Artificial Intelligence Stack Exchange
Artificial Intelligence Stack Exchange

Questions tagged [embeddings]

For questions about embeddings (not necessarily just word embeddings, for which there is a specific tag) in the context of machine learning.

79 questions
36
votes
5 answers

What is the difference between latent and embedding spaces?

In general, the word "latent" means "hidden" and "to embed" means "to incorporate". In machine learning, the expressions "hidden (or latent) space" and "embedding space" occur in several contexts. More specifically, an embedding can refer to a…
nbro
  • 39,006
  • 12
  • 98
  • 176
10
votes
2 answers

Why are embeddings added, not concatenated?

Let's consider the following example from BERT I cannot understand why "the input embeddings are the sum of the token embeddings, the segmentation embeddings, and the position embeddings". The thing is, these embeddings carry different types of…
nalzok
  • 251
  • 2
  • 8
7
votes
3 answers

How to determine the embedding size?

When we are training a neural network, we are going to determine the embedding size to convert the categorical (in NLP, for instance) or continuous (in computer vision or voice) information to hidden vectors (or embeddings), but I wonder if there…
Lerner Zhang
  • 877
  • 1
  • 7
  • 19
5
votes
2 answers

What is the intuition behind how word embeddings bring information to a neural network?

How is it that a word embedding layer (say word2vec) brings more insights to the neural network compared to a simple one-hot encoded layer? I understand how the word embedding carries some semantic meaning, but it seems that this information would…
silkAdmin
  • 209
  • 1
  • 3
5
votes
2 answers

Is an embedding a representation of a word or its meaning?

What does the term "embedding" actually mean? An embedding is a vector, but is that vector a representation of a word or its meaning? Literature loosely uses the word for both purposes. Which one is actually correct? Or is there anything like: A…
hanugm
  • 3,571
  • 3
  • 18
  • 50
4
votes
2 answers

What is the difference between representation and embedding?

As I searched about this two terms, I found they are somehow like each other, both try to create a vector from raw data as I understood. But, what is the difference of this two term?
aliiiiiiiiiiiiiiiiiiiii
  • 43
  • 3
4
votes
1 answer

How does the embeddings work in vision transformer from paper?

I get the part from the paper where the image is split into P say 16x16 (smaller images) patches and then you have to Flatten the 3-D (16,16,3) patch to pass it into a Linear layer to get what they call "Liner Projection". After passing from the…
Deshwal
  • 253
  • 1
  • 10
3
votes
1 answer

Match two paragraphs of text

I'm building a friend finder app and I need to match people based on a paragraph of text. Here is an example of what I mean: Person A: I love walking and going to the beach, I also love reading and keeping active. I'm very allergic to dogs, so I…
Dom
  • 31
  • 3
3
votes
2 answers

How embeddings learned from one model can be used in another?

In the website the following explanation is provided about Embedding layer: The Embedding layer is initialized with random weights and will learn an embedding for all of the words in the training dataset. It is a flexible layer that can be used in…
Oculu
  • 43
  • 2
3
votes
1 answer

What are knowledge graph embeddings?

What are knowledge graph embeddings? How are they useful? Are there any extensive reviews on the subject to know all the details? Note that I am asking this question just to give a quick overview of the topic and why it might be interesting or…
nbro
  • 39,006
  • 12
  • 98
  • 176
3
votes
1 answer

In the machine learning literature, what does it mean to say that something is "embedded" in some space?

In the machine learning literature, I often see it said that something is "embedded" in some space. For instance, that something is "embedded" in feature space, or that our data are "embedded" in dot product space, etc. However, I've never actually…
The Pointer
  • 527
  • 3
  • 17
2
votes
0 answers

Nearest neighbour search in high dimension retrieves certain points too often

We represent some catalogue items (documents, music tracks, videos, whatever) as vectors embedded in R^d and use them to retrieve nearest neighbours to users query. The typical scenario is that users can input any query and the search results are…
Peter Franek
  • 432
  • 1
  • 4
  • 11
2
votes
1 answer

Sequence Embedding using embedding layer: how does the network architecture influence it?

I want to obtain a dense vector representation of protein sequences so that I can meaningfully represent them in an embedding space. We can consider them as sequences of letters, in particular there are 21 unique symbols which are the amino acids…
HelpNeederStudent
  • 145
  • 8
2
votes
1 answer

Perform clustering on high dimensional data

Recently I trained a BYOL model on a set of images to learn an embedding space where similar vectors are close by. The performance was fantastic when I performed approximate K-nearest neighbours search. Now the next task, where I am facing a problem…
VEDANT JOSHI
  • 31
  • 1
2
votes
0 answers

Is there any way to force one input have more effect on model?

Now I am working on building a deep learning model for a regression problem. I used 50 inputs and try to add one new categorical input. The problem is that this one input is much more important than other inputs. I want to make it more influential…
taegyun kim
  • 21
  • 2
1
2 3 4 5 6