Artificial Intelligence Stack Exchange
Artificial Intelligence Stack Exchange

Questions tagged [curse-of-dimensionality]

For questions related to the concept of "curse of dimensionality", which refers to the problem of an exponential increase in volume which occurs when adding extra dimensions to the Euclidean (or input) space. In machine learning and statistics, the curse of dimensionality implies that more data is required to achieve statistical significance, as the number of dimensions of the input increases. The expression was introduced by Richard Bellman in 1957.

For more info, see e.g. https://en.wikipedia.org/wiki/Curse_of_dimensionality.

7 questions
12
votes
4 answers

What are the purposes of autoencoders?

Autoencoders are neural networks that learn a compressed representation of the input in order to later reconstruct it, so they can be used for dimensionality reduction. They are composed of an encoder and a decoder (which can be separate neural…
nbro
  • 39,006
  • 12
  • 98
  • 176
2
votes
0 answers

Nearest neighbour search in high dimension retrieves certain points too often

We represent some catalogue items (documents, music tracks, videos, whatever) as vectors embedded in R^d and use them to retrieve nearest neighbours to users query. The typical scenario is that users can input any query and the search results are…
Peter Franek
  • 432
  • 1
  • 4
  • 11
2
votes
1 answer

Why the number of training points to densely cover the space grows exponentially with the dimension?

In this lecture (minute 42), the professor says that the number of training examples we need to densely cover the space of training vectors grows exponentially with the dimension of the space. So we need $4^2=16$ training data points if we're…
Daviiid
  • 563
  • 3
  • 15
1
vote
0 answers

Dimensionality Limitations

I just started learning about AI and have been reading a book called "Foundations of Machine Learning" by Mehryar Mohri so that I can try to create my own. I had a question come up recently: Can I create a machine learning algorithm that can…
Sophia
  • 11
  • 3
1
vote
0 answers

Is it feasible to train a DQN with thousands of input ports?

I designed a DQN architecture for some problem. The problem has a parameter $m$ as the number of clients. In my situation, $m$ is large, $m\in\{100,200,\ldots,1000\}$. For this situation, the number of input ports of the DQN is some few thousand,…
zdm
  • 299
  • 2
  • 8
0
votes
0 answers

Very high dimensional optimization with large budget, requiring high quality solutions

What would be theoretically the best performing optimization algorithm(s) in this case? Very high dimensional problem: 250-500 parameters Goal is to obtain very high quality solutions, not just "good" solutions Parameters form multiple…
Charly Empereur-mot
  • 101
  • 3
0
votes
0 answers

What is the intuition about the fact that a set of images will live on a three dimensional manifold?

The following question refers to a dissertation in Bishop's book (see the attachment) Can someone give me an intuition about the fact that a set of images will live on a three dimensional manifold? I've understood the general context and that each…
tech_1994
  • 1