0

In deep learning, models may learn the probability distribution that generated the dataset. Observe the following paragraph from Chapter 5: Machine Learning Basics from the book titled Deep Learning (by Aaron Courville et al.)

Unsupervised learning algorithms experience a dataset containing many features, then learn useful properties of the structure of this dataset. In the context of deep learning, we usually want to learn the entire probability distribution that generated a dataset, whether explicitly, as in density estimation, or implicitly, for tasks like synthesis or denoising. Some other unsupervised learning algorithms perform other roles, like clustering, which consists of dividing the dataset into clusters of similar examples.

I read about density estimation in the same chapter, as given below

In the density estimation problem, the machine learning algorithm is asked to learn a function $p_{model} : R^n \rightarrow R$, where $p_{model}$(x) can be interpreted as a probability density function (if $x$ is continuous) or a probability mass function (if $x$ is discrete) on the space that the examples were drawn from.

This question is focused on explicit probability density estimation in continuous case i.e., learning density function $p_{model}$ directly.

Suppose I have a dataset $D$ with $n$ continuous random variables (features) $X_1, X_2, X_3,\cdots, X_n$. And I don't know anything about the probability density function of individual random variables. That is, I don't know about any information about any $X_i$ , such as, whether $X_i$ follows normal distribution or any other distribution. Then, is it possible to learn density function explicitly? Or do I need to provide some necessary information such as the class of probability distribution function to be learned?

I am thinking as follows:

If I have some information about $X_i$, such as: $X_i$ falls to a well known distribution, then I can learn the parameters of the underlying density function from $D$. So, is it mandatory to know some information about the underlying probability density function.

hanugm
  • 3,571
  • 3
  • 18
  • 50

1 Answers1

0

Neural Networks can approximate any function. Quoting the essence in case the article is removed in the future.

The key to neural networks’ ability to approximate any function is that they incorporate non-linearity into their architecture. Each layer is associated with an activation function that applies a non-linear transformation to the output of that layer.

So no, knowing the class of the probability density function is not required to approximate it via Deep Learning. With a large enough number of samples you could construct an approximation.

tnfru
  • 348
  • 1
  • 12