Conditional Variational Autoencoders (CVAE) and Mixture Density Networks (MDN) are supposed to address this issue. However, these models provide the distribution parameters, e.g., mean and standard deviation, for each given sample, while I need a single underlying distribution that the given data is generated from.
To put it simply, I would like to find the parameters of a normal distribution that estimates $P(Y \mid X)$, given $X$ and $Y$. Let's imagine the given data, $X$, have an $(n, m)$ dimension, where $n$ and $m$ indicate the number of samples, and features, respectively. Using CVAE/MDN I get the parameters with dimension $(n, d)$, where $d$ is the dimension of the parameters. But I am looking for a model to provide parameters with dimensions $(1, d)$.
