For questions related to the concept of generative machine learning models, such as the Restricted Boltzmann Machine (RBM), the Variational Autoencoder (VAE), the Generative Adversarial Network (GAN), Large Language Models (LLMs), and Diffusion Models.
Questions tagged [generative-model]
147 questions
20
votes
3 answers
How can we process the data from both the true distribution and the generator?
I'm struggling to understand the GAN loss function as provided in Understanding Generative Adversarial Networks (a blog post written by Daniel Seita).
In the standard cross-entropy loss, we have an output that has been run through a sigmoid function…
tryingtolearn
- 385
- 1
- 2
- 10
11
votes
1 answer
What is the meaning of $V(D,G)$ in the GAN objective function?
Here is the GAN objective function.
$$\min _{G} \max _{D} V(D, G)=\mathbb{E}_{\boldsymbol{x} \sim p_{\text {data }}(\boldsymbol{x})}[\log D(\boldsymbol{x})]+\mathbb{E}_{\boldsymbol{z} \sim p_{\boldsymbol{z}}(\boldsymbol{z})}[\log…
i_rezic
- 245
- 1
- 6
9
votes
3 answers
Why is the variational auto-encoder's output blurred, while GANs output is crisp and has sharp edges?
I observed in several papers that the variational autoencoder's output is blurred, while GANs output is crisp and has sharp edges.
Can someone please give some intuition why that is the case? I did think a lot but couldn't find any logic.
Trect
- 269
- 1
- 4
- 7
9
votes
2 answers
Is plain autoencoder a generative model?
I am wondering how a plain auto encoder is a generative model though its version might be but how can a plain auto encoder can be generative. I know that Vaes which is a version of the autoencoder is generative as it generates distribution for…
Nervous Hero
- 145
- 4
7
votes
2 answers
What are the best machine learning models for music composition?
What are the best machine learning models that have been used to compose music? Are there some good research papers (or books) on this topic out there?
I would say, if I use a neural network, I would opt for a recurrent one, because it needs to have…
Ben
- 425
- 3
- 10
6
votes
1 answer
Are deep learning models suitable for training with sparse data?
I am training a generative adversarial network (GAN) to generate images given edge histogram descriptor (EHD) features of the image. The EHD features are themselves sparse (meaning they contain a lot of zeroes). While training the generator loss and…
varsh
- 562
- 7
- 19
6
votes
2 answers
What is the exact role of model $p_\theta$ in diffusion models for the reverse process?
I'm reading this interesting blog post explaining diffusion probabilistic models and trying to understand the following.
In order to compute the reverse process, we need to consider the posterior distribution $q(\textbf{x}_{t-1} | \textbf{x}_t)$…
James Arten
- 297
- 1
- 8
6
votes
0 answers
Are generative models actually used in practice for industrial drug design?
I just finished reading this paper MoFlow: An Invertible Flow Model for Generating Molecular Graphs.
The paper, which is about generating molecular graphs with certain chemical properties improved the SOTA at the time of writing by a bit and used a…
Adriaan
- 61
- 2
5
votes
1 answer
Does MMD-VAE solve the problem of blurred images of vanilla VAEs?
I understand that with vanilla VAEs, there are a few reasons justifying the production of blurred out images. The InfoVAE paper describes the case when the decoder is flexible enough to ignore the latent attributes and generate an averaged out image…
Ananda
- 148
- 9
5
votes
1 answer
Context-based gap-fill face posture-mapper GAN
These images are handmade, not auto-generated like they will be in production. Apologies for inaccuracies in the graph overlay.
I am trying to build an AI like that displayed in the diagram: when given a training set of images with their…
Geza Kerecsenyi
- 51
- 6
4
votes
1 answer
Looking to build, compile, and/or find dataset for serial-parallelized code examples
I'm looking to perform two tasks:
Train a classifier to classify code as serial or parallel
Train a generative algorithm to generate parallel code from serial
For the first task a simple scraper can scrape random C and C++ code from git, however…
JMed
- 76
- 3
4
votes
2 answers
How to generate new data given a trained VAE - sample from the learned latent space or from multivariate Gaussian?
To generate synthetic dataset using a trained VAE, there is confusion between two approaches:
Use learned latent space: z = mu + (eps * log_var) to generate (theoretically, infinite amounts of) data. Here, we are learning mu and log_var vectors…
Arun
- 225
- 1
- 8
4
votes
1 answer
What is the input for the prior model of VQ-VAE?
I'm trying to implement the VQ-VAE model. In there, a continuous variable $x$ is encoded in an array $z$ of discrete latent variables $z_i$ that are mapped each to an embedding vector $e_i$. These vectors can be used to generate an $\hat{x}$ that…
Diego Gomez
- 395
- 4
- 14
4
votes
1 answer
What parameters can be tweaked to avoid a generator or discriminator loss collapsing to zero when training a DC-GAN?
Sometimes when I am training a DC-GAN on an image dataset, similar to the DC-GAN PyTorch example (https://pytorch.org/tutorials/beginner/dcgan_faces_tutorial.html), either the Generator or Discriminator will get stuck in a large value while the…
jman4162
- 41
- 4
3
votes
1 answer
Why is the last layer of a DBN or DBM used for classification task?
I understand why deep generative models like DBN ( deep belief nets ) or DBM ( deep boltzmann machines ) are able to capture underlying structures in data and use it for various tasks ( classification, regression, multimodal representations etc…
Ravid
- 31
- 3