Artificial Intelligence Stack Exchange
Artificial Intelligence Stack Exchange

Questions tagged [diffusion-models]

43 questions
10
votes
2 answers

Using AI to extend an imagine pattern

I have created some nice patterns using the MidJourney tool. I'd like to find a way to extend these patterns, and I was thinking about an AI tool that takes one of these patterns and extends it in all directions surrounding the original…
Nicola Lepetit
  • 203
  • 1
  • 8
3
votes
0 answers

Relation between SDE diffusion and DDPM/DDIM

Background & Definitions In DDPM, the diffusion backward step is described as follows (where $z\sim \mathcal{N}(0,I)$ and $x_{T}\sim \mathcal{N}(0,I)$): and in DDIM we have while in the SDE formulation (from the Fokker-Planck equation) the step…
snatchysquid
  • 129
  • 5
3
votes
1 answer

What's the architecture that allows the generation of new images based on input image in tools like Midjourney?

I understand that the high-level architecture of tools like Midjourney use diffusion models to generate images from text. What I don't understand is which type of network architecture allow the second step of their workflow - generating new, similar…
emilaz
  • 133
  • 4
2
votes
2 answers

About cosine noise schedule in Diffusion Model

Could you provide the explanation of Figure 4 from the paper at Improved Denoising Diffusion Probabilistic Models? (1) The paper says, "the end of the forward noising process is too noisy,and so doesn’t contribute very much to sample quality". But…
diffusion stable
  • 41
  • 6
2
votes
1 answer

Clarification on the training objective of denoising diffusion models

I'm reading the Denoising Diffusion Probabilistic Models paper (Ho et al. 2020). And I am puzzled about the training objective. I understood (I think) the trick regarding the reparametrization of the variance in terms of the noise: $$\mu_\theta(x_t,…
user3903647
  • 21
  • 1
2
votes
0 answers

Reverse Process in Diffusion Model Doesn't Return Original Image

I am attempting to program a Denoising Diffusion Model based on the one introduced in the article by Ho et al. (2020). However, I have run into issues while testing the reverse diffusion process. Walking through my PyTorch code, I first load an…
cabralpinto
  • 21
  • 4
2
votes
3 answers

Forward Diffusion Process Derivation In Diffusion Models

In papers and other material regarding diffusion models the forward diffusion process is defined by adding a small amount of Gaussian noise to an image $x_0$ for $T$ time steps. In each time step the noise has a variance of $\beta_t$. This process…
nkam
  • 21
  • 1
1
vote
1 answer

In-depth understanding of formulation and guidance mechanisms in Diffusion models

I've been reading a research paper titled High-Resolution Image Synthesis with Latent Diffusion Models by Robin Rombach et al. and came across an a concept related to diffusion models (DMs). In the abstract, the authors state: By decomposing the…
hanugm
  • 3,571
  • 3
  • 18
  • 50
1
vote
1 answer

Modern graduate-level machine learning books with focus on generative models

I'm looking for a modern machine learning book with graduate-level treatment of more recent topics such as diffusion and generative models, transformers etc. I have a hard copy of Deep Learning by Goodfellow and Bengio; while I liked the book and…
user74376
  • 111
  • 2
1
vote
0 answers

Samples from a reverse diffusion process with cosine noise schedule blow up

I have implemented a diffusion probabilistic model, and I am finding some of the model behavior unexpected. When I draw samples from an untrained reverse diffusion process with 20 denoising steps using a cosine noise schedule, I find that the…
Jabby
  • 11
  • 2
1
vote
0 answers

Latent Diffusion Model Can't Learn the Latent Space of a VAE for the MNIST-Fashion Dataset

I'm currently playing around with LDMs on the MNIST-Fashion dataset. I thought the VQVAEs used in the original paper were a bit overkill for what I'm doing (and I don't fully understand how they construct the discretized codebook latent space), so I…
sb3
  • 137
  • 7
1
vote
2 answers

Reverse Distribution in Denoising Diffusion Models is Simple

In explanations of denoising diffusion models it is stated that $q(x_{t-1}|x_t)$ is intractable. This is often justified via Bayes' rule, i.e. $$ q(x_{t-1}|x_t) \propto q(x_t|x_{t-1})q(x_{t-1}) $$ and the marginal $q(x_{t-1})$ is unknown. But I'm…
Patrick Johnstone
  • 21
  • 3
1
vote
0 answers

Is it possible to combine DDPM with GAN?

From what I understand in GAN, the main idea is that you have a generator and a discriminator network that are "competing" with each other. The generator trying to make images that the discriminator is not able to distinguish from real images. Since…
Nikita Belooussov
  • 123
  • 3
1
vote
1 answer

How is the variance for a diffusion kernel derived for a diffusion model?

So I'm watching this video tutorial from CVPR this year on diffusion models, and I am confused by the variance term in the distribution on the left on the video. I understand that in the forward process, we can track intermediate…
Cynthia Kim
  • 11
  • 2
0
votes
0 answers

Where to find all the documentation regarding StableDiffusion's special prompts (like mixing two persons)

I am currently playing on ClipDrop.co with Stable Diffusion, and I am confused with the tutorials I can find online. For instance, for mixing two celebrities in one pics, I read all the following syntaxes: [Person A:Person B:0.5], where 0.5 is the…
Myoch
  • 101
1
2 3