Artificial Intelligence Stack Exchange
Artificial Intelligence Stack Exchange

Questions tagged [bayesian-deep-learning]

For questions related to Bayesian deep learning, that is, Bayesian techniques applied to deep learning models (i.e. neural networks).

22 questions
37
votes
6 answers

Why do CNN's sometimes make highly confident mistakes, and how can one combat this problem?

I trained a simple CNN on the MNIST database of handwritten digits to 99% accuracy. I'm feeding in a bunch of handwritten digits, and non-digits from a document. I want the CNN to report errors, so I set a threshold of 90% certainty below which my…
Alexander Soare
  • 1,319
  • 2
  • 11
  • 26
7
votes
1 answer

How should the neural network deal with unexpected inputs?

I recently wrote an application using a deep learning model designed to classify inputs. There are plenty of examples of this using images of irises, cats, and other objects. If I trained a data model to identify and classify different types of…
Jeff
  • 173
  • 5
5
votes
1 answer

What is the intuition behind variational inference for Bayesian neural networks?

I'm trying to understand the concept of Variational Inference for BNNs. My source is this work. The aim is to minimize the divergence between the approx. distribution and the true posterior $$\text{KL}(q_{\theta}(w)||p(w|D) = \int q_{\theta}(w) \…
f_3464gh
  • 99
  • 6
3
votes
0 answers

Why does this formula $\sigma^2 + \frac{1}{T}\sum_{t=1}^Tf^{\hat{W_t}}(x)^Tf^{\hat{W_t}}(x_t)-E(y)^TE(y)$ approximate the variance?

How does: $$\text{Var}(y) \approx \sigma^2 + \frac{1}{T}\sum_{t=1}^Tf^{\hat{W_t}}(x)^Tf^{\hat{W_t}}(x_t)-E(y)^TE(y)$$ approximate variance? I'm currently reading What Uncertainties Do We Need in Bayesian Deep Learning for Computer Vision, and the…
user8714896
  • 717
  • 1
  • 4
  • 21
3
votes
1 answer

Is there any research on models that provide uncertainty estimation?

Is there any research on machine learning models that provide uncertainty estimation? If I train a denoising autoencoder on words and put through a noised word, I'd like it to return a certainty that it is correct given the distribution of data it…
user8714896
  • 717
  • 1
  • 4
  • 21
3
votes
1 answer

How can supervised learning be viewed as a conditional probability of the labels given the inputs?

In the literature and textbooks, one often sees supervised learning expressed as a conditional probability, e.g., $$\rho(\vec{y}|\vec{x},\vec{\theta})$$ where $\vec{\theta}$ denotes a learned set of network parameters, $\vec{x}$ is an arbitrary…
Jammy
  • 33
  • 3
2
votes
0 answers

Today's Practicality of Bayesian Neural Networks

Just having heard lately about BNNs (wow, ANNs and CNNs are clear; now there's a B? What's that? Ahh, Bayesian ;-)) and quickly getting their main idea and focus, that is, weights not being pure static numbers anymore, but instead described by a…
Mathy
  • 121
  • 2
2
votes
0 answers

What are the practical problems where full bayesian treatment is affordable?

Suppose, I have a problem, where there is rather a small number of training samples, and transfer learning from ImageNet or some huge NLP dataset is not relevant for this task. Due to the small number of data, say several hundred samples, the use of…
spiridon_the_sun_rotator
  • 2,454
  • 8
  • 16
2
votes
1 answer

How could Bayesian neural networks be used for transfer learning?

In transfer learning, we use big data from similar tasks to learn the parameters of a neural network, and then fine-tune the neural network on our own task that has little data available for it. Here, we can think of the transfer learning step as…
samsambakster
  • 71
  • 5
2
votes
0 answers

How to add prior information when predicting using deep learning models?

Background I'm building a binary classification model for a pair match problem using CNN, e.g. whether person A1 likes product B1 or not. Model input features are sequence features (text descriptions) of the person and the product. The model…
user3915365
  • 21
  • 2
2
votes
1 answer

Why is neural networks being a deterministic mapping not always considered a good thing?

Why is neural networks being a deterministic mapping not always considered a good thing? So I'm excluding models like VAEs since those aren't entirely deterministic. I keep thinking about this and my conclusion is that often times neural networks…
user8714896
  • 717
  • 1
  • 4
  • 21
2
votes
0 answers

Are bayesian neural networks suited for text (or document) classification?

I've tried to do my research on Bayesian neural networks online, but I find most of them are used for image classification. This is probably due to the nature of Bayesian neural networks, which may be significantly slower than traditional artificial…
Nicole
  • 21
  • 1
1
vote
0 answers

Samples from a reverse diffusion process with cosine noise schedule blow up

I have implemented a diffusion probabilistic model, and I am finding some of the model behavior unexpected. When I draw samples from an untrained reverse diffusion process with 20 denoising steps using a cosine noise schedule, I find that the…
Jabby
  • 11
  • 2
1
vote
0 answers

variational inference but with a weighted loglikelihood

I would like to know if it's correct if I substitute in the ELBO formula a weighted sum of the loglikelihood $$\sum E_{q_{\theta}(w)}[w_i \ln{p(y_i|f^{w}(x_i))}]$$ in place of the traditional sum. My problem is that my dataset comes with the…
Alucard
  • 111
  • 1
1
vote
0 answers

Does MobileNet SSD v2 only capture aleatoric uncertainty (and so not the epistemic one)?

Regarding the MobileNet SSD v2 model, I was wondering to what extend it captures uncertainty of the predictions. There are 2 types of uncertainty, data uncertainty (aleatoric) and model uncertainty (epistemic). The model outputs bounding boxes with…
Baka
  • 11
  • 1
1
2