1

Let be a Bayesian multivariate normal distribution classifier with distinct covariance matrices for each class and isotropic, i.e. with equal values over the entire diagonal and zero otherwise, $\mathbf{\Sigma}_i=\sigma_i^2\mathbf{I},~\forall i$.

How can I compute the equation for estimating the parameter $\sigma_{i}$ by the maximum likelihood method? Here $\sigma_{i,j}$ is is the covariance between $x_i$ and $x_j$. So $\sigma_i$ is just the variance of $x_i$.

Attempt:

Suppose $\mathcal{X}_i = \{x^t_i\}^N_{t=1}$ i.i.d, $x_i^t$ is in the class $C_i$ and $x_i^t \sim \mathcal{N}(\mu, \sigma^2)$.

Do I have to find the log-likelihood under $p(x) = \frac{1}{\sqrt{2 \pi} \sigma} \exp \left[ -\frac{(x-\mu)^2}{2 \sigma^2}\right]$, find the derivative and put it equal to $0$ to find the maximum?

EDIT

Suppose my data points are $m$-dimensional, and I have $K$ classes.

David
  • 113
  • 3
  • Comments are not for extended discussion; this conversation has been [moved to chat](https://chat.stackexchange.com/rooms/113411/discussion-on-question-by-david-estimating-sigma-i-according-to-maximum-likel). – nbro Sep 25 '20 at 20:40
  • The problem here is how will you estimate the MLE. It is pretty difficult estimation problem (computationally) and thus has tradeoffs in space vs computational complexity. http://proceedings.mlr.press/v75/hazan18a/hazan18a.pdf A good place to start your problwm. Although I would warn, its a pretty difficult problem requiring a very good theoretical knowledge. –  Sep 25 '20 at 20:55
  • How can your data points be $m$ dimensional if they follow a univariate normal distribution? – David Sep 25 '20 at 21:52

0 Answers0