3

Apologies for the lengthy title. My question is about the weight update rule for logistic regression using stochastic gradient descent.

I have just started experimenting on Logistic Regression. I came across two weight update expressions and did not know which one is more accurate and why they are different.

The first Method:

Source: (Book) Artificial Intelligence: A Modern Approach by Norvig, Russell on page 726-727: using the L2 loss function:

enter image description here

enter image description here

where g stands for the logistic function g' stands for g's derivative w stands for weight hw(x) represents the logistic regression hypothesis

The other method:

Source (Paper authored by Charles Elkan): Logistic Regression and Stochastic Gradient Training.

can be found here

enter image description here

DukeZhou
  • 6,237
  • 5
  • 25
  • 53
GYSHIDO
  • 51
  • 4
  • How do you like that AI book? It’s on my reading list but I’ve done a fair amount of machine learning, deep learning, and now some RL, so I didn’t know how much would be new info. Would be interested in your take on it. – Hanzy May 07 '19 at 18:58
  • @Hanzy Hi, well to be honest, my strategy is to search the subject at hand over several books. Thus, I don’t go through a whole book. However, what I likes about it is the clarity and it was used by an AI veteran of https://machinelearningmastery.com. – GYSHIDO May 08 '19 at 07:17

0 Answers0