For questions about the concept of (information) entropy in the context of artificial intelligence.
Questions tagged [entropy]
10 questions
3
votes
2 answers
How to calculate the entropy in the ID3 decision tree algorithm?
Here is the definition of the entropy
$$H(S)=-\sum_{x \in X} p(x) \log _{2} p(x)$$
Wikipedia's description of entropy breaks down the formula, but I still don't know how to determine the values of $X$, defined as
The set of classes in $S$
and…
Z. Reticulan
- 131
- 1
- 2
1
vote
1 answer
Why is Soft Q Learning not an Actor Critic method?
I've been reading these two papers from Haarnoja et. al.:
Soft Actor-Critic: Off-Policy Maximum Entropy Deep Reinforcement
Learning with a Stochastic Actor
Reinforcement Learning with Deep Energy-Based Policies
As far as I can tell, Soft…
frances_farmer
- 11
- 1
1
vote
0 answers
Can entropy bonus be used with state-independent log std for stochastic policies?
In this blog article by openai, they say the std of the exploration distribution must be state-dependent, i.e. an output of the policy network, so it works with the entropy bonus, which is an integral part of the SAC algorithm.
My question is: Does…
flxh
- 131
- 2
1
vote
0 answers
Why exclude the first entropy bonus term in the soft Q-function in SAC?
Based on OpenAI Spinning Up description of Soft Actor Critic (SAC) the soft Q-function is defined as
and as they say
Q value is changed to include the entropy bonuses from every timestep except the first.
I feel like it should make sense somehow,…
Daniel
- 111
- 2
1
vote
0 answers
How does NN follows law of energy conservation?
Communication requires energy, and using energy requires communication. According to Shannon, the entropy value of a piece of information provides an absolute limit on the shortest possible average length of a message without losing information as…
Satyam Kumar
- 67
- 6
0
votes
1 answer
How do I add Entropy to a PPO algorithm?
I learned about adding entropy to RL algorithms through the notes provided in SpinningUp. They explained how entropy is added to the SAC algorithm. Here is my understanding - In entropy regularized RL, one adds an entropy bonus $H$ to the reward…
desert_ranger
- 586
- 3
- 19
0
votes
0 answers
What is the relationship between entropy in thermodynamics and entropy in information theory?
BACKGROUND: In thermodynamics, entropy $S$ is a measure of disorder and is given by
$${\displaystyle S=k_B\log(W)},$$
where $k_B$ is Boltzman's constant and $W$ is the number of microstates.
In information theory, (Shannon) entropy $H$ is a measure…
Snehal Patel
- 912
- 1
- 1
- 25
0
votes
0 answers
Is there any point in continuing the training of an agent when entropy (of output probabilities) is low?
I'm working with a PPO agent with a small, discrete action space (3 possible actions, 1 of which is always masked depending on the state).
Premise 1:
My understanding is that the "entropy" of output probabilities is calculated according to the…
Vladimir Belik
- 342
- 2
- 12
0
votes
1 answer
What does the product of probabilities raised to own powers used for entropy calculation quantify?
Suppose $X$ is a random variable taking $k$ values.
$$Val(X) = \{x_1, x_2, x_3, \cdots, x_k\} $$
Then what is the following expression of $N(X)$ called in literature if exists? What does it signify?
$$ N(X) = \prod \limits_{i = 1}^{k} …
hanugm
- 3,571
- 3
- 18
- 50
0
votes
1 answer
How does high entropy targets relate to less variance of the gradient between training cases?
I've been trying to understand the Distilling the Knowledge in a Neural Network paper by Hinton et al. But I cannot fully understand this:
When the soft targets have high entropy, they provide much more information per training case than hard…
