For questions related to the minimum description length (MDL) principle, which is a formalization of Occam's razor in which the best hypothesis (a model and its parameters) for a given set of data is the one that leads to the best compression of the data. MDL was introduced in the paper "Modeling by shortest data description" by Jorma Rissanen in 1978.
Questions tagged [minimum-description-length]
3 questions
1
vote
0 answers
How does NN follows law of energy conservation?
Communication requires energy, and using energy requires communication. According to Shannon, the entropy value of a piece of information provides an absolute limit on the shortest possible average length of a message without losing information as…
Satyam Kumar
- 67
- 6
1
vote
1 answer
How can a machine learning problem be reduced as a communication problem?
I once heard that the problem of approximating an unknown function can be modeled as a communication problem. How is this possible?
Raphael Augusto
- 187
- 7
1
vote
1 answer
Can feature engineering change the selection of the model according to the minimum description length?
The definition of MDL according to these slides is:
The minimum description length (MDL) criteria in machine learning says that the best description of the data is given by the model which compresses it the best. Put another way, learning a model…
user9947