1

In Driven by Compression Progress: A Simple Principle Explains Essential Aspects of Subjective Beauty, Novelty, Surprise, Interestingness, Attention, Curiosity, Creativity, Art, Science, Music, Jokes Jürgen Schmidhuber describes, using the idea of using compression progress on perceived data as an intrinsic reward signal:

[Artists] create action sequences yielding interesting inputs, where interestingness is a measure of learning progress, for example [...] the saved number of bits needed to encode the data.

Has such a (purely generative) system been created before? Say, one that creates 2d images?

nbro
  • 39,006
  • 12
  • 98
  • 176
2080
  • 121
  • 3
  • Hello. Welcome to Artificial Intelligence Stack Exchange. Are you familiar with or heard of (variational) autoencoders? – nbro Dec 20 '21 at 00:24
  • @nbro Although variational autoencoders inherently use compression (and better compression arguably leads to better results in many cases), the loss signal is not directly related to degree of compression, but fidelity of reproduction using a fixed size compression. I think someone could answer using VAEs as an example, but it should be caveated as these don't quite reach Schmidhuber's design. – Neil Slater Dec 20 '21 at 09:28
  • @NeilSlater Well, the KL divergence and cross-entropy (which form the ELBO objective function to train VAEs) can have an information-theoretic perspective. See [my answer](https://ai.stackexchange.com/a/20574/2444) if you are interested in more details. However, I don't know the details of Schmidhuber's design, but this answer also doesn't provide more details. That's why I was asking if the OP was aware of VAEs. If yes, maybe he had in mind something different and could provide more details about Schmidhuber's design. – nbro Dec 20 '21 at 09:33
  • @NeilSlater By the way, if I remember correctly, World Models (co-authored by Schmidhuber) use the VAE. So, maybe that goes more in this direction, as the OP is talking about rewarding and intrinsic rewards. Anyway, I think this question is vague, so it needs more details to be answered appropriately, so I don't plan to give an answer now. – nbro Dec 20 '21 at 09:45

0 Answers0