1

It sounds like a straight-forward task for DALL-E (and GPT?) to present a painting and ask to repaint it "in the style of Leonardo da Vinci". Like one can present texts and ask to rewrite them in the style of some author. Or even better: to present two paintings and ask to repaint the first in the style of the second. (You may replace "paint" by "draw".)

Can this already be achieved - and how? Or is it possibly in the pipeline?

Hans-Peter Stricker
  • 811
  • 1
  • 8
  • 20

1 Answers1

3

This is quite a specific task and there is a solution: CycleGAN.

Of course, DALL-E, GPT-4 and similar systems can also do it, but if you are looking for the "how?", CycleGAN is specifically designed to solve this task and it is more st to understand.

How does CycleGAN work?

CycleGAN is specifically trained on two domains, e.g. Photos and Van Gogh Paintings. CycleGAN consists of 4 Neural Networks: One that transforms images of the first domain (e.g. photos) into images of the second domain (e.g. Van Gogh-Paintings). A second network does the backward transformation (e.g. Van Gogh --> Photo).

To ensure, that a transformed image still represents the same content, CycleGAN has Cyclic Objectiv that requires an image that is transformed forward and backward (e.g. Photos --> Van Gogh --> Photo) to be close to the original Photo. The same should also hold for backward-forward-transformations.

Two networks serve as discriminators that ensure that a transformed image look like an image of the other domain (e.g. a transformed photo looks like a Van Gogh).

For further readings:

How Good is CyclicGAN:

These Examples are taken from the Paper: Examples of CycleGAN outputs

Broele
  • 551
  • 2
  • 12
  • Hi, thanks for the answer. Could you please include some explanation of how CycleGAN works within the answer itself? We generally expect all answers on Stack Exchange to be self-contained as much as possible, and saying "there's an explanation over there" is much less useful than giving an overview within the answer itself. Thank you! – Mithical Mar 21 '23 at 06:07
  • Hi Mithical, of course, I can include some explaination, but I hesitate a bit to do so (and this is also the reason I did not do it in the first place): The question seems to be mainly about the abilities of AI Systems. Explaining a concrete Network Architecture would be on a much more technical level. To me this feel out of scope of this question. So "A network called CycleGAN" would (here) have been my answer to this question of "How". The links are ment as external references. Having this said: I would appreciate your oppinion on this. – Broele Mar 21 '23 at 08:24
  • @Mithical: I edited the answer. I would appreciate some feedback from you if this meets your expectations. – Broele Mar 23 '23 at 10:51
  • That's much better, thank you! – Mithical Mar 23 '23 at 11:08