1

Refik Anadol has machines view actual pictures and then has the machine create its own images. This video shows some of the stuff he does.

What kind of out-of-the-box tools (e.g. a Python package) or algorithms produce similar things to what he does?

I am hoping to play around with it and see what can happen. Not sure what to even Google for or search for.

nbro
  • 39,006
  • 12
  • 98
  • 176
confused
  • 143
  • 6
  • I have provided an answer covering the work with some ideas, but I suspect that what you are really interested in is the GANs. Hence I provided a link to a library for those and not as much detail on the other parts. – Neil Slater Oct 16 '21 at 17:28
  • @confused, try Clip Interrogator: https://huggingface.co/spaces/pharma/CLIP-Interrogator converts his work to a description, which can then be drawn in that style: https://youtu.be/rQq5i0lTMNE – Rob Nov 14 '22 at 00:52

1 Answers1

0

For a Google term you could use "computational creativity". It covers a wide range of ideas, and the artist here is not using one single tool or approach.

There are clearly a range of different techniques that went into the artist's installations that were shown on the video. I have an idea about a few of them:

  • Some are from basic animation and story-telling, and not "generated" art - at one stage the installation lists all the files that were used for instance. This is a way to communicate something about how the piece was produced. It would have been constructed into a video sequence by the artist, probably without any AI, although it is always possible to write a script that produces "decisions" from some kind of arbitrary mapping, and an artist may decide to cede control in that way as part of a piece.

  • There is some kind of particle system or physics-based simulation behind the flowing coloured clouds of cubes (note the physics does not have to be real physics, it often is not close, but inspired by some physical process such as flowing liquids, or maybe biological such as swarming insects). There a few different frameworks to generate these - Blender could do it for example, plus many games engines are capable of such renders. It is also common to have these types of systems hooked up to a data source that provides input of perturbations - movement, forces, shapes - that drive the dynamics, and are somehow meaninful in the context of the piece. It is not clear whether the artist has done that in this case, but it is also common to hook these systems up to data inputs in real time, such as location of gallery visitors, weather data from around the world etc. Choices of colour gradients, size and shape of elements etc will be curated by the artist to capture a certain mood.

    • Given the rest of this installation, the flowing cubes animation could well have something to do with the photos and how they were processed. However, it is not at all clear from viewing it whether this is the case.
  • The mutating photos look a lot like the output of GANs (generative adversarial networks) or VAEs (variational autoencoders). The liquid-like flow between images is caused by picking a trajectory through a latent space that these image generators learn from observing many images. A popular Python implementation of StyleGAN 2 would definitley be capable of producing output of the quality seen in the video.

  • Potentially the overall mix of the artwork, transitions and layouts could be a curated video sequence by the artist, or it could be dynamically mixed by a function reacting to a live data feed. In this case, the mixing seems more on the human-curated side - the installation could be e.g. a 15 minute video loop showcasing a fixed rendering of interesting parts of the spaces selected by the artist.

There are probably other things. For instance, the artist has gone to some trouble to curate their data, also using machine learning techniques when they discuss filtering out images of people. I think the effort that has gone into the installation, and the desire of the artist to showcase that is why some of the installation includes storytelling about how it was made.

Neil Slater
  • 28,678
  • 3
  • 38
  • 60
  • Thanks! Yeah I was mostly interested in how to get a computer to recognize images and I guess learn from it so it can create it's own. The after effects I can probably figure out - plus that will depend on style. But this is very useful starting point on things to look at. – confused Oct 16 '21 at 17:33
  • @confused [If you think this answer is useful, you should upvote it](https://ai.stackexchange.com/help/privileges/vote-up). – nbro Oct 19 '21 at 12:48