Creating Creatures with AI

I’ve enjoyed playing with Tensorflow over the years in a very recreational way. This Colab: Creating New Objects: Blending BigGAN Categories with Tensorflow is one that has both disturbed and delighted me more than once.

The concept is not too strange: the BigGAN project includes a set of image training data of many types of objects. So, for example, based on what the model ‘knows’ about any of these ~1000 different kinds of objects you can generate a new image of any of them. So here are 20 entirely fake volcanoes.

…and here are twenty espressos,

You can see very well in the espresso images that these models are not perfect. Some of these images do look a lot like espressos, but many really don’t. Look at the cup handles and the objects around them in the images. But even when they go wrong, you can kind of understand why it looks like an espresso anyway. This is the uncanny valley of AI that often occurs in such demonstrations, and which often makes machine-learning tools like this both funny and disturbing.

Given a random sample of noise, we can use BigGAN to conjure up a new image of an objects. As I understand it, the code does this by repeatedly making some random image noise ‘look more like’ the object in question. However we don’t need to be so linear here. We can also ask it to go halfway to an object, or to create an image exactly halfway between two different categories.

In this image you see three attempts are stepping from a generated Komodo Dragon image on the left, into an Emperor Penguin on the right. None of the images are real of course (I mean, do they look like Komodo Dragons really?) but the column in the middle are the ‘Komodo Penguins’, the images halfway between Dragon and Penguin. There are so many combinations:

The Centipede Retriever is really quite disturbing, but I rather like the Marmoset Corgi on the bottom row. We don’t have to mix animals with animals though. Here’s a Baboon and a Beer. The poor creature in dead centre should either be avoided or looked after, not sure which:

Finally, as a dog owner I couldn’t resist looking at the huge selection of dog breeds in the BigGAN image data set. Particularly fun is to create ML crossbreeds and see if they look like real cross breeds. So here is the code’s attempt at a French Bulldog shifting into a Border Terrier, and then a Bernese Mountain Dog crossing with a Schnauzer:

I personally think this works rather well. But to sign-off it would be a shame not to see if my own dog Rufus can be recreated in the same way. Rufus is a Shih-Tzu crossed with a Toy Poodle. The generative dogs below are all lighter in colour (bias in training data!) whereas Rufus is all black. But I think you can see that the code does a reasonable prediction of the crossbreed appearance.

If you want to try all this out yourself, then make a copy of the Colab: Creating New Objects: Blending BigGAN Categories with Tensorflow. Connect to the runtime, change the dropdowns and play about with all the sliders.