“It’s been really fun to see all the stuff people are making with it,” Pinkney wrote in an email to The Washington Post. “I’m always impressed by how it seems to capture the essence of famous people, but in a very weird Pokémon form.”
Very weird is an apt way to put it. People have used it to generate all sorts of Pokémon in the bizarre likeness of famous figures. Here’s what it spit out using Hollywood superstar Dwayne Johnson as a prompt:
Here are some Pokémon variants of Amazon founder Jeff Bezos. (Bezos also owns The Washington Post).
Here, by the way, is Wordle as a Pokémon.
But the generator is also capable of producing some surprisingly poignant creations, as can be seen in this mosaic of sample images shared by Pinkney. The first Pokémon in the lineup was generated from the prompt “Girl with a Pearl Earring,” the famed painting by Dutch master Johannes Vermeer. See if you can guess the others.
This is a Pokémon created from “The Return of the Prodigal Son,” one of the best-known paintings by another Dutch master, Rembrandt Harmenszoon van Rijn.
Pinkney was inspired to create a Pokémon art generator by his 6-year-old daughter, who has recently become a big fan of the franchise.
“I thought it would be a fun project to train a model so she could describe her ideal Pokémon,” he said. “It seems like it worked!”
Text-to-image art generators work through a process called deep learning, in which algorithms make predictions and complete tasks in a process that mimics the human brain’s neurons. In the case of AI-generated art, the generators pull from a database of existing pictures and illustrations to put together a discrete piece based on a user’s prompt. Pinkney explained that his own creation is adapted from an open-source deep-learning model called Stable Diffusion, which already has vast data sets of information. Text-to-Pokémon works by matching Stable Diffusion’s data sets to a data set of 850 Pokémon images from a previous university-run research project, which Pinkney filed using an automated caption system to categorize each image with a text description.
That’s why Text-to-Pokémon can create a more-or-less convincing Pokémon facsimiles of well-known public figures such as Dwayne Johnson or Taylor Swift, but will generate more abstract results for most people using their own names (unless they’re also high-profile celebrities). Pinkney published an extensive write-up on how his generator works on Lambda’s blog.
If you want to try your hand at creating some of your own Pokémon, you can access it here. You’ll need a GitHub account to run the model through the website, but once that’s set up, all you need to do is enter a phrase (Keanu Reeves, “The Last Supper,” “Doom Eternal,” etc.) and hit the submit button to generate one or more Pokémon.
Pinkney intends to keep working on Text-to-Pokémon. He floated the possibility of getting more images of Pokémon to increase his data set with more detailed captions, perhaps as a collaboration with someone with more Pokémon knowledge. He’s also interested in developing art generators that can compile images beyond text prompts.
“I think people have only seen the tip of the iceberg so far from these techniques!” Pinkney wrote. “Personally, although I think text prompts are a nice, easy interface to tell the AI what sort of image you want, I’m really interested in how to add other ways of influencing and controlling the output, to make it much more useful for artists and creatives to guide it to give the sort of images they’re after.”
