What’s happening
Three of the best augmented reality projects from Tribeca Festival are available for anyone — free, no trip to New York required.
Why it matters
Whatever our metaverse future may be, artists’ AR experiences can give you a glimpse of its more thoughtful possibilities.
Augmented reality took a backseat to virtual reality in the pandemic. VR headsets, which block out the real world to immerse you in a virtual one, suddenly commanded new appeal when everything social shut down and we sequestered ourselves at home. But now, AR — which overlays the real world with digital ornaments — has regained one prominent stage to show off its cutting edge again: immersive in-person experiences at big film festivals.
One of the latest, Tribeca Festival in New York last month, revived the full-blown, in-person installations of its Immersive curation for the first time in more than two years. Unless you were there, though, you probably missed them.
But three of Tribeca’s best AR art experiments are available now — free — for anyone with a phone to experience them.
Iago: The Green Eyed Monster reimagines Shakespeare’s villain as a modern military woman.
Tribeca Festival
One app, ReachYou, feels like post-COVID meta therapy guided by a gentle, glitchy transmission from the future. Another, Emerging Radiance, gives voice to murals of Japanese-Americans who were wrongfully incarcerated by the US government during World War II. And a third, Iago the Green Eyed Monster, performs a rock ‘n’ roll anthem reimagining Shakespeare’s infamous villain as a queer woman fed up with arrogant dudes.
These experiences give glimpses of what’s possible with AR, which you’ve likely interacted with through Instagram and other social media filters, or trying out digitized replicas of an Ikea couch in your living room. The technology has often been at the forefront of mixed-reality experimentation, by getting new concepts into the hands of billions of people with phones rather than just a niche minority who bother with headsets or ludicrously expensive prototypes of new hardware.
They’re also a tease of what things look like if our destiny really is to live in a metaverse (debatable).
ReachYou
ReachYou feels like meta-therapy sent from a metaverse.
Described as a “transmission from the future built for the tenderness of the present,” ReachYou is an AR experience that transforms your phone into a portal, both to receive messages sent by a meditative emissary from the future and to record your own contemplations on “the human record.”
Katrina Goldsaito, one of the project creators, said the team behind ReachYou created it “for strangers to connect over the things that are most important about being human and to give people a fierce nostalgia for the world we’re living in right now.”
“If we are living now in a dystopia, then we need more visions of protopias,” she said. The word, coined by Wired founding editor Kevin Kelly, describes a middle ground between the nightmare of dystopia and the unattainability — and stasis — of utopia. Protopias hope for technology to bring about a complicated progress toward a future that is better than present.
In ReachYou’s protopia, progress involves connecting to yourself and others in “authentic and vulnerable” ways, Goldsaito said. The app — which is a free download for iPhones and iPads in Apple’s App store, with a version for Android in the works — also encourages us to slow down, be present and change our relationship with technology, she added.
ReachYou uses AR to let you place a rectangular “beacon” on the floor in front of you, which morphs into a floating onyx cluster and a swirl of glowing orbs before it transforms into a cameo window-portal studded with pinpricks of stars. Inside, a woman contemplatively addresses you about “the great unwinding” and about the nature of time, grief and gratitude. She beckons you to step closer, and your phone trembles at the point of connection. She asks you to gently hum, and you hear your vocal vibrations join a chorus of others.
The end of the experience allows you to wander around floating pyramids that spin and unlock one of these public messages recorded by another Archivist, as Goldsaito calls participants. (You can choose your recordings to be public or for your ears only.) This field remains in the app for you to revisit whenever you like, with new recordings to unlock, but be aware: The full “transmission” will play only once. When you complete it, you can’t experience it again.
ReachYou, however, has fresh transmissions incoming. For those who’ve completed the first, a second was scheduled to arrive Friday; another will come Oct. 15. The experience was built to be episodic.
The ephemeral nature of each transmission was meant to make your moments in the experience more purposeful, according to Goldsaito.
“It’s a reminder that we only have this moment,” she said. “It asks that we set aside the time to make sure we’re listening carefully and fully.”
Emerging Radiance
Emerging Radiance breathes life into painted murals of Japanese-Americans, sharing their testimonials about wrongful incarceration during WWII.
Tribeca Fest
If ReachYou delivers a transmission from the future, then Emerging Radiance reanimates messages once locked in the past.
Putting a twist on Instagram filters, Emerging Radiance’s AR brings to life mural portraits of three Japanese-Americans, who share — in their own words and with their own voices — their experience with forced relocation and wrongful incarceration in US concentration camps during World War II. Creator Tani Ikeda said she wanted their archival recordings, taped by her own father long ago, to find new life in a medium that can carry on these stories from elders to a younger generation.
“We created our AR filters … so that anyone with Instagram can hold up their smartphone and hear each of our ancestors tell their stories,” she said.
All members of Bellevue, Washington’s Nikkei farming community, the protagonists of Emerging Radiance share the personal devastation they experienced because of incarceration. Rae Matsuoka Takekawa, for example, was the daughter of a leader in this community, who helped organize Japanese-American farmers so they could get fair prices for their crops. In Emerging Radiance, Takekawa’s mural recalls the night her father was summarily arrested and abducted by the FBI within hours after the bombing of Pearl Harbor.
Ikeda said stories like Takekawa’s provide “a historical roadmap” for defying the current climate of anti-Asian hate.
“When my grandfather was incarcerated, cameras and anything that could be used to document what was going on inside of the camps were taken away,” she said. “Now that most of the survivors of the concentration camps are ancestors, this augmented reality mural gives our community back the ability to speak.”
Meta Open Arts — the art program of Meta, the owner of Instagram — originally commissioned the murals, by Michelle Kumata, for its office in Bellevue. Ikeda worked with Kamata to craft the AR filters that bring the mural’s subjects to life for people with Instagram installed on their phone. You can hear their stories by scanning any of the three QR codes at the project’s website.
Iago: The Green Eyed Monster
Iago: The Green Eyed Monster turns a character study of Shakespeare’s antagonist into a rock anthem on an AR stage.
Tribeca Fest
Since Shakespeare invented the villain for his tragic play Othello more than 400 years ago, Iago runs deep with DGAF energy. This Iago shows it with crunchy guitars and fuzzed-out vocals instead of soliloquies.
Iago: The Green Eyed Monster reinterprets Iago as a modern female soldier, turning this miniaturized character study into a 3D-animated AR rock ‘n’ roll music video. This Iago may still drop a “doth” and “shall” in her lyrics, but you don’t need to know anything about Shakespeare to get what she’s pissed about. “These men, these men, these men,” she snarls.
Mary Chieffo, one of the creators, portrayed Iago as a man in 2015 in an all-female production of Othello, but she wondered what Iago “would look like as a modern military woman,” she said. “Our female Iago is consumed by her own jealousy and internalized rage over being overlooked, turning her into the ‘green-eyed monster.'”
The creators encourage you to “be your own cinematographer” and explore the AR stage as this Iago’s performance plays out. For Shakespeare fans, the experience is dotted with Easter eggs related to the play. For others, it can be an opportunity to “rock out to Shakespeare verse set to some banging music,” they said.
But beyond that, they hope the project stokes reflection about how patriarchy and other systems of oppression can make “monsters” out of people who behave outside of the boxes they’re prescribed to stay within.
Iago: The Green Eyed monster was produced with support from Verizon, as well as Viola Davis’ JuVee Productions, the company she founded with her husband, actor Julius Tennon. This isn’t the first time the Oscar winner’s production company dabbled in Shakespearean mixed reality: It produced a VR series pilot called Operation: Othello that premiered at the Cannes Film Festival in 2019. Joshua Nelson Youssef, JuVee’s head of immersive and creator behind Green Eyed Monster, said Davis “would retreat into other worlds in her trailer on the set of How to Get Away with Murder” with her VR headset.
Green Eyed Monster’s AR app is available for both Apple iOS and Android devices. The creators recommend an iPad in landscape orientation with headphones for the best experience. (A bigger screen definitely makes a difference.) The song is also available to stream on Spotify and other services. They even have a Snapchat Lens.
Chieffo said watching young women engage with Iago: The Green Eyed Monster during Tribeca Fest made her hopeful that “they can see it as a means to embrace their inner monster in a healthy way, rather than stifle it.”
“I hope this piece does what all great art should do – and certainly what Shakespeare was trying to do in his day: allow us to examine ourselves through heightened expression in order to inspire action in the real world we live in.”
