Card, 73, and her husband, Greg Grace, 75, dashed to their bank in Regina, Saskatchewan, and withdrew 3,000 Canadian dollars ($2,207 in U.S. currency), the daily maximum. They hurried to a second branch for more money. But a bank manager pulled them into his office: Another patron had gotten a similar call and learned the eerily accurate voice had been faked, Card recalled the banker saying. The man on the phone probably wasn’t their grandson.
That’s when they realized they’d been duped.
“We were sucked in,” Card said in an interview with The Washington Post. “We were convinced that we were talking to Brandon.”
As impersonation scams in the United States rise, Card’s ordeal is indicative of a troubling trend. Technology is making it easier and cheaper for bad actors to mimic voices, convincing people, often the elderly, that their loved ones are in distress. In 2022, impostor scams were the second most popular racket in America, with over 36,000 reports of people being swindled by those pretending to be friends and family, according to data from the Federal Trade Commission. Over 5,100 of those incidents happened over the phone, accounting for over $11 million in losses, FTC officials said.
Advancements in artificial intelligence have added a terrifying new layer, allowing bad actors to replicate a voice with just an audio sample of a few sentences. Powered by AI, a slew of cheap online tools can translate an audio file into a replica of a voice, allowing a swindler to make it “speak” whatever they type.
Experts say federal regulators, law enforcement and the courts are ill-equipped to rein in the burgeoning scam. Most victims have few leads to identify the perpetrator and it’s difficult for the police to trace calls and funds from scammers operating across the world. And there’s little legal precedent for courts to hold the companies that make the tools accountable for their use.
“It’s terrifying,” said Hany Farid, a professor of digital forensics at the University of California at Berkeley. “It’s sort of the perfect storm … [with] all the ingredients you need to create chaos.”
Although impostor scams come in many forms, they essentially work the same way: a scammer impersonates someone trustworthy — a child, lover or friend — and convinces the victim to send them money because they’re in distress.
But artificially generated voice technology is making the ruse more convincing. Victims report reacting with visceral horror when hearing loved ones in danger.
It’s a dark impact of the recent rise in generative artificial intelligence, which backs software that creates texts, images or sounds based on data it is fed. Advances in math and computing power have improved the training mechanisms for such software, spurring a fleet of companies to release chatbots, image-creators and voice-makers that are strangely lifelike.
AI voice-generating software analyzes what makes a person’s voice unique — including age, gender and accent — and searches a vast database of voices to find similar ones and predict patterns, Farid said.
It can then re-create the pitch, timber and individual sounds of a person’s voice to create an overall effect that is similar, he added. It requires a short sample of audio, taken from places such as YouTube, podcasts, commercials, TikTok, Instagram or Facebook videos, Farid said.
“Two years ago, even a year ago, you needed a lot of audio to clone a person’s voice,” Farid said. “Now … if you have a Facebook page … or if you’ve recorded a TikTok and your voice is in there for 30 seconds, people can clone your voice.”
Companies such as ElevenLabs, an AI voice synthesizing start-up founded in 2022, transform a short vocal sample into a synthetically generated voice through a text-to-speech tool. ElevenLabs software can be free or cost between $5 and $330 per month to use, according to the site, with higher prices allowing users to generate more audio.
ElevenLabs burst into the news following criticism of it’s tool, which has been used to replicate voices of celebrities saying things they never did, such as Emma Watson falsely reciting passages from Adolf Hitler’s “Mein Kampf.” ElevenLabs did not return a request for comment, but in a Twitter thread the company said it’s incorporating safeguards to stem misuse, including banning free users from creating custom voices and launching a tool to detect AI-generated audio.
But such safeguards are too late for victims like Benjamin Perkin, whose elderly parents lost thousands of dollars to a voice scam.
His voice-cloning nightmare started when his parents received a phone call from an alleged lawyer, saying their son had killed a U.S. diplomat in a car accident. Perkin was in jail and needed money for legal fees.
The lawyer put Perkin, 39, on the phone, who said he loved them, appreciated them and needed the money. A few hours later, the lawyer called Perkin’s parents again, saying their son needed $21,000 ($15,449) before a court date later that day.
Perkin’s parents later told him the call seemed unusual, but they couldn’t shake the feeling they’d really talked to their son.
The voice sounded “close enough for my parents to truly believe they did speak with me,” he said. In their state of panic, they rushed to several banks to get cash and sent the lawyer the money through a bitcoin terminal.
When the real Perkin called his parents that night for a casual check-in, they were confused.
It’s unclear where the scammers got his voice, although Perkin has posted YouTube videos talking about his snowmobiling hobby. The family has filed a police report with Canada’s federal authorities, Perkin said, but that hasn’t brought the cash back.
“The money’s gone,” he said. “There’s no insurance. There’s no getting it back. It’s gone.”
Will Maxson, an assistant director at the FTC’s division of marketing practices, said tracking down voice scammers can be “particularly difficult” because they could be using a phone based anywhere in the world, making it hard to even identify which agency has jurisdiction over a particular case.
Maxson urged constant vigilance. If a loved one tells you they need money, put that call on hold and try calling your family member separately, he said. If a suspicious call comes from a family member’s number, understand that too can be spoofed. Never pay people in gift cards, because those are hard to trace, he added, and be wary of any requests for cash.
Eva Velasquez, the chief executive of the Identity Theft Resource Center, said it’s difficult for law enforcement to track down voice-cloning thieves. Velasquez, who spent 21 years at the San Diego District Attorney’s Office investigating consumer fraud, said police departments might not have enough money and staff to fund a unit dedicated to tracking fraud.
Larger departments have to triage resources to cases that can be solved, she said. Victims of voice scams might not have much information to give police for investigations, making it tough for officials to dedicate much time or staff power, particularly for smaller losses.
“If you don’t have any information about it,” she said. “Where do they start?”
Farid said the courts should hold AI companies liable if the products they make result in harms. Jurists, such as Supreme Court Justice Neil M. Gorsuch, said in February that legal protections that shield social networks from lawsuits might not apply to work created by AI.
For Card, the experience has made her more vigilant. Last year, she talked with her local newspaper, the Regina Leader-Post, to warn people about these scams. Because she didn’t lose any money, she didn’t report it to the police.
Above all, she said, she feels embarrassed.
“It wasn’t a very convincing story,” she said. “But it didn’t have to be any better than what it was to convince us.”
