John Herrman, The New York Times
Published: 2019-07-26 11:38:52 BdST
I would also like to ask him about FaceApp, his indifferent creator. Aside from being a decent icebreaker, it would be a way into something 2019 me is quite curious about: The group of millennium-era technologies known colloquially as the internet, and where they are taking us.
It’s hard enough to tell, looking back over just a few weeks, how we got here with FaceApp. After going viral for the second or third time in the United States, the novelty app became, in the space of a few days, an avatar for deceptive user agreements, a player in an insinuated global conspiracy and a cautionary tale about how little it takes to convince tens of millions of people to give up their likenesses for processing. Sen. Chuck Schumer of New York, a Democrat, said in a letter that he would like the FBI and FTC to investigate, writing that it is “deeply troubling” that Americans’ personal data had been transferred to a “hostile foreign power.” This week, Sen. Rick Scott of Florida, a Republican, introduced a bill that would require app stores to list software’s country of origin, citing FaceApp.
It’s not hard to understand why people fell for the app. Its facial filters are often good enough to be plausible, and novel enough to be surprising, creating the impression that this particular artificial intelligence knows something we don’t. It’s a cartoonish but illustrative example of automation, in that it takes something rare and specialised — age-advanced portraits — and makes them available to everyone at no upfront cost. The app also set up the perfect lightly self-deprecating joke for the countless attractive celebrities who took part in the FaceApp Challenge. (The challenge’s directive being: post your FaceApp.)
The FaceApp backlash is more complicated. Users of the app agreed to grant the company a license to their photos that is “perpetual, irrevocable, nonexclusive, royalty-free, worldwide, fully-paid,” which is indeed quite expansive. Just as the app went viral, so did some of the fine print.
Of course, the agreement was boilerplate. Assumptions about the motives of Russian developers were just that; while anyone operating an app that asks for the permissions granted to FaceApp has comprehensive access to users’ devices, close traffic analysis found that it was, for now, doing what it claimed: collecting millions of uploaded user photos, analysing them and then serving them back older, younger, gender-swapped, smiling or styled.
After a few days of hazy panic came some attempts to restore clarity: There are many apps we already use that are at least as invasive as FaceApp, and that use facial recognition, and that are already known to have breached the trust of their users. Worries about FaceApp were, in other words, disproportionate to its significance. “Think FaceApp Is Scary? Wait Till You Hear About Facebook,” read a headline in Wired. The situation was also, in part, our fault: “You downloaded FaceApp. Here’s what you just did to your privacy,” another ominously stated in The Washington Post.
It’s true that tens of millions of people downloaded the app. And we do live in a world in which egregious privacy violations are the norm. But reflexive calls to calm down, or see the bigger picture, were part of the slow, complicated process that made them normal.
The fact that a frictionless process enjoyed between friends, motivated by reasons both silly and poignant, entails granting a legal license to media containing your likeness is plainly absurd. How did this happen? Just because privacy violations are routine doesn’t mean they make sense, or that anyone actually asked for them. No wonder our embrace and rejection of FaceApp seems confused.
“We have this underlying infection, and every once in a while it breaks out, and suddenly you have a spike of fever,” said Shoshana Zuboff, author of “The Age of Surveillance Capitalism.”
“You treat the symptom, or you don’t,” she said, “and you stay oblivious to the underlying cause.”
“The default economic model for almost every app in existence is to not only take what you give it, but to take its privileged position on your phone, in your computer, to secretly take much more than you’ve given, and to use that much more in ways that are ultimately monetisable,” Zuboff added. It’s not what these companies have done, or haven’t done, or what they might do in the future. It’s what they’re able to do, and how little say we have in it.
In a generation, large-scale facial recognition has gone from the sort of thing that might be abused by governments to a technology that widely is. It’s also both legally and practically possible for any company that gives people a way to have fun for a few minutes on their smartphones.
Discussion about the dangers of an app like FaceApp have revolved around competing possible future violations: users’ images being sold as stock photos, or used in an ad; a massive data set being sold to a company with different ambitions; a hack. But the real violation is right there in the concept, and in the name.
FaceApp, in order to do the innocent thing that it advertises, must collect data so personal that its frequent surrender and seizure could soon result in the end of anonymous free movement on Earth. This is what the app economy, often a synonym for the new economy, demands. You can make the most innocent assumptions about FaceApp and its creators and still arrive at the conclusion that it should not exist, and yet here it is, the perfect smartphone toy, with nearly a million reviews in the App Store, and a rating of 4.7/5 stars. Sound crazy? Try it again this way: How did you make that photo? Oh my God. FaceApp? I’ve got to download that. “Allow access to your photos”? OK, “Allow.” “Enable access to the camera”? OK, “Enable.”
If the old man in my FaceApp could talk, he might scream: “FaceApp is the singularity! FaceApp is the end of the world.” More likely, though, he wouldn’t remember FaceApp at all. If anything, he’d probably say that FaceApp was just a fun app whose makers, after a bad year or two, maybe began to wonder just how much the things they already had access to were worth, and to whom, and who, like us, didn’t really feel the need to think about just how completely we had been habituated to not think about how, whether an app failed and disappeared or became incredibly successful, the data we had given it years ago would be put to uses beyond our control or imagination.
The story of FaceApp has been short, confused and speculative, but of course it has. The digital world that produced it configured to make us feel like we’re losing our minds. Its story, as easy as it is to dismiss, and as conspicuously frivolous as it is, is not excused, but rather given power by how well it rhymes with the much longer stories of Facebook and Google. It was Facebook to which we were uploading photos, for reasons silly and poignant, 15 years ago. And it was Facebook that started asking us to tag them, and then which started tagging them itself. It was Google that started as one thing and become many things, each bigger than the first, carrying with it whatever data we gave it, and whatever permissions we granted to its unrecognisable former selves. And while Facebook’s betrayals, both prosecutable and more general, are both more established and far larger than anything a gimmick app like FaceApp could aspire to, Facebook was a gimmick site too, for a while.
The Facebook backlash has been intense, confused and frequently criticized as misguided, as it backtracks up seemingly countless branches of a diagnostic tree. This backlash, like the FaceApp backlash, is locked within the industrial and legal framework that produced these apps in the first place. One that devised, as punishment for Facebook’s handling of user data, a $5 billion fine that the company can easily pay through the further handling of user data. One that takes for granted that industry has a right to use poorly understood forms of personal data into perpetuity. And one that doesn’t dare to entertain the possibility that it has created something that, in its popular form, simply should not exist.