Facial recognition is helping Putin curb dissent with the aid of US tech

A review of more than 2,000 court cases shows how Russia uses facial recognition to identify and sweep up the Kremlin's opponents

Andrey Chernyshov had just entered a Moscow metro station on his way to an anti-war protest last May, when police officers stopped him, informed him he was on a wanted list and, without further explanation, escorted him to a police office inside the station.

There officers told the 51-year-old bank employee that the metro's facial recognition system had flagged him for detention because of his political activism. A little over a week earlier Chernyshov stood alone by a fountain in central Moscow's Pushkin Square and held up a home-made poster that said "Peace to Ukraine," "No War" and "Freedom for Russia."

Released without charge after a few hours, Chernyshov was detained again later the same day as he travelled home. This time he was questioned about his views on the war in Ukraine and President Vladimir Putin. A man in plain clothes identified himself as an official from Russia's Centre for Combating Extremism, known by Russians as Centre E, and advised Chernyshov to refrain from joining future demonstrations because he had a young child to care for.

"I took his words as a threat," said Chernyshov, who has a 5-year-old son.

Chernyshov spent seven hours with police that day.

And that wasn't the end of it. Police detained Chernyshov again in the metro in June, August and September, and they twice visited his home to warn him against protesting. In June, he had defied authorities by handing out badges to passers-by that read "No to War" and "Russia will be free."

Russian authorities did not respond to questions from about Chernyshov's brushes with law enforcement.

​​It's no secret that the Russian government uses facial recognition to keep an eye on citizens. In 2017, the city of Moscow announced the launch of one of the world's largest facial recognition video surveillance networks. In a news release at the time, Moscow's Department of Information Technologies said 160,000 cameras across the city - more than 3,000 of them connected to the facial recognition system - would help law enforcement.

Now a review of more than 2,000 court cases shows these cameras have played an important role in the arrests of hundreds of protesters. Most of these people were detained in 2021 after they joined anti-government demonstrations, court records show. But after Russia invaded Ukraine in February 2022, authorities began using facial recognition to prevent people from protesting in the first place, according to interviews with more than two dozen detainees and information gathered by a Russian monitoring group. Facial recognition is now helping police to identify and sweep up the Kremlin's opponents as a preventive measure, whenever they choose.

"It's a new practice, which is being used to chilling effect, especially in Moscow where protests have been the largest and where people know that they are being watched by facial recognition cameras," said Daria Korolenko, a lawyer with OVD-Info, an independent human rights group that monitors repression in Russia.

Western technology has aided the crackdown. The facial recognition system in Moscow is powered by algorithms produced by one Belarusian company and three Russian firms. At least three of the companies have used chips from US firms Nvidia Corp or Intel Corp in conjunction with their algorithms, found. There is no suggestion that Nvidia or Intel have breached sanctions.

The Russian and Belarusian companies which participated in a US facial-recognition test program, aimed at evaluating emerging technologies and run by an offshoot of the Department of Commerce. One of the firms received $40,000 in prize money awarded by an arm of US intelligence.

Approached for comment, Nvidia and Intel said they halted all shipments to Russia in March 2022 after the United States tightened export restrictions. They added that they can't always know how their products are used.

A spokesperson for Commerce's National Institute of Standards and Technology said participation in its assessments is not a seal of approval. The Intelligence Advanced Research Projects Activity, which awarded the prizes, said challenges are for "market analysis" and "IARPA has no access to or role in an organisation's continued development of a technology after the close of a prize challenge."

Surveillance cameras are just one piece of the Russian government's campaign to suppress opposition to the war, according to OVD-Info and more than a dozen activists interviewed. It is illegal to engage in "public actions aimed at discrediting the use of the armed forces of the Russian Federation" and to deliberately and publicly spread false information about the Russian armed forces. Such actions are punishable by fines and prison sentences ranging from three to 15 years.

Chernyshov is among at least 141 people who were detained preventively in 2022, according to data collected by OVD-Info. They were stopped in the metro on national holidays, when authorities were expecting protests, or at other times when anti-war sentiment was running high, for example after Russia announced a mass draft of men into the military in September.

Reuters interviewed 29 people who were stopped by police in Moscow metro stations. All but one said they understood from officers that they were flagged for detention by facial recognition. At least 14 said officials referred to a system called Sfera, or Sphere. Moscow's department for transportation stated in a December 2022 news release that its Sfera video analytics system uses VisionLabs technology. It couldn't determine which other products underpin Sfera.

Those detained included students, pensioners, a scientist, an academic researcher and a courier. Some were commuting to or from work. One man was going to the theatre with his wife and children. A woman was taking her mother to a doctor's appointment. All had one thing in common - they were critical of the Kremlin. Most had previously joined anti-government protests.

These people said they were held at a metro station or police station for periods ranging from 10 minutes to 18 hours. Seven said that police informed them they were picked up to stop them protesting. At least 12 said that before their release they signed a document: either undertaking not to protest or acknowledging they had received a warning against protesting. None was charged with any offence.

The Kremlin referred detailed questions for this article to the Moscow mayor's office which did not respond. Russia's Interior Ministry, which oversees law enforcement, also did not respond to questions sent via the Kremlin.

LIGHTNING SPEED

The Moscow metro uses facial recognition as part of its fare payment system as well as for security. Passengers are photographed as they walk through the gates and a computer algorithm compares the face to a pictures database. If the system flags a passenger for detention, police respond within seconds or minutes, according to the 29 people who were detained in this way. Alexander Zharov, a 32-year-old civil rights activist, recorded the moment of his detention on his cellphone in August last year and shared the footage.

Facial recognition technology uses artificial intelligence algorithms to analyse and identify faces. The Moscow metro has deployed algorithms from three Russian companies, according to a 2021 news release on the city of Moscow's website. These firms are Tevian, an artificial intelligence company founded by staff at Moscow State University; NtechLab, which from 2018 was part-owned by Russian defence firm Rostec State Corp; and VisionLabs, owned by Russia's largest mobile operator, MTS, and headquartered in the Netherlands.

VisionLabs' executive Anton Nazarkin said the company provides facial recognition to Moscow via a third party, which he declined to name. He said VisionLabs' algorithm has been used in Moscow's facial recognition system since it was rolled out across the capital during the 2018 FIFA World Cup, which Russia hosted, and is also used in the metro's voluntary Face Pay payment system. Nazarkin said he was not aware of the company's technology being used in other ways in the metro. MTS did not respond to a request for comment.

An NtechLab spokesperson said that as of 2022, the Moscow metro no longer uses the firm's technology. He declined to go into detail about its wider deployment in the capital, citing a non-disclosure agreement. He noted, however, that software is independently managed by the customer.

Tevian did not respond to requests for comment.

Contracts published in 2022 on Russia's state tender website provide further technical details about the capital's facial recognition system. They show Moscow uses VisionLabs' Luna Platform, NtechLab's FindFace and Tevian's FaceSDK, as well as facial recognition software called Kipod, made by Synesis, a Belarusian company. Synesis is under American, EU and British sanctions for its role in suppressing pro-democracy movements in Belarus and Russia. Synesis has rejected the sanctions as "unfounded." It didn't comment for this article.

To speed image-matching, NtechLab and VisionLabs have turned to US technology. Both firms have used graphics processing units (GPUs) made by Santa Clara, California-based Nvidia, according to Nvidia's website. GPUs are powerful chips originally designed to improve image rendering in video games.

Nazarkin, the VisionLabs executive, said that Nvidia GPUs are the "industry standard" for training a facial recognition system to accurately identify images. "Pretty much any company out there that is doing any kind of AI application is utilising Nvidia GPUs," said Nazarkin in a phone interview from Moscow.

In a statement, Nvidia said it halted sales to Russia in March 2022 after the US imposed extensive export controls and sanctions but cannot track every downstream use of its products. A spokesperson said Nvidia had a brief engagement with VisionLabs and NtechLab that concluded before February 2022.

Russian customs records show that at least 129 shipments of Nvidia products reached Russia via third parties between April 1 and Oct. 31, 2022, however. Records for at least 57 of these shipments stated that they contained GPUs. In response to these findings, the spokesperson said, "We comply with all applicable laws, and insist our customers do the same. If we learn that any Nvidia customer has violated US export laws and shipped our products to Russia, we will cease doing business with them."

Facial recognition algorithms also run on high-speed central processing units (CPUs), the chips that provide the processing power a computer needs to complete its tasks.

A document that appeared on Intel's website as recently as October last year said Synesis, the Belarusian firm, tested Intel technology in 2019 and was using Intel CPUs to improve the performance of its Kipod platform "for law enforcement."

The European Union and Britain sanctioned Synesis in December 2020, saying Kipod was used to track and repress civil-society and pro-democracy activists in Belarus. Synesis has called the assertions "absurd." After Russia's invasion of Ukraine in February 2022, the US Department of the Treasury announced it too was sanctioning Synesis because Russian and Belarusian authorities were using the firm's video surveillance system to persecute protesters.

Intel said that its sales in Russia for over a decade have been through distributors that are required to comply with US export controls. It said it suspended all shipments to customers in Russia and Belarus after Russia's invasion of Ukraine. At least $457 million worth of Intel products arrived in Russia between Apr 1 and Oct 31, 2022, according to Russian customs records. "We take reports of continued availability of our products seriously and we are looking into the matter," an Intel spokesperson said.

RIGHTS IN PERIL

To be sure, the Russian and Belarusian companies began using Western technology before the latest export restrictions were imposed. But as far back as 2009, Intel joined the United Nations Global Compact that says companies should not be complicit in human rights violations. Nvidia joined the compact in 2022. In its human rights policy, Nvidia says it endorses the United Nations Guiding Principles on Business and Human Rights, which require companies to mitigate the risk that their products could be used for rights abuses. Intel's human rights policy also says it embodies the UN principles.

In response to findings, seven lawyers said the use of facial recognition against protesters and activists in Moscow likely constitutes human rights violations.

Klara Polackova Van der Ploeg, head of the Business, Trade and Human Rights Unit at the University of Nottingham's Human Rights Law Centre, said, "An appropriate process for those companies would be to know who their business partners are, learn how their products are used, and exercise leverage to prevent their technology from being utilised in violations of human rights." Intel and Nvidia declined to comment on this point.

The Russian and Belarusian firms have previously had some contact with US government agencies. All have taken part in facial recognition tests by the National Institute of Standards and Technology (NIST), an agency of the US Department of Commerce, documents show. In 2017, NtechLab received two prizes worth a total of $25,000 in a facial recognition contest organised and funded by Intelligence Advanced Research Projects Activity (IARPA), an arm of the Office of the Director of National Intelligence, which oversees the US intelligence community. Two years later, NtechLab won a $15,000 second-place prize in another IARPA technology challenge.

"Participation in the assessment does not indicate NIST approval of the business or other practices of any participant," NIST said in an email. An IARPA spokesperson said its work requires it to maintain awareness of the world's leading innovations and that awards do not signal government endorsement.

The CEOs and founders of VisionLabs, NtechLab and Tevian are now on a list of people who jailed Russian opposition leader Alexei Navalny's Anti-Corruption Foundation believes should be sanctioned. The foundation points to the companies' involvement in Moscow's video surveillance system, which it says is used to persecute political activists. The companies did not respond to questions about their inclusion on the list.

UNDER OBSERVATION

The preventive detentions, coupled with other pressure from police, so upended the lives of some of its targets, they say they felt driven to leave Russia.

Luba Krutenko, a 32-year-old architect who was in police sights because of two prior arrests for protesting, said police showed up at her home seven or eight times in March and April last year. In early March, they came three times in two days.

"They were just giving me warnings, documents saying that I shouldn't go to rallies and if I go, a criminal case could be opened," she said.

She stopped answering the door to avoid the encounters. Then they began to call her.

During a phone call in April, a police officer told her they knew she was home. "They told me they can check the footage from the cameras by the entrance to my building and know if I'm inside," she said. She showed screenshots of several missed calls from police and of a text message informing her that they were standing outside her home.

She stopped answering the phone and went on vacation in the town of Onega in northern Russia. When she got back to Moscow, on May 6, police were waiting for her on the platform as she exited the train.

"They knew what carriage I was in because they waited right outside it," she said. Krutenko showed a video of the encounter. Police asked her to sign a document warning her against protesting. Krutenko shared a copy of the signed document and said she signed six such documents during different encounters with the police last year.

The next encounter came three days later, just after she entered a Moscow metro station on Russia's Victory Day. Two police officers approached, asked her to show her ID and informed her that the security camera by the metro's payment gate had recognised her. They told her she was on a wanted list and escorted her to a police point in the metro station. After about 40 minutes, two more officers arrived and told her she was being detained and would be taken to the police station. One of the officers carried a machine gun, she said.

At the police station, she demanded an explanation for her detention. She told one officer they were violating her rights. "I was angry. I told her 'you're keeping me here. This is my weekend. I have plans with friends'." Krutenko said the officer replied that people like her have no rights and called her a traitor. Another police officer told her that they were detaining her because an anti-war protest was planned in the city centre that day and that they had a list of would-be protesters – including her – whom they were trying to prevent from attending.

She was allowed to leave after about three hours, with another warning that she would risk jail if she was charged again with protesting.

"Because police are present in your life all the time, it makes it unpredictable," she said, describing the repeated encounters. "You think you can be detained at any time."

The surveillance and risk of detention made her decide to leave Russia in September. She now lives in Bonn, Germany, where she is in the process of completing an integration course.

On May 9, the same day Krutenko was detained, a police officer approached Sergei Pinchuk, a 27-year-old courier, seconds after he entered the Moscow metro. Pinchuk, who wore blue and yellow, the colours of the Ukrainian flag, that day, said police had a handheld electronic device with about 10 photos of him, all seemingly taken by metro security cameras on different dates.

At a nearby police station, a detective pushed him into the wall, grabbed him by the neck and called him names, he said.

A month prior, Pinchuk had stood alone in front of the Kremlin building holding a sign with the words '353 Criminal Code of Russia. Stop Putin,' referencing a Russian law that says waging an aggressive war is punishable by up to 20 years in prison. The detective asked about this. "He said 'why did you do that? It's a hard time for our country'," said Pinchuk, who had been arrested, charged and fined for the protest. The detective also threatened to jail him for years and create problems for his family, he said.

In mid-August, Pinchuk and his friend climbed a cell tower in his town Naro-Fominsk, southwest of Moscow, and placed a Ukrainian flag at the top. The next day, while Pinchuk was visiting his parents, his brother called and informed him that police had stopped by Pinchuk's home and were looking for him. A few hours later, Pinchuk was at the airport, catching the first flight he could find to Tbilisi, Georgia. Later police called his mother and texted his friends asking where he was. He decided to relocate to the United States where he said he felt safer. He now lives in Seattle and is seeking asylum.

Chernyshov, the protester detained twice on the same day, left Russia and is in Minneapolis with his wife and son. He decided to leave after police again detained him in the metro on Sept. 1 on his way to work. He said they kept him handcuffed in a cell overnight and tased him with a stun gun when he asked them to loosen or remove the cuffs. They didn't allow him to drink or use the bathroom and kept the lights on all night, he said. The next morning, they released him without charge. Chernyshov provided photos of several of his detentions and protests.

More than two weeks later, he said, as he was about to get into his car to return home from the gym, a hooded man sprayed paint on his face. The paint stung his eyes and left marks on his face. After the incident, he received an anonymous letter threatening to harm or jail him if he didn't stop expressing his political views. The letter contained a photo of the attack.

"So I realised that I needed to leave Russia as quickly as I could," he said.