Amazon’s Orwellian “Rekognition” System Falsely Matches 28 Lawmakers to Mugshots
Amazon's facial recognition system known as Rekognition has remained at the center of several controversies for its surveillance capabilities. The company aggressively advertises its surveillance technology to police, enticing them with spotting up to 100 faces in a single image and tracking people in real time. From its own employees and shareholders, the company has also faced criticism from the civil society and academia.
Adding to the previous concerns, the American Civil Liberties Union (ACLU) said Thursday that Rekognition falsely identified 28 members of Congress as people who have been previously arrested for a crime.
ACLU said that it took 25,000 mugshots from a "public source" and ran the official photos of all 535 members of Congress through Rekognition. The software was then asked to match them with any of the mugshots, and it did end up matching 28 members.
We used Amazon’s facial recognition tool to compare photos of members of Congress to a database of mugshots — we got 28 false matches.
And even though they only make up 20% of Congress, nearly 40% of the false matches in our test were members of color. https://t.co/WdNRWtqZfa
— ACLU (@ACLU) July 26, 2018
"To conduct our test, we used the exact same facial recognition system that Amazon offers to the public, which anyone could use to scan for matches between images of faces," ACLU wrote, adding that running the test cost it just $12.33 - "less than a large pizza."
Amazon argues the test was wrongly conducted
The company has come under fire in the past for enabling a system where the fictitious "big brother" can watch over anyone at any time using a cheap facial recognition system. The company previously justified the use of its facial recognition system by saying it must be good if it was used at the royal wedding. Leaked communications have revealed that some in the law enforcement are also worried that Rekognition takes surveillance to another level.
With the traditional use of spying or surveillance programs, police needs a warrant to watch someone, however, with software like Rekognition they don't need to ask anyone for permission.
In response to today's ACLU report, Amazon issued a statement saying that the not-for-profit didn't conduct the study properly. Amazon hasn't yet responded to questions on how it ensures that the law enforcement agencies use its Rekognition system with its recommended settings to avoid innocents being caught in the fire thanks to its facial recognition system.
Here's the complete statement issued by the e-commerce giant (via BI):
"We have seen customers use the image and video analysis capabilities of Amazon Rekognition in ways that materially benefit both society (e.g. preventing human trafficking, inhibiting child exploitation, reuniting missing children with their families, and building educational apps for children), and organizations (enhancing security through multi-factor authentication, finding images more easily, or preventing package theft). We remain excited about how image and video analysis can be a driver for good in the world, including in the public sector and law enforcement.
"With regard to this recent test of Amazon Rekognition by the ACLU, we think that the results could probably be improved by following best practices around setting the confidence thresholds (this is the percentage likelihood that Rekognition found a match) used in the test. While 80% confidence is an acceptable threshold for photos of hot dogs, chairs, animals, or other social media use cases, it wouldn't be appropriate for identifying individuals with a reasonable level of certainty. When using facial recognition for law enforcement activities, we guide customers to set a threshold of at least 95% or higher.
"Finally, it is worth noting that in real world scenarios, Amazon Rekognition is almost exclusively used to help narrow the field and allow humans to expeditiously review and consider options using their judgement (and not to make fully autonomous decisions), where it can help find lost children, restrict human trafficking, or prevent crimes."