Rollout of up to 40,000 cameras equipped with AI technology in São Paulo is also raising concerns on security, privacy.
One of the key issues highlighted by experts regarding Smart Sampa pertains to the negative consequences the system could generate, particularly for groups such as the Black community, which constitutes 56 percent of Brazil’s population, according to the Brazilian Institute of Geography and Statistics (IBGE). Experts argue the project could undermine the right to non-discrimination and challenge the principle of presumption of innocence.
According to Fernanda Rodrigues, a digital rights lawyer and research coordinator at the Institute for Research on Internet and Society (IRIS), facial recognition technology has the potential to lead to false positives – wrongly matching a person’s face with an image in the database – and drive mass incarceration of Black individuals as a result.
“As well as the risks that the information fed to these platforms may not be accurate and the system itself might fail, there is a problem that precedes technology implications, which is racism”, Rodrigues said.
“We know the penal system in Brazil is selective, so we can conclude that [use of surveillance with facial recognition] is all about augmenting the risks and harms to this population”, Rodrigues added, referring to the high representation of Black individuals in Brazilian jails, who account for over 67 percent of the imprisoned population, according to 2022 data from the Brazilian Public Security Forum.
A study conducted by the Center for Studies on Public Security and Citizenship (CESeC), which monitors the impact of facial recognition use by police nationwide, revealed that more than 90 percent of the individuals arrested through decisions based on facial recognition in Brazil are Black. In the state of Rio de Janeiro, the percent of unjust arrests based on photographic recognition and involving Black individuals reached 81 percent in 2021, according to data from the Public Defenders Chamber of Rio de Janeiro.
Another problem relates to the lack of openness regarding the project. One of the organisations behind the national campaign “Take My Face Out of Your Aim” and other initiatives calling for a ban on facial recognition, CESeC and other nonprofit bodies are involved in a number of legal actions against governments that choose to roll out the technology for urban security and other purposes.
Last year the São Paulo City Hall held a virtual public consultation over the last two weeks of August, inviting experts to contribute their views, with a single day set aside for people to get their queries answered.
“Participation was limited and the suggestions made were largely ignored,” said Celina Bottino, project director at the Institute for Technology & Society of Rio de Janeiro.
In response to the public pushback, the Smart Sampa tender was updated with a study on the impact of the technology, which acknowledged shortcomings such as a high probability of biases in the facial recognition features, as well as the risk of unauthorised use and exposure of personal data, as well as the likelihood of privacy violations.
To address these risks, the report noted the platform would only consider detections with 90 percent parity, and all alerts issued would be analysed by trained personnel to mitigate injustices, as well as an advanced data protection and access control system.