Left Menu
Development News Edition

Now, AI can detect racial, gender discrimination

PTI | Washington DC | Updated: 11-07-2019 12:37 IST | Created: 11-07-2019 12:28 IST
Now, AI can detect racial, gender discrimination
Image Credit: Pixabay

Scientists have developed a new artificial intelligence (AI) tool for detecting unfair discrimination -- such as on the basis of race or gender. Preventing unfair treatment of individuals on the basis of race, gender or ethnicity, for example, has been a long-standing concern of civilized societies.

However, detecting such discrimination resulting from decisions, whether by human decision-makers or automated AI systems, can be extremely challenging. "Artificial intelligence systems -- such as those involved in selecting candidates for a job or for admission to a university -- are trained on large amounts of data," said Vasant Honavar, a professor at Pennsylvania State University (Penn State) in the US.

"But if these data are biased, they can affect the recommendations of AI systems," Honavar said. He said if a company historically has never hired a woman for a particular type of job, then an AI system trained on this historical data will not recommend a woman for a new job.

"There's nothing wrong with the machine learning algorithm itself," said Honavar. "It's doing what it's supposed to do, which is to identify good job candidates based on certain desirable characteristics. But since it was trained on historical, biased data it has the potential to make unfair recommendations," he said.

The team created an AI tool for detecting discrimination with respect to a protected attribute, such as race or gender, by human decision-makers or AI systems. "We can minimize gender-based discrimination in salary if we ensure that similar men and women receive similar salaries," said Aria Khademi, a graduate student at Penn State.

The researchers tested their method using various types of available data, such as income data from the US Census Bureau to determine whether there is gender-based discrimination in salaries. They also tested their method using the New York City Police Department's stop-and-frisk program data to determine whether there is discrimination against people of color in arrests made after stops.

"We analyzed an adult income data set containing salary, demographic and employment-related information for close to 50,000 individuals," said Honavar. "We found evidence of gender-based discrimination in salary. Specifically, we found that the odds of a woman having a salary greater than USD 50,000 per year is only one-third that for a man.

"This would suggest that employers should look for and correct, when appropriate, gender bias in salaries," he said. Although the team's analysis of the New York stop-and-frisk dataset -- which contains demographic and other information about drivers stopped by the New York City police force -- revealed evidence of possible racial bias against Hispanics and African American individuals, it found no evidence of discrimination against them on average as a group.

"You cannot correct for a problem if you don't know that the problem exists," said Honavar. "To avoid discrimination on the basis of race, gender or other attributes you need effective tools for detecting discrimination. Our tool can help with that," he said.


TRENDING

OPINION / BLOG / INTERVIEW

5G will be the key driving force for COVID-19 recovery: Here's how?

... ...

Canada’s COVID-19 pitfalls highlight need for integrated health information system

In the globalized world of today where outbreaks can spread far and wide within a matter of days, a global-level integrated health information system is ideal but Canadas provincial barriers show that the country lags much behind in deployi...

Pandemic must be impetus, not obstacle, for clean water access

To make matters worse, there are suspicions that the inadequacy of wastewater treatment methods in California, the rest of the USA, and indeed around the world may help to propagate the disease even more widely. ...

3D printing and the future of manufacturing post COVID-19

The on-demand, customizable, and localized manufacturing of product components facilitated by 3D printing has the potential to redefine manufacturing but there are certain technical, mechanical, and legal limitations that, unless ...

Videos

Latest News

Children in Beirut suffer from trauma after deadly blast

When the huge explosion ripped through Beirut last week, it shattered the glass doors near where 3-year-old Abed Achi was playing with his Lego blocks. He suffered a head injury and cuts on his tiny arms and feet, and he was taken to the em...

Aircraft Accident Investigation Bureau (AAIB) chief says too premature to make initial assessment of Kozhikode plane crash.

Aircraft Accident Investigation Bureau AAIB chief says too premature to make initial assessment of Kozhikode plane crash....

Chhattisgarh: Four Naxals killed in encounter with security forces in Sukma

Four Naxals were killed in an exchange of fire in Sukma district of Chhattisgarh, the police said. The gun-battle between the security personnel and the Naxals took place in the forest area of Jagargunda in the Sukma district.Four Naxals ki...

Soccer-Adelaide in A-League limbo after last gasp equaliser

Adelaide Uniteds players will become supporters this week after a sterling fight back in their last match of the regular season kept their A-League playoff hopes alive but reliant on other results. Ben Halloran scored a stoppage time equali...

Give Feedback