No, MIT’s new AI cannot determine a person’s race based on medical images

MIT researchers recently made one of the boldest claims we’ve seen yet regarding artificial intelligence: They believe they’ve built an AI that can identify a person’s race using only medical images. And according to the popular mediathey have no idea how it works!

Secure. And I’d like to sell you an NFT of the Brooklyn Bridge.

Let’s be clear in advance, per the team’s newspaperthe model can to predict be a person self-reported race:

In our study, we show that standard AI deep learning models can be trained to predict race based on high-performance medical images across multiple imaging modalities.

Greetings humanoids

Sign up now for a weekly recap of our favorite AI stories

Prediction and identification are two completely different things. If a prediction is wrong, it is still a prediction. If an identification is wrong, it is a wrong identification. These are important awards.

AI models can be refined to predict anything, even concepts that are not real.

Here’s an old analogy I like to use in these situations:

I can predict with 100% accuracy how many lemons in a lemon tree are aliens from another planet.

Since I’m the only person who can see the aliens in the lemons, I’m what you call a “database.”

I could stand there, next to your AI, pointing to all the lemons with aliens in them. The AI ​​would try to figure out what it is with the lemons I’m pointing at, which makes me think there are aliens in them.

Eventually, the AI ​​would look at a new lemon tree and try to guess which lemons I think contained aliens.

If it were 70% accurate to guess that, it would still be 0% accurate to determine which lemons contain aliens. Because lemons don’t contain aliens.

In other words, you can train an AI to predict anything as long as you:

  • Don’t give it the opportunity to say, “I don’t know.”
  • Continue tuning the parameters of the model until you get the answer you want.

As accurate as an AI system is at predicting a label, if it can’t demonstrate how it arrived at its prediction, those predictions are useless for identification purposes, especially when it comes to matters pertaining to individual people.

Furthermore, claims of “accuracy” don’t mean what the media seem to think they do when it comes to these kinds of AI models.

The MIT model achieves an accuracy of less than 99% on labeled data. This means that in the wild (looking at images without labels), we can never be sure that the AI ​​made the right judgment unless a human judges the results.

Even with 99% accuracy, MIT’s AI would still mislabel 79 million people if it were given a database with an image for every living human being. And worse, we definitely wouldn’t know which 79 million people it mislabeled unless we go to all the 7.9 billion people on the planet and ask them to confirm the AI’s assessment of their particular image. This would defeat the purpose of using AI in the first place.

Most importantly, teaching an AI to identify the labels in a database is a trick that can be applied to any database of all labels† It is not a method by which an AI can determine or identify a specific object in a database; it just tries to predict – guess – what label the human developers were using.

The MIT team concluded in their paper that their model could be dangerous in the wrong hands:

The results of our study highlight that the ability of AI deep learning models to predict self-reported race is of no importance per se.

Our finding that AI can accurately predict self-reported race even from damaged, cropped and noiseless medical images, often when clinical experts cannot, creates a huge risk for all model implementations in medical imaging.

It is important for AI developers to consider the potential risks of their creations. But this particular warning is in reality unfounded.

The model the MIT team built can achieve benchmark accuracy on large databases, but as explained above, there is absolutely no way to tell if the AI ​​is correct unless you already know the ground truth.

In short, MIT warns us of the possibility for evil doctors and medical technicians to practice racial discrimination on a large scale, using a similar system.

But this AI cannot determine race. It predicts labels in specific data sets. The only way this model (or a similar one) can be used to discriminate is with a wide net, and only if the discriminator doesn’t really care how many times the machine gets it wrong.

The only thing you can be sure of is that you couldn’t trust an individual result without checking it again for the truth. And the more images the AI ​​processes, the more mistakes it will make.

In summary, MIT’s ‘new’ AI is nothing more than a magician’s illusion. It’s a good one, and models like this are often incredibly useful when it’s not that important to get things right, but there’s no reason to believe that bad actors can use this as a race detector.

MIT could apply the exact same modeI to a grove of lemon trees and using the database of labels I created it can be trained to predict which lemons contain aliens with 99% accuracy.

This AI can only predict labels. It does not identify a race.