Studies Show Medical AI Is Racist. As a Black Woman, This Terrifies Me.
As a Black woman, medical AI scares me, and research indicates I should be afraid.
When I go to the doctor, I expect to encounter a very knowledgeable professional who is focused on my condition and is using their training to find the best treatments. And if they do not know how to treat me, they will do the necessary work to research until they have an answer. I also expect them to use reliable data-driven sources. I am afraid this may not always be the case, especially now with AI so readily available. Doctors are unwittingly and neglectfully falling into a trap that leads to medical bias against Black people. People like me.
It has been widely reported, including by The New York Times, that doctors in training and licensed doctors hold onto racist medical opinions of Black people that were debunked decades ago. These racist myths include believing that our brains are smaller, that our skin is thicker, and that we can withstand more pain, leading to improper treatment strategies being implemented. None of this is borne out by research, but doctors believe these racist theories, and they impact patient care, doing harm to Black patients.
Now after years of hard work to undo the racist theories, Artificial Intelligence (AI) is rearing its ugly head. Researchers studied the most popular AI apps, including Chat GPT, and found that the algorithm's answers to basic medical questions are riddled with falsehoods about Black people and how to treat us medically. This is worrisome since many physicians, medical systems and medical contractors are using AI to do various tasks to make life easier and under the guise of improving patient care.
According to a shocking report published in Fortune, when medical contracting companies are confronted with the claim that their algorithms are racist, they say they are working to root out these biases, but admit this is hard to do since their databases rely on publicly available information on the internet and textbooks, all of which can contain bias. Racism, sexism, homophobia, and ableism are being literally encoded into the large language models that AI programs rely on to generate text.
The bottom line of why these biases are being baked into algorithms is just that: the bottom line. There's an AI venture capital gold rush, and competition is high to get large language models to market and in the hands of sometimes unsuspecting doctors. The money in this industry is driving companies to take shortcuts and put Black and other marginalized patients’ lives at risk.
There has got to be another way to improve doctors' experiences without compromising the health of patients. Right now, medical AI is writing patient notes and reviewing charts to make recommendations. If doctors trust these recommendations as facts, people will be harmed. There will be malpractice due to AI; if it hasn't happened already, it certainly will.
I wonder how I, a Black chronically ill patient who engages with doctors on a regular basis, am supposed to make sure AI is not informing my treatment in a way that is detrimental to my health. There is really no way to determine this without asking how medical AI is involved in your treatment, and, even if you did find out, there is no guarantee you can get the doctor to stop using it. They may not even have a choice but to use it depending on who they work for.
The best I can say is to do your own research. If something doesn't seem right, question it. Be your own best advocate and do not let the doctor run roughshod over you. Ask them how they determined your treatment plan and whether any of it came from computerized recommendations. They might not even realize that the database they are using could contain biases.
AI is a new and growing field. It has some great implications and applications. It can also have some very dangerous ramifications for Black people like me. I don't know where all of this is headed, but those of us who belong to minority and marginalized communities need to be aware and alert.
Stay safe out there.
For more about the negative impact of AI in the medical and disability realm, read about how search algorithms suppress articles written by disabled people and the recording of online therapy sessions to build AI language models.
I am Black, lesbian, disabled, mentally ill, fat, a birth mom, mom and grandmom (grand ma Coco to be exact) and Funny. I am a woman who is constantly fighting for my and your liberation.
I have a history of working for those living at the margins mostly in activist and nonprofit spaces. I currently work in the mental health field serving those who have been convicted of felonies and are in mental heath court. I am also a writer. I write about disabilities, chronic illness, mental health, racial trauma, sexual violence and disordered eating. I am also a public community speaker on the same topics. Hit me up if you need my writing or speaking skills.
Please use she or her pronouns when referring to or about me.
