Hospitals and health care systems are increasingly turning to artificial intelligence to help summarize doctors’ notes and analyze health records.

Powered by AI models trained on troves of text pulled from the internet, chatbots such as ChatGPT and Google’s Bard responded to the researchers’ questions with a range of misconceptions and falsehoods about Black patients, sometimes including fabricated, race-based equations, according to the study published Friday in the academic journal Digital Medicine.

Experts worry these systems could cause real-world harms and amplify forms of medical racism that have persisted for generations as more physicians use chatbots for help with daily tasks such as emailing patients or appealing to health insurers.

The report found that all four models tested — ChatGPT and the more advanced GPT-4, both from OpenAI; Google’s Bard, and Anthropic’s Claude — failed when asked to respond to medical questions about kidney function, lung capacity and skin thickness. In some cases, they appeared to reinforce long-held false beliefs about biological differences between Black and white people that experts have spent years trying to eradicate from medical institutions.

  • Dr. Dabbles@lemmy.worldEnglish
    22·
    1 year ago

    I guess you’re just a big anti AI person and that’s fine.

    Nope, I live in reality and work in the hardware industry. I presume you’re an ML specialist of some sort?

    I understand the limitations of the tech

    It really doesn’t seem like you do based on what you’ve said in this thread.

    especially in its earliest stages when it’s the most unreliable

    Yes, in the 1950s it was indeed unreliable. And here in 2023 it’s still unreliable. Again, based solely on what you’ve said in this thread I don’t think you understand the history, the current state of the art, or the future of any of this work. Let alone limitations.

    But this tech is here and it’s not going anywhere.

    …until a significantly more power efficient development comes along which will make current methods look foolish. Then it’s going away instantly. Also “this tech” has evolved so dramatically over the past 60-something years that even addressing it as “this tech” completely misses the point, and saying it isn’t going anywhere entirely ignores the developments we’ve had.

    Which tech specifically isn’t going anywhere? The hardware? The software? The networks themselves? Using activation functions as a concept in software?

    Just casting all of these unavoidable truths aside

    You don’t understand what you’re talking about, so I don’t think you’re in a position to tell me what is a truth or not.

    No doubt you’re a specialist, though, so I look forward to you describing in detail which “tech” you think isn’t going anywhere and how it’s going to develop in the future.

    • RaincoatsGeorge@lemmy.zipEnglish
      12·
      1 year ago

      Your argument is that ai from the 50s was bad so this is also bad and it will soon be replaced by like… actual ai so this doesn’t matter at all.

      Okie dokie. This has been fun.

      • Dr. Dabbles@lemmy.worldEnglish
        22·
        1 year ago

        Maybe you should read what I wrote. Try answering my questions too. Or maybe you can’t and the snark is a deflection? Yeah, I’m guessing that’s it.