Generative AI may be riddled with hallucinations, misinformation, and bias, but that didn't stop over half of respondents in a recent global study from saying that they would use this nascent technology for sensitive areas like financial planning and medical advice.

New study shows large language models have high toxic probabilities and leak private information::Generative AI may be riddled with hallucinations, misinformation, and bias, but that didn’t stop over half of respondents in a recent global study from saying that they would use this nascent technology for sensitive areas …

  • @KerfuffleEnglish
    1•10 months ago

    The problem is not really the LLM itself - it’s how some people are trying to use it.

    This I can definitely agree with.

    ChatGPT cannot discern between instructions from the developer and those from the user

    I don’t know about ChatGPT, but this problem probably isn’t really that hard to deal with. You might already know text gets encoded to token ids. It’s also possible to have special token ids like start of text, end of text, etc. Using those special non-text token ids and appropriate training, instructions can be unambiguously separated from something like text to summarize.

    The bad summary gets circulated around to multiple other sites by users and automated scraping, and now there’s a real mess of misinformation out there.

    Ehh, people do that themselves pretty well too. The LLM possibly is more susceptible to being tricked but people are more likely to just do bad faith stuff deliberately.

    Not really because of this specific problem, but I’m definitely not a fan of auto summaries (and bots that wander the internet auto summarizing stuff no one actually asked them to). I’ve seen plenty of examples where the summary is wrong or misleading without any weird stuff like hidden instructions.