A couple days ago I reported a survey saying that most IT professional are worried about the security of LLMs. They have every right to be. There seems to be an endless number of ways of attacking them. In my forthcoming book, Taming Silicon Valley, I describe two examples. The first sometimes gets an LLM to disgorge private information:
  • mindbleach
    1·
    6 months ago

    The death of LLMs… for stupid purposes, pretending they’ll never be wrong. Morally or factually.

    It’s still a chatbot. It works fine for that. You shouldn’t have been using it for a lot of this shit in the first place. And your attack is getting it to spit out information that’s supposed to be public knowledge. There’s where all the training data came from. (Or where it fucking ought to have come from.) If you treat this technology like a dodgy android in a comedy about the future then it’s great for whatever you’d entrust to that slightly malfunctioning idiot. It has approximate knowledge of many things.

    … is Gary Marcus’s whole career just shitting on AI? This post about Sora is all the oddities I’d point out as mildly amusing and inconvenient, but he phrases each one like a damning condemnation of the entire technology. Like it’s indefensible that you can see signs of how this jawdropping witchcraft works. Like he hasn’t been hyperfixated on it for at least a year, and watched most of his prior objections addressed and surmounted.

    Dude’s riding an “AI is a bubble!” bubble.