• Australis13
    link
    fedilink
    1472 months ago

    This is why you do staged rollouts of updates… not the entire planet at once.

      • shameless
        link
        fedilink
        English
        212 months ago

        So true, this really highlights the risk of updates impacting critical systems vs critical systems being exposed to critical vulnerabilities. Its a real balancing act.

        • @[email protected]
          link
          fedilink
          English
          12 months ago

          It actually highlights the risk of having unaudited third party software running on your critical infrastructure

      • @[email protected]
        link
        fedilink
        English
        132 months ago

        I don’t know exactly how crowd strike works, but this sounded like a “virus signatures” update (IE not a software update per se). And thats what caused the issue.

        I think “real time virus protection” is why people use it so they expect the signatures to get updated asap/with little to no human intervention.

        This is a crowd strike epic fail…for how they let their software blue screen systems with a virus signature update.

  • DarkThoughts
    link
    fedilink
    962 months ago

    Can someone in non marketing terms explain what the fuck CrowdStrike Falcon Sensor is? I literally never heard of this company or product before.

    • @[email protected]OP
      link
      fedilink
      English
      1112 months ago

      It’s basically corporate anti-virus software. Intended to detect and prevent malware.

      • Alimentar
        link
        fedilink
        English
        292 months ago

        Apparently it’s the next iteration of AI based antivirus where it uses smart algorithms to detect system behaviours and makes assessments on whether they’re malicious or not

        • GreyBeard
          link
          fedilink
          English
          262 months ago

          I know there is a lot of marketing fluff, but yes, it is an EDR. Which means instead of just checking file signatures against a database if known bad stuff, it actually examines what applications do and makes a sort of judgement on if it is acting maliciously or not. I use a similar product. Although the false positives can sometimes be baffling, it honestly can catch a legit program misbehaving.

          On top of that, everything is logged. Every file, network connection, or registry key that every process on the computer touches is logged. That means when something happens, you can see the full and complete list of actions taken by the malicious system. Thus can actually be a drain on the computer, but modern systems handle it well enough.

            • GreyBeard
              link
              fedilink
              English
              52 months ago

              SentinelOne. They are more reseller/MSP friendly, but the product is very similar to CrowdStrike.

              • ditty
                link
                fedilink
                English
                22 months ago

                We also use S1 and while it does often flag false positives, that’s a whole heck of a lot better than the alternative. Also I have not noticed it being very resource intensive.

                • GreyBeard
                  link
                  fedilink
                  English
                  32 months ago

                  It’s overhead is more subtle than task manager can tell. Because of all its watching and monitoring, it slows down applications themselves. Task take longer. Sometime it is by a trivial amount, but I’ve been able to measure a notable difference in some task with and without S1, even if task manager says all is well.

        • sukotai
          link
          fedilink
          English
          42 months ago

          obviously, A.I consider microsoft as a malicious software. Sometimes, A.I is very accurate 😁

      • @[email protected]
        link
        fedilink
        English
        12 months ago

        Can you tell whether this update was delivered by Crowdstrike’s own update delivery pipeline of via Window’s update pipeline?

    • @[email protected]
      link
      fedilink
      English
      42 months ago

      It’s software put on every machine so that the company can quickly isolate it if/when something bad happens (or it falls out of security compliance). To do this is requires a constant Internet connection, insanely high privileges on the machine and frequent updates to be appraised of risks.

      That risk update went off the rails and into the next state.

  • @[email protected]
    link
    fedilink
    English
    642 months ago

    I work in QA on the night shift at a video game company. It was absolute chaos at work tonight lmao we only had a grand total of 6 working PCs between all of us

  • @[email protected]
    link
    fedilink
    English
    52
    edit-2
    2 months ago

    Company spyware. We have that on our devices. They used to have an “about” stored locally on the app, but removed it and a web connection is required to view the docs. Basically says it downloads/sees everything on your device and checks for threats. Thing is a few people have been fired for having things in their devices they shouldn’t. I didn’t ask what it was, nor did I hear how these things were “threats”, but nonetheless they were fired. Too many people treat company hardware like “free device, bro!” and put all sorts of personal stuff on the device. Most industries it’s probably not too big of a deal, but for mine if there’s an incident that happens when you were busy watching Netflix or something instead of doing your job you’re fucked. First thing they’ll do is check your device and crowdstrike to see what you were doing, and even if you weren’t watching Netflix all your personal data will be exposed.

    • @[email protected]
      link
      fedilink
      English
      202 months ago

      They definitely could, but most cybersecurity departments are paid too much to worry about minor items like that. If HR tells us to look into a specific user and gets the proper approvals so that everything is in compliance, we’ll definitely get someone on the team to do it, but otherwise if we happen to see evidence of unapproved usage, we’re mostly going to overlook it unless it could lead to something dangerous to your machine or the company as a whole.

      EDRs like Crowdstrike can see very very nearly everything you do though, definitely everything you would care about.

  • @[email protected]
    link
    fedilink
    English
    492 months ago

    Yikes. I feel sorry for all the help desk and support staff that has to deal with this chaotic mess all day.

      • @[email protected]
        link
        fedilink
        English
        152 months ago

        What kind of criminally incompetent psychopath rolls out a global update on a fucking Friday afternoon?

        Is the CEO of CrowdStrike Satan?

        • @[email protected]
          link
          fedilink
          English
          2
          edit-2
          2 months ago

          I have heard that Friday afternoon can be better because it gives you a full weekend to put out any fires before business hours start again.

          That’s assuming it’s a small error that doesn’t roll on into the week anyway.

    • @[email protected]
      link
      fedilink
      English
      72 months ago

      Yup, my phone is nonstop going off with slack messages and tickets. Time to mute it for now

  • RubberDuck
    link
    fedilink
    English
    302 months ago

    This is going to turn out it was a hack in several months right?

    • @[email protected]
      link
      fedilink
      English
      532 months ago

      Won’t take that long, security researchers are already decompiling the update to see if it was malicious or incompetence.

      • RubberDuck
        link
        fedilink
        English
        262 months ago

        This is going to be Solarwinds all over again I can just smell it.

      • @[email protected]
        link
        fedilink
        English
        -202 months ago

        You won’t find the incompetence in the software no matter what.

        If you fail to assume that the software contains issues – if you fail to understand that your software is made by humans and humans make mistakes, not because they’re bad but because they’re human – and if you fail to implement mechanisms to feel gracefully with inevitable failures, THAT is the incompetence.

        Failures are systemic.

        • @[email protected]
          link
          fedilink
          English
          332 months ago

          Oh yes I make those failures myself, testing and staging and limited release schedules save my human failures from breaking the world

          • @[email protected]
            link
            fedilink
            English
            12 months ago

            One funny thing about humans is that they aren’t just gloriously fallible: they also get quite upset when that’s pointed out. :)

            Unfortunately, that’s also how you end up with blameful company cultures that actively make reliability worse, because then your humans make just the same amounts of mistakes, but they hide them – and you never get a chance to evolve your systems with the safeguards that would have prevented these.

      • RubberDuck
        link
        fedilink
        English
        5
        edit-2
        2 months ago

        A lot of companies will get calls from the “provider” offering help with mitigation so that additional features can also be installed. This is a time to be extra wary.

        Edited: spelling

      • @[email protected]
        link
        fedilink
        English
        02 months ago

        Think big. This may have had a target. But hitting the target only wasn’t possible so everyone got hit.

        It’s possible those responsible only had this weapon that was capable of hitting the target, maybe the plan was to disrupt world flights to make someone late tomorrow, who knows. Maybe poo-tin or Xi-the-Pooh wanted to hit America and its allies?

        • @[email protected]
          link
          fedilink
          English
          5
          edit-2
          2 months ago

          A state actor will use more precise techniques to attack specific targets. Think SolarWinds and Stuxnet.

          Ransomware doesn’t apply here and tends to depend on phishing first anyway.

          Even terrorists have specific targets in mind.

          So it’s either Bond villains or incompetence.

          Edit: The only way i can fit your comment would be an incompetent script kiddy. Even then, doesn’t make sense as all systems were not directly attacked, as would be the case, but rather through what would have to be a side-channel attack, so no.

    • @[email protected]
      link
      fedilink
      English
      152 months ago

      Never attribute to maliciousness that which can be explained by incompetence.

      That said, I’m sure the Crowdstrike CEO is currently on a phone call with three of their pet Congresscritters asking if they can get a $100M grant to harden their systems against Russia/China/NKorea/Antifa interference right now.

      • @[email protected]
        link
        fedilink
        English
        32 months ago

        If I ever become a super 1337 hacker I’m going to setup all of my exploits to look like it could be regular mismanagement, thanks for the advice

      • @[email protected]
        link
        fedilink
        English
        22 months ago

        While being simultaneously gang…handled by the unnamed 3-letter agencies representatives