• Australis13@fedia.io
    link
    fedilink
    arrow-up
    149
    arrow-down
    2
    ·
    4 months ago

    This is why you do staged rollouts of updates… not the entire planet at once.

      • shameless@lemmy.world
        link
        fedilink
        English
        arrow-up
        21
        ·
        4 months ago

        So true, this really highlights the risk of updates impacting critical systems vs critical systems being exposed to critical vulnerabilities. Its a real balancing act.

        • Thann@lemmy.ml
          link
          fedilink
          English
          arrow-up
          1
          ·
          4 months ago

          It actually highlights the risk of having unaudited third party software running on your critical infrastructure

      • cybersandwich@lemmy.world
        link
        fedilink
        English
        arrow-up
        13
        ·
        4 months ago

        I don’t know exactly how crowd strike works, but this sounded like a “virus signatures” update (IE not a software update per se). And thats what caused the issue.

        I think “real time virus protection” is why people use it so they expect the signatures to get updated asap/with little to no human intervention.

        This is a crowd strike epic fail…for how they let their software blue screen systems with a virus signature update.

      • Thann@lemmy.ml
        link
        fedilink
        English
        arrow-up
        2
        arrow-down
        13
        ·
        4 months ago

        No, you run Linux with automatic secutity updates turned on

  • DarkThoughts@fedia.io
    link
    fedilink
    arrow-up
    96
    arrow-down
    1
    ·
    4 months ago

    Can someone in non marketing terms explain what the fuck CrowdStrike Falcon Sensor is? I literally never heard of this company or product before.

    • farcaster@lemmy.worldOP
      link
      fedilink
      English
      arrow-up
      111
      ·
      4 months ago

      It’s basically corporate anti-virus software. Intended to detect and prevent malware.

      • Alimentar@lemm.ee
        link
        fedilink
        English
        arrow-up
        31
        arrow-down
        2
        ·
        4 months ago

        Apparently it’s the next iteration of AI based antivirus where it uses smart algorithms to detect system behaviours and makes assessments on whether they’re malicious or not

        • GreyBeard@lemmy.one
          link
          fedilink
          English
          arrow-up
          26
          ·
          4 months ago

          I know there is a lot of marketing fluff, but yes, it is an EDR. Which means instead of just checking file signatures against a database if known bad stuff, it actually examines what applications do and makes a sort of judgement on if it is acting maliciously or not. I use a similar product. Although the false positives can sometimes be baffling, it honestly can catch a legit program misbehaving.

          On top of that, everything is logged. Every file, network connection, or registry key that every process on the computer touches is logged. That means when something happens, you can see the full and complete list of actions taken by the malicious system. Thus can actually be a drain on the computer, but modern systems handle it well enough.

            • GreyBeard@lemmy.one
              link
              fedilink
              English
              arrow-up
              5
              ·
              4 months ago

              SentinelOne. They are more reseller/MSP friendly, but the product is very similar to CrowdStrike.

              • ditty@lemm.ee
                link
                fedilink
                English
                arrow-up
                2
                ·
                4 months ago

                We also use S1 and while it does often flag false positives, that’s a whole heck of a lot better than the alternative. Also I have not noticed it being very resource intensive.

                • GreyBeard@lemmy.one
                  link
                  fedilink
                  English
                  arrow-up
                  3
                  ·
                  4 months ago

                  It’s overhead is more subtle than task manager can tell. Because of all its watching and monitoring, it slows down applications themselves. Task take longer. Sometime it is by a trivial amount, but I’ve been able to measure a notable difference in some task with and without S1, even if task manager says all is well.

        • sukotai@lemmy.world
          link
          fedilink
          English
          arrow-up
          11
          arrow-down
          7
          ·
          4 months ago

          obviously, A.I consider microsoft as a malicious software. Sometimes, A.I is very accurate 😁

      • xavier666@lemm.ee
        link
        fedilink
        English
        arrow-up
        1
        ·
        4 months ago

        Can you tell whether this update was delivered by Crowdstrike’s own update delivery pipeline of via Window’s update pipeline?

    • send_me_your_ink@lemmynsfw.com
      link
      fedilink
      English
      arrow-up
      4
      ·
      4 months ago

      It’s software put on every machine so that the company can quickly isolate it if/when something bad happens (or it falls out of security compliance). To do this is requires a constant Internet connection, insanely high privileges on the machine and frequent updates to be appraised of risks.

      That risk update went off the rails and into the next state.

  • rockSlayer@lemmy.world
    link
    fedilink
    English
    arrow-up
    64
    ·
    4 months ago

    I work in QA on the night shift at a video game company. It was absolute chaos at work tonight lmao we only had a grand total of 6 working PCs between all of us

  • RememberTheApollo_@lemmy.world
    link
    fedilink
    English
    arrow-up
    58
    arrow-down
    6
    ·
    edit-2
    4 months ago

    Company spyware. We have that on our devices. They used to have an “about” stored locally on the app, but removed it and a web connection is required to view the docs. Basically says it downloads/sees everything on your device and checks for threats. Thing is a few people have been fired for having things in their devices they shouldn’t. I didn’t ask what it was, nor did I hear how these things were “threats”, but nonetheless they were fired. Too many people treat company hardware like “free device, bro!” and put all sorts of personal stuff on the device. Most industries it’s probably not too big of a deal, but for mine if there’s an incident that happens when you were busy watching Netflix or something instead of doing your job you’re fucked. First thing they’ll do is check your device and crowdstrike to see what you were doing, and even if you weren’t watching Netflix all your personal data will be exposed.

    • BarbecueCowboy@lemmy.world
      link
      fedilink
      English
      arrow-up
      20
      ·
      4 months ago

      They definitely could, but most cybersecurity departments are paid too much to worry about minor items like that. If HR tells us to look into a specific user and gets the proper approvals so that everything is in compliance, we’ll definitely get someone on the team to do it, but otherwise if we happen to see evidence of unapproved usage, we’re mostly going to overlook it unless it could lead to something dangerous to your machine or the company as a whole.

      EDRs like Crowdstrike can see very very nearly everything you do though, definitely everything you would care about.

  • fuzzywombat@lemmy.world
    link
    fedilink
    English
    arrow-up
    49
    ·
    4 months ago

    Yikes. I feel sorry for all the help desk and support staff that has to deal with this chaotic mess all day.

      • WhatAmLemmy@lemmy.world
        link
        fedilink
        English
        arrow-up
        16
        arrow-down
        1
        ·
        4 months ago

        What kind of criminally incompetent psychopath rolls out a global update on a fucking Friday afternoon?

        Is the CEO of CrowdStrike Satan?

        • catloaf@lemm.ee
          link
          fedilink
          English
          arrow-up
          12
          ·
          4 months ago

          They push updates every day. Attackers don’t take Fridays off.

        • Excrubulent@slrpnk.net
          link
          fedilink
          English
          arrow-up
          2
          ·
          edit-2
          4 months ago

          I have heard that Friday afternoon can be better because it gives you a full weekend to put out any fires before business hours start again.

          That’s assuming it’s a small error that doesn’t roll on into the week anyway.

    • worldeater@lemmy.ml
      link
      fedilink
      English
      arrow-up
      7
      ·
      4 months ago

      Yup, my phone is nonstop going off with slack messages and tickets. Time to mute it for now

  • falx@lemmy.ml
    link
    fedilink
    English
    arrow-up
    36
    arrow-down
    2
    ·
    4 months ago

    What a striking name… CrowdStrike heh. They definitely live up to it!

    • Munkisquisher@lemmy.nz
      link
      fedilink
      English
      arrow-up
      53
      ·
      4 months ago

      Won’t take that long, security researchers are already decompiling the update to see if it was malicious or incompetence.

      • Balinares@pawb.social
        link
        fedilink
        English
        arrow-up
        13
        arrow-down
        33
        ·
        4 months ago

        You won’t find the incompetence in the software no matter what.

        If you fail to assume that the software contains issues – if you fail to understand that your software is made by humans and humans make mistakes, not because they’re bad but because they’re human – and if you fail to implement mechanisms to feel gracefully with inevitable failures, THAT is the incompetence.

        Failures are systemic.

        • Munkisquisher@lemmy.nz
          link
          fedilink
          English
          arrow-up
          33
          ·
          4 months ago

          Oh yes I make those failures myself, testing and staging and limited release schedules save my human failures from breaking the world

          • Balinares@pawb.social
            link
            fedilink
            English
            arrow-up
            3
            arrow-down
            2
            ·
            4 months ago

            One funny thing about humans is that they aren’t just gloriously fallible: they also get quite upset when that’s pointed out. :)

            Unfortunately, that’s also how you end up with blameful company cultures that actively make reliability worse, because then your humans make just the same amounts of mistakes, but they hide them – and you never get a chance to evolve your systems with the safeguards that would have prevented these.

      • RubberDuck@lemmy.world
        link
        fedilink
        English
        arrow-up
        7
        arrow-down
        2
        ·
        edit-2
        4 months ago

        A lot of companies will get calls from the “provider” offering help with mitigation so that additional features can also be installed. This is a time to be extra wary.

        Edited: spelling

      • Evil_incarnate@lemm.ee
        link
        fedilink
        English
        arrow-up
        2
        arrow-down
        2
        ·
        4 months ago

        Think big. This may have had a target. But hitting the target only wasn’t possible so everyone got hit.

        It’s possible those responsible only had this weapon that was capable of hitting the target, maybe the plan was to disrupt world flights to make someone late tomorrow, who knows. Maybe poo-tin or Xi-the-Pooh wanted to hit America and its allies?

        • 0x0@programming.dev
          link
          fedilink
          English
          arrow-up
          5
          ·
          edit-2
          4 months ago

          A state actor will use more precise techniques to attack specific targets. Think SolarWinds and Stuxnet.

          Ransomware doesn’t apply here and tends to depend on phishing first anyway.

          Even terrorists have specific targets in mind.

          So it’s either Bond villains or incompetence.

          Edit: The only way i can fit your comment would be an incompetent script kiddy. Even then, doesn’t make sense as all systems were not directly attacked, as would be the case, but rather through what would have to be a side-channel attack, so no.

    • UnderpantsWeevil@lemmy.world
      link
      fedilink
      English
      arrow-up
      18
      arrow-down
      3
      ·
      4 months ago

      Never attribute to maliciousness that which can be explained by incompetence.

      That said, I’m sure the Crowdstrike CEO is currently on a phone call with three of their pet Congresscritters asking if they can get a $100M grant to harden their systems against Russia/China/NKorea/Antifa interference right now.