• drathvedro@lemm.ee
    link
    fedilink
    arrow-up
    36
    arrow-down
    1
    ·
    1 年前

    The biggest offender is, surprisingly, cloudflare. They will straight up refuse to serve you any site if your user agent is not one of the mainstream ones. It’s not even “find the traffic light to prove you’re human”, but a page basically saying “fuck you, go away”.

      • lseif@sopuli.xyz
        link
        fedilink
        arrow-up
        36
        arrow-down
        2
        ·
        1 年前

        what is more likely to be a bot? a unique and trackable useragent for a semi-niche browser engine, or a vanilla Chromium+Windows which half of everyone uses ?

        • lud@lemm.ee
          link
          fedilink
          arrow-up
          3
          arrow-down
          1
          ·
          1 年前

          Most semi and fully legitimate bots use a custom user agent.

          • lseif@sopuli.xyz
            link
            fedilink
            arrow-up
            5
            ·
            1 年前

            what about malicious/unwanted bots? if cloudflare is trying to block bots, the bots will want to not look like bots. the easiest way to do that is to use a common user agent.