If you think this post would be better suited in a different community, please let me know.


Topics could include (this list is not intending to be exhaustive — if you think something is relevant, then please don’t hesitate to share it):

  • Moderation
  • Handling of illegal content
  • Server structure (system requirements, configs, layouts, etc.)
  • Community transparency/communication
  • Server maintenance (updates, scaling, etc.)

Cross-posts
  1. https://sh.itjust.works/post/27913098
  • KalciferOP
    link
    fedilink
    English
    arrow-up
    13
    arrow-down
    1
    ·
    2 days ago

    Run the software that scans images for CSAM.

    Which software is that?

    • walden@sub.wetshaving.social
      link
      fedilink
      English
      arrow-up
      14
      ·
      2 days ago

      It’s called Lemmy-Safety of Fedi-Safety depending on where you look.

      One thing to note, I wasn’t able to get it running on a VPS because it requires some sort of GPU.

      • KalciferOP
        link
        fedilink
        English
        arrow-up
        10
        ·
        2 days ago

        One thing to note, I wasn’t able to get it running on a VPS because it requires some sort of GPU.

        This is good to know. I know that you can get a VPS with a GPU, but they’re usually rather pricey. I wonder if there’s one where the GPU’s are shared, and you only get billed by how much the GPU is used. So if there is an image upload, the GPU would kick on to check it, you get billed for that GPU time, then it turns off and waits for the next image upload.

        • pe1uca@lemmy.pe1uca.dev
          link
          fedilink
          English
          arrow-up
          5
          ·
          2 days ago

          I don’t think there are services like that, since usually this means deploying and destructing an instance, which takes a few minutes (if you just turn off the instance you still get billed).
          Probably the best option would be to have a snapshot, which costs way less than the actual instance, and create from it each day or so yo run on the images since it was last destroyed.

          This is kind of what I do with my media collection, I process it on my main machine with a GPU, and then just serve it from a low-power one with Jellyfin.

          • KalciferOP
            link
            fedilink
            English
            arrow-up
            2
            ·
            edit-2
            1 day ago

            create from it each day or so yo run on the images since it was last destroyed.

            Unfortunately, for this usecase, the GPU needs to be accessible in real time; there is a 10 second window when an image is posted for it to be processed [1].

            References
            1. “I just developed and deployed the first real-time protection for lemmy against CSAM!”. @[email protected]. [email protected]. Divisions by zero. Published: 2023-09-20T08:38:09Z. Accessed: 2024-11-12T01:28Z. https://lemmy.dbzer0.com/post/4500908.
              • §“For lemmy admins:”

                […]

                • fedi-safety must run on a system with GPU. The reason for this is that lemmy provides just a 10-seconds grace period for each upload before it times out the upload regardless of the results. [1]

                […]

            • db0@lemmy.dbzer0.com
              link
              fedilink
              English
              arrow-up
              1
              ·
              edit-2
              1 day ago

              You can actually run it in async model without pictrs safety and just have it scan your newly uploaded images directly from storage. It just doesn’t prevent upload this way, just deletes them.

          • KalciferOP
            link
            fedilink
            English
            arrow-up
            1
            ·
            1 day ago

            Probably the best option would be to have a snapshot

            Could you point me towards some documentation so that I can look into exactly what you mean by this? I’m not sure I understand the exact procedure that you are describing.

        • db0@lemmy.dbzer0.com
          link
          fedilink
          English
          arrow-up
          1
          ·
          2 days ago

          The software is setup in such a way that you can run it on your pc if you have a local gpu. It only needs like 2 gb vram

          • KalciferOP
            link
            fedilink
            English
            arrow-up
            1
            ·
            edit-2
            2 days ago

            That is a cool feature, but that would mean that all of the web traffic would get returned to my local network (assuming that the server is set up on a remote VPS), which I really don’t want to have happen. There is also the added downtime potential cause by the added point of failure of the GPU being hosted in a much more volatile environment (ie not, for example, a tier 3 data center).

            • db0@lemmy.dbzer0.com
              link
              fedilink
              English
              arrow-up
              2
              ·
              2 days ago

              Not all web traffic, just the images to check. With any decent bandwidth, it shouldn’t be an issue for most. It also setup in such a way as to not cause a downtime if the checker goes down.

              • KalciferOP
                link
                fedilink
                English
                arrow-up
                1
                ·
                1 day ago

                It also setup in such a way as to not cause a downtime if the checker goes down.

                Oh? Would the fallback be that it simply doesn’t do a check? Or perhaps it could disable image uploads if the checker is down? Something else? Presumably, this would be configurable.

              • KalciferOP
                link
                fedilink
                English
                arrow-up
                1
                ·
                1 day ago

                With any decent bandwidth, it shouldn’t be an issue for most.

                It’s not only the bandwidth; I just fundamentally don’t relish the idea of public traffic being directed to my local network.

              • KalciferOP
                link
                fedilink
                English
                arrow-up
                1
                ·
                1 day ago

                Not all web traffic, just the images to check.

                Ah, yeah, my bad this was a lack of clarity on my part; I meant all image traffic.