As the reddit mods gets ready for the June 12-14 black-out, there some anticipation that an influx in user base will shift over to many of the lemmy instances as user seek out a home to post their internet memes and discuss their interests.

In anticipation of this increased volume I will be growing our current instance from

  • 16 CPU
  • 8 GB ram

to

  • 24 CPU
  • 64 GB ram

This server is currently equipped with SSDs that are configured in a raid 10 array (NVMEs will come in the next gen that get deployed)

Earlier today I also configured some monitoring that I’ll be watching closely in order to have a better understanding on how the lemmy platform does under stress (for science!)

I’ll be sharing graphs and some other insights in this thread for everyone that is interested. Feel free to ask anything you might be interested in knowing more of!

EDIT: I’ll be posting and updating the graphs in this main post periodically! Last updated: 6:21AM ET June 12th

CPU - 48 hours

Memory - 48 hours

Network - 48 hours

Load Average - 48 hours

System Disk I/O - 48 hours

  • ArtisanalRuntime
    link
    fedilink
    English
    arrow-up
    32
    ·
    1 year ago

    For the icon: How about a Satisfied Seal? Because when sh.itjust.works, you know that feel! satisfied_seal

      • ArtisanalRuntime
        link
        fedilink
        English
        arrow-up
        9
        ·
        1 year ago

        Let me know if you want me to tweak the design any and I can send an SVG as well. No copyright/usage restrictions - free to use or modify as you wish.

    • TheDudeOPMA
      link
      fedilink
      English
      arrow-up
      9
      ·
      1 year ago

      That would be great! I checked your link and will work on completing the steps shortly :)

      • noodle
        link
        fedilink
        English
        arrow-up
        4
        ·
        1 year ago

        let’s goo! Great instance that deserves some growth! 🚀

      • Acheron
        link
        fedilink
        English
        arrow-up
        4
        ·
        1 year ago

        Just joined you from that featured site, it just works. People are complaining about how hard it is to switch, I genuinely don’t know what the difficulty is

        • planish
          link
          fedilink
          English
          arrow-up
          3
          ·
          1 year ago

          This is one of the easier instances to join; most of even the big ones have a manual approval step/application process that slows things down.

          @[email protected] that might be a good thing to note about the different instances? Or would that just tell the spammers where to spam?

          • God
            link
            fedilink
            English
            arrow-up
            4
            ·
            1 year ago

            Saw some dude who had spent a week on and off trying every couple of days to join and all the servers rejected him and he had given up lol. Most users are lazy fucks. If you want quality users, it’s okay to hide the goods and let the best people find them. If you want mass adoption you need to adapt to the lowest common demoninator.

            • roomey
              link
              fedilink
              English
              arrow-up
              2
              ·
              1 year ago

              Well that and you can’t even tell if you are rejected, and why. I waited over 24hours for one, and I still can’t tell if I was rejected or what. It is a nice feeling to just be able to use this without jumping through hoops

              • God
                link
                fedilink
                English
                arrow-up
                2
                ·
                1 year ago

                In the end I was accepted. I noticed because I tried to log in and succeeded. Now I have two accounts I guess.

        • Toebeans
          link
          fedilink
          English
          arrow-up
          2
          ·
          1 year ago

          Tbf this is the sh.it just.works instance - some of the others a more hit and miss

    • God
      link
      fedilink
      English
      arrow-up
      6
      ·
      1 year ago

      Joined from recommended a bit less than an hour ago. Tried lemmy.ml, got rejected, went for this one cuz it was top recommended and I figured with the name and the user count the likelihood of rejection was smaller.

  • pax
    link
    fedilink
    English
    arrow-up
    13
    ·
    1 year ago

    I am from reddit migration. it’s 100000000000 times better, e.g. more accessible with my screenreader, on reddit, I needed to use special app, called dystopia which handled voiceover more gracefully.

  • TheDudeOPMA
    link
    fedilink
    English
    arrow-up
    8
    ·
    edit-2
    1 year ago

    A little update for all of you interested. I allocated the additional resources to the VM and will post some updated graphs once they update with the new configurations.

    For those who are like me and like looking at graphs here are some prior to the upgrade.

    CPU - 48 hours

    Memory - 48 hours

    Network - 48 hours

    • this
      link
      fedilink
      English
      arrow-up
      4
      ·
      1 year ago

      so if I’m reading this right, less than 10% CPU capacity and about 1/8th ram at peak times, before upgrades? gotta give you credit where it’s due, that thing looks ready to take some abuse.

      • TheDudeOPMA
        link
        fedilink
        English
        arrow-up
        3
        ·
        1 year ago

        The only part I don’t have graphed yet is the disk IOs. I’m going to need to invest a little more time to get that metric going captured.

        Say you through 1000s of active users to this instance… what would cause the bottle neck first? CPU, memory, network, disk? I’m thinking probably disk due to database optimization that need to be reworked on lemmy afterwards CPU and then memory.

        • Echolot
          link
          fedilink
          English
          arrow-up
          1
          ·
          1 year ago

          Yeah disk iops is probably is gonna be the biggest hurdle, but so far so good…

    • Srootus
      link
      fedilink
      English
      arrow-up
      3
      ·
      1 year ago

      Hello Lemmy! I’ve just made an account to get my foot in the door incase the Reddit execs don’t roll back after the blackout, let’s sit back and watch the fireworks.

    • TheDudeOPMA
      link
      fedilink
      English
      arrow-up
      8
      ·
      1 year ago

      Sure lets see how far we can get this server going!

  • Condodog
    link
    fedilink
    English
    arrow-up
    5
    ·
    1 year ago

    Just discovered this place. Nice to see you guys are getting ready for the influx of users.

  • manifexM
    link
    fedilink
    English
    arrow-up
    5
    ·
    1 year ago

    Thanks @TheDude. I work in enterprise network/systems/cloud operations as a network/security engineer. Would love to contribute monetarily or with time.

  • phase_change
    link
    fedilink
    English
    arrow-up
    5
    ·
    1 year ago

    I’d love to see a few additional charts with the next update:

    • Disk space consumed
    • Subscribers
    • Communities federated

    My guess is that’s not available vie SNMP for Librenms. :)

    • TheDudeOPMA
      link
      fedilink
      English
      arrow-up
      9
      ·
      1 year ago

      I’m being a little private about the total amount of disk assigned to this instance for now. I do plan on sharing these details when I have implemented a viable solution. That being said I can tell you that with the amount of current users and activities this instance is growing by about 20GB per day in disk size.

      • LetterboxPancake
        link
        fedilink
        English
        arrow-up
        5
        ·
        1 year ago

        Holy shit… You tell us when you’ll accept donations, right? This doesn’t have to be your financial responsibility.

        Or do you and I just haven’t found it yet?

        • TheDudeOPMA
          link
          fedilink
          English
          arrow-up
          4
          ·
          1 year ago

          Not taking donations at the moment. Still need to figure out the best way to collect donations and haven’t had the chance to think about it! I will eventually

  • PCChipsM922U
    link
    fedilink
    English
    arrow-up
    3
    ·
    1 year ago

    Can this thing run on a Pi clone with 4GB RAM and 4 cores?

    • TheDudeOPMA
      link
      fedilink
      English
      arrow-up
      4
      ·
      1 year ago

      It can definitely run on a Pi instance. The storage medium is going to be important as it will need to house a PostgreSQL database. Would be fine for a few users but not sure how many it would be able to handle concurrently.

      • PCChipsM922U
        link
        fedilink
        English
        arrow-up
        2
        ·
        1 year ago

        It’s got 4 cores at 2.16GHz (I think 🤔), 4GB of RAM. Regarding the storage, that can be arranged. Currently there’s a 32GB SD card in it, it can be swapped for a 512GB one, no prolem.

        So, how many users can an instance on a Pi like that serve?

        • TheDudeOPMA
          link
          fedilink
          English
          arrow-up
          3
          ·
          1 year ago

          My biggest concern with your setup is the SD card. SD cards aren’t typically known for their write durability. There is a way to use a SSD drive instead however its been a while since I checked the pi stats and might limited at speeds of usb 2. You’ll be fine for a small amount of users but you’re going to hit a bottleneck pretty quickly if you make your instance open to all.

          • PCChipsM922U
            link
            fedilink
            English
            arrow-up
            1
            ·
            1 year ago

            This is actually a Pi clone, BannanaPi, it has more I/Os than a Pi, but it doesn’t have SATA… so, I can’t plug in an SSD, the best I can do is plug in a USB and save and load from that, but, as you said, it’s limited to USB 2.0 transfer speed.

            Sorry, I just don’t have the funds to run a full blown server at home. A Pi is the best I can do 🤷.

            Maybe ask someone from r/homelab to run an instance or two, those people have like a lot of extra hardware, not to mention they don’t mind spending a lot of money on electricity bills.

    • boydster
      link
      fedilink
      English
      arrow-up
      2
      ·
      1 year ago

      I was looking at that earlier too. I found LemmyNet on GitHub but haven’t had a chance to test it. But “can it run” is also a fundamentally different question than “can it run smoothly for end users as it gets hit with a huge influx of users,” and given the larger context of a potential Reddit exodus, that scalability concern is probably not negligible.

      • PCChipsM922U
        link
        fedilink
        English
        arrow-up
        1
        ·
        edit-2
        1 year ago

        My point was, those that can, make one, fill it till it can serve users, then just disable registrations, so it doesn’t overload.

    • TheDudeOPMA
      link
      fedilink
      English
      arrow-up
      3
      ·
      1 year ago

      There are many to choose from. Where are you located?

    • donut
      link
      fedilink
      English
      arrow-up
      2
      ·
      edit-2
      1 year ago

      Check out Vultr. Super customizable. Have not gone down since I got mine (currently has 900 days uptime).

    • God
      link
      fedilink
      English
      arrow-up
      1
      ·
      edit-2
      1 year ago

      I’ve never hosted anything big but cloud fanatic is cheaper than digital ocean. I migrated from the latter to the former earlier this year to save. And then someone told me AWS is basically free up to a certain usage point. $200 free credits/mo? I can’t remember what the “free” part implied. As always, do your own research, but you could join one and switch servers later on I guess.

      • DevoidWisdom
        link
        fedilink
        English
        arrow-up
        1
        ·
        1 year ago

        +1 Cloud Fanatic. They were Server Cheap until a few months ago, but I guess that name turned people off. I don’t have any heavy usage so I cant really speak to that aspect, but I’ve never been over billed or unable to connect in 3 years.

    • crius
      link
      fedilink
      English
      arrow-up
      1
      ·
      1 year ago

      I’ve recently migrated a forum community (vbulletin -> discourse) using a vpn by Contabo (germany hosting). Their price are incredible and while you don’t get an official SLA, it’s around 99.5%.