134
Kaity A (@supakaity)
blahaj.zone**State of the ~~nation~~ instance**
So over the last month our instances have grown substantially and while Ada and I are privileged enough to be able to cover the cost of the small instance we initially envisioned, what it has become is somewhat bigger than that.
The Lemmy side of things is currently at 6,500 users and is hosting a larger meme community which involves a lot of image downloads and a fairly substantial bandwidth cost for serving those downloads.
As we are hosting on AWS, we pay a premium for all the resources we use and the results of this are that the hosting this month has presented us with a surprise.
Our monthly bill has jumped from about $1,000 to about $3,000 a month. (See attached image)
Unfortunately while we can manage this as a once off, I can't afford to continue in this fashion, month after month, so we will have to investigate solutions to help reduce operating costs.
This will involve moving services off AWS to less premium providers, and the downtime involved while these migrations take place and we retune everything.
I apologize in advance for the interruption while we we rejig everything through this process.
Thankyou all for understanding. Please let @ada or myself know of any issues that you notice, as glitches may occur when moving things around.
$[x2 :Blobhaj_Plead_2:] (📎1)
Discussion of the current situation with the Blåhaj instances, and upcoming maintenance.
How much bandwidth does the instance use a day?
Over a terabyte
That is insane
Yep!
That’s crazy is there a breakdown of the bandwidth? Like is it federation, images or ddos using so much bandwidth?
I know you are looking/found a solution but if the main culprit is images would it be possible to offload it to a few servers? I feel like running multiple image server instances would be cheaper for bandwidth. Oracle cloud offers 10tb outbound a month for free per region/month on the free teir. Or is an unmetered host viable? From my understanding Lemmy is pretty light in terms of CPU/memory or does that fall apart at scale?
If you don’t mind and it’s not too much trouble would you be willing to share stuff like load average/disk use?
Edit I looked this over again. Not an endorsement for giving oracle or Larry Ellison money, they are awful. I was just was trying to say the free stuff is generous. You can use 40tb outbound without paying on the free resources. As always aggressive support for you Ada! Thanks for the transparency in your community!
It’s most images from lemmy. We’ve set up a caching server for pictrs, and that has got the bandwidth part under control.
Next step, get everything else off of AWS :)
Wow! Infrastructure surely is complicated! Thank you for doing this amazing work for us ❤️